CircleCI excels with its setup process. All that's needed is a GitHub login and CircleCI automatically detects the settings for Ruby, Python, Node.js, Java and Clojure. The setup process is their most widely praised feature.
CircleCI can be connected to any project that is hosted on GitHub by logging in using the GitHub OAuth and adding the desired repository.
Whenever a new commit is pushed to GitHub, CircleCI runs the tests that have been already defined and if none of them fails, the build is deployed to the runtime environment.
Circle CI's web UI is clean and easy to use.
It gives all the information for a single build in a feed and gives the explanation for each step of the build, what it's doing and what the step is related to. On the top it displays author information and the time and date when the build was started and finished.
This is all done by giving only the most essential information without clogging the screen.
It als has support for: MySQL, MongoDB, PostgreSQL, Cassandra, Riak, Redis, SQLite, Solr, CouchDB, ElasticSearch, Neo4j, Couchbase, Lucene, Sphinx, ThriftDB, Memcache.
Support for Heroku, AWS, Engine Yard, dotCloud, Fabric, Nodejitsu, AppFog, Capistrano, Rockspace, Joynet.
Integration with Heroku is solid with the ability to automatically deploy or merge branches.
CircleCI is also very flexible with the deployment arrangement allowing SSH key management, deployment freedom including directly to a PaaS, using Capistrano, Fabric, arbitrary bash commands, or by auto-merging to another branch, or packaging code up to S3.
Currently (October 5th 2016), Docker installed on the VM is: 1.9.1-circleci-cp-workaround, build 517b158, and docker-compose is 1.5.2, build 7240ff3. docker-compose in particular is almost too old to be used.
CircleCI has support only for projects hosted on GitHub so teams that use BitBucket or any other alternative to GitHub are forced to rely on another CI tool or use third-party solutions to be able to integrate CircleCI with BitBucket. One of those solutions may be Cloudpipes.
The fact that Shippable runs inside of Docker means that it keeps a persistent state and every build will not have to revert to initial state where it needs to install every dependency from the ground up. Classic CI tools that run on virtual machines need to reset their environment every time and every time install the gems, packages and services needed.
All Shippable needs for it's setup is a shippable.yml file in the root of the repository that needs to be built. The bare minimum Shippable needs is the language and the version number specified in that file.
Shippable is built using Docker, a popular open source Linux container. It was originally built using it's own container but when that started to become too complex, they switched to using Docker. Since the beginning Shippable was different from other CI tools because while Shippable uses a container (Docker), traditionally CI tools have used virtual machines to manage their workloads.
Builds are described in the shippable.yml file located in the root of your project. This empowers engineers to take responsibility for code delivery. If you are coming from Travis CI, Shippable reads your .travis.yml file directly so you can try it out painlessly.
Currently, Shippable does not allow for build artifacts to be natively deployed to S3. This can be gotten around, however it is a rather large hole when compared to Travis.
In order to deploy to S3 you have to add a couple of lines to the yml file. For example:
#secure variable contains values for AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
- secure: HKwYujx/qmsyQQdHvR2myu8HLUDtcLeDyYV149YJuxIV4J7Hk3SxeY8X3D6aTlR8mvMnd/ZFY+tGNUh4G0xtLLjjZcPsBgvFlB
- aws s3 sync $SHIPPABLE_BUILD_DIR "s3://bucket_name" --region "us-east-1"
Shippable runs inside Docker containers. Docker has some specific security measures which may or may not become a hindrance in using Shippable. It may be harder for users who are not very comfortable with a Linux container environment and that can create some security problems. Even for more advanced users, it's still something more that they have to address while using Shippable.
When registering with Bitbucket Shippable it requests way too many permissions, even "Read and write to your team's projects and move repositories between them". Before giving all these permissions you have to be sure you can trust this service.
Semaphore is quite easy to configure and work with. It easily integrates with GitHub and a first build is only a few clicks away.
Semaphore is configured using .yaml configuration files which can be added from the web UI. There are a lot of tutorials out there that help developers configure Semaphore to their preference.
Semaphore is not free and nor is it open source. Pricing starts at $29 per month. However, there is a free option for private projects which have less than 100 builds per month and it's free for open source projects.
When registering with Bitbucket Codeship it requests way to many permissions, even "Read and write to your team's projects and move repositories between them". Before giving all these permissions you have to be sure you can trust this service.
Support for e-mail, HipChat, Slack, Campfire, Flowdock, Grove, Webhook, Github Status API.
Support for Ruby, Python, Node, Dart, PHP, Java, Scala, Groovy, Clojure, Go.
Support for: PostgreSQL, MySQL, MongoDB, Redis, Memcached, ElasticSearch, SQLite.
Wercker offers free unlimited private repositories as long as they are still in their beta stage. But once that's over, they will most likely change this to a paid service and no one knows how much it will cost.
GitLab CI's tests run parallel to each other distributed on different machines. Developers can add as many machines as they want or need, this makes GitLab CI highly scalable to the development team's needs.
There are various types of supported builders, including builders running the builds inside a Docker image specified in the project (under source control). You can have various steps run on different images, including private ones.
Bamboo allows using Docker containers to create build agents. Using Docker agents lets you run multiple remote agents on the same host without conflicting requirements. It makes it easier to duplicate and distribute changes to build agents, and to use scripts for creating and maintaining agents.
How can you define and build your own image and push it to a registry to share? This is when Bamboo’s Docker tasks come into play. Docker tasks make it possible to build an image, run a container, and push a Docker image to a registry from within your build or deployment project.
When connecting Bamboo with Stash and JIRA, details like JIRA issues, commits, reviews and approvals follow each release from development to production. If HipCHat is part of the integration, team members get notified right away in addition to email notifications.
Bamboo is the only build server to offer first-class support for the "delivery" aspect of continuous delivery. Deployment projects automate the tedium right out of releasing into each environment, while letting you control the flow with per-environment permissions.
Avoid plugin hell by having most important capabilities as out-of-the-box features, not plugins. Bamboo is not just built for teams, but teams-of-teams. It has the administrative features you need to manage and maintain CI at scale. Enterprise model for access control, management, and support.
Deployments are configured to run in sequence, one after one in the build pipeline. Not every environment is updated on every change, this is especially helpful for example for someone who is testing the application and does not want their environment to change while they are doing it.
Snap offers developers a simple and easy to use tool for debugging their applications. Just by typing snap-shell on their pipeline configuration they will have the ability to visit their logs page where they can start debugging. The log page shows a command prompt with some help on how to use it.
Snap watches for any new branch that is created and once it catches one of these new branches, if the master branch is configured for a pipeline it will create a new pipeline around the new branch. Once the branch is deleted or merged with master, the newly created pipeline is deleted by Snap
When registering with Bitbucket drone.io it requests way to many permissions, even "Read and write to your team's projects and move repositories between them". Before giving all these permissions you have to be sure you can trust this service.
There's no configuration files to write, nothing to download or install, nothing to really configure. Setting it up takes basically only adding your Git repository and waiting for the clone, build and test to finish.
AppVeyor's configuration which is done from the .yaml file in the root of the project is unfortunately very limited. The configuration is either tied to a branch or in other cases it's global. This limits the developer to a single build process.