When comparing Codecov vs Codacy, the Slant community recommends Codecov for most people. In the question“What are the best code coverage services?” Codecov is ranked 1st while Codacy is ranked 5th. The most important reason people chose Codecov is:
Great detailed pull request comments including Codecov's Coverage Diff which breaks down coverage changes in the pull request. Comments are updated after every successful CI build.
Specs
Ranked in these QuestionsQuestion Ranking
Pros
Pro Detailed pull request comments
Great detailed pull request comments including Codecov's Coverage Diff which breaks down coverage changes in the pull request. Comments are updated after every successful CI build.
Pro Free version
Codecov is free for up to 5 users and open source repositories on Github, Bitbucket, and Gitlab.
Pro Grouping Coverage Reports
Codecov groups coverage reports to isolate coverage metrics. For example: unittest
vs functional
. Learn more here.
Pro Github commit status
Commit statuses are posted to maintain minimum coverage percentage.
Pro Coverage reports can be integrated into Github and Bitbucket with a browser extension
Overlay coverage reports directly in Github and Bitbucket for seamless integration into your workflow.
Pro Good customer support
Codecov has great customer support. Especially because it uses Intercom for communication which makes it easy to ask questions or make feature requests.
Pro Free for open-source projects
Pro Tons of features, plugins to check code
The list is never ending: they use a lot of open sourced linters for many different languages, allowing to configure them one by one, add custom rules... It checks duplicates, issues, security AND is customizable.
Cons
Con Expensive - paid per user/month
Per-user pricing increases the pricing for us by 2600% compared to before.
Con Frequent downtime not reflected in status page
Often codecov.io URL returns 504 or 503 errors to the build scripts resulting in failed CI builds. Downtime is never reflected in their status page.
Con Node package doesn't support piped input
Most Node coverage uploaders allow you to pipe directly from a coverage tool into the coverage uploader, but Codecov seems to require that coverage is written to a file.
Con Browser extension not available for Safari
Con Reports tons of noisy, irrelevant "quality" metrics
You want code coverage, not someone to tell you where to put semicolons in your markdown documents. It spots dozens of "issues" in your code and Markdown sources, none of which are in any way valid, most of which were random opinions from people with no justification and not accepted best practices. (In fairness, on the Python side, this was due largely to Pylint itself being way too prescriptive.)
Con Can't handle combined coverage
When there are multiple environments that contribute coverage to different branches it just picks one at random and says that the rest of the code isn't covered at all.
Con Dashboard causes Safari to freeze, crash
Con API Tokens need to be hard coded
Most CIs have options for that in safe environment variables, but that makes it unavailable in PRs anyways. The support says they're working on it.