Botify is a crawler and site analysis tool for optimizing SEO of large and complex websites.
Ranked in these QuestionsQuestion Ranking
Pro Very powerful once combined with Googlebot log data
Botify lets you compare what urls googlebot is actually crawling vs the compliant urls you're trying to get indexed. This is an incredibly powerful feature for complex/large websites where you're trying to get longtail content ranked. Using this data you can optimise your sitemaps etc to point google to content it is currently missing.
Pro Lets you compare the differences between crawls
The data in botify is broken up into current urls, disappeared urls and new urls. This will let you keep track of changes made to your site between crawls.
Pro Fantastic UI for filtering URLS/data
The botify UI for building reports is very polished and powerful. It supports regex, multiple levels of AND/OR and lets you filter on pretty much any field/data you can think of.
Pro Configurable crawls
You can set how deep Botify will crawl into your website, as well as how fast and how many URLs. Alternatively, you can set only certain sections of your website to be crawled. Previous crawls can be compared to new crawls to show differences and improvements.
Pro Easy sharing
You can easily share reports with co-workers, and set each user to an authorization level (guest/admin/owner) to ensure the right people can make edits, and others can only view the information.
Pro Crawls through competitors sites
Botify will also crawl through competing sites, allowing you to compare your SEO to theirs, and see where would be most beneficial for you to improve your site's SEO.
Con Very expensive subscriptions
While SMB or freelancer plans are available starting at $49/month, the regular plans cost $569, $999, or $2500 per month which is out of reach of most website developers.