JSDoc only provides a linear output of the API and not really a visually appealing or any information other than the methods and what they do. It's very technical so it may be hard for beginners to read documentation generated by it and understand it.
Since Docco doesn't enforce any comment style or rules, it's great for beginners who can instantly pick up without having to learn any new syntax or style.
Since Docco only accepts comment lines starting with double slashes, it does not really enforce any pattern in writing comments, at least not as much as it would if it would use comment blocks. This can bring a huge disparity and even make it harder for Docco to generate good documentation as the project grows larger.
dox has built-in support for all jsDoc tags and allows you to specify complex object types including optional flag =, nullable ?, non-nullable ! and variable arguments ...
dox only generates a JSON representation of the documentation. In order to have a fully readable documentation you need to integrate it with some other library that parses the JSON and returns HTML or markdown.