Add to Chrome
Add to Edge
Add to Firefox
Add to Opera
Add to Brave
Add to SafariWhen comparing Fluentd vs Kafka, the Slant community recommends Fluentd for most people. In the question“What are the best log management, aggregation & monitoring tools?” Fluentd is ranked 4th while Kafka is ranked 8th. The most important reason people chose Fluentd is:
Gives structure to unstructured logs.
Specs
Question Ranking
Pros
Pro Logs everything in JSON
Gives structure to unstructured logs.
Pro Logs stored to FS buffer while network is down
Logs aren't lost due to network issues.
Pro In-stream processing
With a list of 150+ plugins, Fluentd can perform all kinds of in-stream data processing tasks.
Pro Huge plugin ecosystem
Fluentd has a plugin ecosystem that has resulted in developers creating over 150 plugins for the service.
Pro Prioritizes simplicity and robustness
For example, inputs and outputs have built-in support for buffering, load-balancing, timeouts and retries so to be able to deliver data reliably.
Pro Free and open source
Licensed under Apache 2.0.
Pro Routing based on tags
Pro Exponential retry wait
Pro Copy to multiple storages
Pro Based on CRuby
Pro Optimized for performance
Kafka is optimized for supporting a huge number of users.
Pro Native mirroring support
Kafka has native support for mirroring.
Pro Native compression support
Kafka has native support for compression.
Cons
Con Difficult to setup
Requires a significant time investment to get up and running.
Con Java is a resource hog
Java is a resource hog, making this far too slow unless you have money to throw at multiple servers with 1/2TB of ram.
Con Need Zookeeper
Kafka can only work with Zookeeper
