Add to Chrome
Add to Edge
Add to Firefox
Add to Opera
Add to Brave
Add to SafariWhen comparing Flume vs Kafka, the Slant community recommends Kafka for most people. In the question“What are the best log management, aggregation & monitoring tools?” Kafka is ranked 8th while Flume is ranked 14th. The most important reason people chose Kafka is:
Kafka is optimized for supporting a huge number of users.
Specs
Question Ranking
Pros
Pro Flume is transactional (no lost when duplicating streams), and can be backuped by Kafka
Pro Hadoop integration
Flume was created to efficiently move log data to Apache Hadoop's HDFS.
Pro Free and open source
Licensed under Apache 2.0.
Pro Optimized for performance
Kafka is optimized for supporting a huge number of users.
Pro Native mirroring support
Kafka has native support for mirroring.
Pro Native compression support
Kafka has native support for compression.
Cons
Con Hard to manage
Since Flume cannot do multiplex connections, it's extremely hard to manage. It's essentially one socket per log type.
Con Java is a resource hog
Java is a resource hog, making this far too slow unless you have money to throw at multiple servers with 1/2TB of ram.
Con Need Zookeeper
Kafka can only work with Zookeeper
