Introducing
The Slant team built an AI & it’s awesome
Find the best product instantly
Add to Chrome
Add to Edge
Add to Firefox
Add to Opera
Add to Brave
Add to Safari
Try it now
4.7 star rating
0
Development
What are the best processing tools for Big Data?
4
Options
Considered
4
User
Recs.
May 8, 2022
Last
Updated
Related Questions
Activity
Have feedback or ideas?
Join our community
on Discord
Ad
4
Options
Considered
Best processing tools for Big Data
Price
Last Updated
--
Apache Spark
-
Mar 4, 2018
--
H20.ai
-
May 8, 2022
--
Pachyderm
-
Nov 26, 2021
--
Apache Flink
-
Aug 17, 2015
See Full List
--
Apache Spark
My Rec
ommendation
for
Apache Spark
My Recommendation for
Apache Spark
All
3
Pros
2
Cons
1
Top
Pro
•••
High level API that scales
Pandas-like data frame syntax for succinctly doing powerful aggregations, with no need to pollute your code with batching or scaling logic.
See More
Top
Con
•••
High learning curve
Takes time to understand memory & time complexity of transforms, which leads to a rocky start with Spark while you keep spilling memory.
See More
Top
Pro
•••
Cheap clusters with spot instances
Using AWS spot instances or GCE preemptible VMs allows for starting cheap clusters, averaging ~10% the price of a on demand instance.
See More
Hide
See All
Get it
here
Recommend
2
--
H20.ai
My Rec
ommendation
for
H20.ai
My Recommendation for
H20.ai
Hide
Get it
here
Recommend
--
Pachyderm
My Rec
ommendation
for
Pachyderm
My Recommendation for
Pachyderm
All
2
Cons
2
Top
Con
•••
Reinventing the wheel
Recreated a "modern" hadoop mapreduce & HDFS, except with git-like data. This ties you to their system, and cannot not run any tools such as Spark/Hive on top.
See More
Top
Con
•••
Low level MapReduce API
Only ways to express transforms are: map & reduce. Developer time will be spent expressing high level concepts such as count, windowing, graph traversal in this framework, vs having Spark/Hive do it.
See More
Hide
Get it
here
Recommend
1
1
--
Apache Flink
My Rec
ommendation
for
Apache Flink
My Recommendation for
Apache Flink
Hide
Get it
here
Recommend
Don't see your favorite option? Add it.
Built By the Slant team
Find the best product instantly.
4.7 star rating
Add to Chrome
Add to Edge
Add to Firefox
Add to Opera
Add to Brave
Add to Safari
Try it now - it's free
One sec!
Are you sure that you want to abandon your hard work?
Delete Work
Continue working
{}
undefined
url next
price drop