the core of hadoop is currently hdfs + mapreduce. the more appropriate question is if it will become hdfs + spark. so will spark overtake mapreduce as the dominant computational engine? its a very serious candidate for that i think. it can do many things mapreduce cannot do, and has an awesome api.
it's missing a few things to truly replace mapreduce: * handling data that does not fit in memory per key/reducer * security support (integrate with hdfs authorization/authentication) * scalability??? (has spark been tested on 1000 machines) On Mon, Nov 18, 2013 at 12:38 AM, jamal sasha <jamalsha...@gmail.com> wrote: > I found this interesting question on quora.. and thought of sharing here. > https://www.quora.com/Apache-Hadoop/Will-spark-ever-overtake-hadoop > So.. is spark missing any capabilty? > >
