Beam is a project from Google and its been used internally at Google for a while and was recently donated to Apache by Google.
I have limited experience working with it, and presently prototyping it at work for a project. You may want to check out the below links. https://cloud.google.com/blog/big-data/2016/05/why-apache-beam-a-google-perspective https://www.oreilly.com/ideas/future-proof-and-scale-proof-your-code http://data-artisans.com/why-apache-beam/ https://cloud.google.com/dataflow/blog/dataflow-beam-and-spark-comparison On Mon, Jul 18, 2016 at 2:02 PM, Ellison Anne Williams < [email protected]> wrote: > Hi Guys, > > Moving the Beam disc to a new thread as Suneel suggested... > > I am in favor of integrating with Beam, but not depending on it until we > have a chance to really check it out with Pirk. I have not been able to > find developer good info on Beam, but their website states that the project > is currently 'bootstrapping'. > > I propose that we try a Beam on the existing Spark batch process to see if > we can achieve good scalability. > > Suneel - Do you have direct experience working with Beam? > > Thanks! > > Ellison Anne >
