Thanks for sharing Emanuele! Looking forward to providing built-in Docker support in Beam.
On Fri, Jun 24, 2016 at 9:30 AM, Amit Sela <[email protected]> wrote: > You're right about standalone, I know many companies (small-medium) > companies that prefer spawning standalone per use case/s. I'm currently > biased now towards large clusters because of my current work place ;) which > relates better to my previous comment. > > > On Fri, Jun 24, 2016, 03:42 Emanuele Cesena <[email protected]> wrote: >> >> Thanks Amit! >> >> I chose Flink because of the current capability support and for the nicer >> front end UI, but I have nothing against Spark — actually I’m using Spark in >> my daily job, and chances are that if we’ll use Beam, it will be on Spark >> first. >> >> I can also tell you that I know of 2 instances (MemSQL, that distribute >> its own Spark, and our parent company SK Planet in Korea) that prefer Spark >> standalone, mostly for performance and easy of setup. So I can see a lot of >> potential even in production environments. >> >> Best, >> >> >> > On Jun 23, 2016, at 3:42 PM, Amit Sela <[email protected]> wrote: >> > >> > Thanks for sharing Emanuele, I will definitely look into trying >> > something like that with Spark as well :) >> > While production clusters (usually) use YARN/Mesos to manage resources, >> > this could be really great for developers to use on a virtual environment. >> > Really interesting! >> > >> > On Thu, Jun 23, 2016 at 7:21 PM Emanuele Cesena <[email protected]> >> > wrote: >> > Thank you Aljoscha! >> > >> > > On Jun 23, 2016, at 1:19 AM, Aljoscha Krettek <[email protected]> >> > > wrote: >> > > >> > > It's a very nice write up indeed! Thanks for sharing. :-) >> > > >> > > On Thu, 23 Jun 2016 at 07:35 Jean-Baptiste Onofré <[email protected]> >> > > wrote: >> > > Hi Emanuele, >> > > >> > > this is a great example ! >> > > >> > > It shows Beam with Flink. Maybe we can enhance a bit showing how the >> > > same pipeline can result to different docker depending of the backend. >> > > >> > > I'm working on new "concrete" Beam samples showing that: >> > > >> > > https://github.com/jbonofre/beam-samples >> > > >> > > Great work anyway ! >> > > >> > > Regards >> > > JB >> > > >> > > On 06/22/2016 10:18 PM, Emanuele Cesena wrote: >> > > > Hi, >> > > > >> > > > I just published a "quick start" with Beam and wanted to share: >> > > > >> > > > https://medium.com/@ecesena/a-quick-demo-of-apache-beam-with-docker-da98b99a502a >> > > > >> > > > Related repos: >> > > > https://github.com/ecesena/docker-beam-flink >> > > > https://github.com/ecesena/beam-starter >> > > > >> > > > Any feedback is more than welcome! >> > > > >> > > > Best, >> > > > E. >> > > > >> > > >> > > -- >> > > Jean-Baptiste Onofré >> > > [email protected] >> > > http://blog.nanthrax.net >> > > Talend - http://www.talend.com >> > >> >
