Hi Sergio, It was great talking with you in Vancouver. As of today, the Python SDK is here, [1], [2]. Wasn't that fast enough ;)
Davor [1] https://github.com/apache/incubator-beam/pull/461 [2] https://github.com/apache/incubator-beam/tree/python-sdk/sdks/python On Tue, Jun 14, 2016 at 3:45 AM, Jean-Baptiste Onofré <[email protected]> wrote: > Hi Sergio, > > Welcome aboard, and good to discuss with you during ApacheCon. > > Distribution of the resources is a point related to runner, and more > specifically to the execution environment of the runner. Each > runner/backend will implement their own logic. > > I don't know Keras enough to provide a strong advice. > > Regarding the Python SDK, we discussed about that last week: it's on the > way. We should have the Python SDK very soon (we were busy with the first > release). > > Regards > JB > > > On 06/14/2016 12:38 PM, Sergio Fernández wrote: > >> Hi guys, >> >> I'm newbie in the Beam community, but as someone who has used DataFlow in >> the past I've been following the podling since you came to ASK. I'm very >> happy to see that 0.1.0-incubating is finally going out, congratulations >> for such great milestone. >> >> I discussed with some of you guys in the last ApacheCon, and for me was >> good to know the Python SDK was just a matter of time and should come to >> Beam at some point. So coming back to the original plans < >> >> http://beam.incubator.apache.org/beam/python/sdk/2016/02/25/python-sdk-now-public.html >> >, >> do you manage any timeline to bring the Python SDK to Beam? >> >> So I'd like to bring a question how Beam plans to deal with the >> distribution of resources across all nodes, something I know it not really >> clean with some runners (e.g., Spark). More concretely, we're using Keras >> < >> http://keras.io/>, a deep learning Python library that is capable of >> running on top of either TensorFlow or Theano. Historically I know >> DataFlow >> and TensorFlow are not very compatible. But I wonder if the project has >> already discussed how to support running Keras (TensorFlow) tasks on Beam. >> For us is more for querying than for training, so I'd like to know if the >> Beam Model could natively support the distribution of the models >> (sometimes >> several GB). >> >> Thanks in advance. >> >> Cheers, >> >> > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
