Hi, This is related with the implementation of a beam interpreter for Apache zeppelin. I think for the first phase, DirectPipelineRunner will do the job :) Please let me know if there is anything which can be helpful.
Thanks and regards. Minudika Minudika Malshan Undergraduate Department of Computer Science and Engineering University of Moratuwa Sri Lanka. On Thu, Mar 10, 2016 at 12:11 PM, Jean-Baptiste Onofré <j...@nanthrax.net> wrote: > By the way, on my side, I will work on a Karaf/OSGi ( > http://karaf.apache.org) runner for Beam (with shell commands, features, > etc). > I will start it just after the work on new IOs. > > Regards > JB > > > On 03/09/2016 08:01 PM, Minudika Malshan wrote: > >> Hi, >> >> Thanks a lot for your quick responses. >> I will refer those resources. >> >> Regards, >> Minudika >> >> Minudika Malshan >> Undergraduate >> Department of Computer Science and Engineering >> University of Moratuwa >> Sri Lanka. >> >> >> >> >> On Thu, Mar 10, 2016 at 12:24 AM, Lukasz Cwik <lc...@google.com.invalid> >> wrote: >> >> There are currently two implementations which do not require the cloud: >>> >>> The DirectPipelineRunner >>> < >>> >>> https://github.com/apache/incubator-beam/blob/master/sdk/src/main/java/com/google/cloud/dataflow/sdk/runners/DirectPipelineRunner.java >>> >>>> >>>> which is mainly used for testing and local development. This runner has >>> several limits (data size, no support for unbounded collections, ...) and >>> is being expanded to support more use cases, for example adding unbounded >>> PCollection support <https://issues.apache.org/jira/browse/BEAM-22>. >>> >>> The FlinkPipelineRunner >>> <https://github.com/apache/incubator-beam/tree/master/runners/flink> >>> which >>> can be used to execute locally or on a Flink cluster. >>> >>> There is also ongoing work to bring Spark >>> <https://issues.apache.org/jira/browse/BEAM-6> into the mix as a runner >>> and >>> suggestions to for other runners such as GearPump >>> <https://github.com/gearpump/gearpump>. >>> >>> On Wed, Mar 9, 2016 at 10:37 AM, Minudika Malshan <minudika...@gmail.com >>> > >>> wrote: >>> >>> Hi all, >>>> >>>> As per my knowledge about Apache beam and data flow sdk, at the first >>>> >>> data >>> >>>> flow sdk has been developed targeting google cloud platform. >>>> So we have to deploy pipelines in the cloud. >>>> >>>> But my question is, can not we use this sdk for standalone >>>> >>> implementations >>> >>>> without cloud. If so, I would love to have a look at some examples of >>>> >>> such >>> >>>> implementations. >>>> Your kind help is much appreciated. >>>> >>>> Regards, >>>> Minudika >>>> >>>> Minudika Malshan >>>> Undergraduate >>>> Department of Computer Science and Engineering >>>> University of Moratuwa >>>> Sri Lanka. >>>> >>>> >>> >> > -- > Jean-Baptiste Onofré > jbono...@apache.org > http://blog.nanthrax.net > Talend - http://www.talend.com >
