Re: Python SDK worker / portable Flink runner performance improvements

2018-10-21 Thread Thomas Weise
Regarding the functionality: https://s.apache.org/apache-beam-portability-support-table While we still have a good chunk of work to do, the MVP feature set is in place and allows to run pipelines. Before we check P2 (feature complete), I would like to see (in addition to what Max mentioned): *

Re: [Proposal] Euphoria DSL - looking for reviewers

2018-10-21 Thread Kenneth Knowles
This discussion veered into territory reserved for priv...@beam.apache.org [1]. But the PMC has agreed that an update is deserved here: We are taking it very seriously that you have placed faith in Beam and that you need to be able to effectively continue development of Euphoria. We are actively

Re: Docker missing on Beam15

2018-10-21 Thread Thomas Weise
There are two issues with https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/ currently: 1) The mentioned issue with docker on beam15 - Jason, can you possibly advise how to deal with it? 2) Frequent failure due to "Segmentation fault (core dumped)", as exhibited by

Possible memory leak in Direct Runner unbounded

2018-10-21 Thread Martin Procházka
Hello, I have got an application, which utilizes Beam pipeline - Direct Runner. It contains an unbounded source. I have got a frontend, which manually adds some data into the pipeline with the same timestamp in order to be processed in the same window. The pipeline runs well, however it

2 tier input

2018-10-21 Thread Chaim Turkel
hi, I have the following flow i need to implement. >From the bigquery i run a query and get a list of id's then i need to load from mongo all the documents based on these id's and export them as an xml file. How do you suggest i go about doing this? chaim -- Loans are funded by FinWise