Re: Help with Dynamic writing

yes. You were right, I had to put back the pc. I am working on the partition function and try to debug it without running a pipeline on dataflow (dataflow execution takes at list 8 minutes for any size of data), based on the link:

Re: beam 3?

Probably too quickly done but can be used as a start here is a first list: https://gist.github.com/rmannibucau/ab7543c23b6f57af921d98639fbcd436 Romain Manni-Bucau @rmannibucau | Blog | Old Blog

Re: beam 3?

Note that for the 2.0 release, we tracked this list of changes in JIRA under "backward-incompatible" labels. https://issues.apache.org/jira/browse/BEAM-2427?jql=project%20%3D%20BEAM%20AND%20labels%20in%20(backwards-incompatible%2C%20backwards-compatibility%2C%20backward-incompatible) We could do

Re: beam 3?

I, too, think it's way to early to move master to 3.0.0. Especially if this involves reworking everything from the API to runners, possibly from scratch. Right now, I think the most important and urgent work is the many efforts underway to fully realize the portability story. I'm also concerned

Re: beam 3?

Le 21 mars 2018 18:25, "Lukasz Cwik" a écrit : I think its immature to start a new major version at this point in time. * Apache Beam 2.x series is less then a year old. * Many features that users want can be built on top of the existing APIs. Oki, how do you see: 1.

debugging python jenkins precommit

Hi, I'm trying to debug a jenkins precommit error for PR #4877. (Side rant: It's taking a long time to run precommits (between 24m and 3h), and I don't have access to the jenkins VM to debug things locally.) Partial log: py27-cython create:

Re: beam 3?

I think its immature to start a new major version at this point in time. * Apache Beam 2.x series is less then a year old. * Many features that users want can be built on top of the existing APIs. On Wed, Mar 21, 2018 at 1:31 AM Jean-Baptiste Onofré wrote: > Hi, > > Starting

Re: Help with Dynamic writing

On Wed, Mar 21, 2018 at 7:53 AM OrielResearch Eila Arich-Landkof < e...@orielresearch.org> wrote: > Hi Cham, > > *all_data = pcollections | beam.Flatten()* > > fires an error: > > TypeError: 'Read' object is not iterable > > > pcollections is the following list: > > [

Re: From Beam Summit - On SDKs and Contributor Experience

Hello Pablo, Thank you for sharing this! As a recent contributor I can confirm most of the issues that you raised in this document, especially Jira and PR unclearness. In the same time, current Beam contribution guide is one of the most detailed and full that I’ve ever seen before and it

Re: Help with Dynamic writing

Hi Cham, *all_data = pcollections | beam.Flatten()* fires an error: TypeError: 'Read' object is not iterable pcollections is the following list: [, ,

Re: From Beam Summit - On SDKs and Contributor Experience

Thanks for sharing, this is very usefull for those who were not there. I left a couple of comments. Etienne Le mardi 20 mars 2018 à 22:40 +, Pablo Estrada a écrit : > Hello everyone, > at the Beam Summit in San Francisco, a number of folks had a breakout session > where we considered

Re: Help with Dynamic writing

very helpful!!! i will keep you posted if I have any issue / question Best, Eila On Tue, Mar 20, 2018 at 5:08 PM, Chamikara Jayalath wrote: > > > On Tue, Mar 20, 2018 at 12:54 PM OrielResearch Eila Arich-Landkof < > e...@orielresearch.org> wrote: > >> Hi Cham, >> >>

Re: [Proposal] Defining and Adding SDK Metrics - Looking for Feedback

Hi Alex, I left a minor comment as well (but shows as "Anonymous") Etienne Le mercredi 14 mars 2018 à 03:47 +, Alex Amato a écrit : > Hello beam community, > > I have put together a proposal, and I would like to get some initial feedback > to improve upon the ideas here. > > The proposal

Build failed in Jenkins: beam_Release_NightlySnapshot #720

See Changes: [herohde] [BEAM-3817] Switch BQ write to not use side input [herohde] Add TODO to revert Go IO to use side input [axelmagn] Fix StateRequestHandler interface to be idiomatic [herohde]

Re: beam 3?

Hi, Starting from scratch is an option, but don't you think it's a huge effort ? Anyway, we will reuse part of the existing codebase. Let's see what the team is thinking. Regards JB On 03/21/2018 09:26 AM, Romain Manni-Bucau wrote: > > > 2018-03-21 9:13 GMT+01:00 Jean-Baptiste Onofré

Re: beam 3?

2018-03-21 9:13 GMT+01:00 Jean-Baptiste Onofré : > Hi Romain, > > We didn't define a date yet. > > However, I think it makes lot of sense to think about that. > > What about creating a beam-2.x branch and move master version to > 3.0.0-SNAPSHOT ? > Do we want to "move" master

Re: beam 3?

Hi Romain, We didn't define a date yet. However, I think it makes lot of sense to think about that. What about creating a beam-2.x branch and move master version to 3.0.0-SNAPSHOT ? I almost agree with your point even if I would suggest you to use a more positive tone: being sharp never

beam 3?

Hi guys, it got mentionned but without any concrete dates: when beam 3 work will be started? I'm very interested in: 1. reworking the whole DAG API to ensure it is instrumentable (today the dag uses a tons of static utilities and internals which makes it not industrializable at all as soon as