Re: [Proposal] Add exception handling option to MapElements

It sounds interesting. Thanks ! Regards JB On 03/10/2018 01:49, Jeff Klukas wrote: > I've seen a few Beam users mention the need to handle errors in their > transforms by using a try/catch and routing to different outputs based > on whether an exception was thrown. This was particularly nicely wr

Re: SplittableDoFn

Nice one Alex ! Thanks Regards JB On 02/10/2018 23:19, Alex Van Boxel wrote: > Don't want to crash the tech discussion here, but... I just gave a > session at the Beam Summit about Splittable DoFn's as a users > perspective (from things I could gather from the documentation and > experimentation)

Re: Source or ParDo for reading data in PostgreSQL in Python

I found that I forgot to perform a beam.Create() first... So no need to answer me, this solves my problem. On 2018/10/02 08:40:46, Jonathan Perron wrote: > Hello,> > > I am looking for some way to access data stored in PostgreSQL and don't > > know if I should go for a Sink or ParDo operation

Re: Purpose of GcpApiSurfaceTest in google-cloud-platform SDK

Worth noting that these API surface tests are not in a great state; they test everything on the class path rather than just true dependencies. I don't know that they still ensure the desired property; they certainly reject things that they need not. From your description, it sounds like in your cas

Re: JIRA permission request

Thanks Kenneth, filed and submitted :) Kengo Seki On Wed, Oct 3, 2018 at 11:21 AM Kenneth Knowles wrote: > > Done. Thanks in advance for your contribution! > > On Tue, Oct 2, 2018 at 7:05 PM Kengo Seki wrote: >> >> Hi, >> >> I found a minor issue in JdbcIO Javadoc and would like to submit a PR

Re: [Proposal] Add exception handling option to MapElements

Great essay. Thanks for the link! I think this idea is very cool. It takes idiomatic Java exception-throwing in myFunctionThatThrows and turns it into the FP fundamentals, where success/failure(s) form a disjoint union and you partition on tag. I'm interested in the details. Maybe also worth doing

Re: JIRA permission request

Done. Thanks in advance for your contribution! On Tue, Oct 2, 2018 at 7:05 PM Kengo Seki wrote: > Hi, > > I found a minor issue in JdbcIO Javadoc and would like to submit a PR for > it. > Would you add me to the contributor list? My JIRA id is "sekikn". > > Thanks! > > Kengo Seki >

JIRA permission request

Hi, I found a minor issue in JdbcIO Javadoc and would like to submit a PR for it. Would you add me to the contributor list? My JIRA id is "sekikn". Thanks! Kengo Seki

[Proposal] Add exception handling option to MapElements

I've seen a few Beam users mention the need to handle errors in their transforms by using a try/catch and routing to different outputs based on whether an exception was thrown. This was particularly nicely written up in a post by Vallery Lancey: https://medium.com/@vallerylancey/error-handling-ele

Re: SplittableDoFn

Very cool, thanks Alex! On Tue, Oct 2, 2018 at 2:19 PM Alex Van Boxel wrote: > Don't want to crash the tech discussion here, but... I just gave a session > at the Beam Summit about Splittable DoFn's as a users perspective (from > things I could gather from the documentation and experimentation).

Jenkins build is back to normal : beam_SeedJob #2734

See

Build failed in Jenkins: beam_SeedJob #2733

See -- GitHub pull request #6549 of commit f844fb681ac7154e371f9fe92b25e36be331c937, no merge conflicts. Setting status of f844fb681ac7154e371f9fe92b25e36be331c937 to PENDING with url https

Re: SplittableDoFn

Don't want to crash the tech discussion here, but... I just gave a session at the Beam Summit about Splittable DoFn's as a users perspective (from things I could gather from the documentation and experimentation). Her is the slides deck, maybe it could be useful: https://docs.google.com/presentatio

Re: Purpose of GcpApiSurfaceTest in google-cloud-platform SDK

Hi Ken, My understanding is that this test is intended to prevent other packages from appearing on the public API surface of the Beam package. For example, guava can't appear on the Beam public API. This is to enable users to depend on different versions of these packages then what Beam depends on

Purpose of GcpApiSurfaceTest in google-cloud-platform SDK

Hi folks, I'm working on adding support for a new API to the google-cloud-platform SDK, and some of my changes have caused the API surface test to start f

Re: Python 3: final step

Very cool : ) I'm also available to review / merge if you need help from my side. Best -P. On Tue, Oct 2, 2018 at 7:45 AM Rakesh Kumar wrote: > Hi Rob, > > I am, Rakesh Kumar, using Beam SDK for one of my projects at Lyft. I have > been working closely with Thomas Weise. I have already met a cou

Re: Python 3: final step

Hi Rob, I am, Rakesh Kumar, using Beam SDK for one of my projects at Lyft. I have been working closely with Thomas Weise. I have already met a couple of Python SDK developers in person. I am interested to help migrate to Python 3. You can assign me PRs for review. I am also more than happy to take

Re: Python typing library is not provisional in Python 3.7

For record, I logged https://issues.apache.org/jira/browse/BEAM-5607 and sent a PR. This may also help anyone who run the tests locally in Python 3.7. 在 2018年9月29日 +0800 AM3:06，Valentyn Tymofieiev ，写道： > Hi Manu, > > I second what Ahmet said - thanks for the pointers. Python 3.7 support can > com

Re: Source or ParDo for reading data in PostgreSQL in Python

Hello again, Thanks to https://github.com/apache/beam/blob/master/sdks/python/apache_beam/examples/complete/estimate_pi.py, I saw that my first step must be a beam.Create() with an iterable. Doing so solved my problem. Sorry for my mistake. On 2018/10/02 08:40:46, Jonathan Perron wrote: >

Re: Source or ParDo for reading data in PostgreSQL in Python

Hi Jonathan, I had a similar requirement as yours, but for mongoDB and I tentatively wrote an IO Connector for it that you can find here: https://github.com/PEAT-AI/beam-extended It is working with the DirectRunner in Read mode (I need to do some test then on DataFlow). But I faced some issue with

Source or ParDo for reading data in PostgreSQL in Python

Hello, I am looking for some way to access data stored in PostgreSQL and don't know if I should go for a Sink or ParDo operations. It is stated that ParDo could be used but I'm not sure this is what will solve my problem, so here I am !I managed to write in the database with only ParDo operat