Re: (java) stream & beam?

The stream API was looked at way back when we were designing the API; one of the primary reasons it was not further pursued at the time was the demand for Java 7 compatibility. It is also much more natural with lambdas, but unfortunately the Java compiler discards types in this case, making coder

Re: Splittable DoFN in Spark discussion

How would timers be implemented? By outputing and reprocessing, the same way you proposed for SDF? On Wed, Mar 14, 2018 at 7:25 PM Holden Karau wrote: > So the timers would have to be in our own code. > > On Wed, Mar 14, 2018 at 5:18 PM Eugene Kirpichov

Re: Splittable DoFN in Spark discussion

So the timers would have to be in our own code. On Wed, Mar 14, 2018 at 5:18 PM Eugene Kirpichov wrote: > Does Spark have support for timers? (I know it has support for state) > > On Wed, Mar 14, 2018 at 4:43 PM Reuven Lax wrote: > >> Could we

Re: Beam 2.4.0

Fully agree. If nobody jumps on it, I will tackle that tomorrow. Regards JB Le 14 mars 2018 à 18:03, à 18:03, Reuven Lax a écrit: >Can you add a description and assign a reviewer (using R:). I think >this >was basically ready to merge before modulo breaking compilation. > >

Re: Beam 2.4.0

Can you add a description and assign a reviewer (using R:). I think this was basically ready to merge before modulo breaking compilation. On Wed, Mar 14, 2018 at 10:01 AM Romain Manni-Bucau wrote: > up, know it missed the 2.4 but can >

Re: Splittable DoFN in Spark discussion

Does Spark have support for timers? (I know it has support for state) On Wed, Mar 14, 2018 at 4:43 PM Reuven Lax wrote: > Could we alternatively use a state mapping function to keep track of the > computation so far instead of outputting V each time? (also the progress so >

Splittable DoFN in Spark discussion

So we had a quick chat about what it would take to add something like SplittableDoFns to Spark. I'd done some sketchy thinking about this last year but didn't get very far. My back-of-the-envelope design was as follows: For input type T Output type V Implement a mapper which outputs type (T, V)

Re: New beam contributor experience?

Good point. As mentioned, maybe the contribution guide is not enough, we also have the "low hanging fruit" tagged Jira that new contributors can take a look. Maybe for both users and new contributors, it would be great to have additional concrete examples. Today, the game examples is great

Re: New beam contributor experience?

Some things that we have found working with our friends at IBM are: - No Python install from code documentation (which I've brought up before, and I believe is a known issue) - The process to pick up a JIRA is not very clear. It seems that there are a lot of implicit conventions (is discussion

Re: New beam contributor experience?

To add more to what Anton said, the 'mvn clean verify' step takes hours and fails frequently due to bad tests. I spent the first few days working with beam trying to figure out what was wrong with my system when I was just hitting test flaps. If we're going to gradle that would be a great place to

Re: New beam contributor experience?

Not sure if it was mentioned in other threads, but it probably makes sense to add gradle instructions there. On Wed, Mar 14, 2018 at 11:48 AM Alan Myrvold wrote: > There is a contribution guide at > https://beam.apache.org/contribute/contribution-guide/ > Has anyone had

New beam contributor experience?

There is a contribution guide at https://beam.apache.org/contribute/contribution-guide/ Has anyone had challenges / pain points when getting started with new contributions? Any suggestions for making this better? Alan

New Slack Workspace

Apache Beam is migrating to the ASF hosted Slack workspace. Feel free to self-enroll via this handy link[1] and join the #beam[2] channel. 1: https://s.apache.org/slack-invite 2: https://the-asf.slack.com/messages/C9H0YNP3P/

Re: slack @the-asf?

An ASF admin graciously created the self enrollment link. I updated the existing channel header. Please share https://s.apache.org/slack-invite with people who want to migrate. On Wed, Mar 14, 2018 at 10:28 AM Jean-Baptiste Onofré wrote: > I will. Just need sometime as we

Re: slack @the-asf?

I will. Just need sometime as we are at the Apache Beam Summit  Le 14 mars 2018 à 10:21, à 10:21, Lukasz Cwik a écrit: >An ASF admin has to create a self enrollment link >https://the-asf.slack.com/admin/shared_invites and share it with >somebody >in the Apache Beam community.

Re: slack @the-asf?

What's the plan for users without `@apache.org` email? The page says to contact a workspace administrator for an invitation. Will all existing users be automatically invited to the new workspace? On Wed, Mar 14, 2018 at 9:58 AM Thomas Weise wrote: > After you enter the ASF ID

Re: Beam 2.4.0

up, know it missed the 2.4 but can https://github.com/apache/beam/pull/4790 have some love? it really makes beam pretty unusable with direct runner, I start to have "// workaround for BEAM-3409" everywhere in my codebase which is quite bothering after 3 months :( Romain Manni-Bucau @rmannibucau

Re: slack @the-asf?

After you enter the ASF ID on https://the-asf.slack.com/signup an email will go to your ASF address. Just completed that successfully. -- sent from mobile On Wed, Mar 14, 2018, 9:37 AM Romain Manni-Bucau wrote: > > > 2018-03-14 17:28 GMT+01:00 Lukasz Cwik

Re: slack @the-asf?

The site shows "Sign in to ASF, the-asf.slack.com, Enter your email address and password." I enter in the my apache account and credentials (I tested it on id.apache.org to make sure it was correct) and get "Sorry, you entered an incorrect email address or password." On Wed, Mar 14, 2018 at

Re: slack @the-asf?

2018-03-14 17:28 GMT+01:00 Lukasz Cwik : > Telling people to migrate without updating the website is not friendly to > the community or providing the self enrollment link is not friendly to > those that helped invite all those existing users. > We should provide users on the old

Re: slack @the-asf?

Do you want me to prepare the site PR with you ? Regards JB Le 14 mars 2018 à 09:12, à 09:12, Romain Manni-Bucau a écrit: >updated the slack message (feel free to rephrase/reformat it, the >length is >quite limited so it is ok) and will send the message just after this

Re: slack @the-asf?

Please update the Apache Beam slack channel link on the website and also the self enrollment link. Note that Slack started supporting self enrollment[1] for anyone and this seems better then self enrollment for just Apache folks. 1: https://issues.apache.org/jira/browse/BEAM-3846 On Wed, Mar

Re: slack @the-asf?

updated the slack message (feel free to rephrase/reformat it, the length is quite limited so it is ok) and will send the message just after this one on #general will need help for the website update on the 26th ;) Romain Manni-Bucau @rmannibucau | Blog

Re: slack @the-asf?

Same for me but better to have Apache "hosted"  Regards JB Le 14 mars 2018 à 08:48, à 08:48, Reuven Lax a écrit: >I don't have a strong feeling here. As long as there are instructions >on >how to use the new slack channel, sounds good to me! > > >On Wed, Mar 14, 2018 at 1:35

Re: slack @the-asf?

I don't have a strong feeling here. As long as there are instructions on how to use the new slack channel, sounds good to me! On Wed, Mar 14, 2018 at 1:35 AM Romain Manni-Bucau wrote: > Hi guys, > > What do you think to migrate to the standard asf slack? I would make it

Re: slack @the-asf?

Go for it! Thanks Romain! On Wed, Mar 14, 2018 at 1:35 AM, Romain Manni-Bucau wrote: > Hi guys, > > What do you think to migrate to the standard asf slack? I would make it a > bit more easy to find beam channel IMHO and it would stay consistent with > others. It also

slack @the-asf?

Hi guys, What do you think to migrate to the standard asf slack? I would make it a bit more easy to find beam channel IMHO and it would stay consistent with others. It also allows to auto join for asf guys. If you think it is the way to go we can do: 1. put a message on current slack channel

Re: (java) stream & beam?

Hi Jan, The wrapping is almost exactly what I had un mind (I would pass the expected Class to support a bit more like in most jre or javax API but that's a detail) but I would really try to align it on java stream just to keep the dev comfortable:

Re: (java) stream & beam?

Hi all, the are actually some steps taken in this direction - a few emails already went to this channel about donation of Euphoria API (https://github.com/seznam/euphoria) to Apache Beam. SGA has already been signed, currently there is work in progress for porting all Euphoria's features to