Re: [VOTE] Release 2.16.0, release candidate #1

2019-10-03 Thread Ahmet Altay
I see most of the release validations have been completed and marked in the spreadsheet. Thank you all for doing that. If you have not validated/voted yet please take a look at the release candidate. On Thu, Oct 3, 2019 at 7:59 AM Thomas Weise wrote: > I think there is a different reason why

Feature addition to java CassandraIO connector

2019-10-03 Thread Vincent Marquez
Currently the CassandraIO connector allows a user to specify a table, and the CassandraSource object generates a list of queries based on token ranges of the table, along with grouping them by the token ranges. I often need to run (generated, sometimes a million+) queries against a subset of a

Re: Share Outreachy project progress

2019-10-03 Thread Kenneth Knowles
Some links for reference: https://www.outreachy.org/docs/applicant/#outreachy-schedule https://www.outreachy.org/communities/cfp/apache/ Kenn On Wed, Oct 2, 2019 at 11:37 AM Rui Wang wrote: > Hi Community, > > I submitted an Outreachy project proposal on behalf of Apache Beam(it's > related

Re: Multiple iterations after GroupByKey with SparkRunner

2019-10-03 Thread Kenneth Knowles
On Tue, Oct 1, 2019 at 5:35 PM Robert Bradshaw wrote: > For this specific usecase, I would suggest this be done via > PTranform URNs. E.g. one could have a GroupByKeyOneShot whose > implementation is > > input > .apply(GroupByKey.of() > .apply(kv -> KV.of(kv.key(), kv.iterator()) > This

Re: Outreachy applicant

2019-10-03 Thread Rui Wang
Hi Ismael, Sorry I wasn't aware that you also has a project. Carlos has contacted me on SQL before. Next time I will ask people to also include their interest of project in introduction emails. -Rui On Thu, Oct 3, 2019 at 9:47 AM Rui Wang wrote: > Hi Isamel, > > Carlos is an Outreachy

Re: Outreachy applicant

2019-10-03 Thread Rui Wang
Hi Isamel, Carlos is an Outreachy applicant so I will take care of starter tasks. -Rui On Thu, Oct 3, 2019 at 6:31 AM Ismaël Mejía wrote: > Hello Carlos! > > Just added you as a contributor so you can self assign the tickets you > want to work on > What project are you interested in? > >

Re: Multiple iterations after GroupByKey with SparkRunner

2019-10-03 Thread Reuven Lax
Ok - now I see what you're talking about. You are focusing on the Java types in the Java SDK, where the output of GBK is an Iterable type (which should always be reiterable). I was talking more abstractly about the programming model, i.e. the portability representation of the graph. In this case

Re: [DISCUSS] Beam SQL filter push-down

2019-10-03 Thread Kenneth Knowles
** Bumping this thread especially if you are an IO author ** Really glad you are working on this. The basic idea in your doc seems good. It seems mostly that Beam SQL contributors have commented on it. There are many more people who may be interested in this and have valuable feedback, such as

Re: Multiple iterations after GroupByKey with SparkRunner

2019-10-03 Thread Reuven Lax
Putting a stateful dofn after a GBK is not completely redundant - the element type changes, so it is different than just having .a stateful dofn. However it is a weird thing to do, and usually not optimal (especially because many runners might insert two shuffles in this case). On Wed, Oct 2,

Re: [VOTE] Release 2.16.0, release candidate #1

2019-10-03 Thread Thomas Weise
I think there is a different reason why the release manager should probably merge/approve all PRs that go into the release branch while the release is in progress: If/when the need arises for another RC, then only those changes should be included that are deemed blockers or explicitly agreed.

Re: Introduction + Support in Comms for Beam!

2019-10-03 Thread Cyrus Maden
Welcome, Maria! On Thu, Oct 3, 2019 at 9:40 AM Ismaël Mejía wrote: > Hello and welcome Maria! > > Great to see you at dev@. Thanks for sharing the link on the comm > framework. Now I am curious on what's next and how this will adapt to > our community. > > Ismaël. > > > On Tue, Oct 1, 2019 at

Re: Introduction + Support in Comms for Beam!

2019-10-03 Thread Ismaël Mejía
Hello and welcome Maria! Great to see you at dev@. Thanks for sharing the link on the comm framework. Now I am curious on what's next and how this will adapt to our community. Ismaël. On Tue, Oct 1, 2019 at 12:15 AM María Cruz wrote: > > Hi everyone, > my name is María Cruz, I am from Buenos

Re: Outreachy applicant

2019-10-03 Thread Ismaël Mejía
Hello Carlos! Just added you as a contributor so you can self assign the tickets you want to work on What project are you interested in? Regards, Ismaël On Thu, Oct 3, 2019 at 8:27 AM Carlos Oceguera wrote: > > I forgot to add my jira user: "chefoce", sorry > > -- Forwarded message

Re: Beam 2.15.0 SparkRunner issues

2019-10-03 Thread Jan Lukavský
Hi Tim, can you please elaborate more about some parts? 1) What happens actually in your case? What is the specific settings you use? 3) Can you share stacktrace? Is it always the same, or does it change? The mentioned GroupCombineFunctions.java:202 comes from a Reshuffle, which seems to

Re: Beam 2.15.0 SparkRunner issues

2019-10-03 Thread Jozef Vilcek
We do have 2.15.0 Beam batch jobs running on Spark runner. I did have a bit of tricky time with spark.default.parallelism, but at the end it works fine for us (custom parallelism on source stages and spark.default.parallelism on all other stages after shuffles) Tricky part in my case was

Re: Introduction + Support in Comms for Beam!

2019-10-03 Thread Maximilian Michels
Hi Mujuzi Moses, Welcome! I've given you contributor permissions in JIRA. Cheers, Max On 03.10.19 01:07, Mujuzi Moses wrote: Hello, i am requesting to be added to the contributors list, i am an Outreachy applicant, Regards, Mujuzi Moses JIRA Username: iamMujuziMoses On Wed, Oct 2, 2019,

Re: [VOTE] Release 2.16.0, release candidate #1

2019-10-03 Thread Maximilian Michels
> For the next time, may I suggest asking release manager to do the > merging to the release branch. We do not know whether there will be an > RC2 or not. And if there will not be an RC2 release branch as of now > does not directly correspond to what will be released. The ground truth for

Re: Introduction + Support in Comms for Beam!

2019-10-03 Thread Mujuzi Moses
Hello, i am requesting to be added to the contributors list, i am an Outreachy applicant, Regards, Mujuzi Moses JIRA Username: iamMujuziMoses On Wed, Oct 2, 2019, 10:10 PM Kenneth Knowles wrote: > Welcome to dev@beam! And thanks for the interesting link. > > Kenn > > On Tue, Oct 1, 2019 at

Re: Outreachy applicant

2019-10-03 Thread Mujuzi Moses
Username: iamMujuziMoses On Wed, Oct 2, 2019, 11:47 PM Rui Wang wrote: > Can you copy your username? That link directs me to my own profile. > > -Rui > > On Wed, Oct 2, 2019 at 1:34 PM Mujuzi Moses wrote: > >> JIRA: https://issues.apache.org/jira/secure/ViewProfile.jspa >> >> On Wed, Oct 2,

Outreachy applicant

2019-10-03 Thread Carlos Oceguera
Hi my name is Carlos, i'm from Mexico, i want to contribute in the project, i will very grateful for the opportunity.

Fwd: Outreachy applicant

2019-10-03 Thread Carlos Oceguera
I forgot to add my jira user: "chefoce", sorry -- Forwarded message - De: Carlos Oceguera Date: jue., 3 de octubre de 2019 12:24 a. m. Subject: Outreachy applicant To: Hi my name is Carlos, i'm from Mexico, i want to contribute in the project, i will very grateful for the