Re: [ANNOUNCE] New committer: Chad Dombrova

2020-02-25 Thread Valentyn Tymofieiev
Congratulations, Chad!

On Tue, Feb 25, 2020 at 9:36 AM Chamikara Jayalath 
wrote:

> Congrats Chad!
>
> On Tue, Feb 25, 2020 at 6:00 AM jincheng sun 
> wrote:
>
>> Congratulations Chad!
>>
>> Best,
>> Jincheng
>>
>>
>> Jan Lukavský  于2020年2月25日周二 下午5:05写道:
>>
>>> Congrats Chad!
>>> On 2/25/20 9:48 AM, Gleb Kanterov wrote:
>>>
>>> Congratulations!
>>>
>>> On Tue, Feb 25, 2020 at 9:44 AM Ismaël Mejía  wrote:
>>>
 Congratulations, so well deserved for the lots of amazing work and new
 perspectives you have broght into the project !!!

 On Tue, Feb 25, 2020 at 8:24 AM Austin Bennett <
 whatwouldausti...@gmail.com> wrote:

> Hooray!
>
> On Mon, Feb 24, 2020, 11:21 PM Alex Van Boxel 
> wrote:
>
>> Congrats!
>>
>>  _/
>> _/ Alex Van Boxel
>>
>>
>> On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise  wrote:
>>
>>> Congratulations!
>>>
>>>
>>> On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka 
>>> wrote:
>>>
 Congratulations Chad!

 On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay 
 wrote:

> Congratulations!
>
> On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne 
> wrote:
>
>> Nice one Chad. Your typing efforts are very welcomed.
>>
>> On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang 
>> wrote:
>>
>>> Congratulations, Chad!
>>>
>>> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw <
>>> rober...@google.com> wrote:
>>>
 Well deserved, Chad. Congratulations!

 On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni 
 wrote:
 >
 > Congratulations! :-)
 >
 > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova <
 chad...@gmail.com> wrote:
 >>
 >> Thanks, folks!  I'm very excited to "retest this" :)
 >>
 >> Especially big thanks to Robert and Udi for all their hard
 work reviewing my PRs.
 >>
 >> -chad
 >>
 >>
 >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette <
 bhule...@google.com> wrote:
 >>>
 >>> Congratulations Chad! Thanks for all your contributions :)
 >>>
 >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver <
 kcwea...@google.com> wrote:
 
  Well-deserved, thanks for your dedication to the project
 Chad. :)
 
  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri <
 eh...@google.com> wrote:
 >
 > Congrats and welcome, Chad!
 >
 > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada <
 pabl...@google.com> wrote:
 >>
 >> Hi everyone,
 >>
 >> Please join me and the rest of the Beam PMC in welcoming
 a new committer: Chad Dombrova
 >>
 >> Chad has contributed to the project in multiple ways,
 including improvements to the testing infrastructure, and adding 
 type
 annotations throughout the Python SDK, as well as working closely 
 with the
 community on these improvements.
 >>
 >> In consideration of his contributions, the Beam PMC
 trusts him with the responsibilities of a Beam Committer[1].
 >>
 >> Thanks Chad for your contributions!
 >>
 >> -Pablo, on behalf of the Apache Beam PMC.
 >>
 >> [1]
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>


Re: [ANNOUNCE] New committer: Jincheng Sun

2020-02-25 Thread Valentyn Tymofieiev
Congratulations, Jincheng!

On Tue, Feb 25, 2020 at 5:02 PM Chamikara Jayalath 
wrote:

> Congrats Jincheng!
>
> On Tue, Feb 25, 2020 at 10:14 AM Rui Wang  wrote:
>
>> Congrats!
>>
>>
>> -Rui
>>
>> On Mon, Feb 24, 2020 at 11:24 PM Austin Bennett <
>> whatwouldausti...@gmail.com> wrote:
>>
>>> Congrats!
>>>
>>> On Mon, Feb 24, 2020, 11:22 PM Alex Van Boxel  wrote:
>>>
 Congrats!

  _/
 _/ Alex Van Boxel


 On Mon, Feb 24, 2020 at 8:13 PM Kyle Weaver 
 wrote:

> Thanks Jincheng for all your work on Beam and Flink integration.
>
> On Mon, Feb 24, 2020 at 11:02 AM Yichi Zhang 
> wrote:
>
>> Congrats, Jincheng!
>>
>> On Mon, Feb 24, 2020 at 9:45 AM Ahmet Altay  wrote:
>>
>>> Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 6:48 AM Thomas Weise  wrote:
>>>
 Congratulations!


 On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía 
 wrote:

> Congrats Jincheng!
>
> On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov 
> wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng 
>> wrote:
>>
>>> Congratulations Jincheng, well deserved!
>>>
>>> Best,
>>> Hequn
>>>
>>> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni 
>>> wrote:
>>>
 Congrats!

 On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský 
 wrote:

> Congrats Jincheng!
>
>   Jan
>
> On 2/24/20 11:55 AM, Maximilian Michels wrote:
> > Hi everyone,
> >
> > Please join me and the rest of the Beam PMC in welcoming a
> new
> > committer: Jincheng Sun 
> >
> > Jincheng has worked on generalizing parts of Beam for
> Flink's Python
> > API. He has also picked up other issues, like fixing
> documentation,
> > implementing missing features, or cleaning up code [1].
> >
> > In consideration of his contributions, the Beam PMC trusts
> him with
> > the responsibilities of a Beam committer [2].
> >
> > Thank you for your contributions Jincheng!
> >
> > -Max, on behalf of the Apache Beam PMC
> >
> > [1]
> >
> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
> > [2]
> >
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>



Re: [ANNOUNCE] New committer: Jincheng Sun

2020-02-25 Thread Chamikara Jayalath
Congrats Jincheng!

On Tue, Feb 25, 2020 at 10:14 AM Rui Wang  wrote:

> Congrats!
>
>
> -Rui
>
> On Mon, Feb 24, 2020 at 11:24 PM Austin Bennett <
> whatwouldausti...@gmail.com> wrote:
>
>> Congrats!
>>
>> On Mon, Feb 24, 2020, 11:22 PM Alex Van Boxel  wrote:
>>
>>> Congrats!
>>>
>>>  _/
>>> _/ Alex Van Boxel
>>>
>>>
>>> On Mon, Feb 24, 2020 at 8:13 PM Kyle Weaver  wrote:
>>>
 Thanks Jincheng for all your work on Beam and Flink integration.

 On Mon, Feb 24, 2020 at 11:02 AM Yichi Zhang  wrote:

> Congrats, Jincheng!
>
> On Mon, Feb 24, 2020 at 9:45 AM Ahmet Altay  wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 6:48 AM Thomas Weise  wrote:
>>
>>> Congratulations!
>>>
>>>
>>> On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía 
>>> wrote:
>>>
 Congrats Jincheng!

 On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov 
 wrote:

> Congratulations!
>
> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng 
> wrote:
>
>> Congratulations Jincheng, well deserved!
>>
>> Best,
>> Hequn
>>
>> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni 
>> wrote:
>>
>>> Congrats!
>>>
>>> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský 
>>> wrote:
>>>
 Congrats Jincheng!

   Jan

 On 2/24/20 11:55 AM, Maximilian Michels wrote:
 > Hi everyone,
 >
 > Please join me and the rest of the Beam PMC in welcoming a
 new
 > committer: Jincheng Sun 
 >
 > Jincheng has worked on generalizing parts of Beam for Flink's
 Python
 > API. He has also picked up other issues, like fixing
 documentation,
 > implementing missing features, or cleaning up code [1].
 >
 > In consideration of his contributions, the Beam PMC trusts
 him with
 > the responsibilities of a Beam committer [2].
 >
 > Thank you for your contributions Jincheng!
 >
 > -Max, on behalf of the Apache Beam PMC
 >
 > [1]
 >
 https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
 > [2]
 >
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>


Re: [DISCUSSION] Use github actions for python wheels ?

2020-02-25 Thread Robert Bradshaw
I'd be in favor of this, assuming it actually simplifies things. (Note
that the wheels are for several variants of linux, presumably we could
do cross-compiles. Also, manylinux is a "minimal" linux specifically
built as to produce shared object libraries compatible with a wide
variety of distributions--we can't just assume that a shared object
library built on one modern linux will just work on another. (But
maybe it's sufficient to do this within a docker environment?)

On Tue, Feb 25, 2020 at 2:23 PM Kenneth Knowles  wrote:
>
> +1 to exploring this.
>
> On bui...@apache.org there is lots of discussion and general approval for 
> trying it. It is enabled and used by some projects. Calcite uses it to build 
> their website, for example.
>
> Kenn
>
>
> On Tue, Feb 25, 2020 at 2:08 PM Ahmet Altay  wrote:
>>
>> Hi all,
>>
>> I recently had a chance to look at the documentation for github actions. I 
>> think we could use github actions instead of travis to for building python 
>> wheels during releases. This will have the following advantages:
>>
>> - We will eliminate one repo. (If you don't know, we have 
>> https://github.com/apache/beam-wheels for the sole purpose of building 
>> wheels file.)
>> - Workflow will be stored in the same repo. This will prevent bit rot that 
>> is only discovered at release times. (happened a few times, although usually 
>> easy to fix.)
>> - github actions supports ubuntu, mac, windows environments. We could try to 
>> build wheels for windows as well. (Travis also supports the same 
>> environments but we only use linux and mac environments. Maybe there are 
>> other blockers for building wheels for Windows.)
>> - We could do more, like daily python builds.
>>
>> Downsides would be:
>> - I do not know if github actions will require some special set of 
>> permissions that require an approval from infra.
>> - Travis works fine most of the time. This might be unnecessary work.
>>
>> What do you think? Is this feasible, would this be useful?
>>
>> Ahmet


Re: [DISCUSSION] Use github actions for python wheels ?

2020-02-25 Thread Kenneth Knowles
+1 to exploring this.

On bui...@apache.org there is lots of discussion and general approval for
trying it. It is enabled and used by some projects. Calcite uses it to
build their website, for example.

Kenn


On Tue, Feb 25, 2020 at 2:08 PM Ahmet Altay  wrote:

> Hi all,
>
> I recently had a chance to look at the documentation for github actions. I
> think we could use github actions instead of travis to for building python
> wheels during releases. This will have the following advantages:
>
> - We will eliminate one repo. (If you don't know, we have
> https://github.com/apache/beam-wheels for the sole purpose of building
> wheels file.)
> - Workflow will be stored in the same repo. This will prevent bit rot that
> is only discovered at release times. (happened a few times, although
> usually easy to fix.)
> - github actions supports ubuntu, mac, windows environments. We could try
> to build wheels for windows as well. (Travis also supports the same
> environments but we only use linux and mac environments. Maybe there are
> other blockers for building wheels for Windows.)
> - We could do more, like daily python builds.
>
> Downsides would be:
> - I do not know if github actions will require some special set of
> permissions that require an approval from infra.
> - Travis works fine most of the time. This might be unnecessary work.
>
> What do you think? Is this feasible, would this be useful?
>
> Ahmet
>


[DISCUSSION] Use github actions for python wheels ?

2020-02-25 Thread Ahmet Altay
Hi all,

I recently had a chance to look at the documentation for github actions. I
think we could use github actions instead of travis to for building python
wheels during releases. This will have the following advantages:

- We will eliminate one repo. (If you don't know, we have
https://github.com/apache/beam-wheels for the sole purpose of building
wheels file.)
- Workflow will be stored in the same repo. This will prevent bit rot that
is only discovered at release times. (happened a few times, although
usually easy to fix.)
- github actions supports ubuntu, mac, windows environments. We could try
to build wheels for windows as well. (Travis also supports the same
environments but we only use linux and mac environments. Maybe there are
other blockers for building wheels for Windows.)
- We could do more, like daily python builds.

Downsides would be:
- I do not know if github actions will require some special set of
permissions that require an approval from infra.
- Travis works fine most of the time. This might be unnecessary work.

What do you think? Is this feasible, would this be useful?

Ahmet


Re: Can any contributor trigger PreCommit tests?

2020-02-25 Thread Liu Wang
How can I become a committer?

On 2020/02/25 19:14:28, Hannah Jiang  wrote: 
> Committers can trigger the tests. Can you share your PRs here so people who
> have permission can trigger tests for your PRs?
> Please refer to this thread
> 
> for more details of the issue.
> 
> 
> 
> 
> On Tue, Feb 25, 2020 at 10:53 AM Wenbing Bai 
> wrote:
> 
> > I have the same issue. Follow this thread.
> >
> > Wenbing
> >
> > On Tue, Feb 25, 2020 at 9:43 AM Liu Wang  wrote:
> >
> >> Hi,
> >>
> >> I'm a contributor but can't trigger tests for my PRs. Do I need to get
> >> any permission for this?
> >> Is it like any contributor can apply for and get the permission?
> >>
> >> Thanks,
> >> Liu
> >>
> >
> >
> > --
> >
> >
> >
> >
> >
> > Wenbing Bai
> >
> > Senior Software Engineer, MLP
> >
> > Cruise
> >
> > Pronouns: She/Her
> >
> >
> >
> > *Confidentiality Note:* We care about protecting our proprietary
> > information, confidential material, and trade secrets. This message may
> > contain some or all of those things. Cruise will suffer material harm if
> > anyone other than the intended recipient disseminates or takes any action
> > based on this message. If you have received this message (including any
> > attachments) in error, please delete it immediately and notify the sender
> > promptly.
> 


Re: Can any contributor trigger PreCommit tests?

2020-02-25 Thread Liu Wang
https://github.com/apache/beam/pull/10951

On 2020/02/25 19:14:28, Hannah Jiang  wrote: 
> Committers can trigger the tests. Can you share your PRs here so people who
> have permission can trigger tests for your PRs?
> Please refer to this thread
> 
> for more details of the issue.
> 
> 
> 
> 
> On Tue, Feb 25, 2020 at 10:53 AM Wenbing Bai 
> wrote:
> 
> > I have the same issue. Follow this thread.
> >
> > Wenbing
> >
> > On Tue, Feb 25, 2020 at 9:43 AM Liu Wang  wrote:
> >
> >> Hi,
> >>
> >> I'm a contributor but can't trigger tests for my PRs. Do I need to get
> >> any permission for this?
> >> Is it like any contributor can apply for and get the permission?
> >>
> >> Thanks,
> >> Liu
> >>
> >
> >
> > --
> >
> >
> >
> >
> >
> > Wenbing Bai
> >
> > Senior Software Engineer, MLP
> >
> > Cruise
> >
> > Pronouns: She/Her
> >
> >
> >
> > *Confidentiality Note:* We care about protecting our proprietary
> > information, confidential material, and trade secrets. This message may
> > contain some or all of those things. Cruise will suffer material harm if
> > anyone other than the intended recipient disseminates or takes any action
> > based on this message. If you have received this message (including any
> > attachments) in error, please delete it immediately and notify the sender
> > promptly.
> 


Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.10.8

2020-02-25 Thread Ismaël Mejía
Thanks Luke there is indeed an extra module-info.class in the vendored jar.
This vote is cancelled. A new vote will be opened once the fix is merged.
PTAL https://github.com/apache/beam/pull/10970

On Tue, Feb 25, 2020 at 6:39 PM Luke Cwik  wrote:

> -1
>
> The jar contains META-INF/versions/9/module-info.class copied over from
> bytebuddy containing:
>
> module net.bytebuddy {
>   requires static java.instrument;
>   requires static jdk.unsupported;
>   requires static net.bytebuddy.agent;
>
>   exports net.bytebuddy;
>   exports net.bytebuddy.agent.builder;
>   exports net.bytebuddy.asm;
>   exports net.bytebuddy.build;
>   exports net.bytebuddy.description;
>   exports net.bytebuddy.description.annotation;
>   exports net.bytebuddy.description.enumeration;
>   exports net.bytebuddy.description.field;
>   exports net.bytebuddy.description.method;
>   exports net.bytebuddy.description.modifier;
>   exports net.bytebuddy.description.type;
>   exports net.bytebuddy.dynamic;
>   exports net.bytebuddy.dynamic.loading;
>   exports net.bytebuddy.dynamic.scaffold;
>   exports net.bytebuddy.dynamic.scaffold.inline;
>   exports net.bytebuddy.dynamic.scaffold.subclass;
>   exports net.bytebuddy.implementation;
>   exports net.bytebuddy.implementation.attribute;
>   exports net.bytebuddy.implementation.auxiliary;
>   exports net.bytebuddy.implementation.bind;
>   exports net.bytebuddy.implementation.bind.annotation;
>   exports net.bytebuddy.implementation.bytecode;
>   exports net.bytebuddy.implementation.bytecode.assign;
>   exports net.bytebuddy.implementation.bytecode.assign.primitive;
>   exports net.bytebuddy.implementation.bytecode.assign.reference;
>   exports net.bytebuddy.implementation.bytecode.collection;
>   exports net.bytebuddy.implementation.bytecode.constant;
>   exports net.bytebuddy.implementation.bytecode.member;
>   exports net.bytebuddy.matcher;
>   exports net.bytebuddy.pool;
>   exports net.bytebuddy.utility;
>   exports net.bytebuddy.utility.privilege;
>   exports net.bytebuddy.utility.visitor;
>   exports net.bytebuddy.jar.asm;
>   exports net.bytebuddy.jar.asm.signature;
>   exports net.bytebuddy.jar.asm.commons;
> }
>
>
> On Tue, Feb 25, 2020 at 8:15 AM Jean-Baptiste Onofré 
> wrote:
>
>> +1
>>
>> Regards
>> JB
>>
>> Le mar. 25 f?vr. 2020 ? 13:43, Isma?l Mej?a  a ?crit :
>>
>>> Please review the release of the following artifacts that we vendor:
>>>  * beam-vendor-bytebuddy-1_10_8
>>>
>>> Hi everyone,
>>> Please review and vote on the release candidate #1 for the version 0.1,
>>> as follows:
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>>
>>> The complete staging area is available for your review, which includes:
>>> * the official Apache source release to be deployed to dist.apache.org
>>> [1], which is signed with the key with fingerprint
>>> 3415631729E15B33051ADB670A9DAF6713B86349 [2],
>>> * all artifacts to be deployed to the Maven Central Repository [3],
>>> * commit hash "0756edfba7c682b1e75f4691c0ecb38a1dc9d16a" [4],
>>>
>>> The vote will be open for at least 72 hours. It is adopted by majority
>>> approval, with at least 3 PMC affirmative votes.
>>>
>>> Thanks,
>>> Release Manager
>>>
>>> [1] https://dist.apache.org/repos/dist/dev/beam/vendor/
>>> [2] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> [3]
>>> https://repository.apache.org/content/repositories/orgapachebeam-1093/
>>> [4]
>>> https://github.com/apache/beam/commit/0756edfba7c682b1e75f4691c0ecb38a1dc9d16a
>>>
>>>


Re: Can any contributor trigger PreCommit tests?

2020-02-25 Thread Boyuan Zhang
Hey Wenbing,

Which tests you are interested in? Most non-dataflow tests can be ran
locally for testing purposes.

On Tue, Feb 25, 2020 at 11:15 AM Hannah Jiang 
wrote:

> Committers can trigger the tests. Can you share your PRs here so people
> who have permission can trigger tests for your PRs?
> Please refer to this thread
> 
> for more details of the issue.
>
>
>
>
> On Tue, Feb 25, 2020 at 10:53 AM Wenbing Bai 
> wrote:
>
>> I have the same issue. Follow this thread.
>>
>> Wenbing
>>
>> On Tue, Feb 25, 2020 at 9:43 AM Liu Wang  wrote:
>>
>>> Hi,
>>>
>>> I'm a contributor but can't trigger tests for my PRs. Do I need to get
>>> any permission for this?
>>> Is it like any contributor can apply for and get the permission?
>>>
>>> Thanks,
>>> Liu
>>>
>>
>>
>> --
>>
>>
>>
>>
>>
>> Wenbing Bai
>>
>> Senior Software Engineer, MLP
>>
>> Cruise
>>
>> Pronouns: She/Her
>>
>>
>>
>> *Confidentiality Note:* We care about protecting our proprietary
>> information, confidential material, and trade secrets. This message may
>> contain some or all of those things. Cruise will suffer material harm if
>> anyone other than the intended recipient disseminates or takes any action
>> based on this message. If you have received this message (including any
>> attachments) in error, please delete it immediately and notify the sender
>> promptly.
>
>


Re: Can any contributor trigger PreCommit tests?

2020-02-25 Thread Hannah Jiang
Committers can trigger the tests. Can you share your PRs here so people who
have permission can trigger tests for your PRs?
Please refer to this thread

for more details of the issue.




On Tue, Feb 25, 2020 at 10:53 AM Wenbing Bai 
wrote:

> I have the same issue. Follow this thread.
>
> Wenbing
>
> On Tue, Feb 25, 2020 at 9:43 AM Liu Wang  wrote:
>
>> Hi,
>>
>> I'm a contributor but can't trigger tests for my PRs. Do I need to get
>> any permission for this?
>> Is it like any contributor can apply for and get the permission?
>>
>> Thanks,
>> Liu
>>
>
>
> --
>
>
>
>
>
> Wenbing Bai
>
> Senior Software Engineer, MLP
>
> Cruise
>
> Pronouns: She/Her
>
>
>
> *Confidentiality Note:* We care about protecting our proprietary
> information, confidential material, and trade secrets. This message may
> contain some or all of those things. Cruise will suffer material harm if
> anyone other than the intended recipient disseminates or takes any action
> based on this message. If you have received this message (including any
> attachments) in error, please delete it immediately and notify the sender
> promptly.


Re: Review Request

2020-02-25 Thread Pablo Estrada
Thanks! I'll take a look today.

On Mon, Feb 24, 2020 at 5:19 PM Wenbing Bai 
wrote:

> Hi there,
>
> Can I get a review on this PR https://github.com/apache/beam/pull/10901?
>
> --
>
>
>
>
>
> Wenbing Bai
>
> Senior Software Engineer, MLP
>
> Cruise
>
> Pronouns: She/Her
>
>
>
> *Confidentiality Note:* We care about protecting our proprietary
> information, confidential material, and trade secrets. This message may
> contain some or all of those things. Cruise will suffer material harm if
> anyone other than the intended recipient disseminates or takes any action
> based on this message. If you have received this message (including any
> attachments) in error, please delete it immediately and notify the sender
> promptly.


Re: Can any contributor trigger PreCommit tests?

2020-02-25 Thread Wenbing Bai
I have the same issue. Follow this thread.

Wenbing

On Tue, Feb 25, 2020 at 9:43 AM Liu Wang  wrote:

> Hi,
>
> I'm a contributor but can't trigger tests for my PRs. Do I need to get any
> permission for this?
> Is it like any contributor can apply for and get the permission?
>
> Thanks,
> Liu
>


-- 





Wenbing Bai

Senior Software Engineer, MLP

Cruise

Pronouns: She/Her

-- 


*Confidentiality Note:* We care about protecting our proprietary 
information, confidential material, and trade secrets. This message may 
contain some or all of those things. Cruise will suffer material harm if 
anyone other than the intended recipient disseminates or takes any action 
based on this message. If you have received this message (including any 
attachments) in error, please delete it immediately and notify the sender 
promptly.


Are there extra Beam Python test matchers available?

2020-02-25 Thread Liu Wang
Hi,

As far as I know, the matchers I can use with assert_that are
equal_to and matches_all in this file:
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/testing/util.py

Is there any matcher that I can test a PCollection contains a subset?
Is there any matcher that I can test a PCollection contains any number of a 
specific element?

For example, I want to test the PCollection [1, 3, 6, 10, 15, 15, 15] contains 
[1, 3, 6, 10], and it contains any number of 15s.

Thanks,
Liu


Re: Create a 2.21.0 tag

2020-02-25 Thread Kenneth Knowles
Done

On Tue, Feb 25, 2020 at 10:25 AM Rui Wang  wrote:

> Hi Community,
>
> Is there a PMC who is able to create 2.21.0 tag to help people move their
> Jiras?
>
>
>
> -Rui
>


Create a 2.21.0 tag

2020-02-25 Thread Rui Wang
Hi Community,

Is there a PMC who is able to create 2.21.0 tag to help people move their
Jiras?



-Rui


Re: Github flaky?

2020-02-25 Thread Robert Burke
I've seen complaints on Twitter about GitHub availablilty today, so it's
not just you.

On Tue, Feb 25, 2020, 10:10 AM Luke Cwik  wrote:

> I have been getting errors from the Github UI when attempting to post a
> comment / merge a commit / 500 errors when reloading a page. Anyone else
> seeing this?
>


Re: [VOTE] Upgrade gradle to 6.2

2020-02-25 Thread Luke Cwik
+1

On Tue, Feb 25, 2020 at 12:49 AM Gleb Kanterov  wrote:

> +1 (non-binding)
>
> On Tue, Feb 25, 2020 at 9:38 AM Ismaël Mejía  wrote:
>
>> +1 great to have our build updated, please share if there are new
>> interesting features/plugin advantages we can benefit from too.
>>
>> On Tue, Feb 25, 2020 at 8:24 AM Jean-Baptiste Onofré 
>> wrote:
>>
>>> Hi Alex
>>>
>>> I also have couple of contacts at Gradle. Let me know if needed.
>>>
>>> Regards
>>> JB
>>>
>>> Le mar. 25 f?vr. 2020 ? 08:20, Alex Van Boxel  a
>>> ?crit :
>>>
 OK, great. I know someone that works at gradle, so I can ping them when
 I have some problems.

 Any other know pitfalls I can expect, let me know :-)

  _/
 _/ Alex Van Boxel


 On Tue, Feb 25, 2020 at 7:20 AM Jean-Baptiste Onofr? 
 wrote:

 +1

 It makes sense.

 Thanks.
 Regards
 JB

 Le lun. 24 f?vr. 2020 ? 22:37, Alex Van Boxel  a
 ?crit :

 Anyone objections that I upgrade gradle to 6.2. If ok this will be done
 over several commits where I will:

- Upgrade plugins
- Upgrade gradle to 6.2
- See where we can use some of the new features


  _/
 _/ Alex Van Boxel




Re: Github flaky?

2020-02-25 Thread Alan Myrvold
There is an active incident, from https://www.githubstatus.com/

On Tue, Feb 25, 2020 at 10:10 AM Luke Cwik  wrote:

> I have been getting errors from the Github UI when attempting to post a
> comment / merge a commit / 500 errors when reloading a page. Anyone else
> seeing this?
>


Re: [ANNOUNCE] New committer: Jincheng Sun

2020-02-25 Thread Rui Wang
Congrats!


-Rui

On Mon, Feb 24, 2020 at 11:24 PM Austin Bennett 
wrote:

> Congrats!
>
> On Mon, Feb 24, 2020, 11:22 PM Alex Van Boxel  wrote:
>
>> Congrats!
>>
>>  _/
>> _/ Alex Van Boxel
>>
>>
>> On Mon, Feb 24, 2020 at 8:13 PM Kyle Weaver  wrote:
>>
>>> Thanks Jincheng for all your work on Beam and Flink integration.
>>>
>>> On Mon, Feb 24, 2020 at 11:02 AM Yichi Zhang  wrote:
>>>
 Congrats, Jincheng!

 On Mon, Feb 24, 2020 at 9:45 AM Ahmet Altay  wrote:

> Congratulations!
>
> On Mon, Feb 24, 2020 at 6:48 AM Thomas Weise  wrote:
>
>> Congratulations!
>>
>>
>> On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía 
>> wrote:
>>
>>> Congrats Jincheng!
>>>
>>> On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov 
>>> wrote:
>>>
 Congratulations!

 On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng 
 wrote:

> Congratulations Jincheng, well deserved!
>
> Best,
> Hequn
>
> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:
>
>> Congrats!
>>
>> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský 
>> wrote:
>>
>>> Congrats Jincheng!
>>>
>>>   Jan
>>>
>>> On 2/24/20 11:55 AM, Maximilian Michels wrote:
>>> > Hi everyone,
>>> >
>>> > Please join me and the rest of the Beam PMC in welcoming a new
>>> > committer: Jincheng Sun 
>>> >
>>> > Jincheng has worked on generalizing parts of Beam for Flink's
>>> Python
>>> > API. He has also picked up other issues, like fixing
>>> documentation,
>>> > implementing missing features, or cleaning up code [1].
>>> >
>>> > In consideration of his contributions, the Beam PMC trusts him
>>> with
>>> > the responsibilities of a Beam committer [2].
>>> >
>>> > Thank you for your contributions Jincheng!
>>> >
>>> > -Max, on behalf of the Apache Beam PMC
>>> >
>>> > [1]
>>> >
>>> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
>>> > [2]
>>> >
>>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>>
>>


Github flaky?

2020-02-25 Thread Luke Cwik
I have been getting errors from the Github UI when attempting to post a
comment / merge a commit / 500 errors when reloading a page. Anyone else
seeing this?


Can any contributor trigger PreCommit tests?

2020-02-25 Thread Liu Wang
Hi,

I'm a contributor but can't trigger tests for my PRs. Do I need to get any 
permission for this?
Is it like any contributor can apply for and get the permission? 

Thanks,
Liu


Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.10.8

2020-02-25 Thread Luke Cwik
-1

The jar contains META-INF/versions/9/module-info.class copied over from
bytebuddy containing:

module net.bytebuddy {
  requires static java.instrument;
  requires static jdk.unsupported;
  requires static net.bytebuddy.agent;

  exports net.bytebuddy;
  exports net.bytebuddy.agent.builder;
  exports net.bytebuddy.asm;
  exports net.bytebuddy.build;
  exports net.bytebuddy.description;
  exports net.bytebuddy.description.annotation;
  exports net.bytebuddy.description.enumeration;
  exports net.bytebuddy.description.field;
  exports net.bytebuddy.description.method;
  exports net.bytebuddy.description.modifier;
  exports net.bytebuddy.description.type;
  exports net.bytebuddy.dynamic;
  exports net.bytebuddy.dynamic.loading;
  exports net.bytebuddy.dynamic.scaffold;
  exports net.bytebuddy.dynamic.scaffold.inline;
  exports net.bytebuddy.dynamic.scaffold.subclass;
  exports net.bytebuddy.implementation;
  exports net.bytebuddy.implementation.attribute;
  exports net.bytebuddy.implementation.auxiliary;
  exports net.bytebuddy.implementation.bind;
  exports net.bytebuddy.implementation.bind.annotation;
  exports net.bytebuddy.implementation.bytecode;
  exports net.bytebuddy.implementation.bytecode.assign;
  exports net.bytebuddy.implementation.bytecode.assign.primitive;
  exports net.bytebuddy.implementation.bytecode.assign.reference;
  exports net.bytebuddy.implementation.bytecode.collection;
  exports net.bytebuddy.implementation.bytecode.constant;
  exports net.bytebuddy.implementation.bytecode.member;
  exports net.bytebuddy.matcher;
  exports net.bytebuddy.pool;
  exports net.bytebuddy.utility;
  exports net.bytebuddy.utility.privilege;
  exports net.bytebuddy.utility.visitor;
  exports net.bytebuddy.jar.asm;
  exports net.bytebuddy.jar.asm.signature;
  exports net.bytebuddy.jar.asm.commons;
}


On Tue, Feb 25, 2020 at 8:15 AM Jean-Baptiste Onofré 
wrote:

> +1
>
> Regards
> JB
>
> Le mar. 25 f?vr. 2020 ? 13:43, Isma?l Mej?a  a ?crit :
>
>> Please review the release of the following artifacts that we vendor:
>>  * beam-vendor-bytebuddy-1_10_8
>>
>> Hi everyone,
>> Please review and vote on the release candidate #1 for the version 0.1,
>> as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>> The complete staging area is available for your review, which includes:
>> * the official Apache source release to be deployed to dist.apache.org
>> [1], which is signed with the key with fingerprint
>> 3415631729E15B33051ADB670A9DAF6713B86349 [2],
>> * all artifacts to be deployed to the Maven Central Repository [3],
>> * commit hash "0756edfba7c682b1e75f4691c0ecb38a1dc9d16a" [4],
>>
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval, with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> Release Manager
>>
>> [1] https://dist.apache.org/repos/dist/dev/beam/vendor/
>> [2] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [3]
>> https://repository.apache.org/content/repositories/orgapachebeam-1093/
>> [4]
>> https://github.com/apache/beam/commit/0756edfba7c682b1e75f4691c0ecb38a1dc9d16a
>>
>>


Re: [ANNOUNCE] New committer: Chad Dombrova

2020-02-25 Thread Chamikara Jayalath
Congrats Chad!

On Tue, Feb 25, 2020 at 6:00 AM jincheng sun 
wrote:

> Congratulations Chad!
>
> Best,
> Jincheng
>
>
> Jan Lukavský  于2020年2月25日周二 下午5:05写道:
>
>> Congrats Chad!
>> On 2/25/20 9:48 AM, Gleb Kanterov wrote:
>>
>> Congratulations!
>>
>> On Tue, Feb 25, 2020 at 9:44 AM Ismaël Mejía  wrote:
>>
>>> Congratulations, so well deserved for the lots of amazing work and new
>>> perspectives you have broght into the project !!!
>>>
>>> On Tue, Feb 25, 2020 at 8:24 AM Austin Bennett <
>>> whatwouldausti...@gmail.com> wrote:
>>>
 Hooray!

 On Mon, Feb 24, 2020, 11:21 PM Alex Van Boxel  wrote:

> Congrats!
>
>  _/
> _/ Alex Van Boxel
>
>
> On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise  wrote:
>
>> Congratulations!
>>
>>
>> On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka  wrote:
>>
>>> Congratulations Chad!
>>>
>>> On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay 
>>> wrote:
>>>
 Congratulations!

 On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne 
 wrote:

> Nice one Chad. Your typing efforts are very welcomed.
>
> On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang 
> wrote:
>
>> Congratulations, Chad!
>>
>> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw <
>> rober...@google.com> wrote:
>>
>>> Well deserved, Chad. Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni 
>>> wrote:
>>> >
>>> > Congratulations! :-)
>>> >
>>> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova <
>>> chad...@gmail.com> wrote:
>>> >>
>>> >> Thanks, folks!  I'm very excited to "retest this" :)
>>> >>
>>> >> Especially big thanks to Robert and Udi for all their hard
>>> work reviewing my PRs.
>>> >>
>>> >> -chad
>>> >>
>>> >>
>>> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette <
>>> bhule...@google.com> wrote:
>>> >>>
>>> >>> Congratulations Chad! Thanks for all your contributions :)
>>> >>>
>>> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver <
>>> kcwea...@google.com> wrote:
>>> 
>>>  Well-deserved, thanks for your dedication to the project
>>> Chad. :)
>>> 
>>>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri 
>>> wrote:
>>> >
>>> > Congrats and welcome, Chad!
>>> >
>>> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada <
>>> pabl...@google.com> wrote:
>>> >>
>>> >> Hi everyone,
>>> >>
>>> >> Please join me and the rest of the Beam PMC in welcoming
>>> a new committer: Chad Dombrova
>>> >>
>>> >> Chad has contributed to the project in multiple ways,
>>> including improvements to the testing infrastructure, and adding 
>>> type
>>> annotations throughout the Python SDK, as well as working closely 
>>> with the
>>> community on these improvements.
>>> >>
>>> >> In consideration of his contributions, the Beam PMC
>>> trusts him with the responsibilities of a Beam Committer[1].
>>> >>
>>> >> Thanks Chad for your contributions!
>>> >>
>>> >> -Pablo, on behalf of the Apache Beam PMC.
>>> >>
>>> >> [1]
>>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>>
>>


[RESULT] [VOTE] Vendored Dependencies Release gRPC 1.26.0 v0.2 for BEAM-9252

2020-02-25 Thread Luke Cwik
I'm happy to announce that we have unanimously approved this release.

There are 7 approving votes, 4 of which are binding:
* Ismaël Mejía
* Robert Bradshaw
* Ahmet Altay
* Luke Cwik

There are no disapproving votes.

Thanks everyone!

On Sat, Feb 22, 2020 at 5:49 AM Ismaël Mejía  wrote:

> +1 (binding)
>
> Verified signatures, produced jar from code and checked that artifact does
> not contain both Main.class and module-info.classes
>
> Thanks for taking care of this Tomo and Luke.
>
>
> On Sat, Feb 22, 2020 at 10:27 AM jincheng sun 
> wrote:
>
>> +1(non-binding)
>>
>> Best,
>> Jincheng
>>
>>
>> Kai Jiang  于2020年2月22日周六 下午2:58写道:
>>
>>> +1 (non-binding)
>>>
>>> On Fri, Feb 21, 2020 at 5:03 PM Robin Qiu  wrote:
>>>
 +1 (verified)

 On Fri, Feb 21, 2020 at 4:55 PM Robert Bradshaw 
 wrote:

> +1 (binding)
>
>
> On Fri, Feb 21, 2020 at 4:48 PM Ahmet Altay  wrote:
> >
> > +1
> >
> > On Fri, Feb 21, 2020 at 4:39 PM Luke Cwik  wrote:
> >>
> >> +1 (binding)
> >> I diffed the binary contents of the 0.1 jar and 0.2 jar with no
> changes to the contents of the files and can confirm that 
> module-info.class
> the offending Main.class and Main$1.class have been removed as well.
> >>
> >> On Fri, Feb 21, 2020 at 4:38 PM Luke Cwik  wrote:
> >>>
> >>> Please review the release of the following artifacts that we
> vendor:
> >>>  * beam-vendor-grpc-1_26_0
> >>>
> >>> Hi everyone,
> >>> Please review and vote on the release candidate #1 for the version
> 0.2, as follows:
> >>> [ ] +1, Approve the release
> >>> [ ] -1, Do not approve the release (please provide specific
> comments)
> >>>
> >>>
> >>> The complete staging area is available for your review, which
> includes:
> >>> * the official Apache source release to be deployed to
> dist.apache.org [1], which is signed with the key with fingerprint
> EAD5DE293F4A03DD2E77565589E68A56E371CCA2 [2],
> >>> * all artifacts to be deployed to the Maven Central Repository [3],
> >>> * commit hash "91125d1d1fc1fe8c5684a486c9b6163c4ec41549" [4],
> >>>
> >>> The vote will be open for at least 72 hours. It is adopted by
> majority approval, with at least 3 PMC affirmative votes.
> >>>
> >>> Thanks,
> >>> Release Manager
> >>>
> >>> [1] https://dist.apache.org/repos/dist/dev/beam/vendor/
> >>> [2] https://dist.apache.org/repos/dist/release/beam/KEYS
> >>> [3]
> https://repository.apache.org/content/repositories/orgapachebeam-1092/
> >>> [4]
> https://github.com/apache/beam/commit/91125d1d1fc1fe8c5684a486c9b6163c4ec41549
>
 ᐧ
>>>
>>


Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.10.8

2020-02-25 Thread Jean-Baptiste Onofré
+1Regards JBLe mar. 25 f?vr. 2020 ? 13:43, Isma?l Mej?a  a ?crit :Please review the release of the following artifacts that we vendor: * beam-vendor-bytebuddy-1_10_8Hi everyone,Please review and vote on the release candidate #1 for the version 0.1, as follows:[ ] +1, Approve the release[ ] -1, Do not approve the release (please provide specific comments)The complete staging area is available for your review, which includes:* the official Apache source release to be deployed to dist.apache.org [1], which is signed with the key with fingerprint 3415631729E15B33051ADB670A9DAF6713B86349 [2],* all artifacts to be deployed to the Maven Central Repository [3],* commit hash "0756edfba7c682b1e75f4691c0ecb38a1dc9d16a" [4],The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes.Thanks,Release Manager[1] https://dist.apache.org/repos/dist/dev/beam/vendor/[2] https://dist.apache.org/repos/dist/release/beam/KEYS[3] https://repository.apache.org/content/repositories/orgapachebeam-1093/[4] https://github.com/apache/beam/commit/0756edfba7c682b1e75f4691c0ecb38a1dc9d16a

Re: [VOTE] Vendored Dependencies Release Byte Buddy 1.10.8

2020-02-25 Thread Reuven Lax
+1

On Tue, Feb 25, 2020 at 4:44 AM Ismaël Mejía  wrote:

> Please review the release of the following artifacts that we vendor:
>  * beam-vendor-bytebuddy-1_10_8
>
> Hi everyone,
> Please review and vote on the release candidate #1 for the version 0.1, as
> follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staging area is available for your review, which includes:
> * the official Apache source release to be deployed to dist.apache.org
> [1], which is signed with the key with fingerprint
> 3415631729E15B33051ADB670A9DAF6713B86349 [2],
> * all artifacts to be deployed to the Maven Central Repository [3],
> * commit hash "0756edfba7c682b1e75f4691c0ecb38a1dc9d16a" [4],
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Thanks,
> Release Manager
>
> [1] https://dist.apache.org/repos/dist/dev/beam/vendor/
> [2] https://dist.apache.org/repos/dist/release/beam/KEYS
> [3] https://repository.apache.org/content/repositories/orgapachebeam-1093/
> [4]
> https://github.com/apache/beam/commit/0756edfba7c682b1e75f4691c0ecb38a1dc9d16a
>
>


Re: [EXTERNAL] Re: Java Build broken

2020-02-25 Thread Pulasthi Supun Wickramasinghe
Hi Stefan,

Yes, I am also still getting this error on my local setup, However,
strangely I am not getting this on my laptop. I tried manually installing
the missing 'error_prone' dependencies to maven but then go some other
error.
Might this be some kind of cache issue?

Best Regards,
Pulasthi

On Tue, Feb 25, 2020 at 5:38 AM Stefan Djelekar 
wrote:

> Hi all,
>
>
>
> No this is not yet fixed.
>
>
>
> @Pulasthi do you still get the same error?
>
>
>
> @Maximilian I don’t have any overrides.
>
> It looks like on localhost build references
> https://oss.sonatype.org/content/repositories/staging/com/google/errorprone/error_prone_check_api/2.3.4/
>
> istead of
>
>
> https://mvnrepository.com/artifact/com.google.errorprone/error_prone_check_api/2.3.4
>
>
>
> and the first link returns 404
>
>
>
>
>
> Can you please advise?
>
>
>
> All the best,
>
> Stefan
>
>
>
> *From:* Pulasthi Supun Wickramasinghe 
> *Sent:* Tuesday, February 18, 2020 5:11 PM
> *To:* dev 
> *Cc:* Stefan Djelekar 
> *Subject:* [EXTERNAL] Re: Java Build broken
>
>
>
> Hi All,
>
>
>
> Was this issue resolved? I started to get the same error on my local build
> suddenly.
>
>
>
> Best Regards,
>
> Pulasthi
>
>
>
> On Thu, Jan 23, 2020 at 10:17 AM Maximilian Michels 
> wrote:
>
> Do you have any overrides in your ~/.m2/settings.xml? The artifacts
> should be found as part of Maven central, e.g.
>
> https://mvnrepository.com/artifact/com.google.errorprone/error_prone_check_api
>
> Cheers,
> Max
>
> On 23.01.20 11:11, Stefan Djelekar wrote:
> > Hi guys,
> >
> > It’s been days now since the build for Java SDK is broken.
> >
> > Namely, pipeline is successful on Jenkins, but it fails in my localhost
> > with error in task model:pipeline:compileJava. As I've seen, last couple
> > of builds were served from cache, so maybe that is the reason why it's
> > green. I confirmed same thing happened to other devs as well.
> >
> > 22:49:34 > Task :model:pipeline:compileJava FROM-CACHE
> >
> > It looks like it’s related to mismatch of com.google.errorprone library
> > version. Can someone please take a look as this is a blocker to
> > localhost development?
> >
> > Cheers,
> >
> > *Stefan Đelekar*
> >
> > Sofware Engineer
> >
> > Mobile +381 65 22 33 293
> >
> > stefan.djele...@redbox.com 
> >
> > Redbox Automated Retail LLC / Interventure d.o.o
> >
> > 16. Oktobra 21/67, 11050 Zvezdara, Serbia
> >
> > djelekar.com
> >
> > --
> >
> > This email is confidential and intended for the recipient specified in
> > the message only.
> > If you have received this message in error, please notify the sender by
> > e-mail, discard any
> > paper copies and delete the mail and all electronic files of the message
> > from your system.
> >
>
>
>
>
> --
>
> Pulasthi S. Wickramasinghe
>
> PhD Candidate  | Research Assistant
>
> School of Informatics and Computing | Digital Science Center
>
> Indiana University, Bloomington
>
> cell: 224-386-9035
>


-- 
Pulasthi S. Wickramasinghe
PhD Candidate  | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington
cell: 224-386-9035


Re: [ANNOUNCE] New committer: Chad Dombrova

2020-02-25 Thread jincheng sun
Congratulations Chad!

Best,
Jincheng


Jan Lukavský  于2020年2月25日周二 下午5:05写道:

> Congrats Chad!
> On 2/25/20 9:48 AM, Gleb Kanterov wrote:
>
> Congratulations!
>
> On Tue, Feb 25, 2020 at 9:44 AM Ismaël Mejía  wrote:
>
>> Congratulations, so well deserved for the lots of amazing work and new
>> perspectives you have broght into the project !!!
>>
>> On Tue, Feb 25, 2020 at 8:24 AM Austin Bennett <
>> whatwouldausti...@gmail.com> wrote:
>>
>>> Hooray!
>>>
>>> On Mon, Feb 24, 2020, 11:21 PM Alex Van Boxel  wrote:
>>>
 Congrats!

  _/
 _/ Alex Van Boxel


 On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise  wrote:

> Congratulations!
>
>
> On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka  wrote:
>
>> Congratulations Chad!
>>
>> On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay  wrote:
>>
>>> Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne 
>>> wrote:
>>>
 Nice one Chad. Your typing efforts are very welcomed.

 On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang 
 wrote:

> Congratulations, Chad!
>
> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw <
> rober...@google.com> wrote:
>
>> Well deserved, Chad. Congratulations!
>>
>> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni 
>> wrote:
>> >
>> > Congratulations! :-)
>> >
>> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova <
>> chad...@gmail.com> wrote:
>> >>
>> >> Thanks, folks!  I'm very excited to "retest this" :)
>> >>
>> >> Especially big thanks to Robert and Udi for all their hard
>> work reviewing my PRs.
>> >>
>> >> -chad
>> >>
>> >>
>> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette <
>> bhule...@google.com> wrote:
>> >>>
>> >>> Congratulations Chad! Thanks for all your contributions :)
>> >>>
>> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver <
>> kcwea...@google.com> wrote:
>> 
>>  Well-deserved, thanks for your dedication to the project
>> Chad. :)
>> 
>>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri 
>> wrote:
>> >
>> > Congrats and welcome, Chad!
>> >
>> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada <
>> pabl...@google.com> wrote:
>> >>
>> >> Hi everyone,
>> >>
>> >> Please join me and the rest of the Beam PMC in welcoming a
>> new committer: Chad Dombrova
>> >>
>> >> Chad has contributed to the project in multiple ways,
>> including improvements to the testing infrastructure, and adding type
>> annotations throughout the Python SDK, as well as working closely 
>> with the
>> community on these improvements.
>> >>
>> >> In consideration of his contributions, the Beam PMC trusts
>> him with the responsibilities of a Beam Committer[1].
>> >>
>> >> Thanks Chad for your contributions!
>> >>
>> >> -Pablo, on behalf of the Apache Beam PMC.
>> >>
>> >> [1]
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>


[VOTE] Vendored Dependencies Release Byte Buddy 1.10.8

2020-02-25 Thread Ismaël Mejía
Please review the release of the following artifacts that we vendor:
 * beam-vendor-bytebuddy-1_10_8

Hi everyone,
Please review and vote on the release candidate #1 for the version 0.1, as
follows:
[ ] +1, Approve the release
[ ] -1, Do not approve the release (please provide specific comments)

The complete staging area is available for your review, which includes:
* the official Apache source release to be deployed to dist.apache.org [1],
which is signed with the key with fingerprint
3415631729E15B33051ADB670A9DAF6713B86349 [2],
* all artifacts to be deployed to the Maven Central Repository [3],
* commit hash "0756edfba7c682b1e75f4691c0ecb38a1dc9d16a" [4],

The vote will be open for at least 72 hours. It is adopted by majority
approval, with at least 3 PMC affirmative votes.

Thanks,
Release Manager

[1] https://dist.apache.org/repos/dist/dev/beam/vendor/
[2] https://dist.apache.org/repos/dist/release/beam/KEYS
[3] https://repository.apache.org/content/repositories/orgapachebeam-1093/
[4]
https://github.com/apache/beam/commit/0756edfba7c682b1e75f4691c0ecb38a1dc9d16a


Re: [PROPOSAL] Vendored bytebuddy dependency release

2020-02-25 Thread Ismaël Mejía
It seems we have a clear consensus so I will proceed with the vote. Thanks!


On Tue, Feb 25, 2020 at 2:19 AM Kenneth Knowles  wrote:

> +1 thanks!
>
> On Mon, Feb 24, 2020 at 1:13 PM Pablo Estrada  wrote:
>
>> +1
>>
>> On Sun, Feb 23, 2020 at 8:31 PM Luke Cwik  wrote:
>>
>>> +1
>>>
>>> On Sun, Feb 23, 2020 at 7:20 PM jincheng sun 
>>> wrote:
>>>
 +1

 Best,
 Jincheng


 Kai Jiang  于2020年2月23日周日 上午2:26写道:

> +1
>
> On Sat, Feb 22, 2020 at 09:50 Alex Van Boxel  wrote:
>
>> +1
>>
>>  _/
>>
>> _/ Alex Van Boxel
>>
>>
>> On Sat, Feb 22, 2020 at 5:45 PM Jean-Baptiste Onofré 
>> wrote:
>>
>>> +1
>>>
>>> Regards
>>> JB
>>>
>>> Le sam. 22 f?vr. 2020 ? 15:02, Isma?l Mej?a  a
>>> ?crit :
>>>
 The version of bytebuddy Beam is vendoring (1.9.3) is more than 16
 months old. I
 would like to propose that we upgrade it [1] to the most recent
 version (1.10.8)
 [2] so we can benefit of the latest improvements for Java 11 and
 upgraded ASM.

 If everyone agrees I would like to volunteer as the release manager
 for this
 upgrade.

 [1] https://issues.apache.org/jira/browse/BEAM-9342
 [2]
 https://github.com/raphw/byte-buddy/blob/master/release-notes.md




Re: SplittableDoFn with Flink fails at checkpointing larger files (200MB)

2020-02-25 Thread Maximilian Michels

Hi Marek,

That's a great question. The answer depends on whether you are using 
portability or the "classic" Runner:


Portability
===

In portability, the SDF functionality includes the option for the Runner 
to split a given bundle such that the remaining current bundle's work 
will be minimized and the deferred remainder of the work will be 
maximized. That would be ideal for checkpointing as soon as possible [1].


Unfortunately, this is not yet implemented in the Flink Runner. That's 
why you are seeing the entire split finishing before the checkpoint.


Implementing this would mean to issue the split call upon checkpointing, 
making sure to checkpoint the remaining work, and resume it after the 
checkpoint has finished.


Perhaps others also could chime in, if there is anything else missing?

Classic
===

AFAIK there is no way to do splitting while processing a split. The best 
option would be to create a custom UnboundedSource which creates smaller 
splits. The default is to use the parallelism for the number of splits. 
Depending on your source this may or may not be trivial.


Cheers,
Max

[1] 
https://github.com/apache/beam/blob/6266296ac037afc775735d4f08d25ffcc1a8e897/model/fn-execution/src/main/proto/beam_fn_api.proto#L421


On 07.02.20 12:18, marek-simu...@seznam.cz wrote:

Hi,
    I am using FileIO with continuously watching folder for new files to 
process. The problem is when flink starts reading 200MB file (around 3M 
elements) and also starts checkpointing. Checkpoint never finishes until 
WHOLE file is processed.


Minimal example :
https://github.com/seznam/beam/blob/simunek/failingCheckpoint/examples/java/src/main/java/org/apache/beam/examples/CheckpointFailingExample.java

My theory what could be wrong from my understanding :
CheckpointMark in this case starts from Create.ofProvider and then its 
propagated to downstream operators where it will be (in queue) behind 
all splits, which means all splits have to be read to successfully 
checkpoint the operator. The problem is even bigger when there are more 
files, then we need to wait for processing all files to successfully 
checkpoint.


1. Are my assumption correct?
2. Is there some possibility to improve behavior of SplittableDoFn (or 
subsequent reading from BoundedSource) for Flink to better propagate 
checkpoint barrier?


For now my fix is reading smaller files (30MB) one by one, by it’s not 
very future proof.


Versions:
Beam 2.17
Flink 1.9

Please correct my poor understanding of checkpointing with Beam and 
Flink and it would be wonderful if you have some advice what to improve 
or where to look.


RE: [EXTERNAL] Re: Java Build broken

2020-02-25 Thread Stefan Djelekar
Hi all,

No this is not yet fixed.

@Pulasthi do you still get the same error?

@Maximilian I don’t have any overrides.
It looks like on localhost build references 
https://oss.sonatype.org/content/repositories/staging/com/google/errorprone/error_prone_check_api/2.3.4/
istead of
https://mvnrepository.com/artifact/com.google.errorprone/error_prone_check_api/2.3.4

and the first link returns 404

[cid:image002.jpg@01D5EBD0.001D1AD0]

Can you please advise?

All the best,
Stefan

From: Pulasthi Supun Wickramasinghe 
Sent: Tuesday, February 18, 2020 5:11 PM
To: dev 
Cc: Stefan Djelekar 
Subject: [EXTERNAL] Re: Java Build broken

Hi All,

Was this issue resolved? I started to get the same error on my local build 
suddenly.

Best Regards,
Pulasthi

On Thu, Jan 23, 2020 at 10:17 AM Maximilian Michels 
mailto:m...@apache.org>> wrote:
Do you have any overrides in your ~/.m2/settings.xml? The artifacts
should be found as part of Maven central, e.g.
https://mvnrepository.com/artifact/com.google.errorprone/error_prone_check_api

Cheers,
Max

On 23.01.20 11:11, Stefan Djelekar wrote:
> Hi guys,
>
> It’s been days now since the build for Java SDK is broken.
>
> Namely, pipeline is successful on Jenkins, but it fails in my localhost
> with error in task model:pipeline:compileJava. As I've seen, last couple
> of builds were served from cache, so maybe that is the reason why it's
> green. I confirmed same thing happened to other devs as well.
>
> 22:49:34 > Task :model:pipeline:compileJava FROM-CACHE
>
> It looks like it’s related to mismatch of com.google.errorprone library
> version. Can someone please take a look as this is a blocker to
> localhost development?
>
> Cheers,
>
> *Stefan Đelekar*
>
> Sofware Engineer
>
> Mobile +381 65 22 33 293
>
> stefan.djele...@redbox.com 
> >
>
> Redbox Automated Retail LLC / Interventure d.o.o
>
> 16. Oktobra 21/67, 11050 Zvezdara, Serbia
>
> djelekar.com
>
> --
>
> This email is confidential and intended for the recipient specified in
> the message only.
> If you have received this message in error, please notify the sender by
> e-mail, discard any
> paper copies and delete the mail and all electronic files of the message
> from your system.
>


--
Pulasthi S. Wickramasinghe
PhD Candidate  | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington
cell: 224-386-9035


Re: [ANNOUNCE] New committer: Chad Dombrova

2020-02-25 Thread Jan Lukavský

Congrats Chad!

On 2/25/20 9:48 AM, Gleb Kanterov wrote:

Congratulations!

On Tue, Feb 25, 2020 at 9:44 AM Ismaël Mejía > wrote:


Congratulations, so well deserved for the lots of amazing work and
new perspectives you have broght into the project !!!

On Tue, Feb 25, 2020 at 8:24 AM Austin Bennett
mailto:whatwouldausti...@gmail.com>>
wrote:

Hooray!

On Mon, Feb 24, 2020, 11:21 PM Alex Van Boxel
mailto:a...@vanboxel.be>> wrote:

Congrats!

 _/
_/ Alex Van Boxel


On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise
mailto:t...@apache.org>> wrote:

Congratulations!


On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka
mailto:goe...@google.com>> wrote:

Congratulations Chad!

On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay
mailto:al...@google.com>> wrote:

Congratulations!

On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne
mailto:samb...@gmail.com>>
wrote:

Nice one Chad. Your typing efforts are
very welcomed.

On Tue, Feb 25, 2020 at 10:16 AM Yichi
Zhang mailto:zyi...@google.com>> wrote:

Congratulations, Chad!

On Mon, Feb 24, 2020 at 3:10 PM Robert
Bradshaw mailto:rober...@google.com>> wrote:

Well deserved, Chad. Congratulations!

On Mon, Feb 24, 2020 at 2:43 PM
Reza Rokni mailto:r...@google.com>> wrote:
>
> Congratulations! :-)
>
> On Tue, Feb 25, 2020 at 6:41 AM
Chad Dombrova mailto:chad...@gmail.com>> wrote:
>>
>> Thanks, folks!  I'm very
excited to "retest this" :)
>>
>> Especially big thanks to Robert
and Udi for all their hard work
reviewing my PRs.
>>
>> -chad
>>
>>
>> On Mon, Feb 24, 2020 at 1:44 PM
Brian Hulette mailto:bhule...@google.com>> wrote:
>>>
>>> Congratulations Chad! Thanks
for all your contributions :)
>>>
>>> On Mon, Feb 24, 2020 at 1:43
PM Kyle Weaver
mailto:kcwea...@google.com>> wrote:

 Well-deserved, thanks for
your dedication to the project
Chad. :)

 On Mon, Feb 24, 2020 at 1:34
PM Udi Meiri mailto:eh...@google.com>> wrote:
>
> Congrats and welcome, Chad!
>
> On Mon, Feb 24, 2020 at 1:21
PM Pablo Estrada
mailto:pabl...@google.com>> wrote:
>>
>> Hi everyone,
>>
>> Please join me and the rest
of the Beam PMC in welcoming a new
committer: Chad Dombrova
>>
>> Chad has contributed to the
project in multiple ways,
including improvements to the
testing infrastructure, and adding
type annotations throughout the
Python SDK, as well as working
closely with the community on
these improvements.
>>
  

Re: [VOTE] Upgrade gradle to 6.2

2020-02-25 Thread Gleb Kanterov
+1 (non-binding)

On Tue, Feb 25, 2020 at 9:38 AM Ismaël Mejía  wrote:

> +1 great to have our build updated, please share if there are new
> interesting features/plugin advantages we can benefit from too.
>
> On Tue, Feb 25, 2020 at 8:24 AM Jean-Baptiste Onofré 
> wrote:
>
>> Hi Alex
>>
>> I also have couple of contacts at Gradle. Let me know if needed.
>>
>> Regards
>> JB
>>
>> Le mar. 25 f?vr. 2020 ? 08:20, Alex Van Boxel  a
>> ?crit :
>>
>>> OK, great. I know someone that works at gradle, so I can ping them when
>>> I have some problems.
>>>
>>> Any other know pitfalls I can expect, let me know :-)
>>>
>>>  _/
>>> _/ Alex Van Boxel
>>>
>>>
>>> On Tue, Feb 25, 2020 at 7:20 AM Jean-Baptiste Onofr? 
>>> wrote:
>>>
>>> +1
>>>
>>> It makes sense.
>>>
>>> Thanks.
>>> Regards
>>> JB
>>>
>>> Le lun. 24 f?vr. 2020 ? 22:37, Alex Van Boxel  a
>>> ?crit :
>>>
>>> Anyone objections that I upgrade gradle to 6.2. If ok this will be done
>>> over several commits where I will:
>>>
>>>- Upgrade plugins
>>>- Upgrade gradle to 6.2
>>>- See where we can use some of the new features
>>>
>>>
>>>  _/
>>> _/ Alex Van Boxel
>>>
>>>


Re: [ANNOUNCE] New committer: Chad Dombrova

2020-02-25 Thread Gleb Kanterov
Congratulations!

On Tue, Feb 25, 2020 at 9:44 AM Ismaël Mejía  wrote:

> Congratulations, so well deserved for the lots of amazing work and new
> perspectives you have broght into the project !!!
>
> On Tue, Feb 25, 2020 at 8:24 AM Austin Bennett <
> whatwouldausti...@gmail.com> wrote:
>
>> Hooray!
>>
>> On Mon, Feb 24, 2020, 11:21 PM Alex Van Boxel  wrote:
>>
>>> Congrats!
>>>
>>>  _/
>>> _/ Alex Van Boxel
>>>
>>>
>>> On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise  wrote:
>>>
 Congratulations!


 On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka  wrote:

> Congratulations Chad!
>
> On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay  wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne  wrote:
>>
>>> Nice one Chad. Your typing efforts are very welcomed.
>>>
>>> On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang 
>>> wrote:
>>>
 Congratulations, Chad!

 On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw <
 rober...@google.com> wrote:

> Well deserved, Chad. Congratulations!
>
> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
> >
> > Congratulations! :-)
> >
> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova 
> wrote:
> >>
> >> Thanks, folks!  I'm very excited to "retest this" :)
> >>
> >> Especially big thanks to Robert and Udi for all their hard work
> reviewing my PRs.
> >>
> >> -chad
> >>
> >>
> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette <
> bhule...@google.com> wrote:
> >>>
> >>> Congratulations Chad! Thanks for all your contributions :)
> >>>
> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver <
> kcwea...@google.com> wrote:
> 
>  Well-deserved, thanks for your dedication to the project
> Chad. :)
> 
>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri 
> wrote:
> >
> > Congrats and welcome, Chad!
> >
> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada <
> pabl...@google.com> wrote:
> >>
> >> Hi everyone,
> >>
> >> Please join me and the rest of the Beam PMC in welcoming a
> new committer: Chad Dombrova
> >>
> >> Chad has contributed to the project in multiple ways,
> including improvements to the testing infrastructure, and adding type
> annotations throughout the Python SDK, as well as working closely 
> with the
> community on these improvements.
> >>
> >> In consideration of his contributions, the Beam PMC trusts
> him with the responsibilities of a Beam Committer[1].
> >>
> >> Thanks Chad for your contributions!
> >>
> >> -Pablo, on behalf of the Apache Beam PMC.
> >>
> >> [1]
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>



Re: [ANNOUNCE] New committer: Chad Dombrova

2020-02-25 Thread Ismaël Mejía
Congratulations, so well deserved for the lots of amazing work and new
perspectives you have broght into the project !!!

On Tue, Feb 25, 2020 at 8:24 AM Austin Bennett 
wrote:

> Hooray!
>
> On Mon, Feb 24, 2020, 11:21 PM Alex Van Boxel  wrote:
>
>> Congrats!
>>
>>  _/
>> _/ Alex Van Boxel
>>
>>
>> On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise  wrote:
>>
>>> Congratulations!
>>>
>>>
>>> On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka  wrote:
>>>
 Congratulations Chad!

 On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay  wrote:

> Congratulations!
>
> On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne  wrote:
>
>> Nice one Chad. Your typing efforts are very welcomed.
>>
>> On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang 
>> wrote:
>>
>>> Congratulations, Chad!
>>>
>>> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw 
>>> wrote:
>>>
 Well deserved, Chad. Congratulations!

 On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
 >
 > Congratulations! :-)
 >
 > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova 
 wrote:
 >>
 >> Thanks, folks!  I'm very excited to "retest this" :)
 >>
 >> Especially big thanks to Robert and Udi for all their hard work
 reviewing my PRs.
 >>
 >> -chad
 >>
 >>
 >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette <
 bhule...@google.com> wrote:
 >>>
 >>> Congratulations Chad! Thanks for all your contributions :)
 >>>
 >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver <
 kcwea...@google.com> wrote:
 
  Well-deserved, thanks for your dedication to the project Chad.
 :)
 
  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri 
 wrote:
 >
 > Congrats and welcome, Chad!
 >
 > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada <
 pabl...@google.com> wrote:
 >>
 >> Hi everyone,
 >>
 >> Please join me and the rest of the Beam PMC in welcoming a
 new committer: Chad Dombrova
 >>
 >> Chad has contributed to the project in multiple ways,
 including improvements to the testing infrastructure, and adding type
 annotations throughout the Python SDK, as well as working closely with 
 the
 community on these improvements.
 >>
 >> In consideration of his contributions, the Beam PMC trusts
 him with the responsibilities of a Beam Committer[1].
 >>
 >> Thanks Chad for your contributions!
 >>
 >> -Pablo, on behalf of the Apache Beam PMC.
 >>
 >> [1]
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>


Re: [VOTE] Upgrade gradle to 6.2

2020-02-25 Thread Ismaël Mejía
+1 great to have our build updated, please share if there are new
interesting features/plugin advantages we can benefit from too.

On Tue, Feb 25, 2020 at 8:24 AM Jean-Baptiste Onofré 
wrote:

> Hi Alex
>
> I also have couple of contacts at Gradle. Let me know if needed.
>
> Regards
> JB
>
> Le mar. 25 f?vr. 2020 ? 08:20, Alex Van Boxel  a ?crit :
>
>> OK, great. I know someone that works at gradle, so I can ping them when I
>> have some problems.
>>
>> Any other know pitfalls I can expect, let me know :-)
>>
>>  _/
>> _/ Alex Van Boxel
>>
>>
>> On Tue, Feb 25, 2020 at 7:20 AM Jean-Baptiste Onofr? 
>> wrote:
>>
>> +1
>>
>> It makes sense.
>>
>> Thanks.
>> Regards
>> JB
>>
>> Le lun. 24 f?vr. 2020 ? 22:37, Alex Van Boxel  a
>> ?crit :
>>
>> Anyone objections that I upgrade gradle to 6.2. If ok this will be done
>> over several commits where I will:
>>
>>- Upgrade plugins
>>- Upgrade gradle to 6.2
>>- See where we can use some of the new features
>>
>>
>>  _/
>> _/ Alex Van Boxel
>>
>>