Re: [ANNOUNCE] New committer: Jincheng Sun

[Austin Bennett]

Congrats!

On Mon, Feb 24, 2020, 11:22 PM Alex Van Boxel  wrote:

> Congrats!
>
>  _/
> _/ Alex Van Boxel
>
>
> On Mon, Feb 24, 2020 at 8:13 PM Kyle Weaver  wrote:
>
>> Thanks Jincheng for all your work on Beam and Flink integration.
>>
>> On Mon, Feb 24, 2020 at 11:02 AM Yichi Zhang  wrote:
>>
>>> Congrats, Jincheng!
>>>
>>> On Mon, Feb 24, 2020 at 9:45 AM Ahmet Altay  wrote:
>>>
 Congratulations!

 On Mon, Feb 24, 2020 at 6:48 AM Thomas Weise  wrote:

> Congratulations!
>
>
> On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía 
> wrote:
>
>> Congrats Jincheng!
>>
>> On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov 
>> wrote:
>>
>>> Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng 
>>> wrote:
>>>
 Congratulations Jincheng, well deserved!

 Best,
 Hequn

 On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:

> Congrats!
>
> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský 
> wrote:
>
>> Congrats Jincheng!
>>
>>   Jan
>>
>> On 2/24/20 11:55 AM, Maximilian Michels wrote:
>> > Hi everyone,
>> >
>> > Please join me and the rest of the Beam PMC in welcoming a new
>> > committer: Jincheng Sun 
>> >
>> > Jincheng has worked on generalizing parts of Beam for Flink's
>> Python
>> > API. He has also picked up other issues, like fixing
>> documentation,
>> > implementing missing features, or cleaning up code [1].
>> >
>> > In consideration of his contributions, the Beam PMC trusts him
>> with
>> > the responsibilities of a Beam committer [2].
>> >
>> > Thank you for your contributions Jincheng!
>> >
>> > -Max, on behalf of the Apache Beam PMC
>> >
>> > [1]
>> >
>> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
>> > [2]
>> >
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>

Re: [VOTE] Upgrade gradle to 6.2

[Jean-Baptiste Onofré]

Hi AlexI also have couple of contacts at Gradle. Let me know if needed. Regards JBLe mar. 25 f?vr. 2020 ? 08:20, Alex Van Boxel  a ?crit :OK, great. I know someone that works at gradle, so I can ping them when I have some problems.Any other know pitfalls I can expect, let me know :-) _/_/ Alex Van BoxelOn Tue, Feb 25, 2020 at 7:20 AM Jean-Baptiste Onofr?  wrote:+1It makes sense. Thanks. Regards JBLe lun. 24 f?vr. 2020 ? 22:37, Alex Van Boxel  a ?crit :Anyone objections that I upgrade gradle to 6.2. If ok this will be done over several commits where I will:Upgrade pluginsUpgrade gradle to 6.2See where we can use some of the new features _/_/ Alex Van Boxel

Re: [ANNOUNCE] New committer: Chad Dombrova

[Austin Bennett]

Hooray!

On Mon, Feb 24, 2020, 11:21 PM Alex Van Boxel  wrote:

> Congrats!
>
>  _/
> _/ Alex Van Boxel
>
>
> On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise  wrote:
>
>> Congratulations!
>>
>>
>> On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka  wrote:
>>
>>> Congratulations Chad!
>>>
>>> On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay  wrote:
>>>
 Congratulations!

 On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne  wrote:

> Nice one Chad. Your typing efforts are very welcomed.
>
> On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang 
> wrote:
>
>> Congratulations, Chad!
>>
>> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw 
>> wrote:
>>
>>> Well deserved, Chad. Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
>>> >
>>> > Congratulations! :-)
>>> >
>>> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova 
>>> wrote:
>>> >>
>>> >> Thanks, folks!  I'm very excited to "retest this" :)
>>> >>
>>> >> Especially big thanks to Robert and Udi for all their hard work
>>> reviewing my PRs.
>>> >>
>>> >> -chad
>>> >>
>>> >>
>>> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette <
>>> bhule...@google.com> wrote:
>>> >>>
>>> >>> Congratulations Chad! Thanks for all your contributions :)
>>> >>>
>>> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver 
>>> wrote:
>>> 
>>>  Well-deserved, thanks for your dedication to the project Chad.
>>> :)
>>> 
>>>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri 
>>> wrote:
>>> >
>>> > Congrats and welcome, Chad!
>>> >
>>> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada <
>>> pabl...@google.com> wrote:
>>> >>
>>> >> Hi everyone,
>>> >>
>>> >> Please join me and the rest of the Beam PMC in welcoming a
>>> new committer: Chad Dombrova
>>> >>
>>> >> Chad has contributed to the project in multiple ways,
>>> including improvements to the testing infrastructure, and adding type
>>> annotations throughout the Python SDK, as well as working closely with 
>>> the
>>> community on these improvements.
>>> >>
>>> >> In consideration of his contributions, the Beam PMC trusts
>>> him with the responsibilities of a Beam Committer[1].
>>> >>
>>> >> Thanks Chad for your contributions!
>>> >>
>>> >> -Pablo, on behalf of the Apache Beam PMC.
>>> >>
>>> >> [1]
>>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>>
>>

Re: [ANNOUNCE] New committer: Jincheng Sun

[Alex Van Boxel]

Congrats!

 _/
_/ Alex Van Boxel


On Mon, Feb 24, 2020 at 8:13 PM Kyle Weaver  wrote:

> Thanks Jincheng for all your work on Beam and Flink integration.
>
> On Mon, Feb 24, 2020 at 11:02 AM Yichi Zhang  wrote:
>
>> Congrats, Jincheng!
>>
>> On Mon, Feb 24, 2020 at 9:45 AM Ahmet Altay  wrote:
>>
>>> Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 6:48 AM Thomas Weise  wrote:
>>>
 Congratulations!


 On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía  wrote:

> Congrats Jincheng!
>
> On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov 
> wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng  wrote:
>>
>>> Congratulations Jincheng, well deserved!
>>>
>>> Best,
>>> Hequn
>>>
>>> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:
>>>
 Congrats!

 On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský 
 wrote:

> Congrats Jincheng!
>
>   Jan
>
> On 2/24/20 11:55 AM, Maximilian Michels wrote:
> > Hi everyone,
> >
> > Please join me and the rest of the Beam PMC in welcoming a new
> > committer: Jincheng Sun 
> >
> > Jincheng has worked on generalizing parts of Beam for Flink's
> Python
> > API. He has also picked up other issues, like fixing
> documentation,
> > implementing missing features, or cleaning up code [1].
> >
> > In consideration of his contributions, the Beam PMC trusts him
> with
> > the responsibilities of a Beam committer [2].
> >
> > Thank you for your contributions Jincheng!
> >
> > -Max, on behalf of the Apache Beam PMC
> >
> > [1]
> >
> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
> > [2]
> >
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>

Re: [ANNOUNCE] New committer: Chad Dombrova

[Alex Van Boxel]

Congrats!

 _/
_/ Alex Van Boxel


On Tue, Feb 25, 2020 at 6:21 AM Thomas Weise  wrote:

> Congratulations!
>
>
> On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka  wrote:
>
>> Congratulations Chad!
>>
>> On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay  wrote:
>>
>>> Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne  wrote:
>>>
 Nice one Chad. Your typing efforts are very welcomed.

 On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang  wrote:

> Congratulations, Chad!
>
> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw 
> wrote:
>
>> Well deserved, Chad. Congratulations!
>>
>> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
>> >
>> > Congratulations! :-)
>> >
>> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova 
>> wrote:
>> >>
>> >> Thanks, folks!  I'm very excited to "retest this" :)
>> >>
>> >> Especially big thanks to Robert and Udi for all their hard work
>> reviewing my PRs.
>> >>
>> >> -chad
>> >>
>> >>
>> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette 
>> wrote:
>> >>>
>> >>> Congratulations Chad! Thanks for all your contributions :)
>> >>>
>> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver 
>> wrote:
>> 
>>  Well-deserved, thanks for your dedication to the project Chad. :)
>> 
>>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri 
>> wrote:
>> >
>> > Congrats and welcome, Chad!
>> >
>> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada <
>> pabl...@google.com> wrote:
>> >>
>> >> Hi everyone,
>> >>
>> >> Please join me and the rest of the Beam PMC in welcoming a new
>> committer: Chad Dombrova
>> >>
>> >> Chad has contributed to the project in multiple ways,
>> including improvements to the testing infrastructure, and adding type
>> annotations throughout the Python SDK, as well as working closely with 
>> the
>> community on these improvements.
>> >>
>> >> In consideration of his contributions, the Beam PMC trusts him
>> with the responsibilities of a Beam Committer[1].
>> >>
>> >> Thanks Chad for your contributions!
>> >>
>> >> -Pablo, on behalf of the Apache Beam PMC.
>> >>
>> >> [1]
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>

Re: [VOTE] Upgrade gradle to 6.2

[Alex Van Boxel]

OK, great. I know someone that works at gradle, so I can ping them when I
have some problems.

Any other know pitfalls I can expect, let me know :-)

 _/
_/ Alex Van Boxel


On Tue, Feb 25, 2020 at 7:20 AM Jean-Baptiste Onofré 
wrote:

> +1
>
> It makes sense.
>
> Thanks.
> Regards
> JB
>
> Le lun. 24 f?vr. 2020 ? 22:37, Alex Van Boxel  a ?crit :
>
>> Anyone objections that I upgrade gradle to 6.2. If ok this will be done
>> over several commits where I will:
>>
>>- Upgrade plugins
>>- Upgrade gradle to 6.2
>>- See where we can use some of the new features
>>
>>
>>  _/
>> _/ Alex Van Boxel
>>
>

Re: [VOTE] Upgrade gradle to 6.2

[Jean-Baptiste Onofré]

+1It makes sense. Thanks. Regards JBLe lun. 24 f?vr. 2020 ? 22:37, Alex Van Boxel  a ?crit :Anyone objections that I upgrade gradle to 6.2. If ok this will be done over several commits where I will:Upgrade pluginsUpgrade gradle to 6.2See where we can use some of the new features _/_/ Alex Van Boxel

Re: [ANNOUNCE] New committer: Chad Dombrova

[Thomas Weise]

Congratulations!


On Mon, Feb 24, 2020, 3:38 PM Ankur Goenka  wrote:

> Congratulations Chad!
>
> On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay  wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne  wrote:
>>
>>> Nice one Chad. Your typing efforts are very welcomed.
>>>
>>> On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang  wrote:
>>>
 Congratulations, Chad!

 On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw 
 wrote:

> Well deserved, Chad. Congratulations!
>
> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
> >
> > Congratulations! :-)
> >
> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova 
> wrote:
> >>
> >> Thanks, folks!  I'm very excited to "retest this" :)
> >>
> >> Especially big thanks to Robert and Udi for all their hard work
> reviewing my PRs.
> >>
> >> -chad
> >>
> >>
> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette 
> wrote:
> >>>
> >>> Congratulations Chad! Thanks for all your contributions :)
> >>>
> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver 
> wrote:
> 
>  Well-deserved, thanks for your dedication to the project Chad. :)
> 
>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri 
> wrote:
> >
> > Congrats and welcome, Chad!
> >
> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada <
> pabl...@google.com> wrote:
> >>
> >> Hi everyone,
> >>
> >> Please join me and the rest of the Beam PMC in welcoming a new
> committer: Chad Dombrova
> >>
> >> Chad has contributed to the project in multiple ways, including
> improvements to the testing infrastructure, and adding type annotations
> throughout the Python SDK, as well as working closely with the community 
> on
> these improvements.
> >>
> >> In consideration of his contributions, the Beam PMC trusts him
> with the responsibilities of a Beam Committer[1].
> >>
> >> Thanks Chad for your contributions!
> >>
> >> -Pablo, on behalf of the Apache Beam PMC.
> >>
> >> [1]
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>

Re: Please comment on draft comms strategy by Oct 16

[María Cruz]

Hi Matthias,
thank you for the detailed feedback on the comms strategy. I can definitely
help with the Youtube channel revamp and social media copy to promote that
content. I will connect off-list with you about this.

All: given that we are moving forward with this strategy, I created a pull
request  to add this work to
Beam's repository. Please let me know if the PR is missing information.

Thanks,

María


On Tue, Feb 18, 2020 at 10:51 PM Matthias Baetens 
wrote:

> To make things more concrete, we could start by giving the YouTube channel
> more structure & drafting Tweets to go out on the different channels. Is
> this something you'd be able to help out with?
>
> Thanks!
> Matthias
>
> On Sun, 16 Feb 2020 at 10:29, Matthias Baetens 
> wrote:
>
>> Hey Maria,
>>
>> Great work!! This provides a nice overview of all the channels and how
>> they contribute to the overall project.
>>
>> I will comment on the sections I am most familiar with:
>> - Website / blog: +1 for reducing the number of outlets to 1. The Beam
>> Summit website repository could use a clean-up and this can be part of that
>> effort.
>> - Workshops: Pablo & Austin have done some great work in this space, but
>> even more dedicated time & attention to this would greatly benefit the
>> project - and as Pablo raised this might be separate from project
>> transparency but directly contribute to increased usage.
>> -- For topics/format, it's split up in how to use and how to contribute.
>> How to use I would split up in a beginner training ("set-up and write your
>> first pipeline"), advanced training ("use some of the more exotic Beam
>> features / optimise your pipeline") and office hours (solve specific
>> problems people have which can feedback to the development team as well in
>> terms of missing features or potential improvement of documentation). Imho,
>> classroom style with an intro to Beam and concepts / advanced topics,
>> combined with do-it-yourself where people actually do some coding with
>> something to walk away is good for onboarding. A more fun and game-like
>> set-up can be done for users that already use Beam where we can accommodate
>> discussions or have them solve a specific use-case in groups.
>> - +1 for increased distribution of the talks. We should actively engage
>> the speakers as well to share it on their personal channels, and surface
>> the talks on the website and Twitter handles. Also a curated list as
>> blogpost would be a great addition.
>>
>> I think a more consistent feedback mechanism and maybe idea box at events
>> might also benefit the community.
>>
>> Cheers,
>> Matthias
>>
>>
>> On Wed, 15 Jan 2020 at 22:46, Austin Bennett 
>> wrote:
>>
>>> Hi Kenn,
>>>
>>> We had workshop on this very topic (how to contribute to Beam) at our
>>> Berlin Summit:  https://www.youtube.com/watch?v=PtPslSdAPcM There's is
>>> certainly room for me (or anyone) to cleanup and formalize that a bit
>>> more.  Though, the views of that are relatively small, which either indeed
>>> points to a lack of appetite and/or that it wasn't well publicized (I
>>> suspect both).
>>>
>>> Cheers,
>>> Austin
>>>
>>> On Thu, Jan 9, 2020 at 6:53 PM Kenneth Knowles  wrote:
>>>
 Wow, this is great work. I looked at the graphical maps when you sent
 them but forgot to reply on thread. They really distill a lot of
 possibilities and help to think about the current state.

 These three action items seem good and doable. Thanks for highlighting
 those. The only one that isn't obvious to me is "workshop on how to
 contribute to Beam". Is there enough appetite / audience to make this a
 workshop? What forms could this take? A live coding demonstration in a
 normal talk slot at an OSS or data conference seems like a possibility.
 Whatever we do, we should record and distribute for sure, because when
 someone wants to contribute, they need to find the resources at that 
 moment.

 Kenn

 On Wed, Jan 8, 2020 at 1:01 PM María Cruz  wrote:

> Hi everyone,
> I'm writing to send an update about the communication strategy for
> Beam. In a nutshell, I have 3 proposed changes (copied from the md file
> here:
> https://github.com/macruzbar/beam/blob/master/Communication-strategy-DRAFT.md
> ).
>
> While all the channels are connected to a specific function in the
> short, medium, and long term, some areas have redundancies, and some other
> areas could use more exposure. In order to continue to grow the project,
> there are 3 proposals we need to focus on (click on the link on each
> section to read more):
>
> 1. Blog post categories, frequency and distribution.
> 
>  Reduce
> to one blogging space (on Beam Website). Incorporate 3 categories to the

Re: [VOTE] Upgrade gradle to 6.2

[Luke Cwik]

+1

Can you try testing removing -Ppublishing and no parallel build requirement
when doing the release?
(some of the Gradle release related plugins had issues when running in
parallel which manifested as build errors when doing the release).

On Mon, Feb 24, 2020 at 5:43 PM Kenneth Knowles  wrote:

> +1
>
> On Mon, Feb 24, 2020 at 5:42 PM Kenneth Knowles  wrote:
>
>> If I recall correctly, upgrading gradle would unblock getting some better
>> static analysis plugins. That would be stellar, since we lost a lot of our
>> analysis when we switched from findbugs to spotbugs (it seems to just not
>> care as much about some errors, which have since flourished)
>>
>> Kenn
>>
>> On Mon, Feb 24, 2020 at 1:38 PM Alex Van Boxel  wrote:
>>
>>> Anyone objections that I upgrade gradle to 6.2. If ok this will be done
>>> over several commits where I will:
>>>
>>>- Upgrade plugins
>>>- Upgrade gradle to 6.2
>>>- See where we can use some of the new features
>>>
>>>
>>>  _/
>>> _/ Alex Van Boxel
>>>
>>

Re: [VOTE] Upgrade gradle to 6.2

[Kenneth Knowles]

+1

On Mon, Feb 24, 2020 at 5:42 PM Kenneth Knowles  wrote:

> If I recall correctly, upgrading gradle would unblock getting some better
> static analysis plugins. That would be stellar, since we lost a lot of our
> analysis when we switched from findbugs to spotbugs (it seems to just not
> care as much about some errors, which have since flourished)
>
> Kenn
>
> On Mon, Feb 24, 2020 at 1:38 PM Alex Van Boxel  wrote:
>
>> Anyone objections that I upgrade gradle to 6.2. If ok this will be done
>> over several commits where I will:
>>
>>- Upgrade plugins
>>- Upgrade gradle to 6.2
>>- See where we can use some of the new features
>>
>>
>>  _/
>> _/ Alex Van Boxel
>>
>

Re: [VOTE] Upgrade gradle to 6.2

[Kenneth Knowles]

If I recall correctly, upgrading gradle would unblock getting some better
static analysis plugins. That would be stellar, since we lost a lot of our
analysis when we switched from findbugs to spotbugs (it seems to just not
care as much about some errors, which have since flourished)

Kenn

On Mon, Feb 24, 2020 at 1:38 PM Alex Van Boxel  wrote:

> Anyone objections that I upgrade gradle to 6.2. If ok this will be done
> over several commits where I will:
>
>- Upgrade plugins
>- Upgrade gradle to 6.2
>- See where we can use some of the new features
>
>
>  _/
> _/ Alex Van Boxel
>

Re: [PROPOSAL] Vendored bytebuddy dependency release

[Kenneth Knowles]

+1 thanks!

On Mon, Feb 24, 2020 at 1:13 PM Pablo Estrada  wrote:

> +1
>
> On Sun, Feb 23, 2020 at 8:31 PM Luke Cwik  wrote:
>
>> +1
>>
>> On Sun, Feb 23, 2020 at 7:20 PM jincheng sun 
>> wrote:
>>
>>> +1
>>>
>>> Best,
>>> Jincheng
>>>
>>>
>>> Kai Jiang  于2020年2月23日周日 上午2:26写道：
>>>
 +1

 On Sat, Feb 22, 2020 at 09:50 Alex Van Boxel  wrote:

> +1
>
>  _/
>
> _/ Alex Van Boxel
>
>
> On Sat, Feb 22, 2020 at 5:45 PM Jean-Baptiste Onofré 
> wrote:
>
>> +1
>>
>> Regards
>> JB
>>
>> Le sam. 22 f?vr. 2020 ? 15:02, Isma?l Mej?a  a
>> ?crit :
>>
>>> The version of bytebuddy Beam is vendoring (1.9.3) is more than 16
>>> months old. I
>>> would like to propose that we upgrade it [1] to the most recent
>>> version (1.10.8)
>>> [2] so we can benefit of the latest improvements for Java 11 and
>>> upgraded ASM.
>>>
>>> If everyone agrees I would like to volunteer as the release manager
>>> for this
>>> upgrade.
>>>
>>> [1] https://issues.apache.org/jira/browse/BEAM-9342
>>> [2] https://github.com/raphw/byte-buddy/blob/master/release-notes.md
>>>
>>>

Review Request

[Wenbing Bai]

Hi there,

Can I get a review on this PR https://github.com/apache/beam/pull/10901?

-- 





Wenbing Bai

Senior Software Engineer, MLP

Cruise

Pronouns: She/Her

-- 


*Confidentiality Note:* We care about protecting our proprietary 
information, confidential material, and trade secrets. This message may 
contain some or all of those things. Cruise will suffer material harm if 
anyone other than the intended recipient disseminates or takes any action 
based on this message. If you have received this message (including any 
attachments) in error, please delete it immediately and notify the sender 
promptly.

Re: Contributor permission for Beam Jira tickets

[Yu Watanabe]

Thank you .
Sorry for the late reply .

On Sun, Feb 16, 2020, 18:25 Ismaël Mejía  wrote:

> done
>
> On Sun, Feb 16, 2020 at 9:02 AM Yu Watanabe  wrote:
>
>> Hi,
>>
>> This is Yu Watanabe from Creationline.
>> I would like to contribute issues related to elasticsearch and kafka.
>> Can someone add me as a contributor for Beam's Jira issue
>> tracker? I would like to create/assign tickets for my work.
>>
>> My username in JIRA is "y-watanabe"
>>
>> Thanks,
>> Yu
>>
>> --
>> Yu Watanabe
>>
>> linkedin: www.linkedin.com/in/yuwatanabe1/
>> twitter:   twitter.com/yuwtennis
>>
>

Re: Docker images are migrated to Apache org.

[Hannah Jiang]

Thanks everyone for reporting issues and potential problems etc.
Please feel free to let me know if you have any questions or see additional
issues!


On Mon, Feb 24, 2020 at 4:17 PM Pablo Estrada  wrote:

> Thanks Hannah! This is great : )
> -P.
>
> On Mon, Feb 24, 2020 at 12:47 PM Robert Burke  wrote:
>
>> It was only merged in an hour ago. That explains why I still saw it as
>> broken before lunch. Thanks again!
>>
>> On Mon, Feb 24, 2020, 12:42 PM Robert Burke  wrote:
>>
>>> NVM. I had stale pages for some reason. Hannah fixed them already. :D
>>>
>>> On Mon, Feb 24, 2020, 12:39 PM Robert Burke  wrote:
>>>
 Looks like the change broke the go post commits. I've filed BEAM-9374
 for it. I think I know how to fix it though.

 On Mon, Feb 24, 2020, 12:18 PM Kyle Weaver  wrote:

> I wonder if searching for "apache beam" (instead of "apache/beam")
> will ever work on Docker hub once the new images start getting more
> downloads. Currently no relevant results are included. Unfortunately this
> might not be something we can control, as it seems Docker hub's search is
> not as sophisticated as I would like.
>
> Anyway, this is still a great improvement. Thanks Hannah for making
> this happen.
>
> On Fri, Feb 21, 2020 at 11:58 AM Ahmet Altay  wrote:
>
>> Thank you, Hannah! This is great.
>>
>> On Fri, Feb 21, 2020 at 11:24 AM Hannah Jiang 
>> wrote:
>>
>>> Hello team
>>>
>>> Docker SDK images (Python, Java, Go) and Flink job server images are
>>> migrated to Apache org[1].
>>> I confirmed digests of all the images of the two repos are
>>> exactly the same.
>>> In addition, I updated readme pages of the new repos. The readma
>>> pages match to the ones at github.
>>>
>>> New images will be deployed to Apache org from v2.20.0. I added
>>> notices to the original repos[2] about the changes.
>>> Spark job server images will be added from v2.20 as well, thanks for 
>>> @Kyle
>>> Weaver  to make it happen[3].
>>>
>>> Thanks,
>>> Hannah
>>>
>>> 1. https://hub.docker.com/search?q=apache%2Fbeam=image
>>> 2. https://hub.docker.com/search?q=apachebeam/=image
>>> 3. https://github.com/apache/beam/pull/10921
>>>
>>

Re: Request to be added as contributor

[Emiliano Capoccia]

Thanks Ahmet,

best

Emiliano

On 2020/02/24 23:57:03, Ahmet Altay  wrote: 
> Done. Added you as a contributor.
> 
> On Mon, Feb 24, 2020 at 2:54 PM Emiliano Capoccia <
> emiliano.capoc...@gmail.com> wrote:
> 
> > Hi Luke,
> >
> > that is "ecapoccia"
> >
> > Thanks, I look forward to hearing from you
> >
> > Emiliano
> >
> > On 2020/02/24 17:03:53, Luke Cwik  wrote:
> > > Please provide your user id associated with your JIRA account.
> > >
> > > On Mon, Feb 24, 2020 at 8:38 AM Emiliano Capoccia  > >
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > This is Emiliano from JP Morgan Chase. I develop applications in Beam
> > and
> > > > run them on a Spark cluster on Kubernetes. Can someone add me as a
> > > > contributor for Beam's Jira issue tracker? I would like to
> > create/assign
> > > > tickets for my work.
> > > >
> > > > Thanks,
> > > >
> > > > Emiliano
> > > >
> > >
> >
> 

Re: Docker images are migrated to Apache org.

[Pablo Estrada]

Thanks Hannah! This is great : )
-P.

On Mon, Feb 24, 2020 at 12:47 PM Robert Burke  wrote:

> It was only merged in an hour ago. That explains why I still saw it as
> broken before lunch. Thanks again!
>
> On Mon, Feb 24, 2020, 12:42 PM Robert Burke  wrote:
>
>> NVM. I had stale pages for some reason. Hannah fixed them already. :D
>>
>> On Mon, Feb 24, 2020, 12:39 PM Robert Burke  wrote:
>>
>>> Looks like the change broke the go post commits. I've filed BEAM-9374
>>> for it. I think I know how to fix it though.
>>>
>>> On Mon, Feb 24, 2020, 12:18 PM Kyle Weaver  wrote:
>>>
 I wonder if searching for "apache beam" (instead of "apache/beam") will
 ever work on Docker hub once the new images start getting more downloads.
 Currently no relevant results are included. Unfortunately this might not be
 something we can control, as it seems Docker hub's search is not as
 sophisticated as I would like.

 Anyway, this is still a great improvement. Thanks Hannah for making
 this happen.

 On Fri, Feb 21, 2020 at 11:58 AM Ahmet Altay  wrote:

> Thank you, Hannah! This is great.
>
> On Fri, Feb 21, 2020 at 11:24 AM Hannah Jiang 
> wrote:
>
>> Hello team
>>
>> Docker SDK images (Python, Java, Go) and Flink job server images are
>> migrated to Apache org[1].
>> I confirmed digests of all the images of the two repos are
>> exactly the same.
>> In addition, I updated readme pages of the new repos. The readma
>> pages match to the ones at github.
>>
>> New images will be deployed to Apache org from v2.20.0. I added
>> notices to the original repos[2] about the changes.
>> Spark job server images will be added from v2.20 as well, thanks for 
>> @Kyle
>> Weaver  to make it happen[3].
>>
>> Thanks,
>> Hannah
>>
>> 1. https://hub.docker.com/search?q=apache%2Fbeam=image
>> 2. https://hub.docker.com/search?q=apachebeam/=image
>> 3. https://github.com/apache/beam/pull/10921
>>
>

Re: Request to be added as contributor

[Ahmet Altay]

Done. Added you as a contributor.

On Mon, Feb 24, 2020 at 2:54 PM Emiliano Capoccia <
emiliano.capoc...@gmail.com> wrote:

> Hi Luke,
>
> that is "ecapoccia"
>
> Thanks, I look forward to hearing from you
>
> Emiliano
>
> On 2020/02/24 17:03:53, Luke Cwik  wrote:
> > Please provide your user id associated with your JIRA account.
> >
> > On Mon, Feb 24, 2020 at 8:38 AM Emiliano Capoccia  >
> > wrote:
> >
> > > Hi,
> > >
> > > This is Emiliano from JP Morgan Chase. I develop applications in Beam
> and
> > > run them on a Spark cluster on Kubernetes. Can someone add me as a
> > > contributor for Beam's Jira issue tracker? I would like to
> create/assign
> > > tickets for my work.
> > >
> > > Thanks,
> > >
> > > Emiliano
> > >
> >
>

Re: [ANNOUNCE] New committer: Chad Dombrova

[Ankur Goenka]

Congratulations Chad!

On Mon, Feb 24, 2020 at 3:34 PM Ahmet Altay  wrote:

> Congratulations!
>
> On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne  wrote:
>
>> Nice one Chad. Your typing efforts are very welcomed.
>>
>> On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang  wrote:
>>
>>> Congratulations, Chad!
>>>
>>> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw 
>>> wrote:
>>>
 Well deserved, Chad. Congratulations!

 On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
 >
 > Congratulations! :-)
 >
 > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova 
 wrote:
 >>
 >> Thanks, folks!  I'm very excited to "retest this" :)
 >>
 >> Especially big thanks to Robert and Udi for all their hard work
 reviewing my PRs.
 >>
 >> -chad
 >>
 >>
 >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette 
 wrote:
 >>>
 >>> Congratulations Chad! Thanks for all your contributions :)
 >>>
 >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver 
 wrote:
 
  Well-deserved, thanks for your dedication to the project Chad. :)
 
  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri 
 wrote:
 >
 > Congrats and welcome, Chad!
 >
 > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada 
 wrote:
 >>
 >> Hi everyone,
 >>
 >> Please join me and the rest of the Beam PMC in welcoming a new
 committer: Chad Dombrova
 >>
 >> Chad has contributed to the project in multiple ways, including
 improvements to the testing infrastructure, and adding type annotations
 throughout the Python SDK, as well as working closely with the community on
 these improvements.
 >>
 >> In consideration of his contributions, the Beam PMC trusts him
 with the responsibilities of a Beam Committer[1].
 >>
 >> Thanks Chad for your contributions!
 >>
 >> -Pablo, on behalf of the Apache Beam PMC.
 >>
 >> [1]
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>

Re: [ANNOUNCE] New committer: Chad Dombrova

[Ahmet Altay]

Congratulations!

On Mon, Feb 24, 2020 at 3:25 PM Sam Bourne  wrote:

> Nice one Chad. Your typing efforts are very welcomed.
>
> On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang  wrote:
>
>> Congratulations, Chad!
>>
>> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw 
>> wrote:
>>
>>> Well deserved, Chad. Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
>>> >
>>> > Congratulations! :-)
>>> >
>>> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova 
>>> wrote:
>>> >>
>>> >> Thanks, folks!  I'm very excited to "retest this" :)
>>> >>
>>> >> Especially big thanks to Robert and Udi for all their hard work
>>> reviewing my PRs.
>>> >>
>>> >> -chad
>>> >>
>>> >>
>>> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette 
>>> wrote:
>>> >>>
>>> >>> Congratulations Chad! Thanks for all your contributions :)
>>> >>>
>>> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver 
>>> wrote:
>>> 
>>>  Well-deserved, thanks for your dedication to the project Chad. :)
>>> 
>>>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri  wrote:
>>> >
>>> > Congrats and welcome, Chad!
>>> >
>>> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada 
>>> wrote:
>>> >>
>>> >> Hi everyone,
>>> >>
>>> >> Please join me and the rest of the Beam PMC in welcoming a new
>>> committer: Chad Dombrova
>>> >>
>>> >> Chad has contributed to the project in multiple ways, including
>>> improvements to the testing infrastructure, and adding type annotations
>>> throughout the Python SDK, as well as working closely with the community on
>>> these improvements.
>>> >>
>>> >> In consideration of his contributions, the Beam PMC trusts him
>>> with the responsibilities of a Beam Committer[1].
>>> >>
>>> >> Thanks Chad for your contributions!
>>> >>
>>> >> -Pablo, on behalf of the Apache Beam PMC.
>>> >>
>>> >> [1]
>>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>>
>>

Re: [ANNOUNCE] New committer: Chad Dombrova

[Sam Bourne]

Nice one Chad. Your typing efforts are very welcomed.

On Tue, Feb 25, 2020 at 10:16 AM Yichi Zhang  wrote:

> Congratulations, Chad!
>
> On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw 
> wrote:
>
>> Well deserved, Chad. Congratulations!
>>
>> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
>> >
>> > Congratulations! :-)
>> >
>> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova 
>> wrote:
>> >>
>> >> Thanks, folks!  I'm very excited to "retest this" :)
>> >>
>> >> Especially big thanks to Robert and Udi for all their hard work
>> reviewing my PRs.
>> >>
>> >> -chad
>> >>
>> >>
>> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette 
>> wrote:
>> >>>
>> >>> Congratulations Chad! Thanks for all your contributions :)
>> >>>
>> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver 
>> wrote:
>> 
>>  Well-deserved, thanks for your dedication to the project Chad. :)
>> 
>>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri  wrote:
>> >
>> > Congrats and welcome, Chad!
>> >
>> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada 
>> wrote:
>> >>
>> >> Hi everyone,
>> >>
>> >> Please join me and the rest of the Beam PMC in welcoming a new
>> committer: Chad Dombrova
>> >>
>> >> Chad has contributed to the project in multiple ways, including
>> improvements to the testing infrastructure, and adding type annotations
>> throughout the Python SDK, as well as working closely with the community on
>> these improvements.
>> >>
>> >> In consideration of his contributions, the Beam PMC trusts him
>> with the responsibilities of a Beam Committer[1].
>> >>
>> >> Thanks Chad for your contributions!
>> >>
>> >> -Pablo, on behalf of the Apache Beam PMC.
>> >>
>> >> [1]
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>

Re: [ANNOUNCE] New committer: Chad Dombrova

[Yichi Zhang]

Congratulations, Chad!

On Mon, Feb 24, 2020 at 3:10 PM Robert Bradshaw  wrote:

> Well deserved, Chad. Congratulations!
>
> On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
> >
> > Congratulations! :-)
> >
> > On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova  wrote:
> >>
> >> Thanks, folks!  I'm very excited to "retest this" :)
> >>
> >> Especially big thanks to Robert and Udi for all their hard work
> reviewing my PRs.
> >>
> >> -chad
> >>
> >>
> >> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette 
> wrote:
> >>>
> >>> Congratulations Chad! Thanks for all your contributions :)
> >>>
> >>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver 
> wrote:
> 
>  Well-deserved, thanks for your dedication to the project Chad. :)
> 
>  On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri  wrote:
> >
> > Congrats and welcome, Chad!
> >
> > On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada 
> wrote:
> >>
> >> Hi everyone,
> >>
> >> Please join me and the rest of the Beam PMC in welcoming a new
> committer: Chad Dombrova
> >>
> >> Chad has contributed to the project in multiple ways, including
> improvements to the testing infrastructure, and adding type annotations
> throughout the Python SDK, as well as working closely with the community on
> these improvements.
> >>
> >> In consideration of his contributions, the Beam PMC trusts him with
> the responsibilities of a Beam Committer[1].
> >>
> >> Thanks Chad for your contributions!
> >>
> >> -Pablo, on behalf of the Apache Beam PMC.
> >>
> >> [1]
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>

Re: [ANNOUNCE] New committer: Chad Dombrova

[Robert Bradshaw]

Well deserved, Chad. Congratulations!

On Mon, Feb 24, 2020 at 2:43 PM Reza Rokni  wrote:
>
> Congratulations! :-)
>
> On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova  wrote:
>>
>> Thanks, folks!  I'm very excited to "retest this" :)
>>
>> Especially big thanks to Robert and Udi for all their hard work reviewing my 
>> PRs.
>>
>> -chad
>>
>>
>> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette  wrote:
>>>
>>> Congratulations Chad! Thanks for all your contributions :)
>>>
>>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver  wrote:

 Well-deserved, thanks for your dedication to the project Chad. :)

 On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri  wrote:
>
> Congrats and welcome, Chad!
>
> On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada  wrote:
>>
>> Hi everyone,
>>
>> Please join me and the rest of the Beam PMC in welcoming a new 
>> committer: Chad Dombrova
>>
>> Chad has contributed to the project in multiple ways, including 
>> improvements to the testing infrastructure, and adding type annotations 
>> throughout the Python SDK, as well as working closely with the community 
>> on these improvements.
>>
>> In consideration of his contributions, the Beam PMC trusts him with the 
>> responsibilities of a Beam Committer[1].
>>
>> Thanks Chad for your contributions!
>>
>> -Pablo, on behalf of the Apache Beam PMC.
>>
>> [1] 
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

Re: Request to be added as contributor

[Emiliano Capoccia]

Hi Luke, sure, that is:

ecapoccia

Thanks, I look forward to hearing from you

On 2020/02/24 17:03:53, Luke Cwik  wrote: 
> Please provide your user id associated with your JIRA account.
> 
> On Mon, Feb 24, 2020 at 8:38 AM Emiliano Capoccia 
> wrote:
> 
> > Hi,
> >
> > This is Emiliano from JP Morgan Chase. I develop applications in Beam and
> > run them on a Spark cluster on Kubernetes. Can someone add me as a
> > contributor for Beam's Jira issue tracker? I would like to create/assign
> > tickets for my work.
> >
> > Thanks,
> >
> > Emiliano
> >
> 

Re: Request to be added as contributor

[Emiliano Capoccia]

Hi Luke,

that is "ecapoccia"

Thanks, I look forward to hearing from you

Emiliano

On 2020/02/24 17:03:53, Luke Cwik  wrote: 
> Please provide your user id associated with your JIRA account.
> 
> On Mon, Feb 24, 2020 at 8:38 AM Emiliano Capoccia 
> wrote:
> 
> > Hi,
> >
> > This is Emiliano from JP Morgan Chase. I develop applications in Beam and
> > run them on a Spark cluster on Kubernetes. Can someone add me as a
> > contributor for Beam's Jira issue tracker? I would like to create/assign
> > tickets for my work.
> >
> > Thanks,
> >
> > Emiliano
> >
> 

Re: [ANNOUNCE] New committer: Chad Dombrova

[Reza Rokni]

Congratulations! :-)

On Tue, Feb 25, 2020 at 6:41 AM Chad Dombrova  wrote:

> Thanks, folks!  I'm very excited to "retest this" :)
>
> Especially big thanks to Robert and Udi for all their hard work reviewing
> my PRs.
>
> -chad
>
>
> On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette  wrote:
>
>> Congratulations Chad! Thanks for all your contributions :)
>>
>> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver  wrote:
>>
>>> Well-deserved, thanks for your dedication to the project Chad. :)
>>>
>>> On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri  wrote:
>>>
 Congrats and welcome, Chad!

 On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada 
 wrote:

> Hi everyone,
>
> Please join me and the rest of the Beam PMC in welcoming a new
> committer: Chad Dombrova
>
> Chad has contributed to the project in multiple ways, including
> improvements to the testing infrastructure, and adding type annotations
> throughout the Python SDK, as well as working closely with the community 
> on
> these improvements.
>
> In consideration of his contributions, the Beam PMC trusts him with
> the responsibilities of a Beam Committer[1].
>
> Thanks Chad for your contributions!
>
> -Pablo, on behalf of the Apache Beam PMC.
>
> [1]
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>
>

Re: [ANNOUNCE] New committer: Chad Dombrova

[Chad Dombrova]

Thanks, folks!  I'm very excited to "retest this" :)

Especially big thanks to Robert and Udi for all their hard work reviewing
my PRs.

-chad


On Mon, Feb 24, 2020 at 1:44 PM Brian Hulette  wrote:

> Congratulations Chad! Thanks for all your contributions :)
>
> On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver  wrote:
>
>> Well-deserved, thanks for your dedication to the project Chad. :)
>>
>> On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri  wrote:
>>
>>> Congrats and welcome, Chad!
>>>
>>> On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada 
>>> wrote:
>>>
 Hi everyone,

 Please join me and the rest of the Beam PMC in welcoming a new
 committer: Chad Dombrova

 Chad has contributed to the project in multiple ways, including
 improvements to the testing infrastructure, and adding type annotations
 throughout the Python SDK, as well as working closely with the community on
 these improvements.

 In consideration of his contributions, the Beam PMC trusts him with the
 responsibilities of a Beam Committer[1].

 Thanks Chad for your contributions!

 -Pablo, on behalf of the Apache Beam PMC.

 [1]
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer


>>>

Re: [VOTE][BIP-1] Beam Schema Options

[Kenneth Knowles]

+1 (binding)

Added a link to the wiki proposal for SQL/MED (Management of External Data)
which treats some of the same ideas.

Kenn

On Fri, Feb 21, 2020 at 3:02 PM Brian Hulette  wrote:

> +1 (non-binding) thanks for all your work on this Alex :)
>
> On Fri, Feb 21, 2020 at 6:50 AM Alex Van Boxel  wrote:
>
>> +1 (non-binding)
>>
>> I assume I can vote on my own proposal :-)
>>
>>  _/
>> _/ Alex Van Boxel
>>
>>
>> On Fri, Feb 21, 2020 at 6:36 AM Jean-Baptiste Onofre 
>> wrote:
>>
>>> +1 (binding)
>>>
>>> Very interesting. It remembers me when we start the discussion about
>>> schema support ;)
>>>
>>> Regards
>>> JB
>>>
>>> Le 20 févr. 2020 à 08:36, Alex Van Boxel  a écrit :
>>>
>>> Hi all,
>>>
>>> let's do a vote on the very first Beam Improvement Proposal. If you have
>>> a -1 or -1 (binding) please add your concern to the open issues section to
>>> the wiki. Thanks.
>>>
>>> This is the proposal:
>>>
>>> https://cwiki.apache.org/confluence/display/BEAM/%5BBIP-1%5D+Beam+Schema+Options
>>>
>>> Can I have your votes.
>>>
>>>  _/
>>> _/ Alex Van Boxel
>>>
>>>
>>>

Fwd: [ANN] Last few days remaining to submit Outreachy intern project proposals for upcoming round

[Kenneth Knowles]

Hi all,

I don't believe this was announced directly on Beam dev. Sorry for that.
The deadline for project proposals is 2020-Feb-25 16:00Z. That is less than
24 hours from now. If you have something in mind, please do submit!

Kenn

-- Forwarded message -
From: Matt Sicker 
Date: Sat, Feb 22, 2020 at 9:21 PM
Subject: [ANN] Last few days remaining to submit Outreachy intern project
proposals for upcoming round
To: 


Hello all,

The next round for Outreachy, a mentoring program for pairing open
source software projects with mentees who are underrepresented
communities in tech in their locality. These projects cover software
development, data science, documentation, design, and other technical
aspects of OSS. For those who have heard of or participated in Google
Summer of Code before, this works fairly similarly.

We have funding available for several projects to participate this
round, so please take a look at the following instructions on how to
get started as a mentor:

https://s.apache.org/OutreachyMay2020

If you have any questions about eligibility, project guidance, etc.,
please reach out to the Outreachy coordinators on the
d...@diversity.apache.org mailing list.

-- 
Matt Sicker
Secretary, Apache Software Foundation
VP Logging Services, ASF

Re: [ANNOUNCE] New committer: Chad Dombrova

[Brian Hulette]

Congratulations Chad! Thanks for all your contributions :)

On Mon, Feb 24, 2020 at 1:43 PM Kyle Weaver  wrote:

> Well-deserved, thanks for your dedication to the project Chad. :)
>
> On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri  wrote:
>
>> Congrats and welcome, Chad!
>>
>> On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada  wrote:
>>
>>> Hi everyone,
>>>
>>> Please join me and the rest of the Beam PMC in welcoming a new
>>> committer: Chad Dombrova
>>>
>>> Chad has contributed to the project in multiple ways, including
>>> improvements to the testing infrastructure, and adding type annotations
>>> throughout the Python SDK, as well as working closely with the community on
>>> these improvements.
>>>
>>> In consideration of his contributions, the Beam PMC trusts him with the
>>> responsibilities of a Beam Committer[1].
>>>
>>> Thanks Chad for your contributions!
>>>
>>> -Pablo, on behalf of the Apache Beam PMC.
>>>
>>> [1]
>>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>>
>>>
>>

Re: [ANNOUNCE] New committer: Chad Dombrova

[Kyle Weaver]

Well-deserved, thanks for your dedication to the project Chad. :)

On Mon, Feb 24, 2020 at 1:34 PM Udi Meiri  wrote:

> Congrats and welcome, Chad!
>
> On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada  wrote:
>
>> Hi everyone,
>>
>> Please join me and the rest of the Beam PMC in welcoming a new committer:
>> Chad Dombrova
>>
>> Chad has contributed to the project in multiple ways, including
>> improvements to the testing infrastructure, and adding type annotations
>> throughout the Python SDK, as well as working closely with the community on
>> these improvements.
>>
>> In consideration of his contributions, the Beam PMC trusts him with the
>> responsibilities of a Beam Committer[1].
>>
>> Thanks Chad for your contributions!
>>
>> -Pablo, on behalf of the Apache Beam PMC.
>>
>> [1]
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>>
>

[VOTE] Upgrade gradle to 6.2

[Alex Van Boxel]

Anyone objections that I upgrade gradle to 6.2. If ok this will be done
over several commits where I will:

   - Upgrade plugins
   - Upgrade gradle to 6.2
   - See where we can use some of the new features


 _/
_/ Alex Van Boxel

Re: [ANNOUNCE] New committer: Chad Dombrova

[Udi Meiri]

Congrats and welcome, Chad!

On Mon, Feb 24, 2020 at 1:21 PM Pablo Estrada  wrote:

> Hi everyone,
>
> Please join me and the rest of the Beam PMC in welcoming a new committer:
> Chad Dombrova
>
> Chad has contributed to the project in multiple ways, including
> improvements to the testing infrastructure, and adding type annotations
> throughout the Python SDK, as well as working closely with the community on
> these improvements.
>
> In consideration of his contributions, the Beam PMC trusts him with the
> responsibilities of a Beam Committer[1].
>
> Thanks Chad for your contributions!
>
> -Pablo, on behalf of the Apache Beam PMC.
>
> [1]
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>
>


smime.p7s
Description: S/MIME Cryptographic Signature

[ANNOUNCE] New committer: Chad Dombrova

[Pablo Estrada]

Hi everyone,

Please join me and the rest of the Beam PMC in welcoming a new committer:
Chad Dombrova

Chad has contributed to the project in multiple ways, including
improvements to the testing infrastructure, and adding type annotations
throughout the Python SDK, as well as working closely with the community on
these improvements.

In consideration of his contributions, the Beam PMC trusts him with the
responsibilities of a Beam Committer[1].

Thanks Chad for your contributions!

-Pablo, on behalf of the Apache Beam PMC.

[1]
https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

Re: [PROPOSAL] Vendored bytebuddy dependency release

[Pablo Estrada]

+1

On Sun, Feb 23, 2020 at 8:31 PM Luke Cwik  wrote:

> +1
>
> On Sun, Feb 23, 2020 at 7:20 PM jincheng sun 
> wrote:
>
>> +1
>>
>> Best,
>> Jincheng
>>
>>
>> Kai Jiang  于2020年2月23日周日 上午2:26写道：
>>
>>> +1
>>>
>>> On Sat, Feb 22, 2020 at 09:50 Alex Van Boxel  wrote:
>>>
 +1

  _/

 _/ Alex Van Boxel


 On Sat, Feb 22, 2020 at 5:45 PM Jean-Baptiste Onofré 
 wrote:

> +1
>
> Regards
> JB
>
> Le sam. 22 f?vr. 2020 ? 15:02, Isma?l Mej?a  a
> ?crit :
>
>> The version of bytebuddy Beam is vendoring (1.9.3) is more than 16
>> months old. I
>> would like to propose that we upgrade it [1] to the most recent
>> version (1.10.8)
>> [2] so we can benefit of the latest improvements for Java 11 and
>> upgraded ASM.
>>
>> If everyone agrees I would like to volunteer as the release manager
>> for this
>> upgrade.
>>
>> [1] https://issues.apache.org/jira/browse/BEAM-9342
>> [2] https://github.com/raphw/byte-buddy/blob/master/release-notes.md
>>
>>

Re: Custom 2.20 failing on Dataflow: what am I doing wrong?

[Luke Cwik]

I look at the gradle task definition to see what additional flags are being
passed whenever trying to rerun/repro an issue. Many of our integration
tests require additional flags/experiments which are unique on a runner per
runner basis.

On Wed, Feb 19, 2020 at 9:15 AM Alex Van Boxel  wrote:

> That's a great idea. I'll do, as some changes needed to be done to the
> gradle files as well
>
> On Wed, Feb 19, 2020, 17:52 Ismaël Mejía  wrote:
>
>> Alex/Gleb can someone of you please add the detailed instructions that
>> worked for you in some section of cwiki.
>> I have the impression that this will benefit us all at some point.
>>
>> Thanks,
>>
>>
>>
>> On Tue, Feb 18, 2020 at 9:46 AM Alex Van Boxel  wrote:
>>
>>> Thanks everyone. This really helped a lot. I used Gleb's tip to make it
>>> work. Successfully validated my Pull Requests against Dataflow!
>>>
>>>  _/
>>> _/ Alex Van Boxel
>>>
>>>
>>> On Mon, Feb 17, 2020 at 11:55 PM Brian Hulette 
>>> wrote:
>>>
 I think if you update past [1] this will go away. We had to build a new
 worker to use with builds on master after [2]. You should be fine running
 from master as long as you aren't using a commit between those two (merge
 commits are 6818560 and bde3031, respectively). Setting the Dataflow worker
 jar would work too.

 [1] https://github.com/apache/beam/pull/10861
 [2] https://github.com/apache/beam/pull/10790

 On Mon, Feb 17, 2020 at 2:14 AM Gleb Kanterov  wrote:

> You need to pass custom Dataflow worker jar. One of the ways of doing
> that is adding it as a dependency, and using following code snippet:
>
> opts.setDataflowWorkerJar(
>   BatchDataflowWorker.class
>   .getProtectionDomain()
>   .getCodeSource()
>   .getLocation()
>   .toString());
> opts.setWorkerHarnessContainerImage("");
>
> Coming with the disclaimer that it isn't for production :)
>
> On Mon, Feb 17, 2020 at 8:34 AM Alex Van Boxel 
> wrote:
>
>> Yes, running it manually with the normal parameters as I do for
>> production Dataflow. I'm probably a bit ignorant on that, and I
>> probably need to provide my own worker.
>>
>> Thanks for the hint... I'll dive into that.
>>
>>  _/
>> _/ Alex Van Boxel
>>
>>
>> On Mon, Feb 17, 2020 at 8:16 AM Reuven Lax  wrote:
>>
>>> Are you running things manually? This probably means you are using
>>> an out-of-date Dataflow worker. I believe that all tests on Jenkins will
>>> build the Dataflow worker from head to prevent exactly this problem.
>>>
>>> On Sun, Feb 16, 2020 at 11:10 PM Alex Van Boxel 
>>> wrote:
>>>
 Digging further in the traces, it seems like a result of changes to
 the model:

 Caused by: java.lang.ClassNotFoundException:
 org.apache.beam.model.pipeline.v1.StandardWindowFns$SessionsPayload$Enum

 I see changes by Lukasz Cwik. Will this be a problem for the
 release?

  _/
 _/ Alex Van Boxel


 On Sun, Feb 16, 2020 at 12:11 PM Alex Van Boxel 
 wrote:

> Hey,
>
> I'm testing my own PR's against Dataflow, something I've done in
> the past with success seem to fail now. I get this error:
>
> java.lang.NoClassDefFoundError: Could not initialize class
> org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.construction.WindowingStrategyTranslation
>
>1.
>
>
> Am I doing something wrong?
>
>  _/
> _/ Alex Van Boxel
>

Re: Docker images are migrated to Apache org.

[Robert Burke]

It was only merged in an hour ago. That explains why I still saw it as
broken before lunch. Thanks again!

On Mon, Feb 24, 2020, 12:42 PM Robert Burke  wrote:

> NVM. I had stale pages for some reason. Hannah fixed them already. :D
>
> On Mon, Feb 24, 2020, 12:39 PM Robert Burke  wrote:
>
>> Looks like the change broke the go post commits. I've filed BEAM-9374 for
>> it. I think I know how to fix it though.
>>
>> On Mon, Feb 24, 2020, 12:18 PM Kyle Weaver  wrote:
>>
>>> I wonder if searching for "apache beam" (instead of "apache/beam") will
>>> ever work on Docker hub once the new images start getting more downloads.
>>> Currently no relevant results are included. Unfortunately this might not be
>>> something we can control, as it seems Docker hub's search is not as
>>> sophisticated as I would like.
>>>
>>> Anyway, this is still a great improvement. Thanks Hannah for making this
>>> happen.
>>>
>>> On Fri, Feb 21, 2020 at 11:58 AM Ahmet Altay  wrote:
>>>
 Thank you, Hannah! This is great.

 On Fri, Feb 21, 2020 at 11:24 AM Hannah Jiang 
 wrote:

> Hello team
>
> Docker SDK images (Python, Java, Go) and Flink job server images are
> migrated to Apache org[1].
> I confirmed digests of all the images of the two repos are exactly the
> same.
> In addition, I updated readme pages of the new repos. The readma pages
> match to the ones at github.
>
> New images will be deployed to Apache org from v2.20.0. I added
> notices to the original repos[2] about the changes.
> Spark job server images will be added from v2.20 as well, thanks for @Kyle
> Weaver  to make it happen[3].
>
> Thanks,
> Hannah
>
> 1. https://hub.docker.com/search?q=apache%2Fbeam=image
> 2. https://hub.docker.com/search?q=apachebeam/=image
> 3. https://github.com/apache/beam/pull/10921
>

Re: Docker images are migrated to Apache org.

[Robert Burke]

NVM. I had stale pages for some reason. Hannah fixed them already. :D

On Mon, Feb 24, 2020, 12:39 PM Robert Burke  wrote:

> Looks like the change broke the go post commits. I've filed BEAM-9374 for
> it. I think I know how to fix it though.
>
> On Mon, Feb 24, 2020, 12:18 PM Kyle Weaver  wrote:
>
>> I wonder if searching for "apache beam" (instead of "apache/beam") will
>> ever work on Docker hub once the new images start getting more downloads.
>> Currently no relevant results are included. Unfortunately this might not be
>> something we can control, as it seems Docker hub's search is not as
>> sophisticated as I would like.
>>
>> Anyway, this is still a great improvement. Thanks Hannah for making this
>> happen.
>>
>> On Fri, Feb 21, 2020 at 11:58 AM Ahmet Altay  wrote:
>>
>>> Thank you, Hannah! This is great.
>>>
>>> On Fri, Feb 21, 2020 at 11:24 AM Hannah Jiang 
>>> wrote:
>>>
 Hello team

 Docker SDK images (Python, Java, Go) and Flink job server images are
 migrated to Apache org[1].
 I confirmed digests of all the images of the two repos are exactly the
 same.
 In addition, I updated readme pages of the new repos. The readma pages
 match to the ones at github.

 New images will be deployed to Apache org from v2.20.0. I added notices
 to the original repos[2] about the changes.
 Spark job server images will be added from v2.20 as well, thanks for @Kyle
 Weaver  to make it happen[3].

 Thanks,
 Hannah

 1. https://hub.docker.com/search?q=apache%2Fbeam=image
 2. https://hub.docker.com/search?q=apachebeam/=image
 3. https://github.com/apache/beam/pull/10921

>>>

Re: Docker images are migrated to Apache org.

[Kyle Weaver]

Is it still failing after https://github.com/apache/beam/pull/10936?

On Mon, Feb 24, 2020 at 12:39 PM Robert Burke  wrote:

> Looks like the change broke the go post commits. I've filed BEAM-9374 for
> it. I think I know how to fix it though.
>
> On Mon, Feb 24, 2020, 12:18 PM Kyle Weaver  wrote:
>
>> I wonder if searching for "apache beam" (instead of "apache/beam") will
>> ever work on Docker hub once the new images start getting more downloads.
>> Currently no relevant results are included. Unfortunately this might not be
>> something we can control, as it seems Docker hub's search is not as
>> sophisticated as I would like.
>>
>> Anyway, this is still a great improvement. Thanks Hannah for making this
>> happen.
>>
>> On Fri, Feb 21, 2020 at 11:58 AM Ahmet Altay  wrote:
>>
>>> Thank you, Hannah! This is great.
>>>
>>> On Fri, Feb 21, 2020 at 11:24 AM Hannah Jiang 
>>> wrote:
>>>
 Hello team

 Docker SDK images (Python, Java, Go) and Flink job server images are
 migrated to Apache org[1].
 I confirmed digests of all the images of the two repos are exactly the
 same.
 In addition, I updated readme pages of the new repos. The readma pages
 match to the ones at github.

 New images will be deployed to Apache org from v2.20.0. I added notices
 to the original repos[2] about the changes.
 Spark job server images will be added from v2.20 as well, thanks for @Kyle
 Weaver  to make it happen[3].

 Thanks,
 Hannah

 1. https://hub.docker.com/search?q=apache%2Fbeam=image
 2. https://hub.docker.com/search?q=apachebeam/=image
 3. https://github.com/apache/beam/pull/10921

>>>

Re: Docker images are migrated to Apache org.

[Robert Burke]

Looks like the change broke the go post commits. I've filed BEAM-9374 for
it. I think I know how to fix it though.

On Mon, Feb 24, 2020, 12:18 PM Kyle Weaver  wrote:

> I wonder if searching for "apache beam" (instead of "apache/beam") will
> ever work on Docker hub once the new images start getting more downloads.
> Currently no relevant results are included. Unfortunately this might not be
> something we can control, as it seems Docker hub's search is not as
> sophisticated as I would like.
>
> Anyway, this is still a great improvement. Thanks Hannah for making this
> happen.
>
> On Fri, Feb 21, 2020 at 11:58 AM Ahmet Altay  wrote:
>
>> Thank you, Hannah! This is great.
>>
>> On Fri, Feb 21, 2020 at 11:24 AM Hannah Jiang 
>> wrote:
>>
>>> Hello team
>>>
>>> Docker SDK images (Python, Java, Go) and Flink job server images are
>>> migrated to Apache org[1].
>>> I confirmed digests of all the images of the two repos are exactly the
>>> same.
>>> In addition, I updated readme pages of the new repos. The readma pages
>>> match to the ones at github.
>>>
>>> New images will be deployed to Apache org from v2.20.0. I added notices
>>> to the original repos[2] about the changes.
>>> Spark job server images will be added from v2.20 as well, thanks for @Kyle
>>> Weaver  to make it happen[3].
>>>
>>> Thanks,
>>> Hannah
>>>
>>> 1. https://hub.docker.com/search?q=apache%2Fbeam=image
>>> 2. https://hub.docker.com/search?q=apachebeam/=image
>>> 3. https://github.com/apache/beam/pull/10921
>>>
>>

Re: Docker images are migrated to Apache org.

[Kyle Weaver]

I wonder if searching for "apache beam" (instead of "apache/beam") will
ever work on Docker hub once the new images start getting more downloads.
Currently no relevant results are included. Unfortunately this might not be
something we can control, as it seems Docker hub's search is not as
sophisticated as I would like.

Anyway, this is still a great improvement. Thanks Hannah for making this
happen.

On Fri, Feb 21, 2020 at 11:58 AM Ahmet Altay  wrote:

> Thank you, Hannah! This is great.
>
> On Fri, Feb 21, 2020 at 11:24 AM Hannah Jiang 
> wrote:
>
>> Hello team
>>
>> Docker SDK images (Python, Java, Go) and Flink job server images are
>> migrated to Apache org[1].
>> I confirmed digests of all the images of the two repos are exactly the
>> same.
>> In addition, I updated readme pages of the new repos. The readma pages
>> match to the ones at github.
>>
>> New images will be deployed to Apache org from v2.20.0. I added notices
>> to the original repos[2] about the changes.
>> Spark job server images will be added from v2.20 as well, thanks for @Kyle
>> Weaver  to make it happen[3].
>>
>> Thanks,
>> Hannah
>>
>> 1. https://hub.docker.com/search?q=apache%2Fbeam=image
>> 2. https://hub.docker.com/search?q=apachebeam/=image
>> 3. https://github.com/apache/beam/pull/10921
>>
>

Re: [ANNOUNCE] New committer: Jincheng Sun

[Kyle Weaver]

Thanks Jincheng for all your work on Beam and Flink integration.

On Mon, Feb 24, 2020 at 11:02 AM Yichi Zhang  wrote:

> Congrats, Jincheng!
>
> On Mon, Feb 24, 2020 at 9:45 AM Ahmet Altay  wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 6:48 AM Thomas Weise  wrote:
>>
>>> Congratulations!
>>>
>>>
>>> On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía  wrote:
>>>
 Congrats Jincheng!

 On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov  wrote:

> Congratulations!
>
> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng  wrote:
>
>> Congratulations Jincheng, well deserved!
>>
>> Best,
>> Hequn
>>
>> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:
>>
>>> Congrats!
>>>
>>> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský 
>>> wrote:
>>>
 Congrats Jincheng!

   Jan

 On 2/24/20 11:55 AM, Maximilian Michels wrote:
 > Hi everyone,
 >
 > Please join me and the rest of the Beam PMC in welcoming a new
 > committer: Jincheng Sun 
 >
 > Jincheng has worked on generalizing parts of Beam for Flink's
 Python
 > API. He has also picked up other issues, like fixing
 documentation,
 > implementing missing features, or cleaning up code [1].
 >
 > In consideration of his contributions, the Beam PMC trusts him
 with
 > the responsibilities of a Beam committer [2].
 >
 > Thank you for your contributions Jincheng!
 >
 > -Max, on behalf of the Apache Beam PMC
 >
 > [1]
 >
 https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
 > [2]
 >
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>

Re: [ANNOUNCE] New committer: Jincheng Sun

[Yichi Zhang]

Congrats, Jincheng!

On Mon, Feb 24, 2020 at 9:45 AM Ahmet Altay  wrote:

> Congratulations!
>
> On Mon, Feb 24, 2020 at 6:48 AM Thomas Weise  wrote:
>
>> Congratulations!
>>
>>
>> On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía  wrote:
>>
>>> Congrats Jincheng!
>>>
>>> On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov  wrote:
>>>
 Congratulations!

 On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng  wrote:

> Congratulations Jincheng, well deserved!
>
> Best,
> Hequn
>
> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:
>
>> Congrats!
>>
>> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský  wrote:
>>
>>> Congrats Jincheng!
>>>
>>>   Jan
>>>
>>> On 2/24/20 11:55 AM, Maximilian Michels wrote:
>>> > Hi everyone,
>>> >
>>> > Please join me and the rest of the Beam PMC in welcoming a new
>>> > committer: Jincheng Sun 
>>> >
>>> > Jincheng has worked on generalizing parts of Beam for Flink's
>>> Python
>>> > API. He has also picked up other issues, like fixing
>>> documentation,
>>> > implementing missing features, or cleaning up code [1].
>>> >
>>> > In consideration of his contributions, the Beam PMC trusts him
>>> with
>>> > the responsibilities of a Beam committer [2].
>>> >
>>> > Thank you for your contributions Jincheng!
>>> >
>>> > -Max, on behalf of the Apache Beam PMC
>>> >
>>> > [1]
>>> >
>>> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
>>> > [2]
>>> >
>>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>>
>>

Re: [ANNOUNCE] New committer: Jincheng Sun

[Ahmet Altay]

Congratulations!

On Mon, Feb 24, 2020 at 6:48 AM Thomas Weise  wrote:

> Congratulations!
>
>
> On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía  wrote:
>
>> Congrats Jincheng!
>>
>> On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov  wrote:
>>
>>> Congratulations!
>>>
>>> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng  wrote:
>>>
 Congratulations Jincheng, well deserved!

 Best,
 Hequn

 On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:

> Congrats!
>
> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský  wrote:
>
>> Congrats Jincheng!
>>
>>   Jan
>>
>> On 2/24/20 11:55 AM, Maximilian Michels wrote:
>> > Hi everyone,
>> >
>> > Please join me and the rest of the Beam PMC in welcoming a new
>> > committer: Jincheng Sun 
>> >
>> > Jincheng has worked on generalizing parts of Beam for Flink's
>> Python
>> > API. He has also picked up other issues, like fixing documentation,
>> > implementing missing features, or cleaning up code [1].
>> >
>> > In consideration of his contributions, the Beam PMC trusts him with
>> > the responsibilities of a Beam committer [2].
>> >
>> > Thank you for your contributions Jincheng!
>> >
>> > -Max, on behalf of the Apache Beam PMC
>> >
>> > [1]
>> >
>> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
>> > [2]
>> >
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>

Re: KafkaIO to read from regex topic

[Luke Cwik]

I have been working on getting unbounded SDFs working within Beam over
portability so if you are interested in writing an SDF KafkaIO
implementation, I would be interested.

On Mon, Feb 24, 2020 at 7:34 AM Alexey Romanenko 
wrote:

> Hi Maulik,
>
> For the moment, KafkaIO doesn’t support reading from dynamic topics
> (add/remove topics/partitions) without pipeline restart. In a few words,
> KafkaIO uses unbounded source API which requires fixed number of splits and
> it doesn’t deal well with watermarks for empty splits. One of the potential
> solution could be to rewrite KafkaIO Read using SDF but afaik nobody worked
> on this.
>
> There is an open Jira about that [1] with more details if you are
> interested.
>
> [1] https://issues.apache.org/jira/browse/BEAM-5786
>
> On 24 Feb 2020, at 11:27, Maulik Soneji  wrote:
>
> Hello All,
>
> I am currently using Beam version 2.19.0 to read data from Kafka using
> KafkaIO.
> Kafka version: 0.11.0.0
> My use case is to read all topics from the kafka cluster.
> Below is the code that reads data from kafka using KafkaIO.
>
> KafkaIO.read()
> .withBootstrapServers(brokers)
> .withTopic(topic)
> .withKeyDeserializer(ByteArrayDeserializer.class)
> .withValueDeserializer(ByteArrayDeserializer.class)
> .updateConsumerProperties(props)
> .commitOffsetsInFinalize();
>
> If I provide topic as a regex like topic-.*,the request fails with:
>
> Exception in thread "main"
> org.apache.beam.sdk.Pipeline$PipelineExecutionException:
> org.apache.kafka.common.errors.InvalidTopicException: Topic topic-.*  is 
> invalid
>
> By looking at the code, I saw that there is a call to fetch partition
> information for the topics at KafkaUnboundSource here
> .
> Because we are fetching partitions for only the topic mentioned in the
> builder, it considers the regex as a topic and tries to fetch partition
> information for it even when it is not a topic but a regex. My requirement
> is that I should read from all topics in kafka cluster and if there are new
> tpoics which are added, they should be considered as well dynamically
> without any process restart. Can someone please share details about how I
> can read data from multiple topics by using a regex. Thanks in advance.
> Thanks and regards, Maulik
>
>
>

Re: Request to be added as contributor

[Luke Cwik]

Please provide your user id associated with your JIRA account.

On Mon, Feb 24, 2020 at 8:38 AM Emiliano Capoccia 
wrote:

> Hi,
>
> This is Emiliano from JP Morgan Chase. I develop applications in Beam and
> run them on a Spark cluster on Kubernetes. Can someone add me as a
> contributor for Beam's Jira issue tracker? I would like to create/assign
> tickets for my work.
>
> Thanks,
>
> Emiliano
>

Request to be added as contributor

[Emiliano Capoccia]

Hi,

This is Emiliano from JP Morgan Chase. I develop applications in Beam and run 
them on a Spark cluster on Kubernetes. Can someone add me as a contributor for 
Beam's Jira issue tracker? I would like to create/assign tickets for my work.

Thanks,

Emiliano

Re: KafkaIO to read from regex topic

[Alexey Romanenko]

Hi Maulik,

For the moment, KafkaIO doesn’t support reading from dynamic topics (add/remove 
topics/partitions) without pipeline restart. In a few words, KafkaIO uses 
unbounded source API which requires fixed number of splits and it doesn’t deal 
well with watermarks for empty splits. One of the potential solution could be 
to rewrite KafkaIO Read using SDF but afaik nobody worked on this. 

There is an open Jira about that [1] with more details if you are interested. 

[1] https://issues.apache.org/jira/browse/BEAM-5786 


> On 24 Feb 2020, at 11:27, Maulik Soneji  wrote:
> 
> Hello All,
> 
> I am currently using Beam version 2.19.0 to read data from Kafka using 
> KafkaIO. 
> Kafka version: 0.11.0.0
> My use case is to read all topics from the kafka cluster.
> Below is the code that reads data from kafka using KafkaIO.
> KafkaIO.read()
> .withBootstrapServers(brokers)
> .withTopic(topic)
> .withKeyDeserializer(ByteArrayDeserializer.class)
> .withValueDeserializer(ByteArrayDeserializer.class)
> .updateConsumerProperties(props)
> .commitOffsetsInFinalize();
> If I provide topic as a regex like topic-.*,the request fails with:
> Exception in thread "main" 
> org.apache.beam.sdk.Pipeline$PipelineExecutionException: 
> org.apache.kafka.common.errors.InvalidTopicException: Topic topic-.*  is 
> invalid
> By looking at the code, I saw that there is a call to fetch partition 
> information for the topics
> at KafkaUnboundSource here 
> .
> Because we are fetching partitions for only the topic mentioned in the 
> builder, it considers the regex as a topic 
> and tries to fetch partition information for it even when it is not a topic 
> but a regex.
> 
> My requirement is that I should read from all topics in kafka cluster and if 
> there are new tpoics which are added, 
> they should be considered as well dynamically without any process restart.
> 
> Can someone please share details about how I can read data from multiple 
> topics by using a regex.
> Thanks in advance.
> 
> Thanks and regards,
> Maulik

Re: [ANNOUNCE] New committer: Jincheng Sun

[Thomas Weise]

Congratulations!


On Mon, Feb 24, 2020 at 6:45 AM Ismaël Mejía  wrote:

> Congrats Jincheng!
>
> On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov  wrote:
>
>> Congratulations!
>>
>> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng  wrote:
>>
>>> Congratulations Jincheng, well deserved!
>>>
>>> Best,
>>> Hequn
>>>
>>> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:
>>>
 Congrats!

 On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský  wrote:

> Congrats Jincheng!
>
>   Jan
>
> On 2/24/20 11:55 AM, Maximilian Michels wrote:
> > Hi everyone,
> >
> > Please join me and the rest of the Beam PMC in welcoming a new
> > committer: Jincheng Sun 
> >
> > Jincheng has worked on generalizing parts of Beam for Flink's Python
> > API. He has also picked up other issues, like fixing documentation,
> > implementing missing features, or cleaning up code [1].
> >
> > In consideration of his contributions, the Beam PMC trusts him with
> > the responsibilities of a Beam committer [2].
> >
> > Thank you for your contributions Jincheng!
> >
> > -Max, on behalf of the Apache Beam PMC
> >
> > [1]
> >
> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
> > [2]
> >
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>

Re: [ANNOUNCE] New committer: Jincheng Sun

[Ismaël Mejía]

Congrats Jincheng!

On Mon, Feb 24, 2020 at 1:39 PM Gleb Kanterov  wrote:

> Congratulations!
>
> On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng  wrote:
>
>> Congratulations Jincheng, well deserved!
>>
>> Best,
>> Hequn
>>
>> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:
>>
>>> Congrats!
>>>
>>> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský  wrote:
>>>
 Congrats Jincheng!

   Jan

 On 2/24/20 11:55 AM, Maximilian Michels wrote:
 > Hi everyone,
 >
 > Please join me and the rest of the Beam PMC in welcoming a new
 > committer: Jincheng Sun 
 >
 > Jincheng has worked on generalizing parts of Beam for Flink's Python
 > API. He has also picked up other issues, like fixing documentation,
 > implementing missing features, or cleaning up code [1].
 >
 > In consideration of his contributions, the Beam PMC trusts him with
 > the responsibilities of a Beam committer [2].
 >
 > Thank you for your contributions Jincheng!
 >
 > -Max, on behalf of the Apache Beam PMC
 >
 > [1]
 >
 https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
 > [2]
 >
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>

Re: [ANNOUNCE] New committer: Jincheng Sun

[Gleb Kanterov]

Congratulations!

On Mon, Feb 24, 2020 at 1:18 PM Hequn Cheng  wrote:

> Congratulations Jincheng, well deserved!
>
> Best,
> Hequn
>
> On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:
>
>> Congrats!
>>
>> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský  wrote:
>>
>>> Congrats Jincheng!
>>>
>>>   Jan
>>>
>>> On 2/24/20 11:55 AM, Maximilian Michels wrote:
>>> > Hi everyone,
>>> >
>>> > Please join me and the rest of the Beam PMC in welcoming a new
>>> > committer: Jincheng Sun 
>>> >
>>> > Jincheng has worked on generalizing parts of Beam for Flink's Python
>>> > API. He has also picked up other issues, like fixing documentation,
>>> > implementing missing features, or cleaning up code [1].
>>> >
>>> > In consideration of his contributions, the Beam PMC trusts him with
>>> > the responsibilities of a Beam committer [2].
>>> >
>>> > Thank you for your contributions Jincheng!
>>> >
>>> > -Max, on behalf of the Apache Beam PMC
>>> >
>>> > [1]
>>> >
>>> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
>>> > [2]
>>> >
>>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>>
>>

Beam Dependency Check Report (2020-02-24)

[Apache Jenkins Server]

High Priority Dependency Updates Of Beam Python SDK:


  Dependency Name
  Current Version
  Latest Version
  Release Date Of the Current Used Version
  Release Date Of The Latest Release
  JIRA Issue
  
cachetools
3.1.1
4.0.0
2019-12-23
2019-12-23BEAM-9017
google-cloud-bigquery
1.17.1
1.24.0
2019-09-23
2020-02-10BEAM-5537
google-cloud-datastore
1.7.4
1.10.0
2019-05-27
2019-10-21BEAM-8443
httplib2
0.12.0
0.17.0
2018-12-10
2020-01-27BEAM-9018
mock
2.0.0
3.0.5
2019-05-20
2019-05-20BEAM-7369
oauth2client
3.0.0
4.1.3
2018-12-10
2018-12-10BEAM-6089
PyHamcrest
1.10.1
2.0.0
2020-01-20
2020-01-20BEAM-9155
pytest
4.6.9
5.3.5
2020-01-06
2020-02-03BEAM-8606
Sphinx
1.8.5
2.4.3
2019-05-20
2020-02-24BEAM-7370
tenacity
5.1.5
6.0.0
2019-11-11
2019-11-11BEAM-8607
High Priority Dependency Updates Of Beam Java SDK:


  Dependency Name
  Current Version
  Latest Version
  Release Date Of the Current Used Version
  Release Date Of The Latest Release
  JIRA Issue
  
com.alibaba:fastjson
1.2.49
1.2.62
2018-08-04
2019-10-07BEAM-8632
com.datastax.cassandra:cassandra-driver-core
3.8.0
4.0.0
2019-10-29
2019-03-18BEAM-8674
com.esotericsoftware:kryo
4.0.2
5.0.0-RC4
2018-03-20
2019-04-14BEAM-5809
com.esotericsoftware.kryo:kryo
2.21
2.24.0
2013-02-27
2014-05-04BEAM-5574
com.github.ben-manes.versions:com.github.ben-manes.versions.gradle.plugin
0.20.0
0.28.0
2019-02-11
2020-02-24BEAM-6645
com.github.luben:zstd-jni
1.3.8-3
1.4.4-7
2019-01-29
2020-01-24BEAM-9194
com.github.spotbugs:spotbugs
3.1.12
4.0.0
2019-03-01
2020-02-16BEAM-7792
com.github.spotbugs:spotbugs-annotations
3.1.12
4.0.0
2019-03-01
2020-02-16BEAM-6951
com.google.api.grpc:grpc-google-common-protos
1.12.0
1.17.0
2018-06-29
2019-10-04BEAM-8633
com.google.api.grpc:proto-google-cloud-bigquerystorage-v1beta1
0.85.1
0.90.0
2020-01-08
2020-02-20BEAM-8678
com.google.api.grpc:proto-google-common-protos
1.12.0
1.17.0
2018-06-29
2019-10-04BEAM-6899
com.google.apis:google-api-services-bigquery
v2-rev20190917-1.30.3
v2-rev20191211-1.30.8
2019-10-09
2020-02-08BEAM-8684
com.google.apis:google-api-services-clouddebugger
v2-rev20191003-1.30.3
v2-rev20200102-1.30.8
2019-10-19
2020-02-09BEAM-8750
com.google.apis:google-api-services-cloudresourcemanager
v1-rev20191206-1.30.3
v2-rev20200203-1.30.8
2019-12-17
2020-02-11BEAM-8751
com.google.apis:google-api-services-dataflow
v1b3-rev20190927-1.30.3
v1beta3-rev12-1.20.0
2019-10-11
2015-04-29BEAM-8752
com.google.apis:google-api-services-pubsub
v1-rev2019-1.30.3
v1-rev20200121-1.30.8
2019-11-26
2020-02-12BEAM-8753
com.google.apis:google-api-services-storage
v1-rev20191011-1.30.3
v1-rev20191127-1.30.8
2019-10-30
2020-02-12BEAM-8754
com.google.cloud:google-cloud-bigquery
1.103.0
1.107.0
2020-01-08
2020-02-20BEAM-8687
com.google.cloud:google-cloud-bigquerystorage
0.120.1-beta
0.125.0-beta
2020-01-08
2020-02-20BEAM-8755
com.google.guava:guava
26.0-jre
28.2-jre
2018-08-01
2019-12-27BEAM-5559
com.google.guava:guava-testlib
25.1-jre
28.2-jre
2018-05-23
2019-12-27BEAM-8760
com.google.protobuf.nano:protobuf-javanano
3.0.0-alpha-5
3.2.0rc2
2016-01-06
2017-01-19BEAM-9098
com.hazelcast:hazelcast
3.12
4.0
2019-04-09
2020-02-04BEAM-8636
com.ning:compress-lzf
1.0.3
1.0.4
2014-08-16
2017-03-14BEAM-9100
com.pholser:junit-quickcheck-core
0.8
0.9.1
2018-03-08
2020-01-21BEAM-8699
io.netty:netty-handler
4.1.30.Final
5.0.0.Alpha2
2018-09-27
2015-03-03BEAM-8703
javax.servlet:javax.servlet-api
3.1.0
4.0.1
2013-04-25
2018-04-20BEAM-5750

Re: [ANNOUNCE] New committer: Jincheng Sun

[Hequn Cheng]

Congratulations Jincheng, well deserved!

Best,
Hequn

On Mon, Feb 24, 2020 at 7:21 PM Reza Rokni  wrote:

> Congrats!
>
> On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský  wrote:
>
>> Congrats Jincheng!
>>
>>   Jan
>>
>> On 2/24/20 11:55 AM, Maximilian Michels wrote:
>> > Hi everyone,
>> >
>> > Please join me and the rest of the Beam PMC in welcoming a new
>> > committer: Jincheng Sun 
>> >
>> > Jincheng has worked on generalizing parts of Beam for Flink's Python
>> > API. He has also picked up other issues, like fixing documentation,
>> > implementing missing features, or cleaning up code [1].
>> >
>> > In consideration of his contributions, the Beam PMC trusts him with
>> > the responsibilities of a Beam committer [2].
>> >
>> > Thank you for your contributions Jincheng!
>> >
>> > -Max, on behalf of the Apache Beam PMC
>> >
>> > [1]
>> >
>> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
>> > [2]
>> >
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>

Re: [ANNOUNCE] New committer: Jincheng Sun

[Reza Rokni]

Congrats!

On Mon, Feb 24, 2020 at 7:15 PM Jan Lukavský  wrote:

> Congrats Jincheng!
>
>   Jan
>
> On 2/24/20 11:55 AM, Maximilian Michels wrote:
> > Hi everyone,
> >
> > Please join me and the rest of the Beam PMC in welcoming a new
> > committer: Jincheng Sun 
> >
> > Jincheng has worked on generalizing parts of Beam for Flink's Python
> > API. He has also picked up other issues, like fixing documentation,
> > implementing missing features, or cleaning up code [1].
> >
> > In consideration of his contributions, the Beam PMC trusts him with
> > the responsibilities of a Beam committer [2].
> >
> > Thank you for your contributions Jincheng!
> >
> > -Max, on behalf of the Apache Beam PMC
> >
> > [1]
> >
> https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
> > [2]
> >
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>

Re: [ANNOUNCE] New committer: Jincheng Sun

[Jan Lukavský]

Congrats Jincheng!

 Jan

On 2/24/20 11:55 AM, Maximilian Michels wrote:

Hi everyone,

Please join me and the rest of the Beam PMC in welcoming a new 
committer: Jincheng Sun 


Jincheng has worked on generalizing parts of Beam for Flink's Python 
API. He has also picked up other issues, like fixing documentation, 
implementing missing features, or cleaning up code [1].


In consideration of his contributions, the Beam PMC trusts him with 
the responsibilities of a Beam committer [2].


Thank you for your contributions Jincheng!

-Max, on behalf of the Apache Beam PMC

[1] 
https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
[2] 
https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

[ANNOUNCE] New committer: Jincheng Sun

[Maximilian Michels]

Hi everyone,

Please join me and the rest of the Beam PMC in welcoming a new 
committer: Jincheng Sun 


Jincheng has worked on generalizing parts of Beam for Flink's Python 
API. He has also picked up other issues, like fixing documentation, 
implementing missing features, or cleaning up code [1].


In consideration of his contributions, the Beam PMC trusts him with the 
responsibilities of a Beam committer [2].


Thank you for your contributions Jincheng!

-Max, on behalf of the Apache Beam PMC

[1] 
https://jira.apache.org/jira/browse/BEAM-9299?jql=project%20%3D%20BEAM%20AND%20assignee%20in%20(sunjincheng121)
[2] 
https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

KafkaIO to read from regex topic

[Maulik Soneji]

Hello All,

I am currently using Beam version 2.19.0 to read data from Kafka using
KafkaIO.
Kafka version: 0.11.0.0
My use case is to read all topics from the kafka cluster.
Below is the code that reads data from kafka using KafkaIO.

KafkaIO.read()
.withBootstrapServers(brokers)
.withTopic(topic)
.withKeyDeserializer(ByteArrayDeserializer.class)
.withValueDeserializer(ByteArrayDeserializer.class)
.updateConsumerProperties(props)
.commitOffsetsInFinalize();

If I provide topic as a regex like topic-.*,the request fails with:

Exception in thread "main"
org.apache.beam.sdk.Pipeline$PipelineExecutionException:
org.apache.kafka.common.errors.InvalidTopicException: Topic topic-.*  is invalid

By looking at the code, I saw that there is a call to fetch partition
information for the topics at KafkaUnboundSource here
.
Because we are fetching partitions for only the topic mentioned in the
builder, it considers the regex as a topic and tries to fetch partition
information for it even when it is not a topic but a regex. My requirement
is that I should read from all topics in kafka cluster and if there are new
tpoics which are added, they should be considered as well dynamically
without any process restart. Can someone please share details about how I
can read data from multiple topics by using a regex. Thanks in advance.
Thanks and regards, Maulik