Re: [ANNOUNCE] New committer: Daniel Oliveira

[Tanay Tummalapalli]

Congratulations!

On Mon, Nov 25, 2019 at 11:12 PM Mark Liu  wrote:

> Congratulations, Daniel!
>
> On Mon, Nov 25, 2019 at 9:31 AM Ahmet Altay  wrote:
>
>> Congratulations, Daniel!
>>
>> On Sat, Nov 23, 2019 at 3:47 AM jincheng sun 
>> wrote:
>>
>>>
>>> Congrats, Daniel!
>>> Best,
>>> Jincheng
>>>
>>> Alexey Romanenko  于2019年11月22日周五 下午5:47写道：
>>>
 Congratulations, Daniel!

 On 22 Nov 2019, at 09:18, Jan Lukavský  wrote:

 Congrats Daniel!
 On 11/21/19 10:11 AM, Gleb Kanterov wrote:

 Congratulations!

 On Thu, Nov 21, 2019 at 6:24 AM Thomas Weise  wrote:

> Congratulations!
>
>
> On Wed, Nov 20, 2019, 7:56 PM Chamikara Jayalath 
> wrote:
>
>> Congrats!!
>>
>> On Wed, Nov 20, 2019 at 5:21 PM Daniel Oliveira <
>> danolive...@google.com> wrote:
>>
>>> Thank you everyone! I won't let you down. o7
>>>
>>> On Wed, Nov 20, 2019 at 2:12 PM Ruoyun Huang 
>>> wrote:
>>>
 Congrats Daniel!

 On Wed, Nov 20, 2019 at 1:58 PM Robert Burke 
 wrote:

> Congrats Daniel! Much deserved.
>
> On Wed, Nov 20, 2019, 12:49 PM Udi Meiri  wrote:
>
>> Congrats Daniel!
>>
>> On Wed, Nov 20, 2019 at 12:42 PM Kyle Weaver 
>> wrote:
>>
>>> Congrats Dan! Keep up the good work :)
>>>
>>> On Wed, Nov 20, 2019 at 12:41 PM Cyrus Maden 
>>> wrote:
>>>
 Congratulations! This is great news.

 On Wed, Nov 20, 2019 at 3:24 PM Rui Wang 
 wrote:

> Congrats!
>
>
> -Rui
>
> On Wed, Nov 20, 2019 at 11:48 AM Valentyn Tymofieiev <
> valen...@google.com> wrote:
>
>> Congrats, Daniel!
>>
>> On Wed, Nov 20, 2019 at 11:47 AM Kenneth Knowles <
>> k...@apache.org> wrote:
>>
>>> Hi all,
>>>
>>> Please join me and the rest of the Beam PMC in welcoming a
>>> new committer: Daniel Oliveira
>>>
>>> Daniel introduced himself to dev@ over two years ago and
>>> has contributed in many ways since then. Daniel has contributed 
>>> to general
>>> project health, the portability framework, and all three 
>>> languages: Java,
>>> Python SDK, and Go. I would like to particularly highlight how 
>>> he deleted
>>> 12k lines of dead reference runner code [1].
>>>
>>> In consideration of Daniel's contributions, the Beam PMC
>>> trusts him with the responsibilities of a Beam committer [2].
>>>
>>> Thank you, Daniel, for your contributions and looking
>>> forward to many more!
>>>
>>> Kenn, on behalf of the Apache Beam PMC
>>>
>>> [1] https://github.com/apache/beam/pull/8380
>>> [2]
>>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>>
>>

 --
 
 Ruoyun  Huang

Re: [ANNOUNCE] New committer: Brian Hulette

[Tanay Tummalapalli]

Congratulations!

On Wed, Nov 20, 2019 at 6:15 AM Aizhamal Nurmamat kyzy 
wrote:

> Congratulations, Brian!
>
> On Mon, Nov 18, 2019 at 10:29 AM Łukasz Gajowy 
> wrote:
>
>> Congratulations! :)
>>
>> pt., 15 lis 2019 o 15:52 Chamikara Jayalath 
>> napisał(a):
>>
>>> Congrats, Brian!
>>>
>>> On Fri, Nov 15, 2019 at 11:39 AM Yichi Zhang  wrote:
>>>
 Congratulations, Brian!

 On Fri, Nov 15, 2019 at 11:12 AM Alan Myrvold 
 wrote:

> Congratulations, Brian!
>
> On Fri, Nov 15, 2019 at 11:11 AM Yifan Zou 
> wrote:
>
>> Well deserved. Congratulations!
>>
>> On Fri, Nov 15, 2019 at 11:06 AM Kirill Kozlov <
>> kirillkoz...@google.com> wrote:
>>
>>> Congratulations, Brian!
>>>
>>> On Fri, Nov 15, 2019 at 10:56 AM Udi Meiri  wrote:
>>>
 Congrats Brian!


 On Fri, Nov 15, 2019 at 10:47 AM Ruoyun Huang 
 wrote:

> Congrats Brian!
>
> On Fri, Nov 15, 2019 at 10:41 AM Robin Qiu 
> wrote:
>
>> Congrats, Brian!
>>
>> On Fri, Nov 15, 2019 at 10:02 AM Daniel Oliveira <
>> danolive...@google.com> wrote:
>>
>>> Congratulations Brian! It's well deserved.
>>>
>>> On Fri, Nov 15, 2019, 9:37 AM Alexey Romanenko <
>>> aromanenko@gmail.com> wrote:
>>>
 Congratulations, Brian!

 On 15 Nov 2019, at 18:27, Rui Wang  wrote:

 Congrats!


 -Rui

 On Fri, Nov 15, 2019 at 8:16 AM Thomas Weise 
 wrote:

> Congratulations!
>
>
> On Fri, Nov 15, 2019 at 6:34 AM Connell O'Callaghan <
> conne...@google.com> wrote:
>
>> Well done Brian!!!
>>
>> Kenn thank you for sharing
>>
>> On Fri, Nov 15, 2019 at 6:31 AM Cyrus Maden <
>> cma...@google.com> wrote:
>>
>>> Congrats Brian!
>>>
>>> On Fri, Nov 15, 2019 at 5:25 AM Ismaël Mejía <
>>> ieme...@gmail.com> wrote:
>>>
 Congratulations Brian!
 Happy to see this happening and eager to see more of your
 work!

 On Fri, Nov 15, 2019 at 11:02 AM Ankur Goenka <
 goe...@google.com> wrote:
 >
 > Congrats Brian!
 >
 > On Fri, Nov 15, 2019, 2:42 PM Jan Lukavský <
 je...@seznam.cz> wrote:
 >>
 >> Congrats Brian!
 >>
 >> On 11/15/19 9:58 AM, Reza Rokni wrote:
 >>
 >> Great news!
 >>
 >> On Fri, 15 Nov 2019 at 15:09, Gleb Kanterov <
 g...@spotify.com> wrote:
 >>>
 >>> Congratulations!
 >>>
 >>> On Fri, Nov 15, 2019 at 5:44 AM Valentyn Tymofieiev <
 valen...@google.com> wrote:
 
  Congratulations, Brian!
 
  On Thu, Nov 14, 2019 at 6:25 PM jincheng sun <
 sunjincheng...@gmail.com> wrote:
 >
 > Congratulation Brian!
 >
 > Best,
 > Jincheng
 >
 > Kyle Weaver  于2019年11月15日周五
 上午7:19写道：
 >>
 >> Thanks for your contributions and congrats Brian!
 >>
 >> On Thu, Nov 14, 2019 at 3:14 PM Kenneth Knowles <
 k...@apache.org> wrote:
 >>>
 >>> Hi all,
 >>>
 >>> Please join me and the rest of the Beam PMC in
 welcoming a new committer: Brian Hulette
 >>>
 >>> Brian introduced himself to dev@ earlier this year
 and has been contributing since then. His contributions to 
 Beam include
 explorations of integration with Arrow, standardizing coders, 
 portability
 for schemas, and presentations at Beam events.
 >>>
 >>> In consideration of Brian's contributions, the Beam
 PMC trusts him with the responsibilities of a Beam committer 
 [1].
 >>>
 >>> Thank you, Brian, for your contributions and
 looking forward to many more!
 >>>
 >>> Kenn, on behalf of the Apache Beam PMC
 >>>
 >>> [1]

Re: [VOTE] Sign a pledge to discontinue support of Python 2 in 2020.

[Tanay Tummalapalli]

+1

On Tue, Oct 1, 2019 at 8:19 AM Suneel Marthi  wrote:

> +1
>
> On Mon, Sep 30, 2019 at 10:33 PM Manu Zhang 
> wrote:
>
>> +1
>>
>> On Tue, Oct 1, 2019 at 9:44 AM Austin Bennett <
>> whatwouldausti...@gmail.com> wrote:
>>
>>> +1
>>>
>>> On Mon, Sep 30, 2019 at 5:22 PM Valentyn Tymofieiev 
>>> wrote:
>>>
 Hi everyone,

 Please vote whether to sign a pledge on behalf of Apache Beam to sunset
 Beam Python 2 offering (in new releases) in 2020 on
 http://python3stament.org as follows:

 [ ] +1: Sign a pledge to discontinue support of Python 2 in Beam in
 2020.
 [ ] -1: Do not sign a pledge to discontinue support of Python 2 in Beam
 in 2020.

 The motivation and details for this vote were discussed in [1, 2].
 Please follow up in [2] if you have any questions.

 This is a procedural vote [3] that will follow the majority approval
 rules and will be open for at least 72 hours.

 Thanks,
 Valentyn

 [1]
 https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E
 [2]
 https://lists.apache.org/thread.html/456631fe1a696c537ef8ebfee42cd3ea8121bf7c639c52da5f7032e7@%3Cdev.beam.apache.org%3E
 [3] https://www.apache.org/foundation/voting.html

Re: [ANNOUNCE] New committer: Alan Myrvold

[Tanay Tummalapalli]

Congratulations, Alan!


On Mon, Sep 30, 2019 at 1:03 PM Gleb Kanterov  wrote:

> Congratulations!
>
> On Sat, Sep 28, 2019 at 12:07 AM Valentyn Tymofieiev 
> wrote:
>
>> Congratulations, Alan. Well deserved.
>>
>> On Fri, Sep 27, 2019 at 2:09 PM Chamikara Jayalath 
>> wrote:
>>
>>> Congrats Alan!!
>>>
>>> On Fri, Sep 27, 2019 at 1:49 PM Jan Lukavský  wrote:
>>>
 Congrats Alan!
 On 9/27/19 10:22 PM, Mark Liu wrote:

 Congratulations Alan!!!

 On Fri, Sep 27, 2019 at 12:55 PM Ning Kang  wrote:

> Congrats Alan!
>
> On Fri, Sep 27, 2019 at 12:02 PM Ankur Goenka 
> wrote:
>
>> Congratulations Alan!
>>
>> On Fri, Sep 27, 2019 at 11:17 AM Yichi Zhang 
>> wrote:
>>
>>> Congrats, Alan!
>>>
>>> On Fri, Sep 27, 2019 at 10:26 AM Robin Qiu 
>>> wrote:
>>>
 Congrats, Alan!

 On Fri, Sep 27, 2019 at 10:15 AM Hannah Jiang <
 hannahji...@google.com> wrote:

> Congrats Alan!
>
> On Fri, Sep 27, 2019 at 9:57 AM Ruoyun Huang 
> wrote:
>
>> Congratulations, Alan!
>>
>>
>> On Fri, Sep 27, 2019 at 9:55 AM Rui Wang 
>> wrote:
>>
>>> Congrats!
>>>
>>> -Rui
>>>
>>> On Fri, Sep 27, 2019 at 9:54 AM Pablo Estrada <
>>> pabl...@google.com> wrote:
>>>
 Yooh! : D

 On Fri, Sep 27, 2019 at 9:53 AM Yifan Zou 
 wrote:

> Congratulations, Alan!
>
> On Fri, Sep 27, 2019 at 9:18 AM Ahmet Altay 
> wrote:
>
>> Hi,
>>
>> Please join me and the rest of the Beam PMC in welcoming a
>> new
>> committer: Alan Myrvold
>>
>> Alan has been a long time Beam contributor. His contributions
>> made Beam more productive and friendlier [1] for all 
>> contributors with
>> significant improvements to Beam release process, automation, and
>> infrastructure.
>>
>> In consideration of Alan's contributions, the Beam PMC trusts
>> him
>> with the responsibilities of a Beam committer [2].
>>
>> Thank you, Alan, for your contributions and looking forward
>> to many more!
>>
>> Ahmet, on behalf of the Apache Beam PMC
>>
>> [1]
>> https://beam-summit-na-2019.firebaseapp.com/schedule/2019-09-11?sessionId=1126
>> [2] https://beam.apache.org/contribute/become-a-committer
>> /#an-apache-beam-committer
>>
>
>>
>> --
>> 
>> Ruoyun  Huang
>>
>>
>
> --
> Cheers,
> Gleb
>

Re: [discuss] Auto-close issues with a PR associated?

[Tanay Tummalapalli]

Hey Pablo,

I think we should add the workflow to the contributor's guide[1] since most
new contributors start from there.

Best,
- Tanay

[1] https://beam.apache.org/contribute/

On Wed, Sep 4, 2019 at 11:03 PM Pablo Estrada  wrote:

> Hello all,
> this has been discussed before, and I believe we concluded that we did not
> want to auto-close JIRA issues referenced by a PR.
> I wanted us to revisit this decision, because I don't think everyone has
> adopted the correct workflow:
>
> Workflow A:
> 1. Send a PR to fix BEAM-X
> 2. Go to JIRA, and mark BEAM-X as resolved.
>
> Many of us do (1), but not (2) as much. The main argument was to avoid
> messing with this workflow:
>
> Workflow B:
> 1. Send a PR related to BEAM-X
> 2. Send a PR related to BEAM-X
> 
> N-1. Send a PR that fully fixes BEAM-X
> N. Go to JIRA, and mark BEAM-X as resolved.
>
> But, perhaps we should optimize for the very common case of Workflow A,
> and let community members manually manage the less common case of Workflow
> B.
>
> What do others think?
> -P.
>

Re: Beam at Google Summer of Code 2019

[Tanay Tummalapalli]

Hi everyone,

I've completed Google Summer of Code '19[1].
I had fun working on Beam for the past 3 months and learning about Beam
internals.

Thank you Pablo for everything! None of it would have been possible without
you.
I'd also like to thank the Beam community for the code reviews and being
supportive and encouraging.

I'm moving to Bangalore this month. I'll be back to contributing to Beam
next month.

Thank You
 - Tanay

[1] https://gist.github.com/ttanay/80f84b7b852e0867d5a00d3b345e1dad

On Fri, May 24, 2019 at 12:47 AM Tanay Tummalapalli 
wrote:

> Hi everyone,
>
> I made a Kanban board[1] on Github, on my fork of apache/beam to keep
> track of progress for GSoC '19.
>
> Regards,
> Tanay Tummalapalli
>
> [1] https://github.com/ttanay/beam/projects/1
>
> On Tue, May 7, 2019 at 6:39 PM Tanay Tummalapalli 
> wrote:
>
>> Thank You!
>>
>> I'm really excited to work on Beam!
>> I'd like to thank Pablo, Chamikara Jayalath and Tim Robertson for helping
>> out with my proposal[1].
>>
>> Looking forward to working with everyone and learning a great deal.
>>
>> Regards
>> Tanay Tummalapalli
>> LinkedIn <https://www.linkedin.com/in/ttanay/> | Github
>> <https://github.com/ttanay>
>>
>> [1]
>> https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=sharing
>>
>> On Tue, May 7, 2019 at 12:04 AM Pablo Estrada  wrote:
>>
>>> Hello all,
>>> it is my pleasure to share with everyone that Tanay Tummalapalli has
>>> been accepted as a GSoC student with Beam, to implement support for File
>>> Loads into BigQuery for streaming pipelines[1].
>>>
>>> Tanay wrote a very strong proposal, and showed understanding of the
>>> tricky streaming considerations that will play out in this project.
>>>
>>> I speak on behalf of everyone welcoming you Tanay, and we'll be happy to
>>> see your contributions to Beam. : )
>>> Best
>>> -P.
>>>
>>> [1]
>>> https://summerofcode.withgoogle.com/projects/?sp-search=Tanay#4999837794172928
>>>
>>

Re: [ANNOUNCE] New committer: Valentyn Tymofieiev

[Tanay Tummalapalli]

Congratulations Valentyn!

On Wed, Aug 28, 2019 at 7:16 AM Ruoyun Huang  wrote:

> Congratulations Valentyn!
>
> On Tue, Aug 27, 2019 at 6:16 PM Daniel Oliveira 
> wrote:
>
>> Congratulations Valentyn!
>>
>> On Tue, Aug 27, 2019, 11:31 AM Boyuan Zhang  wrote:
>>
>>> Congratulations!
>>>
>>> On Tue, Aug 27, 2019 at 10:44 AM Udi Meiri  wrote:
>>>
 Congrats!

 On Tue, Aug 27, 2019 at 9:50 AM Yichi Zhang  wrote:

> Congrats Valentyn!
>
> On Tue, Aug 27, 2019 at 7:55 AM Valentyn Tymofieiev <
> valen...@google.com> wrote:
>
>> Thank you everyone!
>>
>> On Tue, Aug 27, 2019 at 2:57 AM Alexey Romanenko <
>> aromanenko@gmail.com> wrote:
>>
>>> Congrats, well deserved!
>>>
>>> On 27 Aug 2019, at 11:25, Jan Lukavský  wrote:
>>>
>>> Congrats Valentyn!
>>> On 8/26/19 11:43 PM, Rui Wang wrote:
>>>
>>> Congratulations!
>>>
>>>
>>> -Rui
>>>
>>> On Mon, Aug 26, 2019 at 2:36 PM Hannah Jiang 
>>> wrote:
>>>
 Congratulations Valentyn, well deserved!

 On Mon, Aug 26, 2019 at 2:34 PM Chamikara Jayalath <
 chamik...@google.com> wrote:

> Congrats Valentyn!
>
> On Mon, Aug 26, 2019 at 2:32 PM Pablo Estrada 
> wrote:
>
>> Thanks Valentyn!
>>
>> On Mon, Aug 26, 2019 at 2:29 PM Robin Qiu 
>> wrote:
>>
>>> Thank you Valentyn! Congratulations!
>>>
>>> On Mon, Aug 26, 2019 at 2:28 PM Robert Bradshaw <
>>> rober...@google.com> wrote:
>>>
 Hi,

 Please join me and the rest of the Beam PMC in welcoming a new
 committer: Valentyn Tymofieiev

 Valentyn has made numerous contributions to Beam over the last
 several
 years (including 100+ pull requests), most recently pushing
 through
 the effort to make Beam compatible with Python 3. He is also an
 active
 participant in design discussions on the list, participates in
 release
 candidate validation, and proactively helps keep our tests
 green.

 In consideration of Valentyn's contributions, the Beam PMC
 trusts him
 with the responsibilities of a Beam committer [1].

 Thank you, Valentyn, for your contributions and looking forward
 to many more!

 Robert, on behalf of the Apache Beam PMC

 [1]
 https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer

>>>
>>>
>
> --
> 
> Ruoyun  Huang
>
>

Re: Late data handling in Python SDK

[Tanay Tummalapalli]

Hi Lukasz,

I'm currently working on BEAM-7742[1].
I'll work on BEAM-3759 afterwards.

Regards,
- Tanay
[1] https://issues.apache.org/jira/browse/BEAM-7742

On Fri, Aug 9, 2019 at 9:25 PM Lukasz Cwik  wrote:

> +dev 
>
> Related JIRA's I found are BEAM-3759 and BEAM-7825. This has been a
> priority thing as the community has been trying to get streaming Python
> execution working on multiple Beam runners.
>
> On Wed, Aug 7, 2019 at 2:31 AM Sam Stephens 
> wrote:
>
>> Hi all,
>>
>> I’ve been reading into, and experimenting with, the Python SDK recently.
>> I see that late data handling is not supported currently, but I can’t find
>> where the progress of this feature is tracked: either in the portability
>> support matrix or in Beam’s Jira. Can anyone help me there?
>>
>> If the answer is that there isn’t a single place to see this, my
>> question is then: what is the current status of this? Is there some
>> underlying difficult problem that has to be solved first or is it just a
>> priority thing?
>>
>> Thanks
>> Sam
>>
>

Hardening BigQuery file loads in Python SDK

[Tanay Tummalapalli]

Hi everyone,

Load jobs into BigQuery are subject to various quotas and limitations.
In the Python SDK, the BigQuery sink that uses file loads does not work
well with these quotas and limitations.
Improvements are needed in the following areas:

   1. Handle per load job limitations on the total size.
   2. Decide when to use temp_tables to load data atomically, at pipeline
   execution time.

I have documented proposed changes in a design doc[1] as well as a draft
pull request[2].

*TL;DR:*
Partition the written files based on:

   1. Total size of files
   2. Number of files

If multiple load jobs are needed to write to a single destination, data
will be loaded to temp tables first. Once all data is loaded to these temp
tables, data in the temp table will be copied to the destination table to
ensure data is loaded into BigQuery atomically.

Would love to get feedback on the proposed changes.

Regards,
- Tanay

[1] https://s.apache.org/beam-bqfl-hardening
[2] https://github.com/apache/beam/pull/9242

Re: [ANNOUNCE] New committer: Kyle Weaver

[Tanay Tummalapalli]

Congratulations!

On Wed, Aug 7, 2019 at 11:27 PM Robin Qiu  wrote:

> Congratulations, Kyle!
>
> On Wed, Aug 7, 2019 at 5:04 AM Valentyn Tymofieiev 
> wrote:
>
>> Congrats, Kyle!
>>
>> On Wed, Aug 7, 2019 at 1:01 PM Ismaël Mejía  wrote:
>>
>>> Congrats Kyle, well deserved :clap: !
>>>
>>> On Wed, Aug 7, 2019, 11:22 AM Gleb Kanterov  wrote:
>>>
 Congratulations!

 On Wed, Aug 7, 2019 at 7:01 AM Connell O'Callaghan 
 wrote:

> Well done congratulations Kyle!!!
>
> On Tue, Aug 6, 2019 at 21:58 Thomas Weise  wrote:
>
>> Congrats!
>>
>> On Tue, Aug 6, 2019, 7:24 PM Reza Rokni  wrote:
>>
>>> Congratz!
>>>
>>> On Wed, 7 Aug 2019 at 06:40, Chamikara Jayalath <
>>> chamik...@google.com> wrote:
>>>
 Congrats!!

 On Tue, Aug 6, 2019 at 3:33 PM Udi Meiri  wrote:

> Congrats Kyle!
>
> On Tue, Aug 6, 2019 at 2:00 PM Melissa Pashniak <
> meliss...@google.com> wrote:
>
>> Congratulations Kyle!
>>
>> On Tue, Aug 6, 2019 at 1:36 PM Yichi Zhang 
>> wrote:
>>
>>> Congrats Kyle!
>>>
>>> On Tue, Aug 6, 2019 at 1:29 PM Aizhamal Nurmamat kyzy <
>>> aizha...@google.com> wrote:
>>>
 Thank you, Kyle! And congratulations :)

 On Tue, Aug 6, 2019 at 10:09 AM Hannah Jiang <
 hannahji...@google.com> wrote:

> Congrats Kyle!
>
> On Tue, Aug 6, 2019 at 9:52 AM David Morávek <
> david.mora...@gmail.com> wrote:
>
>> Congratulations Kyle!!
>>
>> Sent from my iPhone
>>
>> On 6 Aug 2019, at 18:47, Anton Kedin 
>> wrote:
>>
>> Congrats!
>>
>> On Tue, Aug 6, 2019, 9:37 AM Ankur Goenka 
>> wrote:
>>
>>> Congratulations Kyle!
>>>
>>> On Tue, Aug 6, 2019 at 9:35 AM Ahmet Altay 
>>> wrote:
>>>
 Hi,

 Please join me and the rest of the Beam PMC in welcoming a
 new committer: Kyle Weaver.

 Kyle has been contributing to Beam for a while now. And in
 that time period Kyle got the portable spark runner feature 
 complete for
 batch processing. [1]

 In consideration of Kyle's contributions, the Beam PMC
 trusts him with the responsibilities of a Beam committer
  [2].

 Thank you, Kyle, for your contributions and looking forward
 to many more!

 Ahmet, on behalf of the Apache Beam PMC

 [1]
 https://lists.apache.org/thread.html/c43678fc24c9a1dc9f48c51c51950aedcb9bc0fd3b633df16c3d595a@%3Cuser.beam.apache.org%3E
 [2] https://beam.apache.org/contribute/become-a-committer
 /#an-apache-beam-committer

>>>
>>>
>>> --
>>>
>>> This email may be confidential and privileged. If you received this
>>> communication by mistake, please don't forward it to anyone else, please
>>> erase all copies and attachments, and please let me know that it has 
>>> gone
>>> to the wrong person.
>>>
>>> The above terms reflect a potential business arrangement, are
>>> provided solely as a basis for further discussion, and are not intended 
>>> to
>>> be and do not constitute a legally binding obligation. No legally 
>>> binding
>>> obligations will be created, implied, or inferred until an agreement in
>>> final form is executed in writing by all parties involved.
>>>
>>

 --
 Cheers,
 Gleb

>>>

Re: [ANNOUNCE] New committer: Rui Wang

[Tanay Tummalapalli]

Congratulations!

On Wed, Aug 7, 2019 at 11:28 PM Robin Qiu  wrote:

> Congratulations, Rui!
>
> On Wed, Aug 7, 2019 at 5:03 AM Valentyn Tymofieiev 
> wrote:
>
>> Congrats, Rui!
>>
>> On Wed, Aug 7, 2019 at 1:00 PM Ismaël Mejía  wrote:
>>
>>> Congrats Rui!
>>>
>>> On Wed, Aug 7, 2019, 11:37 AM Gleb Kanterov  wrote:
>>>
 Congratulations Rui! Well done!

 On Wed, Aug 7, 2019 at 7:01 AM Connell O'Callaghan 
 wrote:

> Well done Rui!!!
>
> On Tue, Aug 6, 2019 at 15:41 Chamikara Jayalath 
> wrote:
>
>> Congrats Rui.
>>
>> On Tue, Aug 6, 2019 at 2:00 PM Melissa Pashniak 
>> wrote:
>>
>>> Congrats Rui!
>>>
>>> On Tue, Aug 6, 2019 at 1:37 PM Yichi Zhang 
>>> wrote:
>>>
 Congrats Rui!

 On Tue, Aug 6, 2019 at 1:29 PM Aizhamal Nurmamat kyzy <
 aizha...@google.com> wrote:

> Congratulations, Rui! Thank you for your contributions to Beam!
>
> On Tue, Aug 6, 2019 at 10:35 AM sridhar inuog <
> sridharin...@gmail.com> wrote:
>
>> Congratulations Rui!
>>
>> On Tue, Aug 6, 2019 at 12:09 PM Hannah Jiang <
>> hannahji...@google.com> wrote:
>>
>>> Congrats Rui!
>>>
>>> On Tue, Aug 6, 2019 at 9:50 AM Yifan Zou 
>>> wrote:
>>>
 Congratulations Rui!

 On Tue, Aug 6, 2019 at 9:47 AM Anton Kedin 
 wrote:

> Congrats!
>
> On Tue, Aug 6, 2019, 9:36 AM Ankur Goenka 
> wrote:
>
>> Congratulations Rui!
>> Well deserved 
>>
>> On Tue, Aug 6, 2019 at 9:35 AM Ahmet Altay 
>> wrote:
>>
>>> Hi,
>>>
>>> Please join me and the rest of the Beam PMC in welcoming a
>>> new committer: Rui Wang.
>>>
>>> Rui has been an active contributor since May 2018. Rui has
>>> been very active in Beam SQL [1] and continues to help out on 
>>> user@
>>> and StackOverflow. Rui is one of the top answerers for 
>>> apache-beam tag [2].
>>>
>>> In consideration of Rui's contributions, the Beam PMC trusts
>>> him with the responsibilities of a Beam committer [3].
>>>
>>> Thank you, Rui, for your contributions and looking forward
>>> to many more!
>>>
>>> Ahmet, on behalf of the Apache Beam PMC
>>>
>>> [1]
>>> https://github.com/apache/beam/pulls?q=is%3Apr+author%3Aamaliujia
>>> [2] https://stackoverflow.com/tags/apache-beam/topusers
>>> [3] https://beam.apache.org/contribute/become-a-committer
>>> /#an-apache-beam-committer
>>>
>>

 --
 Cheers,
 Gleb

>>>

Re: python precommits failing at head

[Tanay Tummalapalli]

Thanks @Robert! Python PreCommit is green now!

On Mon, Jul 22, 2019 at 5:57 PM Robert Bradshaw  wrote:

> This was due to a bad release artifact push. This has now been fixed
> upstream.
>
> On Mon, Jul 22, 2019 at 11:00 AM Robert Bradshaw 
> wrote:
> >
> > Looks like https://sourceforge.net/p/docutils/bugs/365/
> >
> > On Sun, Jul 21, 2019 at 11:56 PM Tanay Tummalapalli 
> wrote:
> >>
> >> Hi everyone,
> >>
> >> The Python PreCommit from the Jenkins job "beam_PreCommit_Python_Cron"
> is failing[1]. The task :sdks:python:docs is failing with this traceback:
> >>
> >> Traceback (most recent call last):
> >> File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/bin/sphinx-apidoc",
> line 8, in 
> >> from sphinx.apidoc import main
> >> File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/sphinx/apidoc.py",
> line 27, in 
> >> from sphinx.quickstart import EXTENSIONS
> >> File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/sphinx/quickstart.py",
> line 35, in 
> >> from docutils.utils import column_width
> >> File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/docutils/utils/__init__.py",
> line 21, in 
> >> import docutils.io
> >> File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/docutils/io.py",
> line 348
> >> (self.destination.mode, mode)), file=self._stderr)
> >> ^
> >> SyntaxError: invalid syntax
> >>
> >> It seems to be due to a bug in docutils which is a dependency of sphinx.
> >> Unfortunately, I don't have much context about this. I'd love to help
> with fixing this.
> >>
> >> There is another failure in test_wordcount_it of task
> :sdks:python:test-suites:dataflow:preCommitIT, but it seems to be a flaky
> test.
> >>
> >> [1]
> https://scans.gradle.com/s/ygctjiqxcz6qa/console-log?task=:sdks:python:docs
> >>
> >> On Mon, Jul 15, 2019 at 11:16 PM Yifan Zou  wrote:
> >>>
> >>> We saw similar timeouts of the python precommit and it usually break
> the Jenkins build workers. I've run the precommit manually several times.
> It sometimes stuck at :sdks:python:docs and consumes 80G+ memory. Our build
> VMs eventually ran out of memory (104G memory available in total) then
> disconnected. Not sure what happened during that job.
> >>>
> >>>
> >>>
> >>> On Sat, Jul 13, 2019 at 10:27 PM Tanay Tummalapalli <
> ttanay...@gmail.com> wrote:
> >>>>
> >>>> Yes. It passed on the second attempt.
> >>>>
> >>>> But, I'm yet to figure out why it hangs for ~1.5 hours.
> >>>>
> >>>> On Sun, Jul 14, 2019 at 10:36 AM Rakesh Kumar 
> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>> Even I am running into the same issue. Though my test passed but
> somehow the task didn't terminate, eventually the task was aborted.  I have
> already tried a couple of times to retrigger tye python precommit but it
> failed every time.
> >>>>>
> >>>>> @Tanay did it pass it for you?
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Fri, Jul 12, 2019 at 2:22 PM Tanay Tummalapalli <
> ttanay...@gmail.com> wrote:
> >>>>>>
> >>>>>> Thank You Valentyn!
> >>>>>>
> >>>>>> I'll retest it.
> >>>>>> Hopefully, it's a transient issue.
> >>>>>>
> >>>>>> Regards,
> >>>>>> - Tanay Tummalapalli
> >>>>>>
> >>>>>> On Sat, Jul 13, 2019 at 2:39 AM Valentyn Tymofieiev <
> valen...@google.com> wrote:
> >>>>>>>
> >>>>>>> No, we did not reduce the timeout recently. Looking at console
> logs, nothing happened for an hour or so,
> >>>>>>>
> >>>>>

Re: python precommits failing at head

[Tanay Tummalapalli]

Hi everyone,

The Python PreCommit from the Jenkins job "beam_PreCommit_Python_Cron" is
failing[1]. The task :sdks:python:docs is failing with this traceback:

Traceback (most recent call last):
File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/bin/sphinx-apidoc",
line 8, in 
from sphinx.apidoc import main
File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/sphinx/apidoc.py",
line 27, in 
from sphinx.quickstart import EXTENSIONS
File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/sphinx/quickstart.py",
line 35, in 
from docutils.utils import column_width
File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/docutils/utils/__init__.py",
line 21, in 
import docutils.io
File
"/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Cron/src/sdks/python/build/srcs/sdks/python/target/.tox-docs/docs/local/lib/python2.7/site-packages/docutils/io.py",
line 348
(self.destination.mode, mode)), file=self._stderr)
^
SyntaxError: invalid syntax

It seems to be due to a bug in docutils which is a dependency of sphinx.
Unfortunately, I don't have much context about this. I'd love to help with
fixing this.

There is another failure in test_wordcount_it of task
:sdks:python:test-suites:dataflow:preCommitIT,
but it seems to be a flaky test.

[1]
https://scans.gradle.com/s/ygctjiqxcz6qa/console-log?task=:sdks:python:docs

On Mon, Jul 15, 2019 at 11:16 PM Yifan Zou  wrote:

> We saw similar timeouts of the python precommit and it usually break the
> Jenkins build workers. I've run the precommit manually several times. It
> sometimes stuck at :sdks:python:docs and consumes 80G+ memory. Our build
> VMs eventually ran out of memory (104G memory available in total) then
> disconnected. Not sure what happened during that job.
>
>
>
> On Sat, Jul 13, 2019 at 10:27 PM Tanay Tummalapalli 
> wrote:
>
>> Yes. It passed on the second attempt.
>>
>> But, I'm yet to figure out why it hangs for ~1.5 hours.
>>
>> On Sun, Jul 14, 2019 at 10:36 AM Rakesh Kumar 
>> wrote:
>>
>>>
>>>
>>> Even I am running into the same issue. Though my test passed but
>>> <https://builds.apache.org/job/beam_PreCommit_Python_Phrase/656/console>somehow
>>> the task didn't terminate, eventually the task was aborted.  I have already
>>> tried a couple of times to retrigger tye python precommit but it failed
>>> every time.
>>>
>>> @Tanay did it pass it for you?
>>>
>>>
>>>
>>>
>>>
>>> On Fri, Jul 12, 2019 at 2:22 PM Tanay Tummalapalli 
>>> wrote:
>>>
>>>> Thank You Valentyn!
>>>>
>>>> I'll retest it.
>>>> Hopefully, it's a transient issue.
>>>>
>>>> Regards,
>>>> - Tanay Tummalapalli
>>>>
>>>> On Sat, Jul 13, 2019 at 2:39 AM Valentyn Tymofieiev <
>>>> valen...@google.com> wrote:
>>>>
>>>>> No, we did not reduce the timeout recently. Looking at console logs,
>>>>> nothing happened for an hour or so,
>>>>>
>>>>> *06:57:50 py27-cython: commands succeeded 06:57:50 congratulations :)
>>>>> 06:57:50 *
>>>>>
>>>>> *06:57:50* >* Task :sdks:python:preCommitPy2**08:22:33* Build timed out 
>>>>> (after 120 minutes). Marking the build as aborted.
>>>>>
>>>>>
>>>>> However, we can also see in the logs that py36-cython suite never
>>>>> started, not sure way. I assume gradle waited for this suite to finish.
>>>>> Try "retest this please", hopefully this is a transient gradle issue.
>>>>> I did not observe it before.
>>>>>
>>>>> On Fri, Jul 12, 2019 at 1:22 PM Tanay Tummalapalli <
>>>>> ttanay...@gmail.com> wrote:
>>>>>
>>>>>> Hi Udi,
>>>>>>
>>>>>> I rebased another PR[1] onto the fix mentioned above. The lint error
>>>>>> is fixed, but, the "beam_PreCommit_Python_Commit" Jenkins job is failing
>>>>>> because of a timeout at 120 minutes[2].
>>>>>> The log says "Build timed out (after 120 minutes). Marking the build
>>>>>> as aborted."
>>>>>> Another PR's Python PreCommit job aborted with the same error[3].
>>>>>>
>>>>>> I found this issue - "[BEAM-3040] Python precommit timed out after
>>>>>> 150 minutes"[4].
>>>>>> Was the timeout reduced recently?
>>>>>>
>>>>>> Regards,
>>>>>> - Tanay Tummalapalli
>>>>>>
>>>>>> [1] https://github.com/apache/beam/pull/8871
>>>>>> [2]
>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Commit/7412/consoleFull
>>>>>>
>>>>>> [3] https://github.com/apache/beam/pull/9050
>>>>>> [4] https://issues.apache.org/jira/browse/BEAM-3040
>>>>>>
>>>>>> On Fri, Jul 12, 2019 at 5:42 AM Udi Meiri  wrote:
>>>>>>
>>>>>>> This is due to
>>>>>>> https://github.com/apache/beam/pull/8969
>>>>>>> and
>>>>>>> https://github.com/apache/beam/pull/8934
>>>>>>> being merged today.
>>>>>>>
>>>>>>> Fix is here: https://github.com/apache/beam/pull/9044
>>>>>>>
>>>>>>

Re: An update on Python postcommit tests

[Tanay Tummalapalli]

Oh. That wouldn't be ideal.

Thank You for sharing!

Regards,
-Tanay

On Sun, Jul 21, 2019 at 2:20 AM Valentyn Tymofieiev 
wrote:

> The GitHub pull request builder plugin, which we currently use, allow only
> one trigger phrase per Jenkins job [1].
>
> We could have another job that runs all 4 postcommits, but it would appear
> on a PR as a different job, separate from per-minor-version jobs.
>
> [1]
> https://github.com/jenkinsci/ghprb-plugin/blob/f4eb4bc1ee3fd53284d3889c69558c9f36d5379d/src/main/java/org/jenkinsci/plugins/ghprb/GhprbTrigger.java#L124
>
> On Sat, Jul 20, 2019 at 12:52 PM Tanay Tummalapalli 
> wrote:
>
>> Thank You! This will make the testing cycle much faster for contributors.
>>
>> I'm curious; why not still have the original "Run Python PostCommit"
>> trigger phrase?
>> It would be easier to run all the four tests with it when needed.
>>
>> Regards,
>> - Tanay
>>
>>>

Re: Live fixing of a Beam bug on July 25 at 3:30pm-4:30pm PST

[Tanay Tummalapalli]

Thank You! Looking forward to this :)

On Sun, Jul 21, 2019 at 2:35 AM Pablo Estrada  wrote:

> Hello all,
>
> This will be streamed on youtube on this link:
> https://www.youtube.com/watch?v=xpIpEO4PUDo
>
> I think there will be a live chat, so I will hopefully be available to
> answer questions. To be honest, my workflow is not super efficient, but...
> oh well, hopefully it will be at least somewhat helpful to others : )
> Best
> -P.
>
> On Thu, Jul 18, 2019 at 12:59 AM Tim Sell  wrote:
>
>> +1, I'd love to see this as a recording. Will you stick it up on youtube
>> afterwards?
>>
>> On Thu, Jul 18, 2019 at 4:00 AM sridhar inuog 
>> wrote:
>>
>>> Thanks, Pablo! Looking forward to it! Hopefully, it will also be
>>> recorded as well.
>>>
>>> On Wed, Jul 17, 2019 at 2:50 PM Pablo Estrada 
>>> wrote:
>>>
 Yes! So I will be working on a small feature request for Java's
 BigQueryIO: https://issues.apache.org/jira/browse/BEAM-7607

 Maybe I'll do something for Python next month. : )
 Best
 -P.

 On Wed, Jul 17, 2019 at 12:32 PM Rakesh Kumar 
 wrote:

> +1, I really appreciate this initiative. It would be really helpful
> newbies like me.
>
> Is it possible to list out what are the things that you are planning
> to cover?
>
>
>
>
> On Tue, Jul 16, 2019 at 11:19 AM Yichi Zhang 
> wrote:
>
>> Thanks for organizing this Pablo, it'll be very helpful!
>>
>> On Tue, Jul 16, 2019 at 10:57 AM Pablo Estrada 
>> wrote:
>>
>>> Hello all,
>>> I'll be having a session where I live-fix a Beam bug for 1 hour next
>>> week. Everyone is invited.
>>>
>>> It will be on July 25, between 3:30pm and 4:30pm PST. Hopefully I
>>> will finish a full change in that time frame, but we'll see.
>>>
>>> I have not yet decided if I will do this via hangouts, or via a
>>> youtube livestream. In any case, I will share the link here in the next 
>>> few
>>> days.
>>>
>>> I will most likely work on the Java SDK (I have a little feature
>>> request in mind).
>>>
>>> Thanks!
>>> -P.
>>>
>>

Re: [ANNOUNCE] New committer: Robert Burke

[Tanay Tummalapalli]

Congratulations!

On Wed, Jul 17, 2019 at 3:27 AM Aizhamal Nurmamat kyzy 
wrote:

> Congratulations, Rebo
>
> On Tue, Jul 16, 2019 at 1:34 PM Chamikara Jayalath 
> wrote:
>
>> Congrats!!
>>
>> On Tue, Jul 16, 2019 at 1:31 PM Robin Qiu  wrote:
>>
>>> Congrats, Robert!!
>>>
>>> On Tue, Jul 16, 2019 at 1:22 PM Alan Myrvold 
>>> wrote:
>>>
 Congrats, Robert!

 On Tue, Jul 16, 2019 at 11:46 AM Ismaël Mejía 
 wrote:

> Congrats Robert!
>
>
> On Tue, Jul 16, 2019 at 8:19 PM Yichi Zhang  wrote:
> >
> > Congratulations!
> >
> > On Tue, Jul 16, 2019 at 10:51 AM Holden Karau 
> wrote:
> >>
> >> Congratulations! :)
> >>
> >> On Tue, Jul 16, 2019 at 10:50 AM Mikhail Gryzykhin <
> mig...@google.com> wrote:
> >>>
> >>> Congratulations!
> >>>
> >>> On Tue, Jul 16, 2019 at 10:36 AM Ankur Goenka 
> wrote:
> 
>  Congratulations Robert!
> 
>  Go GO!
> 
>  On Tue, Jul 16, 2019 at 10:34 AM Rui Wang 
> wrote:
> >
> > Congrats!
> >
> >
> > -Rui
> >
> > On Tue, Jul 16, 2019 at 10:32 AM Udi Meiri 
> wrote:
> >>
> >> Congrats Robert B.!
> >>
> >> On Tue, Jul 16, 2019 at 10:23 AM Ahmet Altay 
> wrote:
> >>>
> >>> Hi,
> >>>
> >>> Please join me and the rest of the Beam PMC in welcoming a new
> committer: Robert Burke.
> >>>
> >>> Robert has been contributing to Beam and actively involved in
> the community for over a year. He has been actively working on Go SDK,
> helping users, and making it easier for others to contribute [1].
> >>>
> >>> In consideration of Robert's contributions, the Beam PMC
> trusts him with the responsibilities of a Beam committer [2].
> >>>
> >>> Thank you, Robert, for your contributions and looking forward
> to many more!
> >>>
> >>> Ahmet, on behalf of the Apache Beam PMC
> >>>
> >>> [1]
> https://lists.apache.org/thread.html/8f729da2d3009059d7a8b2d8624446be161700dcfa953939dd3530c6@%3Cdev.beam.apache.org%3E
> >>> [2]
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
> >>
> >>
> >>
> >> --
> >> Twitter: https://twitter.com/holdenkarau
> >> Books (Learning Spark, High Performance Spark, etc.):
> https://amzn.to/2MaRAG9
> >> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>

Re: python precommits failing at head

[Tanay Tummalapalli]

Yes. It passed on the second attempt.

But, I'm yet to figure out why it hangs for ~1.5 hours.

On Sun, Jul 14, 2019 at 10:36 AM Rakesh Kumar  wrote:

>
>
> Even I am running into the same issue. Though my test passed but
> <https://builds.apache.org/job/beam_PreCommit_Python_Phrase/656/console>somehow
> the task didn't terminate, eventually the task was aborted.  I have already
> tried a couple of times to retrigger tye python precommit but it failed
> every time.
>
> @Tanay did it pass it for you?
>
>
>
>
>
> On Fri, Jul 12, 2019 at 2:22 PM Tanay Tummalapalli 
> wrote:
>
>> Thank You Valentyn!
>>
>> I'll retest it.
>> Hopefully, it's a transient issue.
>>
>> Regards,
>> - Tanay Tummalapalli
>>
>> On Sat, Jul 13, 2019 at 2:39 AM Valentyn Tymofieiev 
>> wrote:
>>
>>> No, we did not reduce the timeout recently. Looking at console logs,
>>> nothing happened for an hour or so,
>>>
>>> *06:57:50 py27-cython: commands succeeded 06:57:50 congratulations :)
>>> 06:57:50 *
>>>
>>> *06:57:50* >* Task :sdks:python:preCommitPy2**08:22:33* Build timed out 
>>> (after 120 minutes). Marking the build as aborted.
>>>
>>>
>>> However, we can also see in the logs that py36-cython suite never
>>> started, not sure way. I assume gradle waited for this suite to finish.
>>> Try "retest this please", hopefully this is a transient gradle issue. I
>>> did not observe it before.
>>>
>>> On Fri, Jul 12, 2019 at 1:22 PM Tanay Tummalapalli 
>>> wrote:
>>>
>>>> Hi Udi,
>>>>
>>>> I rebased another PR[1] onto the fix mentioned above. The lint error is
>>>> fixed, but, the "beam_PreCommit_Python_Commit" Jenkins job is failing
>>>> because of a timeout at 120 minutes[2].
>>>> The log says "Build timed out (after 120 minutes). Marking the build
>>>> as aborted."
>>>> Another PR's Python PreCommit job aborted with the same error[3].
>>>>
>>>> I found this issue - "[BEAM-3040] Python precommit timed out after 150
>>>> minutes"[4].
>>>> Was the timeout reduced recently?
>>>>
>>>> Regards,
>>>> - Tanay Tummalapalli
>>>>
>>>> [1] https://github.com/apache/beam/pull/8871
>>>> [2]
>>>> https://builds.apache.org/job/beam_PreCommit_Python_Commit/7412/consoleFull
>>>>
>>>> [3] https://github.com/apache/beam/pull/9050
>>>> [4] https://issues.apache.org/jira/browse/BEAM-3040
>>>>
>>>> On Fri, Jul 12, 2019 at 5:42 AM Udi Meiri  wrote:
>>>>
>>>>> This is due to
>>>>> https://github.com/apache/beam/pull/8969
>>>>> and
>>>>> https://github.com/apache/beam/pull/8934
>>>>> being merged today.
>>>>>
>>>>> Fix is here: https://github.com/apache/beam/pull/9044
>>>>>
>>>>

Re: python precommits failing at head

[Tanay Tummalapalli]

Thank You Valentyn!

I'll retest it.
Hopefully, it's a transient issue.

Regards,
- Tanay Tummalapalli

On Sat, Jul 13, 2019 at 2:39 AM Valentyn Tymofieiev 
wrote:

> No, we did not reduce the timeout recently. Looking at console logs,
> nothing happened for an hour or so,
>
> *06:57:50 py27-cython: commands succeeded 06:57:50 congratulations :)
> 06:57:50 *
>
> *06:57:50* >* Task :sdks:python:preCommitPy2**08:22:33* Build timed out 
> (after 120 minutes). Marking the build as aborted.
>
>
> However, we can also see in the logs that py36-cython suite never started,
> not sure way. I assume gradle waited for this suite to finish.
> Try "retest this please", hopefully this is a transient gradle issue. I
> did not observe it before.
>
> On Fri, Jul 12, 2019 at 1:22 PM Tanay Tummalapalli 
> wrote:
>
>> Hi Udi,
>>
>> I rebased another PR[1] onto the fix mentioned above. The lint error is
>> fixed, but, the "beam_PreCommit_Python_Commit" Jenkins job is failing
>> because of a timeout at 120 minutes[2].
>> The log says "Build timed out (after 120 minutes). Marking the build as
>> aborted."
>> Another PR's Python PreCommit job aborted with the same error[3].
>>
>> I found this issue - "[BEAM-3040] Python precommit timed out after 150
>> minutes"[4].
>> Was the timeout reduced recently?
>>
>> Regards,
>> - Tanay Tummalapalli
>>
>> [1] https://github.com/apache/beam/pull/8871
>> [2]
>> https://builds.apache.org/job/beam_PreCommit_Python_Commit/7412/consoleFull
>>
>> [3] https://github.com/apache/beam/pull/9050
>> [4] https://issues.apache.org/jira/browse/BEAM-3040
>>
>> On Fri, Jul 12, 2019 at 5:42 AM Udi Meiri  wrote:
>>
>>> This is due to
>>> https://github.com/apache/beam/pull/8969
>>> and
>>> https://github.com/apache/beam/pull/8934
>>> being merged today.
>>>
>>> Fix is here: https://github.com/apache/beam/pull/9044
>>>
>>

Re: python precommits failing at head

[Tanay Tummalapalli]

Hi Udi,

I rebased another PR[1] onto the fix mentioned above. The lint error is
fixed, but, the "beam_PreCommit_Python_Commit" Jenkins job is failing
because of a timeout at 120 minutes[2].
The log says "Build timed out (after 120 minutes). Marking the build as
aborted."
Another PR's Python PreCommit job aborted with the same error[3].

I found this issue - "[BEAM-3040] Python precommit timed out after 150
minutes"[4].
Was the timeout reduced recently?

Regards,
- Tanay Tummalapalli

[1] https://github.com/apache/beam/pull/8871
[2]
https://builds.apache.org/job/beam_PreCommit_Python_Commit/7412/consoleFull
[3] https://github.com/apache/beam/pull/9050
[4] https://issues.apache.org/jira/browse/BEAM-3040

On Fri, Jul 12, 2019 at 5:42 AM Udi Meiri  wrote:

> This is due to
> https://github.com/apache/beam/pull/8969
> and
> https://github.com/apache/beam/pull/8934
> being merged today.
>
> Fix is here: https://github.com/apache/beam/pull/9044
>

Re: [ANNOUNCE] New committer: Mikhail Gryzykhin

[Tanay Tummalapalli]

Congratulations!

On Fri, Jun 21, 2019 at 10:35 PM Rui Wang  wrote:

> Congrats!
>
>
> -Rui
>
> On Fri, Jun 21, 2019 at 9:58 AM Robin Qiu  wrote:
>
>> Congrats, Mikhail!
>>
>> On Fri, Jun 21, 2019 at 9:12 AM Alexey Romanenko <
>> aromanenko@gmail.com> wrote:
>>
>>> Congrats, Mikhail!
>>>
>>> On 21 Jun 2019, at 18:01, Anton Kedin  wrote:
>>>
>>> Congrats!
>>>
>>> On Fri, Jun 21, 2019 at 3:55 AM Reza Rokni  wrote:
>>>
 Congratulations!

 On Fri, 21 Jun 2019, 12:37 Robert Burke,  wrote:

> Congrats
>
> On Fri, Jun 21, 2019, 12:29 PM Thomas Weise  wrote:
>
>> Hi,
>>
>> Please join me and the rest of the Beam PMC in welcoming a new
>> committer: Mikhail Gryzykhin.
>>
>> Mikhail has been contributing to Beam and actively involved in the
>> community for over a year. He developed the community build dashboard [1]
>> and added substantial improvements to our build infrastructure. Mikhail's
>> work also covers metrics, contributor documentation, development process
>> improvements and other areas.
>>
>> In consideration of Mikhail's contributions, the Beam PMC trusts him
>> with the responsibilities of a Beam committer [2].
>>
>> Thank you, Mikhail, for your contributions and looking forward to
>> many more!
>>
>> Thomas, on behalf of the Apache Beam PMC
>>
>> [1] https://s.apache.org/beam-community-metrics
>> [2]
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>>
>>>

Re: Help triaging Jira issues

[Tanay Tummalapalli]

Hi Kenneth,

I already follow the issues@ mailing list pretty much daily.
I'd like to help with triaging issues, especially ones related to the
Python SDK since I'm most familiar with it.

On Wed, Jun 5, 2019 at 10:26 PM Alex Van Boxel  wrote:

> Hey Kenneth, I help out. I'm planning to contribute more on Beam and it
> seems to be ideal to keep up-to-date with the project.
>
>  _/
> _/ Alex Van Boxel
>
>
> On Wed, Jun 5, 2019 at 6:46 PM Kenneth Knowles  wrote:
>
>> Hi all,
>>
>> I am requesting help in triaging incoming issues. I made a search here:
>> https://issues.apache.org/jira/issues/?filter=12345682
>> 
>>
>> I have a daily email subscription to this filter as a reminder, but
>> rarely can really sit down to do triage for very long. It has grown from
>> just under 200 to just over 200. The rate is actually pretty low but there
>> is a backlog. I also want to start re-triaging stale bugs but priority
>> would be (1) keep up with new bugs (2) clear backlog (3) re-triage stale
>> bugs.
>>
>> Just FYI what I look for before I clicked "Triaged" is:
>>
>>  - correct component
>>  - correct priority
>>  - maybe ping someone in a comment or assign
>>  - write to dev@ if it is a major problem
>>
>> If I can't figure that out, then I ask the reporter for clarification and
>> "Start Watching" the issue so I will receive their response.
>>
>> To avoid duplicate triage work it may help to assign to yourself
>> temporarily during triage phase.
>>
>> Any help greatly appreciated!
>>
>> Kenn
>>
>

Re: Plan for dropping python 2 support

[Tanay Tummalapalli]

We can support Python 2 for some time in 2020, but, we should target a date
no later than 2020 to drop support.
If we do plan to drop support for Python 2 in 2020, we should sign the
Python 3 statement[1], declaring that we will "drop support for Python 2.7
no later than 2020".

In addition to the statement, keeping a target release and date(if
possible) or timeline to drop support would also help users to decide when
they need to work on migrating to Python 3.

Regards,
- TT

[1] https://python3statement.org/

On Wed, Jun 5, 2019 at 4:37 PM Robert Bradshaw  wrote:

> Until Python 3 support for Beam is officially out of beta and
> recommended, I don't think we can tell people to stop using Python 2.
> Given that 2020 is just over 6 months away, that seems a short
> transition time, so I would guess we'll have to continue supporting
> Python 2 sometime into 2020.
>
> A quick survey of users would be valuable here. But first priority is
> making Python 3 rock solid so we can unconditionally recommend it over
> Python 2.
>
> On Wed, Jun 5, 2019 at 12:27 PM Ismaël Mejía  wrote:
> >
> > Python 2 won't be maintained after 2020 [1]. I was wondering what will
> > be our (Beam) plan for this. Other projects [2] have started to alert
> > users that support will be removed so maybe we should decide or policy
> > for this too.
> >
> > [1] https://pythonclock.org/
> > [2]
> https://spark.apache.org/news/plan-for-dropping-python-2-support.html
>

Re: BQ IT tests fail on TestDataflowRunner - Python SDK

[Tanay Tummalapalli]

I didn't have any other changes.
I ran the tests with a clean virtualenv as you suggested and it works now.
:)

Thanks Ahmet and Chamikara!

On Tue, Jun 4, 2019 at 6:36 AM Chamikara Jayalath 
wrote:

> Sounds like your input job was somehow incompatible with the Dataflow
> worker. Running using a clean virtual env should help verify as Ahmet
> mentioned.
>
> On Mon, Jun 3, 2019 at 5:44 PM Ahmet Altay  wrote:
>
>> Do you have any other changes? Are you trying from head with a clean
>> virtual environment?
>>
>> If you can share a link to dataflow job (in the apache-beam-testing GCP
>> project), we can try to look at additional logs as well.
>>
>> On Mon, Jun 3, 2019 at 1:42 PM Tanay Tummalapalli 
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I ran the Integration Tests -
>>> BigQueryStreamingInsertTransformIntegrationTests[1] and
>>> BigQueryFileLoadsIT[2] on the master branch locally, with the following
>>> command:
>>> ./scripts/run_integration_test.sh --test_opts
>>> --tests=apache_beam.io.gcp.bigquery_test:BigQueryStreamingInsertTransformIntegrationTests
>>> The Dataflow jobs for the tests failed with the following error:
>>> root: INFO: 2019-06-03T18:36:53.021Z: JOB_MESSAGE_ERROR: Traceback
>>> (most recent call last):
>>> File
>>> "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
>>> line 649, in do_work
>>> work_executor.execute()
>>> File
>>> "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
>>> line 150, in execute
>>> test_shuffle_sink=self._test_shuffle_sink)
>>> File
>>> "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
>>> line 116, in create_operation
>>> is_streaming=False)
>>> File "apache_beam/runners/worker/operations.py", line 962, in
>>> apache_beam.runners.worker.operations.create_operation
>>> op = BatchGroupAlsoByWindowsOperation(
>>> File "dataflow_worker/shuffle_operations.py", line 219, in
>>> dataflow_worker.shuffle_operations.BatchGroupAlsoByWindowsOperation.
>>> __init__
>>> self.windowing = deserialize_windowing_strategy(self.spec.window_fn)
>>> File "dataflow_worker/shuffle_operations.py", line 207, in
>>> dataflow_worker.shuffle_operations.deserialize_windowing_strategy
>>> return pickler.loads(serialized_data)
>>> File
>>> "/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py",
>>> line 248, in loads
>>> c = base64.b64decode(encoded)
>>> File "/usr/lib/python2.7/base64.py", line 78, in b64decode
>>> raise TypeError(msg)
>>> TypeError: Incorrect padding
>>>
>>>
>>> I tested the same tests on the 2.13.0-RC#2 branch as well and they
>>> passed. These tests also don't fail in the most recent Python post-commit
>>> tests[3-5].
>>>
>>> Keeping in mind the recent b64 changes in BQ, none of the tests in the
>>> test classes mentioned above makes use of a "BYTES" type field.
>>> Would love to get pointers to possible reasons.
>>>
>>> Thank You
>>> - TT
>>>
>>> [1]
>>> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery_test.py#L479-L630
>>> [2]
>>> https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery_file_loads_test.py#L358-L528
>>> [3]
>>> https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/
>>> [4]
>>> https://builds.apache.org/job/beam_PostCommit_Python3_Verify/lastCompletedBuild/
>>> [5]
>>> https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/
>>>
>>

BQ IT tests fail on TestDataflowRunner - Python SDK

[Tanay Tummalapalli]

Hi everyone,

I ran the Integration Tests -
BigQueryStreamingInsertTransformIntegrationTests[1] and BigQueryFileLoadsIT[2]
on the master branch locally, with the following command:
./scripts/run_integration_test.sh --test_opts
--tests=apache_beam.io.gcp.bigquery_test:BigQueryStreamingInsertTransformIntegrationTests
The Dataflow jobs for the tests failed with the following error:
root: INFO: 2019-06-03T18:36:53.021Z: JOB_MESSAGE_ERROR: Traceback (most
recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py",
line 649, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 150, in execute
test_shuffle_sink=self._test_shuffle_sink)
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py",
line 116, in create_operation
is_streaming=False)
File "apache_beam/runners/worker/operations.py", line 962, in
apache_beam.runners.worker.operations.create_operation
op = BatchGroupAlsoByWindowsOperation(
File "dataflow_worker/shuffle_operations.py", line 219, in
dataflow_worker.shuffle_operations.BatchGroupAlsoByWindowsOperation.__init__
self.windowing = deserialize_windowing_strategy(self.spec.window_fn)
File "dataflow_worker/shuffle_operations.py", line 207, in
dataflow_worker.shuffle_operations.deserialize_windowing_strategy
return pickler.loads(serialized_data)
File
"/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py",
line 248, in loads
c = base64.b64decode(encoded)
File "/usr/lib/python2.7/base64.py", line 78, in b64decode
raise TypeError(msg)
TypeError: Incorrect padding


I tested the same tests on the 2.13.0-RC#2 branch as well and they passed.
These tests also don't fail in the most recent Python post-commit
tests[3-5].

Keeping in mind the recent b64 changes in BQ, none of the tests in the test
classes mentioned above makes use of a "BYTES" type field.
Would love to get pointers to possible reasons.

Thank You
- TT

[1]
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery_test.py#L479-L630
[2]
https://github.com/apache/beam/blob/master/sdks/python/apache_beam/io/gcp/bigquery_file_loads_test.py#L358-L528
[3]
https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/
[4]
https://builds.apache.org/job/beam_PostCommit_Python3_Verify/lastCompletedBuild/
[5]
https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/

Re: Support for PaneInfo in Python SDK

[Tanay Tummalapalli]

Hi Pablo,
Thanks for that example, it would be great to be able to use
fileio.WriteToFiles transform to write files with filenames that are based
on their PaneInfo.

Thanks @Charles Chen, for adding the remaining work on the issue - the
emission of PaneInfo in the Python implementation of GBK (either in the
Python FnAPIRunner or the old Python DirectRunner / triggers.py).

I will certainly make sure that [BEAM-3759] is completed after my GSoC
project is implemented. It's a good opportunity to get into the runner code.

Regards
- TT

On Fri, May 31, 2019 at 2:35 AM Pablo Estrada  wrote:

> Hi Tanay,
> thanks for bringing this to the mailing list. I believe this is certainly
> useful, and necessary. As an example, the fileio.WriteToFiles transform
> does not work well without PaneInfo data (since we can't know how many
> firings there are for each window, and we can't give names to files based
> on this).
>
> Best
> -P.
>
> On Thu, May 30, 2019 at 1:00 PM Tanay Tummalapalli 
> wrote:
>
>> Hi everyone,
>>
>> The PR linked in [BEAM-3759] - "Add support for PaneInfo descriptor in
>> Python SDK"[1] was merged, but, the issue is still open.
>> There might be some work left on this for full support for PaneInfo. Eg:
>> Although the PaneInfo class exists, it is not accessible in a DoFn via a
>> kwarg(PaneInfoParam) like TimestampParam or WindowParam.
>>
>> Please let me know the remaining work to be done on this issue as this
>> may be needed in the near future.
>>
>> Regards
>> Tanay Tummalapalli
>>
>> [1] https://issues.apache.org/jira/browse/BEAM-3759
>>
>

Support for PaneInfo in Python SDK

[Tanay Tummalapalli]

Hi everyone,

The PR linked in [BEAM-3759] - "Add support for PaneInfo descriptor in
Python SDK"[1] was merged, but, the issue is still open.
There might be some work left on this for full support for PaneInfo. Eg:
Although the PaneInfo class exists, it is not accessible in a DoFn via a
kwarg(PaneInfoParam) like TimestampParam or WindowParam.

Please let me know the remaining work to be done on this issue as this may
be needed in the near future.

Regards
Tanay Tummalapalli

[1] https://issues.apache.org/jira/browse/BEAM-3759

Re: Beam at Google Summer of Code 2019

[Tanay Tummalapalli]

Hi everyone,

I made a Kanban board[1] on Github, on my fork of apache/beam to keep track
of progress for GSoC '19.

Regards,
Tanay Tummalapalli

[1] https://github.com/ttanay/beam/projects/1

On Tue, May 7, 2019 at 6:39 PM Tanay Tummalapalli 
wrote:

> Thank You!
>
> I'm really excited to work on Beam!
> I'd like to thank Pablo, Chamikara Jayalath and Tim Robertson for helping
> out with my proposal[1].
>
> Looking forward to working with everyone and learning a great deal.
>
> Regards
> Tanay Tummalapalli
> LinkedIn <https://www.linkedin.com/in/ttanay/> | Github
> <https://github.com/ttanay>
>
> [1]
> https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=sharing
>
> On Tue, May 7, 2019 at 12:04 AM Pablo Estrada  wrote:
>
>> Hello all,
>> it is my pleasure to share with everyone that Tanay Tummalapalli has been
>> accepted as a GSoC student with Beam, to implement support for File Loads
>> into BigQuery for streaming pipelines[1].
>>
>> Tanay wrote a very strong proposal, and showed understanding of the
>> tricky streaming considerations that will play out in this project.
>>
>> I speak on behalf of everyone welcoming you Tanay, and we'll be happy to
>> see your contributions to Beam. : )
>> Best
>> -P.
>>
>> [1]
>> https://summerofcode.withgoogle.com/projects/?sp-search=Tanay#4999837794172928
>>
>

Re: [ANNOUNCE] New PMC Member: Pablo Estrada

[Tanay Tummalapalli]

Congratulations Pablo!

On Wed, May 15, 2019, 12:08 Michael Luckey  wrote:

> Congrats, Pablo!
>
> On Wed, May 15, 2019 at 8:21 AM Connell O'Callaghan 
> wrote:
>
>> Awesome well done Pablo!!!
>>
>> Kenn thank you for sharing this great news with us!!!
>>
>> On Tue, May 14, 2019 at 11:01 PM Ahmet Altay  wrote:
>>
>>> Congratulations!
>>>
>>> On Tue, May 14, 2019 at 9:11 PM Robert Burke  wrote:
>>>
 Woohoo! Well deserved.

 On Tue, May 14, 2019, 8:34 PM Reuven Lax  wrote:

> Congratulations!
>
> *From: *Mikhail Gryzykhin 
> *Date: *Tue, May 14, 2019 at 8:32 PM
> *To: * 
>
> Congratulations Pablo!
>>
>> On Tue, May 14, 2019, 20:25 Kenneth Knowles  wrote:
>>
>>> Hi all,
>>>
>>> Please join me and the rest of the Beam PMC in welcoming Pablo
>>> Estrada to join the PMC.
>>>
>>> Pablo first picked up BEAM-722 in October of 2016 and has been a
>>> steady part of the Beam community since then. In addition to technical 
>>> work
>>> on Beam Python & Java & runners, I would highlight how Pablo grows 
>>> Beam's
>>> community by helping users, working on GSoC, giving talks at Beam 
>>> Summits
>>> and other OSS conferences including Flink Forward, and holding training
>>> workshops. I cannot do justice to Pablo's contributions in a single
>>> paragraph.
>>>
>>> Thanks Pablo, for being a part of Beam.
>>>
>>> Kenn
>>>
>>

Fwd: Beam at Google Summer of Code 2019

[Tanay Tummalapalli]

Thank You!

I'm really excited to work on Beam!
I'd like to thank Pablo, Chamikara Jayalath and Tim Robertson for helping
out with my proposal[1].

Looking forward to working with everyone and learning a great deal.

Regards
Tanay Tummalapalli
LinkedIn <https://www.linkedin.com/in/ttanay/> | Github
<https://github.com/ttanay>

[1]
https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=sharing

On Tue, May 7, 2019 at 12:04 AM Pablo Estrada  wrote:

> Hello all,
> it is my pleasure to share with everyone that Tanay Tummalapalli has been
> accepted as a GSoC student with Beam, to implement support for File Loads
> into BigQuery for streaming pipelines[1].
>
> Tanay wrote a very strong proposal, and showed understanding of the tricky
> streaming considerations that will play out in this project.
>
> I speak on behalf of everyone welcoming you Tanay, and we'll be happy to
> see your contributions to Beam. : )
> Best
> -P.
>
> [1]
> https://summerofcode.withgoogle.com/projects/?sp-search=Tanay#4999837794172928
>

Feedback on GSoC Proposal

[Tanay Tummalapalli]

Hi everyone

I'd like to thank Pablo Estrada, Chamikara Jayalath and Tim Robertson for
their suggestions and feedback.

I have made changes to my proposal - "*A Python Sink for BigQuery with File
Loads in Streaming*", and my final proposal is ready.
I'd love to get your final comments and feedback on it, so I can submit the
best possible proposal.

Link to my proposal -
https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=sharing

PS: Sorry for posting again to a new thread. Applications close on 9th
April at 1800 hrs UTC.

Thank You
Tanay Tummalapalli

Re: GSoC Project Proposal

[Tanay Tummalapalli]

Hi everyone

I'd like to thank Pablo Estrada, Chamikara Jayalath and Tim Robertson for
their suggestions and feedback.
I have made changes to my proposal and my final draft is ready.

As the application period for GSoC closes on 9th April, I'd love to get
your feedback on it, so I can submit the best possible proposal.

Link to my proposal -
https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=sharing

Thank You
Tanay Tummalapalli
LinkedIn <https://www.linkedin.com/in/ttanay/> | GitHub
<https://github.com/ttanay>

On Sat, Mar 30, 2019 at 4:18 PM Tanay Tummalapalli 
wrote:

> Thank You Pablo!
>
> On Sat, Mar 30, 2019 at 2:58 AM Pablo Estrada  wrote:
>
>> Hi Tanay,
>> thanks for submitting the proposal. I'll try to take a look on Tuesday to
>> give some feedback.
>> I encourage others to take a look as well : )
>> Best
>> -P.
>>
>> On Thu, Mar 28, 2019 at 11:58 AM Tanay Tummalapalli 
>> wrote:
>>
>>> Hi everyone,
>>>
>>> I am applying for Google Summer of Code with Apache Beam this year.
>>> I have chosen BEAM-6611
>>> <https://issues.apache.org/jira/browse/BEAM-6611> - "A Python Sink for
>>> BigQuery with File Loads in Streaming" as my project.
>>> The draft of my proposal
>>> <https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=sharing>
>>>  is
>>> ready. I would love to get feedback and criticism from you so I can submit
>>> the best possible proposal.
>>>
>>> PFA my project proposal.
>>>
>>> Thank You
>>> Tanay Tummalapalli
>>> LinkedIn <https://www.linkedin.com/in/ttanay/> | GitHub
>>> <https://github.com/ttanay>
>>>
>>>
>>>  GSoC Project proposal - T. Tanay
>>> <https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=drive_web>
>>>
>>>
>>>

Re: GSoC Project Proposal

[Tanay Tummalapalli]

Thank You Pablo!

On Sat, Mar 30, 2019 at 2:58 AM Pablo Estrada  wrote:

> Hi Tanay,
> thanks for submitting the proposal. I'll try to take a look on Tuesday to
> give some feedback.
> I encourage others to take a look as well : )
> Best
> -P.
>
> On Thu, Mar 28, 2019 at 11:58 AM Tanay Tummalapalli 
> wrote:
>
>> Hi everyone,
>>
>> I am applying for Google Summer of Code with Apache Beam this year.
>> I have chosen BEAM-6611 <https://issues.apache.org/jira/browse/BEAM-6611> -
>> "A Python Sink for BigQuery with File Loads in Streaming" as my project.
>> The draft of my proposal
>> <https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=sharing>
>>  is
>> ready. I would love to get feedback and criticism from you so I can submit
>> the best possible proposal.
>>
>> PFA my project proposal.
>>
>> Thank You
>> Tanay Tummalapalli
>> LinkedIn <https://www.linkedin.com/in/ttanay/> | GitHub
>> <https://github.com/ttanay>
>>
>>
>>  GSoC Project proposal - T. Tanay
>> <https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=drive_web>
>>
>>
>>

GSoC Project Proposal

[Tanay Tummalapalli]

Hi everyone,

I am applying for Google Summer of Code with Apache Beam this year.
I have chosen BEAM-6611 <https://issues.apache.org/jira/browse/BEAM-6611> -
"A Python Sink for BigQuery with File Loads in Streaming" as my project.
The draft of my proposal
<https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=sharing>
is
ready. I would love to get feedback and criticism from you so I can submit
the best possible proposal.

PFA my project proposal.

Thank You
Tanay Tummalapalli
LinkedIn <https://www.linkedin.com/in/ttanay/> | GitHub
<https://github.com/ttanay>


 GSoC Project proposal - T. Tanay
<https://docs.google.com/document/d/15Peyd3Z_wu5rvGWw8lMLpZuTyyreM_JOAEFFWvF97YY/edit?usp=drive_web>