Re: Jenkins jobs not running for my PR 10438

[Ismaël Mejía]

done

On Fri, Jan 31, 2020 at 1:51 PM Rehman Murad Ali <
rehman.murad...@venturedive.com> wrote:

> Hi,
>
> I appreciate if someone could trigger the jobs for this PR:
>
> https://github.com/apache/beam/pull/10627
>
> *Thanks*
>
> *Rehman Murad Ali*
> Software Engineer
> Mobile: +92 3452076766
> Skype: rehman.muradali
>
>
> On Thu, Jan 30, 2020 at 10:19 PM Luke Cwik  wrote:
>
>> done
>>
>> On Thu, Jan 30, 2020 at 12:28 AM Shoaib Zafar <
>> shoaib.za...@venturedive.com> wrote:
>>
>>> Hi Beam Committer,
>>>
>>> I appreciate if someone could trigger jobs for
>>> https://github.com/apache/beam/pull/10712.
>>>
>>> Thanks!
>>>
>>> *Shoaib Zafar*
>>>
>>> Software Engineering Lead
>>> Mobile: +92 333 274 6242
>>> Skype: live:shoaibzafar_1
>>>
>>> 
>>>
>>>
>>> On Thu, Jan 30, 2020 at 9:09 AM Boyuan Zhang  wrote:
>>>
 Done : )

 On Wed, Jan 29, 2020 at 7:52 PM Tomo Suzuki  wrote:

> HI Beam committers:
> (Thanks, Luke!)
>
> Can somebody retrigger the following 2 failed checks for
> https://github.com/apache/beam/pull/10714 ?
> Run Java PreCommit
> Run Java_Examples_Dataflow PreCommit
>
> On Wed, Jan 29, 2020 at 4:48 PM Luke Cwik  wrote:
> >
> > done
> >
> > On Wed, Jan 29, 2020 at 11:07 AM Tomo Suzuki 
> wrote:
> >>
> >> Hi Beam committers,
> >>
> >> I appreciate if you can trigger the precommit checks for
> >> https://github.com/apache/beam/pull/10714
> >>
> >> with following 6 additional commands (one command per comment):
> >>
> >> Run Java PostCommit
> >> Run Java HadoopFormatIO Performance Test
> >> Run BigQueryIO Streaming Performance Test Java
> >> Run Dataflow ValidatesRunner
> >> Run Spark ValidatesRunner
> >> Run SQL Postcommit
> >>
> >> Regards,
> >> Tomo
> >>
> >>
>
>
> --
> Regards,
> Tomo
>

Re: Jenkins jobs not running for my PR 10438

[Rehman Murad Ali]

Hi,

I appreciate if someone could trigger the jobs for this PR:

https://github.com/apache/beam/pull/10627

*Thanks*

*Rehman Murad Ali*
Software Engineer
Mobile: +92 3452076766
Skype: rehman.muradali


On Thu, Jan 30, 2020 at 10:19 PM Luke Cwik  wrote:

> done
>
> On Thu, Jan 30, 2020 at 12:28 AM Shoaib Zafar <
> shoaib.za...@venturedive.com> wrote:
>
>> Hi Beam Committer,
>>
>> I appreciate if someone could trigger jobs for
>> https://github.com/apache/beam/pull/10712.
>>
>> Thanks!
>>
>> *Shoaib Zafar*
>>
>> Software Engineering Lead
>> Mobile: +92 333 274 6242
>> Skype: live:shoaibzafar_1
>>
>> 
>>
>>
>> On Thu, Jan 30, 2020 at 9:09 AM Boyuan Zhang  wrote:
>>
>>> Done : )
>>>
>>> On Wed, Jan 29, 2020 at 7:52 PM Tomo Suzuki  wrote:
>>>
 HI Beam committers:
 (Thanks, Luke!)

 Can somebody retrigger the following 2 failed checks for
 https://github.com/apache/beam/pull/10714 ?
 Run Java PreCommit
 Run Java_Examples_Dataflow PreCommit

 On Wed, Jan 29, 2020 at 4:48 PM Luke Cwik  wrote:
 >
 > done
 >
 > On Wed, Jan 29, 2020 at 11:07 AM Tomo Suzuki 
 wrote:
 >>
 >> Hi Beam committers,
 >>
 >> I appreciate if you can trigger the precommit checks for
 >> https://github.com/apache/beam/pull/10714
 >>
 >> with following 6 additional commands (one command per comment):
 >>
 >> Run Java PostCommit
 >> Run Java HadoopFormatIO Performance Test
 >> Run BigQueryIO Streaming Performance Test Java
 >> Run Dataflow ValidatesRunner
 >> Run Spark ValidatesRunner
 >> Run SQL Postcommit
 >>
 >> Regards,
 >> Tomo
 >>
 >>


 --
 Regards,
 Tomo

>>>

Re: Release 2.19.0, release candidate #1

[Ismaël Mejía]

+1 (binding)

- Validated signatures
- Run Python wordcount on Direct runner (from wheels)
- Run Python wordcount on Flink runner with job-server image (via wheels)
- Run Python wordcount on Spark runner with job-server from source (via
wheels)
- Validate no regressions on Nexmark for Spark classic runner (There are
actually improvements!)
- Validate provided artifacts in two external projects beam-samples + one
internal company project


On Fri, Jan 31, 2020 at 2:04 AM Ahmet Altay  wrote:

> +1 -- Verified web page / blog post changes.
>
> Thank you for building this RC and pre-validating a large number of tests.
>
> On Wed, Jan 29, 2020 at 9:03 PM Boyuan Zhang  wrote:
>
>> Hi everyone,
>> Please review and vote on the release candidate #1 for the version
>> 2.19.0, as follows:
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org [2],
>> which is signed with the key with fingerprint D6BF C115 EAA6 9828 5A89
>>  3165 99DF 385D A877 C596[3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.19.0-RC1" [5],
>> * website pull request listing the release [6], publishing the API
>> reference manual [7], and the blog post [8].
>> * Java artifacts were built with Maven MAVEN_VERSION and OpenJDK/Oracle
>> JDK JDK_VERSION.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>> * Validation sheet with a tab for 2.18.0 release to help with validation
>> [9].
>> * Docker images published to Docker Hub [10].
>>
>> The vote will be open for at least 72 hours. It is adopted by majority
>> approval, with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> Release Manager
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12346582
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.19.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4]
>> https://repository.apache.org/content/repositories/orgapachebeam-1091/
>> [5] https://github.com/apache/beam/tree/v2.19.0-RC1
>> [6] https://github.com/apache/beam/pull/10713
>> [7] https://github.com/apache/beam-site/pull/597
>> [8] https://github.com/apache/beam/pull/10722
>> [9]
>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1148160512
>> [10] https://hub.docker.com/u/apachebeam
>>
>

Re: Release 2.19.0, release candidate #1

[Jean-Baptiste Onofré]

+1 (binding)

Checked quickly on beam-samples.

Thanks,
Regards
JB

On 30/01/2020 06:02, Boyuan Zhang wrote:
> Hi everyone,
> Please review and vote on the release candidate #1 for the version
> 2.19.0, as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
> 
> 
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
>  [2], which is signed with the key with
> fingerprint D6BF C115 EAA6 9828 5A89  3165 99DF 385D A877 C596[3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.19.0-RC1" [5],
> * website pull request listing the release [6], publishing the API
> reference manual [7], and the blog post [8].
> * Java artifacts were built with Maven MAVEN_VERSION and OpenJDK/Oracle
> JDK JDK_VERSION.
> * Python artifacts are deployed along with the source release to
> the dist.apache.org  [2].
> * Validation sheet with a tab for 2.18.0 release to help with validation
> [9].
> * Docker images published to Docker Hub [10].
> 
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
> 
> Thanks,
> Release Manager
> 
> [1] 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12346582
> [2] https://dist.apache.org/repos/dist/dev/beam/2.19.0/
> 
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> 
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1091/
> [5] https://github.com/apache/beam/tree/v2.19.0-RC1
> 
> [6] https://github.com/apache/beam/pull/10713
> [7] https://github.com/apache/beam-site/pull/597
> [8] https://github.com/apache/beam/pull/10722
> [9] 
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1148160512
> [10] https://hub.docker.com/u/apachebeam

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Jenkins jobs not running for my PR 10438

[Tomo Suzuki]

HI Beam committers,

Would you re-trigger the 2 failed checks in
https://github.com/apache/beam/pull/10714 ?
Run Java PreCommit
Run Java_Examples_Dataflow PreCommit


On Fri, Jan 31, 2020 at 7:51 AM Rehman Murad Ali <
rehman.murad...@venturedive.com> wrote:

> Hi,
>
> I appreciate if someone could trigger the jobs for this PR:
>
> https://github.com/apache/beam/pull/10627
>
> *Thanks*
>
> *Rehman Murad Ali*
> Software Engineer
> Mobile: +92 3452076766 <+92%20345%202076766>
> Skype: rehman.muradali
>
>
> On Thu, Jan 30, 2020 at 10:19 PM Luke Cwik  wrote:
>
>> done
>>
>> On Thu, Jan 30, 2020 at 12:28 AM Shoaib Zafar <
>> shoaib.za...@venturedive.com> wrote:
>>
>>> Hi Beam Committer,
>>>
>>> I appreciate if someone could trigger jobs for
>>> https://github.com/apache/beam/pull/10712.
>>>
>>> Thanks!
>>>
>>> *Shoaib Zafar*
>>>
>>> Software Engineering Lead
>>> Mobile: +92 333 274 6242
>>> Skype: live:shoaibzafar_1
>>>
>>> 
>>>
>>>
>>> On Thu, Jan 30, 2020 at 9:09 AM Boyuan Zhang  wrote:
>>>
 Done : )

 On Wed, Jan 29, 2020 at 7:52 PM Tomo Suzuki  wrote:

> HI Beam committers:
> (Thanks, Luke!)
>
> Can somebody retrigger the following 2 failed checks for
> https://github.com/apache/beam/pull/10714 ?
> Run Java PreCommit
> Run Java_Examples_Dataflow PreCommit
>
> On Wed, Jan 29, 2020 at 4:48 PM Luke Cwik  wrote:
> >
> > done
> >
> > On Wed, Jan 29, 2020 at 11:07 AM Tomo Suzuki 
> wrote:
> >>
> >> Hi Beam committers,
> >>
> >> I appreciate if you can trigger the precommit checks for
> >> https://github.com/apache/beam/pull/10714
> >>
> >> with following 6 additional commands (one command per comment):
> >>
> >> Run Java PostCommit
> >> Run Java HadoopFormatIO Performance Test
> >> Run BigQueryIO Streaming Performance Test Java
> >> Run Dataflow ValidatesRunner
> >> Run Spark ValidatesRunner
> >> Run SQL Postcommit
> >>
> >> Regards,
> >> Tomo
> >>
> >>
>
>
> --
> Regards,
> Tomo
>


-- 
Regards,
Tomo

Re: Jenkins jobs not running for my PR 10438

[Udi Meiri]

done

On Fri, Jan 31, 2020 at 9:07 AM Tomo Suzuki  wrote:

> HI Beam committers,
>
> Would you re-trigger the 2 failed checks in
> https://github.com/apache/beam/pull/10714 ?
> Run Java PreCommit
> Run Java_Examples_Dataflow PreCommit
>
>
> On Fri, Jan 31, 2020 at 7:51 AM Rehman Murad Ali <
> rehman.murad...@venturedive.com> wrote:
>
>> Hi,
>>
>> I appreciate if someone could trigger the jobs for this PR:
>>
>> https://github.com/apache/beam/pull/10627
>>
>> *Thanks*
>>
>> *Rehman Murad Ali*
>> Software Engineer
>> Mobile: +92 3452076766 <+92%20345%202076766>
>> Skype: rehman.muradali
>>
>>
>> On Thu, Jan 30, 2020 at 10:19 PM Luke Cwik  wrote:
>>
>>> done
>>>
>>> On Thu, Jan 30, 2020 at 12:28 AM Shoaib Zafar <
>>> shoaib.za...@venturedive.com> wrote:
>>>
 Hi Beam Committer,

 I appreciate if someone could trigger jobs for
 https://github.com/apache/beam/pull/10712.

 Thanks!

 *Shoaib Zafar*

 Software Engineering Lead
 Mobile: +92 333 274 6242
 Skype: live:shoaibzafar_1

 


 On Thu, Jan 30, 2020 at 9:09 AM Boyuan Zhang 
 wrote:

> Done : )
>
> On Wed, Jan 29, 2020 at 7:52 PM Tomo Suzuki 
> wrote:
>
>> HI Beam committers:
>> (Thanks, Luke!)
>>
>> Can somebody retrigger the following 2 failed checks for
>> https://github.com/apache/beam/pull/10714 ?
>> Run Java PreCommit
>> Run Java_Examples_Dataflow PreCommit
>>
>> On Wed, Jan 29, 2020 at 4:48 PM Luke Cwik  wrote:
>> >
>> > done
>> >
>> > On Wed, Jan 29, 2020 at 11:07 AM Tomo Suzuki 
>> wrote:
>> >>
>> >> Hi Beam committers,
>> >>
>> >> I appreciate if you can trigger the precommit checks for
>> >> https://github.com/apache/beam/pull/10714
>> >>
>> >> with following 6 additional commands (one command per comment):
>> >>
>> >> Run Java PostCommit
>> >> Run Java HadoopFormatIO Performance Test
>> >> Run BigQueryIO Streaming Performance Test Java
>> >> Run Dataflow ValidatesRunner
>> >> Run Spark ValidatesRunner
>> >> Run SQL Postcommit
>> >>
>> >> Regards,
>> >> Tomo
>> >>
>> >>
>>
>>
>> --
>> Regards,
>> Tomo
>>
>
>
> --
> Regards,
> Tomo
>


smime.p7s
Description: S/MIME Cryptographic Signature

Re: Release 2.19.0, release candidate #1

[Robert Bradshaw]

+1 (binding)

I validated the source tarball and all the signatures/checksums, and
also tried out a Python wheel on a fresh install with some direct
runner pipelines.

On Fri, Jan 31, 2020 at 1:44 AM Jean-Baptiste Onofré  wrote:
>
> +1 (binding)
>
> Checked quickly on beam-samples.
>
> Thanks,
> Regards
> JB
>
> On 30/01/2020 06:02, Boyuan Zhang wrote:
> > Hi everyone,
> > Please review and vote on the release candidate #1 for the version
> > 2.19.0, as follows:
> > [ ] +1, Approve the release
> > [ ] -1, Do not approve the release (please provide specific comments)
> >
> >
> > The complete staging area is available for your review, which includes:
> > * JIRA release notes [1],
> > * the official Apache source release to be deployed to dist.apache.org
> >  [2], which is signed with the key with
> > fingerprint D6BF C115 EAA6 9828 5A89  3165 99DF 385D A877 C596[3],
> > * all artifacts to be deployed to the Maven Central Repository [4],
> > * source code tag "v2.19.0-RC1" [5],
> > * website pull request listing the release [6], publishing the API
> > reference manual [7], and the blog post [8].
> > * Java artifacts were built with Maven MAVEN_VERSION and OpenJDK/Oracle
> > JDK JDK_VERSION.
> > * Python artifacts are deployed along with the source release to
> > the dist.apache.org  [2].
> > * Validation sheet with a tab for 2.18.0 release to help with validation
> > [9].
> > * Docker images published to Docker Hub [10].
> >
> > The vote will be open for at least 72 hours. It is adopted by majority
> > approval, with at least 3 PMC affirmative votes.
> >
> > Thanks,
> > Release Manager
> >
> > [1] 
> > https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12346582
> > [2] https://dist.apache.org/repos/dist/dev/beam/2.19.0/
> > 
> > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > 
> > [4] https://repository.apache.org/content/repositories/orgapachebeam-1091/
> > [5] https://github.com/apache/beam/tree/v2.19.0-RC1
> > 
> > [6] https://github.com/apache/beam/pull/10713
> > [7] https://github.com/apache/beam-site/pull/597
> > [8] https://github.com/apache/beam/pull/10722
> > [9] 
> > https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1148160512
> > [10] https://hub.docker.com/u/apachebeam
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com

New contributor to Beam documentation

[Dave Wrede]

Hi all,

I'm Dave and I recently started working on Dataflow documentation. In
addition, I will also be spending some of my time contributing to the Beam
docs, when needed. I have a strong passion for great learning experiences
and welcome feedback on how we can make Beam more approachable and useful
for all audience types.

Thanks and I look forward to working with you all!

- Dave

Re: Release 2.19.0, release candidate #1

[Valentyn Tymofieiev]

+1.
Ran 1 mobile gaming batch job and 1 streaming job on Dataflow runner under
Python 3.7 using wheels.

On Fri, Jan 31, 2020 at 10:13 AM Robert Bradshaw 
wrote:

> +1 (binding)
>
> I validated the source tarball and all the signatures/checksums, and
> also tried out a Python wheel on a fresh install with some direct
> runner pipelines.
>
> On Fri, Jan 31, 2020 at 1:44 AM Jean-Baptiste Onofré 
> wrote:
> >
> > +1 (binding)
> >
> > Checked quickly on beam-samples.
> >
> > Thanks,
> > Regards
> > JB
> >
> > On 30/01/2020 06:02, Boyuan Zhang wrote:
> > > Hi everyone,
> > > Please review and vote on the release candidate #1 for the version
> > > 2.19.0, as follows:
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide specific comments)
> > >
> > >
> > > The complete staging area is available for your review, which includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release to be deployed to dist.apache.org
> > >  [2], which is signed with the key with
> > > fingerprint D6BF C115 EAA6 9828 5A89  3165 99DF 385D A877 C596[3],
> > > * all artifacts to be deployed to the Maven Central Repository [4],
> > > * source code tag "v2.19.0-RC1" [5],
> > > * website pull request listing the release [6], publishing the API
> > > reference manual [7], and the blog post [8].
> > > * Java artifacts were built with Maven MAVEN_VERSION and OpenJDK/Oracle
> > > JDK JDK_VERSION.
> > > * Python artifacts are deployed along with the source release to
> > > the dist.apache.org  [2].
> > > * Validation sheet with a tab for 2.18.0 release to help with
> validation
> > > [9].
> > > * Docker images published to Docker Hub [10].
> > >
> > > The vote will be open for at least 72 hours. It is adopted by majority
> > > approval, with at least 3 PMC affirmative votes.
> > >
> > > Thanks,
> > > Release Manager
> > >
> > > [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12346582
> > > [2] https://dist.apache.org/repos/dist/dev/beam/2.19.0/
> > > 
> > > [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > > 
> > > [4]
> https://repository.apache.org/content/repositories/orgapachebeam-1091/
> > > [5] https://github.com/apache/beam/tree/v2.19.0-RC1
> > > 
> > > [6] https://github.com/apache/beam/pull/10713
> > > [7] https://github.com/apache/beam-site/pull/597
> > > [8] https://github.com/apache/beam/pull/10722
> > > [9]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=1148160512
> > > [10] https://hub.docker.com/u/apachebeam
> >
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
>

Re: New contributor to Beam documentation

[Kyle Weaver]

Hi Dave,

Welcome to Beam! Looking forward to working with you :)

Kyle

On Fri, Jan 31, 2020 at 11:17 AM Dave Wrede  wrote:

> Hi all,
>
> I'm Dave and I recently started working on Dataflow documentation. In
> addition, I will also be spending some of my time contributing to the Beam
> docs, when needed. I have a strong passion for great learning experiences
> and welcome feedback on how we can make Beam more approachable and useful
> for all audience types.
>
> Thanks and I look forward to working with you all!
>
> - Dave
>
>

What's up with the PreCommit_Java_Examples_Dataflow_Phrase?

[Alex Van Boxel]

It seems to be in an "terminated in state UNRECOGNIZED but did not return a
failure reason"

https://builds.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Phrase/

 _/
_/ Alex Van Boxel

Re: What's up with the PreCommit_Java_Examples_Dataflow_Phrase?

[Luke Cwik]

It has been consistently failing. Opened
https://github.com/apache/beam/pull/10738 to disable and it filed BEAM-9235

On Fri, Jan 31, 2020 at 10:26 AM Alex Van Boxel  wrote:

> It seems to be in an "terminated in state UNRECOGNIZED but did not return
> a failure reason"
>
> https://builds.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Phrase/
>
>  _/
> _/ Alex Van Boxel
>

Re: Extending the Beam IO connectors, how to best share the code

[Luke Cwik]

Since you'll handle the repository and releases, you are governed by the
licenses of your dependencies one of which is Apache Beam that has an ASF
2.0 license. The ASF 2.0 license is fairly liberal as to what you can do
and if you have specific questions you'll want to reach out to ASF directly
and/or your own legal counsel.

I believe we have a spot on the Apache Beam website and/or wiki where we
highlight community projects but where that is escapes me at the moment.

On Fri, Jan 31, 2020 at 1:50 PM Kjetil Halvorsen <
kjetil.halvor...@cognite.com> wrote:

> Hi,
>
> Having our own repository and release process is probably the easiest to
> start with? We still have a lot of changes entering the code base as we
> mature the connectors to their "v1" state. Medium/longer term, we'd be
> happy to contribute the connectors to Beam (given that it doesn't interfere
> too much with your project, of course).
>
> Best,
> Kjetil
>
> On 2020/01/30 19:22:49, Luke Cwik  wrote:
> > Are you looking to contribute your IO connectors to be part of Apache
> Beam>
> > and have the Apache Beam community build/maintain/extend/release them
> or>
> > are you looking to have your own repository and release process?>
> >
> > On Thu, Jan 30, 2020 at 11:15 AM Kjetil Halvorsen <>
> > kjetil.halvor...@cognite.com> wrote:>
> >
> > > Dear Beam dev team,>
> > >>
> > > Our company (www.cognite.com) offers software that is heavily based
> on>
> > > data I/O. As such, we find Beam a very useful framework for building
> data>
> > > pipelines that read/write to our software. We have built a set of
> Beam>
> > > connectors that we (internally per now) use as an extension to the
> Beam SDK>
> > > for building our pipelines.>
> > >>
> > > We would like to make our extensions/connectors available (for free,
> of>
> > > course) to the public. I am looking for your advice on how to best
> share>
> > > our work and stay compliant with whatever requirements the Beam
> ecosystem>
> > > has. Is it as easy as just sharing our code on Github and publish our>
> > > artifacts to Maven?>
> > >>
> > > I apologize for my lack of knowledge on your licensing requirements
> and>
> > > policies in general. We just want to make it easier to use Beam to
> connect>
> > > to our software by making our connectors available--no strings
> attached.>
> > >>
> > > Best regards,>
> > > Kjetil Halvorsen, Cognite>
> > >>
> > > -->
> > >>
> > > *Kjetil Halvorsen*>
> > > Chief Architect, Enterprise Integration>
> > >  | kjetil.halvor...@cognite.com>
> > > www.cognite.com | LIBERATE YOUR DATA™>
> > >>
> > >>
> >
>

Re: New contributor to Beam documentation

[Ahmet Altay]

Welcome!

On Fri, Jan 31, 2020 at 12:03 PM Kyle Weaver  wrote:

> Hi Dave,
>
> Welcome to Beam! Looking forward to working with you :)
>
> Kyle
>
> On Fri, Jan 31, 2020 at 11:17 AM Dave Wrede  wrote:
>
>> Hi all,
>>
>> I'm Dave and I recently started working on Dataflow documentation. In
>> addition, I will also be spending some of my time contributing to the Beam
>> docs, when needed. I have a strong passion for great learning experiences
>> and welcome feedback on how we can make Beam more approachable and useful
>> for all audience types.
>>
>> Thanks and I look forward to working with you all!
>>
>> - Dave
>>
>>

Re: [ANNOUNCE] New committer: Michał Walenia

[Ahmet Altay]

Congratulations!

On Tue, Jan 28, 2020 at 9:16 PM Connell O'Callaghan 
wrote:

> Thank you Pablo for sharing!!!
>
> Well done Michal!!!
>
> On Tue, Jan 28, 2020 at 21:07 jincheng sun 
> wrote:
>
>> Congrats Michał ！
>>
>>
>> Hannah Jiang 于2020年1月29日 周三01:43写道：
>>
>>> Congrats you Michal!
>>>
>>>
>>>
>>> On Tue, Jan 28, 2020 at 9:11 AM Gleb Kanterov  wrote:
>>>
 Congratulations!

 On Tue, Jan 28, 2020 at 6:03 PM Łukasz Gajowy 
 wrote:

> Congratulations Michał! 
>
> wt., 28 sty 2020 o 16:33 Ryan Skraba  napisał(a):
>
>> Congratulations!
>>
>> On Tue, Jan 28, 2020 at 11:26 AM Jan Lukavský 
>> wrote:
>>
>>> Congrats Michał!
>>> On 1/28/20 11:16 AM, Katarzyna Kucharczyk wrote:
>>>
>>> Congratulations Michał!  
>>>
>>> On Tue, Jan 28, 2020 at 9:29 AM Alexey Romanenko <
>>> aromanenko@gmail.com> wrote:
>>>
 Congrats, Michał!

 On 28 Jan 2020, at 09:20, Ismaël Mejía  wrote:

 Congratulations Michał, well deserved!

 On Tue, Jan 28, 2020 at 8:54 AM Kamil Wasilewski <
 kamil.wasilew...@polidea.com> wrote:

> Congrats, Michał!
>
> On Tue, Jan 28, 2020 at 3:03 AM Udi Meiri 
> wrote:
>
>> Congratulations Michał!
>>
>> On Mon, Jan 27, 2020 at 3:49 PM Chamikara Jayalath <
>> chamik...@google.com> wrote:
>>
>>> Congrats Michał!
>>>
>>> On Mon, Jan 27, 2020 at 2:59 PM Reza Rokni 
>>> wrote:
>>>
 Congratulations buddy!

 On Tue, 28 Jan 2020, 06:52 Valentyn Tymofieiev, <
 valen...@google.com> wrote:

> Congratulations, Michał!
>
> On Mon, Jan 27, 2020 at 2:24 PM Austin Bennett <
> whatwouldausti...@gmail.com> wrote:
>
>> Nice -- keep up the good work!
>>
>> On Mon, Jan 27, 2020 at 2:02 PM Mikhail Gryzykhin <
>> mig...@google.com> wrote:
>> >
>> > Congratulations Michal!
>> >
>> > --Mikhail
>> >
>> > On Mon, Jan 27, 2020 at 1:01 PM Kyle Weaver <
>> kcwea...@google.com> wrote:
>> >>
>> >> Congratulations Michał! Looking forward to your future
>> contributions :)
>> >>
>> >> Thanks,
>> >> Kyle
>> >>
>> >> On Mon, Jan 27, 2020 at 12:47 PM Pablo Estrada <
>> pabl...@google.com> wrote:
>> >>>
>> >>> Hi everyone,
>> >>>
>> >>> Please join me and the rest of the Beam PMC in welcoming
>> a new committer: Michał Walenia
>> >>>
>> >>> Michał has contributed to Beam in many ways, including
>> the performance testing infrastructure, and has even spoken at 
>> events about
>> Beam.
>> >>>
>> >>> In consideration of his contributions, the Beam PMC
>> trusts him with the responsibilities of a Beam committer[1].
>> >>>
>> >>> Thanks for your contributions Michał!
>> >>>
>> >>> Pablo, on behalf of the Apache Beam PMC.
>> >>>
>> >>> [1]
>> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>>
>
 --
>>
>> Best,
>> Jincheng
>> -
>> Twitter: https://twitter.com/sunjincheng121
>> -
>>
>

[DISCUSSION] Improve release notes by adding a change list file

[Ahmet Altay]

Hi all,

We currently have two major ways to communicate changes in a release:
- A blog post, to highlight major changes in the release. (Example for
2.17: [1])
- JIRA release notes pages listing all issues tagged for a specific
release. (Example for 2.17 [2]).

There are a few issues with this process:
- It is difficult for the release manager to know what is important, what
is a breaking change, what is dependency change etc. For example, there
were more than 150 Jira issues tagged for 2.17 release.
- Release blog has many items, and does not necessarily communicate
important changes. It is difficult for users to discover major changes
short of going through a large list.
- People involved in authoring or reviewing a PRs usually have the most
context about the change, and they are not necessarily involved in the
release process to provide this additional information.

Would it be helpful if we maintain a simple change list file and update it
as part of the PRs with noteworthy changes? Release managers could use this
information as is in their blog posts (or link to it). Users will have a
single place to find highlights from various versions.

Concretely, I am proposing:
- Adding a CHANGES file to the root of the repository. (Name could be
anything, TFX uses RELEASE.md in their repo. [3])
- Ask PR authors to update this file as part of their PR whenever it makes
sense
- Reference this file during the release process, and a new section for the
next release after each release.

Ahmet

[1] https://beam.apache.org/blog/2020/01/06/beam-2.17.0.html
[2]
https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345970=12319527
[3] https://github.com/tensorflow/tfx/blob/master/RELEASE.md

Re: [ANNOUNCE] New committer: Hannah Jiang

[Ahmet Altay]

Congratulations!

On Wed, Jan 29, 2020 at 4:50 PM Robert Bradshaw  wrote:

> Congratulations, Hannah!
>
> On Wed, Jan 29, 2020 at 3:23 PM Chamikara Jayalath 
> wrote:
>
>> Congrats Hannah!
>>
>> On Wed, Jan 29, 2020 at 9:22 AM Hannah Jiang 
>> wrote:
>>
>>> Thanks everyone!
>>> It is a very rewarding journey and I am happy to be able to achieve a
>>> mini milestone. :)
>>>
>>>
>>> On Wed, Jan 29, 2020 at 7:44 AM Cyrus Maden  wrote:
>>>
 Congrats!!

 On Wed, Jan 29, 2020 at 4:07 AM Michał Walenia <
 michal.wale...@polidea.com> wrote:

> Congratulations! :)
>
> On Wed, Jan 29, 2020 at 9:40 AM Ismaël Mejía 
> wrote:
>
>> Congrats Hannah!
>>
>> On Wed, Jan 29, 2020 at 9:35 AM Katarzyna Kucharczyk <
>> ka.kucharc...@gmail.com> wrote:
>>
>>> Congratulations, Hannah! :)
>>>
>>> On Wed, Jan 29, 2020 at 6:04 AM jincheng sun <
>>> sunjincheng...@gmail.com> wrote:
>>>
 Congrats Hannah!

 Ankur Goenka 于2020年1月29日 周三11:35写道：

> Congrats Hannah!
>
> On Tue, Jan 28, 2020 at 7:30 PM Reza Rokni  wrote:
>
>> Congratz!
>>
>> On Wed, 29 Jan 2020 at 09:52, Valentyn Tymofieiev <
>> valen...@google.com> wrote:
>>
>>> Congratulations, Hannah!
>>>
>>> On Tue, Jan 28, 2020 at 5:46 PM Udi Meiri 
>>> wrote:
>>>
 Welcome and congrats Hannah!

 On Tue, Jan 28, 2020 at 4:52 PM Robin Qiu 
 wrote:

> Congratulations, Hannah!
>
> On Tue, Jan 28, 2020 at 4:50 PM Alan Myrvold <
> amyrv...@google.com> wrote:
>
>> Congrats, Hannah
>>
>> On Tue, Jan 28, 2020 at 4:46 PM Connell O'Callaghan <
>> conne...@google.com> wrote:
>>
>>> Thank you for sharing Luke!!!
>>>
>>> Well done and congratulations Hannah!!
>>>
>>> On Tue, Jan 28, 2020 at 4:45 PM Heejong Lee <
>>> heej...@google.com> wrote:
>>>
 Congratulations! :)

 On Tue, Jan 28, 2020 at 4:43 PM Yichi Zhang <
 zyi...@google.com> wrote:

> Congrats Hannah!
>
> On Tue, Jan 28, 2020 at 3:57 PM Yifan Zou <
> yifan...@google.com> wrote:
>
>> Congratulations Hannah!!
>>
>> On Tue, Jan 28, 2020 at 3:55 PM Boyuan Zhang <
>> boyu...@google.com> wrote:
>>
>>> Thanks for all your contributions! Congratulations~
>>>
>>> On Tue, Jan 28, 2020 at 3:44 PM Pablo Estrada <
>>> pabl...@google.com> wrote:
>>>
 yoooho : D

 On Tue, Jan 28, 2020 at 3:21 PM Luke Cwik <
 lc...@google.com> wrote:

> Hi everyone,
>
> Please join me and the rest of the Beam PMC in
> welcoming a new committer: Hannah Jiang
>
> Hannah has contributed to Beam in many ways, including
> work on building and releasing the Apache Beam SDK 
> containers.
>
> In consideration of their contributions, the Beam PMC
> trusts them with the responsibilities of a Beam 
> committer[1].
>
> Thanks for your contributions Hannah!
>
> Luke, on behalf of the Apache Beam PMC.
>
> [1]
> https://beam.apache.org/contribute/become-a-committer/#an-apache-beam-committer
>

>>
>> --
>>
>> This email may be confidential and privileged. If you received
>> this communication by mistake, please don't forward it to anyone 
>> else,
>> please erase all copies and attachments, and please let me know that 
>> it has
>> gone to the wrong person.
>>
>> The above terms reflect a potential business arrangement, are
>> provided solely as a basis for further discussion, and are not 
>> intended to
>> be and do not constitute a legally binding obligation. No legally 
>> binding
>> obligations will be created, implied, or inferred until an agreement 
>> in
>> final form is executed in writing by all parties involved.
>>
> --

 Best,
 Jincheng
 -
 Twitter: 

Re: Extending the Beam IO connectors, how to best share the code

[Kjetil Halvorsen]

Hi,

Having our own repository and release process is probably the easiest to
start with? We still have a lot of changes entering the code base as we
mature the connectors to their "v1" state. Medium/longer term, we'd be
happy to contribute the connectors to Beam (given that it doesn't interfere
too much with your project, of course).

Best,
Kjetil

On 2020/01/30 19:22:49, Luke Cwik  wrote:
> Are you looking to contribute your IO connectors to be part of Apache
Beam>
> and have the Apache Beam community build/maintain/extend/release them or>
> are you looking to have your own repository and release process?>
>
> On Thu, Jan 30, 2020 at 11:15 AM Kjetil Halvorsen <>
> kjetil.halvor...@cognite.com> wrote:>
>
> > Dear Beam dev team,>
> >>
> > Our company (www.cognite.com) offers software that is heavily based on>
> > data I/O. As such, we find Beam a very useful framework for building
data>
> > pipelines that read/write to our software. We have built a set of Beam>
> > connectors that we (internally per now) use as an extension to the Beam
SDK>
> > for building our pipelines.>
> >>
> > We would like to make our extensions/connectors available (for free,
of>
> > course) to the public. I am looking for your advice on how to best
share>
> > our work and stay compliant with whatever requirements the Beam
ecosystem>
> > has. Is it as easy as just sharing our code on Github and publish our>
> > artifacts to Maven?>
> >>
> > I apologize for my lack of knowledge on your licensing requirements
and>
> > policies in general. We just want to make it easier to use Beam to
connect>
> > to our software by making our connectors available--no strings
attached.>
> >>
> > Best regards,>
> > Kjetil Halvorsen, Cognite>
> >>
> > -->
> >>
> > *Kjetil Halvorsen*>
> > Chief Architect, Enterprise Integration>
> >  | kjetil.halvor...@cognite.com>
> > www.cognite.com | LIBERATE YOUR DATA™>
> >>
> >>
>

Re: What's up with the PreCommit_Java_Examples_Dataflow_Phrase?

[Luke Cwik]

Your right, it looks like it recovered this morning.

On Fri, Jan 31, 2020 at 1:44 PM Luke Cwik  wrote:

> It has been consistently failing. Opened
> https://github.com/apache/beam/pull/10738 to disable and it filed
> BEAM-9235
>
> On Fri, Jan 31, 2020 at 10:26 AM Alex Van Boxel  wrote:
>
>> It seems to be in an "terminated in state UNRECOGNIZED but did not
>> return a failure reason"
>>
>>
>> https://builds.apache.org/job/beam_PreCommit_Java_Examples_Dataflow_Phrase/
>>
>>  _/
>> _/ Alex Van Boxel
>>
>

Re: [DISCUSSION] Improve release notes by adding a change list file

[Kenneth Knowles]

+1

This is a great idea. Hope it can lead to higher-value view of relevant
changes.

I like it being in the root of the repo, so it lives next to the code.

Since the website is also markdown, it could be copied over directly at
release time, so it can be browsed there, too.

Kenn

On Fri, Jan 31, 2020 at 3:16 PM Ahmet Altay  wrote:

> Hi all,
>
> We currently have two major ways to communicate changes in a release:
> - A blog post, to highlight major changes in the release. (Example for
> 2.17: [1])
> - JIRA release notes pages listing all issues tagged for a specific
> release. (Example for 2.17 [2]).
>
> There are a few issues with this process:
> - It is difficult for the release manager to know what is important, what
> is a breaking change, what is dependency change etc. For example, there
> were more than 150 Jira issues tagged for 2.17 release.
> - Release blog has many items, and does not necessarily communicate
> important changes. It is difficult for users to discover major changes
> short of going through a large list.
> - People involved in authoring or reviewing a PRs usually have the most
> context about the change, and they are not necessarily involved in the
> release process to provide this additional information.
>
> Would it be helpful if we maintain a simple change list file and update it
> as part of the PRs with noteworthy changes? Release managers could use this
> information as is in their blog posts (or link to it). Users will have a
> single place to find highlights from various versions.
>
> Concretely, I am proposing:
> - Adding a CHANGES file to the root of the repository. (Name could be
> anything, TFX uses RELEASE.md in their repo. [3])
> - Ask PR authors to update this file as part of their PR whenever it makes
> sense
> - Reference this file during the release process, and a new section for
> the next release after each release.
>
> Ahmet
>
> [1] https://beam.apache.org/blog/2020/01/06/beam-2.17.0.html
> [2]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345970=12319527
> [3] https://github.com/tensorflow/tfx/blob/master/RELEASE.md
>

Re: [DISCUSSION] Improve release notes by adding a change list file

[Robert Bradshaw]

Yes, yes, yes! This is the one model of release notes that I've
actually seen work well at scale.

https://lists.apache.org/thread.html/41e03ace17dbcccf7e267ba6d538736b2a99a8e73e7fb45702766b17%40%3Cdev.beam.apache.org%3E

Let's make it happen.

On Fri, Jan 31, 2020 at 3:47 PM Robert Burke  wrote:
>
> I like this suggestion, Jira titles and commit summaries don't necessarily 
> reflect the user impact for a given change (or set of changes). Being able to 
> see the Forest instead of the trees.
>
> On Fri, Jan 31, 2020, 3:37 PM Kenneth Knowles  wrote:
>>
>> +1
>>
>> This is a great idea. Hope it can lead to higher-value view of relevant 
>> changes.
>>
>> I like it being in the root of the repo, so it lives next to the code.
>>
>> Since the website is also markdown, it could be copied over directly at 
>> release time, so it can be browsed there, too.
>>
>> Kenn
>>
>> On Fri, Jan 31, 2020 at 3:16 PM Ahmet Altay  wrote:
>>>
>>> Hi all,
>>>
>>> We currently have two major ways to communicate changes in a release:
>>> - A blog post, to highlight major changes in the release. (Example for 
>>> 2.17: [1])
>>> - JIRA release notes pages listing all issues tagged for a specific 
>>> release. (Example for 2.17 [2]).
>>>
>>> There are a few issues with this process:
>>> - It is difficult for the release manager to know what is important, what 
>>> is a breaking change, what is dependency change etc. For example, there 
>>> were more than 150 Jira issues tagged for 2.17 release.
>>> - Release blog has many items, and does not necessarily communicate 
>>> important changes. It is difficult for users to discover major changes 
>>> short of going through a large list.
>>> - People involved in authoring or reviewing a PRs usually have the most 
>>> context about the change, and they are not necessarily involved in the 
>>> release process to provide this additional information.
>>>
>>> Would it be helpful if we maintain a simple change list file and update it 
>>> as part of the PRs with noteworthy changes? Release managers could use this 
>>> information as is in their blog posts (or link to it). Users will have a 
>>> single place to find highlights from various versions.
>>>
>>> Concretely, I am proposing:
>>> - Adding a CHANGES file to the root of the repository. (Name could be 
>>> anything, TFX uses RELEASE.md in their repo. [3])
>>> - Ask PR authors to update this file as part of their PR whenever it makes 
>>> sense
>>> - Reference this file during the release process, and a new section for the 
>>> next release after each release.
>>>
>>> Ahmet
>>>
>>> [1] https://beam.apache.org/blog/2020/01/06/beam-2.17.0.html
>>> [2] 
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345970=12319527
>>> [3] https://github.com/tensorflow/tfx/blob/master/RELEASE.md

Re: [DISCUSSION] Improve release notes by adding a change list file

[Ahmet Altay]

Thank you for the quick responses. I sent out
https://github.com/apache/beam/pull/10743 to make this change. Please
provide feedback or directly edit the PR.

On Fri, Jan 31, 2020 at 3:58 PM Robert Bradshaw  wrote:

> Yes, yes, yes! This is the one model of release notes that I've
> actually seen work well at scale.
>
>
> https://lists.apache.org/thread.html/41e03ace17dbcccf7e267ba6d538736b2a99a8e73e7fb45702766b17%40%3Cdev.beam.apache.org%3E
>
> Let's make it happen.
>
> On Fri, Jan 31, 2020 at 3:47 PM Robert Burke  wrote:
> >
> > I like this suggestion, Jira titles and commit summaries don't
> necessarily reflect the user impact for a given change (or set of changes).
> Being able to see the Forest instead of the trees.
> >
> > On Fri, Jan 31, 2020, 3:37 PM Kenneth Knowles  wrote:
> >>
> >> +1
> >>
> >> This is a great idea. Hope it can lead to higher-value view of relevant
> changes.
> >>
> >> I like it being in the root of the repo, so it lives next to the code.
> >>
> >> Since the website is also markdown, it could be copied over directly at
> release time, so it can be browsed there, too.
> >>
> >> Kenn
> >>
> >> On Fri, Jan 31, 2020 at 3:16 PM Ahmet Altay  wrote:
> >>>
> >>> Hi all,
> >>>
> >>> We currently have two major ways to communicate changes in a release:
> >>> - A blog post, to highlight major changes in the release. (Example for
> 2.17: [1])
> >>> - JIRA release notes pages listing all issues tagged for a specific
> release. (Example for 2.17 [2]).
> >>>
> >>> There are a few issues with this process:
> >>> - It is difficult for the release manager to know what is important,
> what is a breaking change, what is dependency change etc. For example,
> there were more than 150 Jira issues tagged for 2.17 release.
> >>> - Release blog has many items, and does not necessarily communicate
> important changes. It is difficult for users to discover major changes
> short of going through a large list.
> >>> - People involved in authoring or reviewing a PRs usually have the
> most context about the change, and they are not necessarily involved in the
> release process to provide this additional information.
> >>>
> >>> Would it be helpful if we maintain a simple change list file and
> update it as part of the PRs with noteworthy changes? Release managers
> could use this information as is in their blog posts (or link to it). Users
> will have a single place to find highlights from various versions.
> >>>
> >>> Concretely, I am proposing:
> >>> - Adding a CHANGES file to the root of the repository. (Name could be
> anything, TFX uses RELEASE.md in their repo. [3])
> >>> - Ask PR authors to update this file as part of their PR whenever it
> makes sense
> >>> - Reference this file during the release process, and a new section
> for the next release after each release.
> >>>
> >>> Ahmet
> >>>
> >>> [1] https://beam.apache.org/blog/2020/01/06/beam-2.17.0.html
> >>> [2]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345970=12319527
> >>> [3] https://github.com/tensorflow/tfx/blob/master/RELEASE.md
>

Re: [DISCUSSION] Improve release notes by adding a change list file

[Robert Burke]

I like this suggestion, Jira titles and commit summaries don't necessarily
reflect the user impact for a given change (or set of changes). Being able
to see the Forest instead of the trees.

On Fri, Jan 31, 2020, 3:37 PM Kenneth Knowles  wrote:

> +1
>
> This is a great idea. Hope it can lead to higher-value view of relevant
> changes.
>
> I like it being in the root of the repo, so it lives next to the code.
>
> Since the website is also markdown, it could be copied over directly at
> release time, so it can be browsed there, too.
>
> Kenn
>
> On Fri, Jan 31, 2020 at 3:16 PM Ahmet Altay  wrote:
>
>> Hi all,
>>
>> We currently have two major ways to communicate changes in a release:
>> - A blog post, to highlight major changes in the release. (Example for
>> 2.17: [1])
>> - JIRA release notes pages listing all issues tagged for a specific
>> release. (Example for 2.17 [2]).
>>
>> There are a few issues with this process:
>> - It is difficult for the release manager to know what is important, what
>> is a breaking change, what is dependency change etc. For example, there
>> were more than 150 Jira issues tagged for 2.17 release.
>> - Release blog has many items, and does not necessarily communicate
>> important changes. It is difficult for users to discover major changes
>> short of going through a large list.
>> - People involved in authoring or reviewing a PRs usually have the most
>> context about the change, and they are not necessarily involved in the
>> release process to provide this additional information.
>>
>> Would it be helpful if we maintain a simple change list file and update
>> it as part of the PRs with noteworthy changes? Release managers could use
>> this information as is in their blog posts (or link to it). Users will have
>> a single place to find highlights from various versions.
>>
>> Concretely, I am proposing:
>> - Adding a CHANGES file to the root of the repository. (Name could be
>> anything, TFX uses RELEASE.md in their repo. [3])
>> - Ask PR authors to update this file as part of their PR whenever it
>> makes sense
>> - Reference this file during the release process, and a new section for
>> the next release after each release.
>>
>> Ahmet
>>
>> [1] https://beam.apache.org/blog/2020/01/06/beam-2.17.0.html
>> [2]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12345970=12319527
>> [3] https://github.com/tensorflow/tfx/blob/master/RELEASE.md
>>
>

Re: [DISCUSS] BIP reloaded

[Kenneth Knowles]

These stages sound like a great starting point to me. Would you be the
volunteer to set up a cwiki page for BIPs?

Kenn

On Mon, Jan 20, 2020 at 3:30 AM Jan Lukavský  wrote:

> I agree that we can take inspiration from other projects. Besides the
> organizational part (what should be part of BIP, where to store it, how to
> edit it and how to make it available to the whole community) - for which
> the cwiki might be the best option - there are still open questions about
> the lifecycle of a BIP:
>
>  a) when to create one?
>
>   - I feel this might be optional, there might be some upper bound of
> features that are really "too big" or "too controversial", so these should
> undergo the BIP process in all cases, otherwise the decision might be part
> of the proposal, the question is how to define those
>
>  b) what are lifecycle stages of a BIP? How to do transitions between
> these stages?
>
>   - From the top of my head this might be:
>
> a) proposal -- not yet accepted
>
> b) accepted -- most probably after vote?
>
> c) in progress -- having assigned JIRA and being worked on
>
> d) done -- after merge to master
>
> e) released -- obvious
>
> WDYT?
>
>  Jan
> On 1/15/20 8:19 PM, Kenneth Knowles wrote:
>
> Focusing this thread on the BIP process seems wise, without changing much
> else in the same thread. I don't think the BIP process has to do with
> exactly how design docs are written or archived, but the ability to *at a
> glance* understand:
>
>  - what are the high level proposals
>  - status of the proposals
>  - who to contact
>  - how to get to more info (links to design docs, thread, Jiras, etc)
>
> A page with a table on cwiki is common and seems good for this. How we
> manage such a table would be a possible next step. I think they should
> focus on large changes that need heavyweight process, so should encourage
> lightweight creation. I think adding heavy process to smaller changes would
> be bad.
>
> 
>
> I have looked multiple times at other projects (linked in prior thread and
> in this thread too but gathering them here)
>
> Spark: https://spark.apache.org/improvement-proposals.html
>  - Jira is not good for "at a glance" reading. The title should have a
> short and easy to understand paragraph.
>
> Kafka:
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+Improvement+Proposals
>  - Quite a lot of content; I would prefer 10s of proposals. But it is
> readable enough. Table lacks important content like links and summaries.
>  - Blends the table with a bunch of header material that IMO ets in the way
>
> Flink:
> https://cwiki.apache.org/confluence/display/FLINK/Flink+Improvement+Proposals
>  - Looks very similar to Kafka
>  - Target Release is too specific, and actual status is missing
>
> Airflow:
> https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals
>  - seems best organized, and the table has more info
>  - having sections for the different status proposals in different tables
> is great
>  - "InRelease" column is left blank
>
> It seems there is a lot of redundancy with Jira fields - owner, release,
> etc. I think that redundancy is good. If it is too much effort to
> redundantly manage to write it in the table then it probably is not
> appropriate for heavyweight process. Anything that is one simple task that
> fits in a Jira that can be passed around from person to person shouldn't be
> a BIP. Probably anything where we can guess the target version isn't big
> enough for a BIP.
>
> Kenn
>
> On Thu, Jan 9, 2020 at 7:59 AM Jan Lukavský  wrote:
>
>> I think that, besides ownership of a feature, a BIP (or whatever document
>> or process) should contain the following:
>>
>>  * description of a problem that the improvement addresses  - this is
>> currently often part of design doc
>>
>>  * description of multiple possible solutions (if multiple exist, which
>> is probably mostly the case)
>>
>>  * justifying choice of a particular solution
>>
>>  * result of a vote - the vote should cover both (a) do we don't this
>> feature in the first place and (b) do we accept the proposed solution
>>
>> This would probably be iterative process involving multiple people,
>> mailing list communication, etc. Pretty much what we do now, just there
>> would be a place to keep track of decisions made throughout the process. I
>> pretty much think that voting on complicated solutions is vital, the soft
>> consensus approach is good for "simple" features (what that means might be
>> subjective), but might fail for features where multiple more or less
>> complex solutions exist. After successful PMC vote, the problem simplifies
>> to reviewing code, the reviewer doesn't have to think about "do we want
>> this feature?". That is given in advance. After we agree on the process and
>> the form it should have I can volunteer to test it by letting proposal of
>> ordered stateful processing pass through it.
>> On 1/9/20 9:11 AM, Alex Van Boxel wrote:
>>
>>