Re: Broken links to releases

2018-06-19 Thread Jean-Baptiste Onofré 
No, that's not the same storage area. So, latest release is on
dist.apache.org (faster) and previous releases on archive.apache.org.

Regards
JB

On 19/06/2018 23:25, Robert Bradshaw wrote:
> In that case, should we always be linking to the archive? Or is it
> standard practice to change your links when things get archived? (You
> can't, of course, change links to announcement emails and it's hard to
> hunt down all blog posts, etc. so I'd much rather have stable release
> pointers from the start.)
> 
> Whatever we decide to do, we should update the release guide. 
> 
> On Tue, Jun 19, 2018 at 2:22 PM Thomas Weise  > wrote:
> 
> It is expected that old releases are removed and
> archived:  http://www.apache.org/legal/release-policy.html#when-to-archive
> 
> Old releases are available from http://archive.apache.org/dist/beam/
> 
> 
> On Tue, Jun 19, 2018 at 11:12 PM, Dan Halperin  > wrote:
> 
> Looks like JB removed these during release of 2.3.0:
> 
> svn commit: r25111 - in /release/beam: ./ 0.1.0-incubating/
> 0.2.0-incubating/ 0.3.0-incubating/ 0.4.0/ 0.5.0/ 0.6.0/
> 2.0.0/ 2.1.0/ 2.1.1/ 2.2.0/ 2.3.0/
> 
> Author: jbonofre
> Date: Sat Feb 17 06:08:19 2018
> New Revision: 25111
> 
> Log:
> Publish 2.3.0 release
> 
> Added:
>     release/beam/2.3.0/
>     release/beam/2.3.0/apache-beam-2.3.0-python.zip   (with
> props)
>     release/beam/2.3.0/apache-beam-2.3.0-python.zip.asc
>     release/beam/2.3.0/apache-beam-2.3.0-python.zip.md5
>     release/beam/2.3.0/apache-beam-2.3.0-python.zip.sha1
>     release/beam/2.3.0/apache-beam-2.3.0-source-release.zip 
>  (with props)
>     release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.asc
>     release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.md5
>     release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.sha1
>     release/beam/latest   (with props)
> Removed:
>     release/beam/0.1.0-incubating/
>     release/beam/0.2.0-incubating/
>     release/beam/0.3.0-incubating/
>     release/beam/0.4.0/
>     release/beam/0.5.0/
>     release/beam/0.6.0/
>     release/beam/2.0.0/
>     release/beam/2.1.0/
>     release/beam/2.1.1/
>     release/beam/2.2.0/
> 
> 
> I don't think this is the intended release process -- don't we
> want them up forever?
> 
> Suggest partial revert of that SVN commit to restore the old
> releases.
> 
> On Tue, Jun 19, 2018 at 1:39 PM Chamikara Jayalath
> mailto:chamik...@google.com>> wrote:
> 
> I noticed that links to source code of releases is broken in
> the following page for everything but the latest release. 
> 
> https://beam.apache.org/get-started/downloads/
> 
> For
> example 
> http://www-eu.apache.org/dist/beam/2.3.0/apache-beam-2.3.0-source-release.zip
> 
> Anybody know how to fix this ?
> 
> - Cham
> 
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Broken links to releases

2018-06-19 Thread Jean-Baptiste Onofré 
That's normal: dist.apache.org should contain only the latest release.
The previous releases are automatically available on archive.apache.org.

So, the website should use archive.apache.org for old release instead of
dist.apache.org.

Regards
JB

On 19/06/2018 23:12, Dan Halperin wrote:
> Looks like JB removed these during release of 2.3.0:
> 
> svn commit: r25111 - in /release/beam: ./ 0.1.0-incubating/
> 0.2.0-incubating/ 0.3.0-incubating/ 0.4.0/ 0.5.0/ 0.6.0/ 2.0.0/
> 2.1.0/ 2.1.1/ 2.2.0/ 2.3.0/
> 
> Author: jbonofre
> Date: Sat Feb 17 06:08:19 2018
> New Revision: 25111
> 
> Log:
> Publish 2.3.0 release
> 
> Added:
>     release/beam/2.3.0/
>     release/beam/2.3.0/apache-beam-2.3.0-python.zip   (with props)
>     release/beam/2.3.0/apache-beam-2.3.0-python.zip.asc
>     release/beam/2.3.0/apache-beam-2.3.0-python.zip.md5
>     release/beam/2.3.0/apache-beam-2.3.0-python.zip.sha1
>     release/beam/2.3.0/apache-beam-2.3.0-source-release.zip   (with
> props)
>     release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.asc
>     release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.md5
>     release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.sha1
>     release/beam/latest   (with props)
> Removed:
>     release/beam/0.1.0-incubating/
>     release/beam/0.2.0-incubating/
>     release/beam/0.3.0-incubating/
>     release/beam/0.4.0/
>     release/beam/0.5.0/
>     release/beam/0.6.0/
>     release/beam/2.0.0/
>     release/beam/2.1.0/
>     release/beam/2.1.1/
>     release/beam/2.2.0/
> 
> 
> I don't think this is the intended release process -- don't we want them
> up forever?
> 
> Suggest partial revert of that SVN commit to restore the old releases.
> 
> On Tue, Jun 19, 2018 at 1:39 PM Chamikara Jayalath  > wrote:
> 
> I noticed that links to source code of releases is broken in the
> following page for everything but the latest release. 
> 
> https://beam.apache.org/get-started/downloads/
> 
> For
> example 
> http://www-eu.apache.org/dist/beam/2.3.0/apache-beam-2.3.0-source-release.zip
> 
> Anybody know how to fix this ?
> 
> - Cham
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #2

2018-06-19 Thread Jean-Baptiste Onofré 
+1 (binding)

Regards
JB

On 17/06/2018 07:18, Jean-Baptiste Onofré wrote:
> Hi everyone,
> 
> Please review and vote on the release candidate #2 for the version
> 2.5.0, as follows:
> 
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
> 
> NB: this is the first release using Gradle, so don't be too harsh ;) A
> PR about the release guide will follow thanks to this release.
> 
> The complete staging area is available for your review, which includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
> [2], which is signed with the key with fingerprint C8282E76 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.5.0-RC2" [5],
> * website pull request listing the release and publishing the API
> reference manual [6].
> * Java artifacts were built with Gradle 4.7 (wrapper) and OpenJDK/Oracle
> JDK 1.8.0_172 (Oracle Corporation 25.172-b11).
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
> 
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
> 
> Thanks,
> JB
> 
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12342847
> [2] https://dist.apache.org/repos/dist/dev/beam/2.5.0/
> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1043/
> [5] https://github.com/apache/beam/tree/v2.5.0-RC2
> [6] https://github.com/apache/beam-site/pull/463
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Where is the mapreduce runner?

2018-06-19 Thread Kenneth Knowles 
The MapReduce runner is still on a feature branch. See
https://beam.apache.org/contribute/#mapreduce-runner

We should remove unreleased runners from the capability matrix.

May I ask how you arrived at
https://beam.apache.org/documentation/runners/mapreduce/ ?

Kenn

On Tue, Jun 19, 2018 at 8:36 PM Reminia Scarlet 
wrote:

> I've found that mapreduce runner is supported  in beam from below links:
>
> https://beam.apache.org/documentation/runners/mapreduce/
>
> https://beam.apache.org/documentation/runners/capability-matrix/
>
> But I cant find the runner on maven.
>

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #2

2018-06-19 Thread Yifan Zou 
follow up the previous email. The automated validation test verified the
wheel below:

apache_beam-2.5.0-cp27-cp27mu-manylinux1_x86_64.whl



On Tue, Jun 19, 2018 at 5:11 PM Yifan Zou  wrote:

> +1
> Verified Python wheels. They are good.
> https://builds.apache.org/job/beam_PostRelease_Python_Candidate/117/
>
>
> On Tue, Jun 19, 2018 at 10:32 AM Udi Meiri  wrote:
>
>> +1
>>
>> On Mon, Jun 18, 2018 at 6:16 PM Boyuan Zhang  wrote:
>>
>>> Follow up with pervious email( sorry for the inconvenience).
>>>
>>> Hey JB,
>>>
>>> If you haven't built python wheels yet, then you can use files in the
>>> previous email to move forward. If you have done all of them, then please
>>> ignore previous email.
>>>
>>> Thanks so much for your help.
>>>
>>> Boyuan
>>>
>>> On Mon, Jun 18, 2018 at 6:08 PM Boyuan Zhang  wrote:
>>>
 Hey JB,

 I built python wheels and you can get them here:
 https://storage.googleapis.com/python_wheels/python_wheels.zip. You
 probably need to sign them and upload them into
 https://dist.apache.org/repos/dist/dev/beam/2.5.0/ like what we did
 for 2.4.0: https://dist.apache.org/repos/dist/dev/beam/2.4.0/ . After
 you upload, we can do more validations on python wheels.

 Boyuan

 On Mon, Jun 18, 2018 at 4:49 PM Pablo Estrada 
 wrote:

> +0.9
>
> I've run the Mobile Gaming examples in Java+Dataflow, and others have
> documented their tests in the spreadsheet[1]. All seems to work well.
> We will need the wheel files to be available, and run a few
> validations on them, but other than that, the release looks good to me.
>
> Best
> -P.
>
> [1]
> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807
>
> On Mon, Jun 18, 2018 at 10:08 AM Jean-Baptiste Onofré 
> wrote:
>
>> Hi,
>>
>> I thought I did it, let me double check.
>>
>> Regards
>> JB
>>
>> On 18/06/2018 19:01, Boyuan Zhang wrote:
>> > Hey JB,
>> >
>> > Could you please also build python wheels for RC2 and stage them in
>> > https://dist.apache.org/repos/dist/dev/beam/2.5.0/? There are some
>> > instructions in this PR:
>> https://github.com/apache/beam-site/pull/467.
>> > Hope it can be helpful.
>> >
>> > Thanks.
>> > Boyuan
>> >
>> > On Sun, Jun 17, 2018 at 9:22 PM Jean-Baptiste Onofré <
>> j...@nanthrax.net
>> > > wrote:
>> >
>> > s/add/had/g
>> >
>> > On 17/06/2018 19:32, Pablo Estrada wrote:
>> > > Going to https://github.com/apache/beam/tree/v2.5.0-RC2 gives
>> me a 404
>> > > error. May need to push the label?
>> > > Best
>> > > -P.
>> > >
>> > > On Sun, Jun 17, 2018 at 1:15 AM Romain Manni-Bucau
>> > > mailto:rmannibu...@gmail.com>
>> > >>
>> wrote:
>> > >
>> > > +1 (same tests than for take #1)
>> > >
>> > > Le dim. 17 juin 2018 07:18, Jean-Baptiste Onofré
>> > mailto:j...@nanthrax.net>
>> > > >> a
>> écrit :
>> > >
>> > > Hi everyone,
>> > >
>> > > Please review and vote on the release candidate #2
>> for the
>> > version
>> > > 2.5.0, as follows:
>> > >
>> > > [ ] +1, Approve the release
>> > > [ ] -1, Do not approve the release (please provide
>> specific
>> > > comments)
>> > >
>> > > NB: this is the first release using Gradle, so don't
>> be too
>> > > harsh ;) A
>> > > PR about the release guide will follow thanks to this
>> release.
>> > >
>> > > The complete staging area is available for your
>> review, which
>> > > includes:
>> > > * JIRA release notes [1],
>> > > * the official Apache source release to be deployed to
>> > > dist.apache.org 
>> > 
>> > > [2], which is signed with the key with fingerprint
>> > C8282E76 [3],
>> > > * all artifacts to be deployed to the Maven Central
>> > Repository [4],
>> > > * source code tag "v2.5.0-RC2" [5],
>> > > * website pull request listing the release and
>> publishing
>> > the API
>> > > reference manual [6].
>> > > * Java artifacts were built with Gradle 4.7 (wrapper)
>> and
>> > > OpenJDK/Oracle
>> > > JDK 1.8.0_172 (Oracle Corporation 25.172-b11).
>> > > * Python artifacts are deployed along with the source
>> >

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #2

2018-06-19 Thread Yifan Zou 
+1
Verified Python wheels. They are good.
https://builds.apache.org/job/beam_PostRelease_Python_Candidate/117/


On Tue, Jun 19, 2018 at 10:32 AM Udi Meiri  wrote:

> +1
>
> On Mon, Jun 18, 2018 at 6:16 PM Boyuan Zhang  wrote:
>
>> Follow up with pervious email( sorry for the inconvenience).
>>
>> Hey JB,
>>
>> If you haven't built python wheels yet, then you can use files in the
>> previous email to move forward. If you have done all of them, then please
>> ignore previous email.
>>
>> Thanks so much for your help.
>>
>> Boyuan
>>
>> On Mon, Jun 18, 2018 at 6:08 PM Boyuan Zhang  wrote:
>>
>>> Hey JB,
>>>
>>> I built python wheels and you can get them here:
>>> https://storage.googleapis.com/python_wheels/python_wheels.zip. You
>>> probably need to sign them and upload them into
>>> https://dist.apache.org/repos/dist/dev/beam/2.5.0/ like what we did for
>>> 2.4.0: https://dist.apache.org/repos/dist/dev/beam/2.4.0/ . After you
>>> upload, we can do more validations on python wheels.
>>>
>>> Boyuan
>>>
>>> On Mon, Jun 18, 2018 at 4:49 PM Pablo Estrada 
>>> wrote:
>>>
 +0.9

 I've run the Mobile Gaming examples in Java+Dataflow, and others have
 documented their tests in the spreadsheet[1]. All seems to work well.
 We will need the wheel files to be available, and run a few validations
 on them, but other than that, the release looks good to me.

 Best
 -P.

 [1]
 https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807

 On Mon, Jun 18, 2018 at 10:08 AM Jean-Baptiste Onofré 
 wrote:

> Hi,
>
> I thought I did it, let me double check.
>
> Regards
> JB
>
> On 18/06/2018 19:01, Boyuan Zhang wrote:
> > Hey JB,
> >
> > Could you please also build python wheels for RC2 and stage them in
> > https://dist.apache.org/repos/dist/dev/beam/2.5.0/? There are some
> > instructions in this PR:
> https://github.com/apache/beam-site/pull/467.
> > Hope it can be helpful.
> >
> > Thanks.
> > Boyuan
> >
> > On Sun, Jun 17, 2018 at 9:22 PM Jean-Baptiste Onofré <
> j...@nanthrax.net
> > > wrote:
> >
> > s/add/had/g
> >
> > On 17/06/2018 19:32, Pablo Estrada wrote:
> > > Going to https://github.com/apache/beam/tree/v2.5.0-RC2 gives
> me a 404
> > > error. May need to push the label?
> > > Best
> > > -P.
> > >
> > > On Sun, Jun 17, 2018 at 1:15 AM Romain Manni-Bucau
> > > mailto:rmannibu...@gmail.com>
> > >>
> wrote:
> > >
> > > +1 (same tests than for take #1)
> > >
> > > Le dim. 17 juin 2018 07:18, Jean-Baptiste Onofré
> > mailto:j...@nanthrax.net>
> > > >> a
> écrit :
> > >
> > > Hi everyone,
> > >
> > > Please review and vote on the release candidate #2 for
> the
> > version
> > > 2.5.0, as follows:
> > >
> > > [ ] +1, Approve the release
> > > [ ] -1, Do not approve the release (please provide
> specific
> > > comments)
> > >
> > > NB: this is the first release using Gradle, so don't
> be too
> > > harsh ;) A
> > > PR about the release guide will follow thanks to this
> release.
> > >
> > > The complete staging area is available for your
> review, which
> > > includes:
> > > * JIRA release notes [1],
> > > * the official Apache source release to be deployed to
> > > dist.apache.org 
> > 
> > > [2], which is signed with the key with fingerprint
> > C8282E76 [3],
> > > * all artifacts to be deployed to the Maven Central
> > Repository [4],
> > > * source code tag "v2.5.0-RC2" [5],
> > > * website pull request listing the release and
> publishing
> > the API
> > > reference manual [6].
> > > * Java artifacts were built with Gradle 4.7 (wrapper)
> and
> > > OpenJDK/Oracle
> > > JDK 1.8.0_172 (Oracle Corporation 25.172-b11).
> > > * Python artifacts are deployed along with the source
> > release to the
> > > dist.apache.org 
> >  [2].
> > >
> > > The vote will be open for at least 72 hours. It is
> adopted by
> > > majority
> > > approval, with at least 3 PMC affirmative votes.
> > >

Re: Broken links to releases

2018-06-19 Thread Chamikara Jayalath 
Thanks. Sent https://github.com/apache/beam-site/pull/473 fixing currently
broken links.

- Cham

On Tue, Jun 19, 2018 at 2:26 PM Robert Bradshaw  wrote:

> In that case, should we always be linking to the archive? Or is it
> standard practice to change your links when things get archived? (You
> can't, of course, change links to announcement emails and it's hard to hunt
> down all blog posts, etc. so I'd much rather have stable release pointers
> from the start.)
>
> Whatever we decide to do, we should update the release guide.
>
> On Tue, Jun 19, 2018 at 2:22 PM Thomas Weise  wrote:
>
>> It is expected that old releases are removed and archived:
>> http://www.apache.org/legal/release-policy.html#when-to-archive
>>
>> Old releases are available from http://archive.apache.org/dist/beam/
>>
>>
>> On Tue, Jun 19, 2018 at 11:12 PM, Dan Halperin 
>> wrote:
>>
>>> Looks like JB removed these during release of 2.3.0:
>>>
>>> svn commit: r25111 - in /release/beam: ./ 0.1.0-incubating/
>>> 0.2.0-incubating/ 0.3.0-incubating/ 0.4.0/ 0.5.0/ 0.6.0/ 2.0.0/ 2.1.0/
>>> 2.1.1/ 2.2.0/ 2.3.0/
>>>
>>> Author: jbonofre
>>> Date: Sat Feb 17 06:08:19 2018
>>> New Revision: 25111
>>>
>>> Log:
>>> Publish 2.3.0 release
>>>
>>> Added:
>>> release/beam/2.3.0/
>>> release/beam/2.3.0/apache-beam-2.3.0-python.zip   (with props)
>>> release/beam/2.3.0/apache-beam-2.3.0-python.zip.asc
>>> release/beam/2.3.0/apache-beam-2.3.0-python.zip.md5
>>> release/beam/2.3.0/apache-beam-2.3.0-python.zip.sha1
>>> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip   (with
>>> props)
>>> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.asc
>>> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.md5
>>> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.sha1
>>> release/beam/latest   (with props)
>>> Removed:
>>> release/beam/0.1.0-incubating/
>>> release/beam/0.2.0-incubating/
>>> release/beam/0.3.0-incubating/
>>> release/beam/0.4.0/
>>> release/beam/0.5.0/
>>> release/beam/0.6.0/
>>> release/beam/2.0.0/
>>> release/beam/2.1.0/
>>> release/beam/2.1.1/
>>> release/beam/2.2.0/
>>>
>>>
>>> I don't think this is the intended release process -- don't we want them
>>> up forever?
>>>
>>> Suggest partial revert of that SVN commit to restore the old releases.
>>>
>>> On Tue, Jun 19, 2018 at 1:39 PM Chamikara Jayalath 
>>> wrote:
>>>
 I noticed that links to source code of releases is broken in the
 following page for everything but the latest release.

 https://beam.apache.org/get-started/downloads/

 For example
 http://www-eu.apache.org/dist/beam/2.3.0/apache-beam-2.3.0-source-release.zip

 Anybody know how to fix this ?

 - Cham

>>>
>>

Re: Suggestions for Building Python Wheels

2018-06-19 Thread Boyuan Zhang 
Sorry for the inconvenience.
Attach link again:
https://docs.google.com/document/d/1HHnUkiFmwAVQwMJ-BjAX3LOmkC44Gb3jvGIJvSMC84s/edit?usp=sharing

On Tue, Jun 19, 2018 at 3:58 PM Austin Bennett 
wrote:

> Hi Boyuan,
>
> Doc is in view only mode.
>
> Best,
> Austin
>
>
>
> On Tue, Jun 19, 2018 at 3:45 PM, Boyuan Zhang  wrote:
>
>> Hey all,
>>
>> I'm Boyuan and I'm currently working on release process. We began to
>> release python wheels since last release so we wanna make this process
>> as easy as possible. This doc
>> 
>> describes how we built wheels for 2.4.0 and 2.5.0. Also we need some more
>> help especially for discussions section from you. Please feel free to
>> comment in the doc or ping this thread to help us make this process better.
>>
>> Thanks so much for your help!
>>
>> Boyuan
>>
>
>

Re: Suggestions for Building Python Wheels

2018-06-19 Thread Austin Bennett 
Hi Boyuan,

Doc is in view only mode.

Best,
Austin



On Tue, Jun 19, 2018 at 3:45 PM, Boyuan Zhang  wrote:

> Hey all,
>
> I'm Boyuan and I'm currently working on release process. We began to
> release python wheels since last release so we wanna make this process
> as easy as possible. This doc
> 
> describes how we built wheels for 2.4.0 and 2.5.0. Also we need some more
> help especially for discussions section from you. Please feel free to
> comment in the doc or ping this thread to help us make this process better.
>
> Thanks so much for your help!
>
> Boyuan
>

Suggestions for Building Python Wheels

2018-06-19 Thread Boyuan Zhang 
Hey all,

I'm Boyuan and I'm currently working on release process. We began to
release python wheels since last release so we wanna make this process
as easy as possible. This doc

describes how we built wheels for 2.4.0 and 2.5.0. Also we need some more
help especially for discussions section from you. Please feel free to
comment in the doc or ping this thread to help us make this process better.

Thanks so much for your help!

Boyuan

Re: Broken links to releases

2018-06-19 Thread Robert Bradshaw 
In that case, should we always be linking to the archive? Or is it standard
practice to change your links when things get archived? (You can't, of
course, change links to announcement emails and it's hard to hunt down all
blog posts, etc. so I'd much rather have stable release pointers from the
start.)

Whatever we decide to do, we should update the release guide.

On Tue, Jun 19, 2018 at 2:22 PM Thomas Weise  wrote:

> It is expected that old releases are removed and archived:
> http://www.apache.org/legal/release-policy.html#when-to-archive
>
> Old releases are available from http://archive.apache.org/dist/beam/
>
>
> On Tue, Jun 19, 2018 at 11:12 PM, Dan Halperin 
> wrote:
>
>> Looks like JB removed these during release of 2.3.0:
>>
>> svn commit: r25111 - in /release/beam: ./ 0.1.0-incubating/
>> 0.2.0-incubating/ 0.3.0-incubating/ 0.4.0/ 0.5.0/ 0.6.0/ 2.0.0/ 2.1.0/
>> 2.1.1/ 2.2.0/ 2.3.0/
>>
>> Author: jbonofre
>> Date: Sat Feb 17 06:08:19 2018
>> New Revision: 25111
>>
>> Log:
>> Publish 2.3.0 release
>>
>> Added:
>> release/beam/2.3.0/
>> release/beam/2.3.0/apache-beam-2.3.0-python.zip   (with props)
>> release/beam/2.3.0/apache-beam-2.3.0-python.zip.asc
>> release/beam/2.3.0/apache-beam-2.3.0-python.zip.md5
>> release/beam/2.3.0/apache-beam-2.3.0-python.zip.sha1
>> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip   (with props)
>> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.asc
>> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.md5
>> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.sha1
>> release/beam/latest   (with props)
>> Removed:
>> release/beam/0.1.0-incubating/
>> release/beam/0.2.0-incubating/
>> release/beam/0.3.0-incubating/
>> release/beam/0.4.0/
>> release/beam/0.5.0/
>> release/beam/0.6.0/
>> release/beam/2.0.0/
>> release/beam/2.1.0/
>> release/beam/2.1.1/
>> release/beam/2.2.0/
>>
>>
>> I don't think this is the intended release process -- don't we want them
>> up forever?
>>
>> Suggest partial revert of that SVN commit to restore the old releases.
>>
>> On Tue, Jun 19, 2018 at 1:39 PM Chamikara Jayalath 
>> wrote:
>>
>>> I noticed that links to source code of releases is broken in the
>>> following page for everything but the latest release.
>>>
>>> https://beam.apache.org/get-started/downloads/
>>>
>>> For example
>>> http://www-eu.apache.org/dist/beam/2.3.0/apache-beam-2.3.0-source-release.zip
>>>
>>> Anybody know how to fix this ?
>>>
>>> - Cham
>>>
>>
>

Re: Broken links to releases

2018-06-19 Thread Thomas Weise 
It is expected that old releases are removed and archived:
http://www.apache.org/legal/release-policy.html#when-to-archive

Old releases are available from http://archive.apache.org/dist/beam/


On Tue, Jun 19, 2018 at 11:12 PM, Dan Halperin  wrote:

> Looks like JB removed these during release of 2.3.0:
>
> svn commit: r25111 - in /release/beam: ./ 0.1.0-incubating/
> 0.2.0-incubating/ 0.3.0-incubating/ 0.4.0/ 0.5.0/ 0.6.0/ 2.0.0/ 2.1.0/
> 2.1.1/ 2.2.0/ 2.3.0/
>
> Author: jbonofre
> Date: Sat Feb 17 06:08:19 2018
> New Revision: 25111
>
> Log:
> Publish 2.3.0 release
>
> Added:
> release/beam/2.3.0/
> release/beam/2.3.0/apache-beam-2.3.0-python.zip   (with props)
> release/beam/2.3.0/apache-beam-2.3.0-python.zip.asc
> release/beam/2.3.0/apache-beam-2.3.0-python.zip.md5
> release/beam/2.3.0/apache-beam-2.3.0-python.zip.sha1
> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip   (with props)
> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.asc
> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.md5
> release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.sha1
> release/beam/latest   (with props)
> Removed:
> release/beam/0.1.0-incubating/
> release/beam/0.2.0-incubating/
> release/beam/0.3.0-incubating/
> release/beam/0.4.0/
> release/beam/0.5.0/
> release/beam/0.6.0/
> release/beam/2.0.0/
> release/beam/2.1.0/
> release/beam/2.1.1/
> release/beam/2.2.0/
>
>
> I don't think this is the intended release process -- don't we want them
> up forever?
>
> Suggest partial revert of that SVN commit to restore the old releases.
>
> On Tue, Jun 19, 2018 at 1:39 PM Chamikara Jayalath 
> wrote:
>
>> I noticed that links to source code of releases is broken in the
>> following page for everything but the latest release.
>>
>> https://beam.apache.org/get-started/downloads/
>>
>> For example http://www-eu.apache.org/dist/beam/2.3.0/apache-
>> beam-2.3.0-source-release.zip
>>
>> Anybody know how to fix this ?
>>
>> - Cham
>>
>

Re: Broken links to releases

2018-06-19 Thread Dan Halperin 
Looks like JB removed these during release of 2.3.0:

svn commit: r25111 - in /release/beam: ./ 0.1.0-incubating/
0.2.0-incubating/ 0.3.0-incubating/ 0.4.0/ 0.5.0/ 0.6.0/ 2.0.0/ 2.1.0/
2.1.1/ 2.2.0/ 2.3.0/

Author: jbonofre
Date: Sat Feb 17 06:08:19 2018
New Revision: 25111

Log:
Publish 2.3.0 release

Added:
release/beam/2.3.0/
release/beam/2.3.0/apache-beam-2.3.0-python.zip   (with props)
release/beam/2.3.0/apache-beam-2.3.0-python.zip.asc
release/beam/2.3.0/apache-beam-2.3.0-python.zip.md5
release/beam/2.3.0/apache-beam-2.3.0-python.zip.sha1
release/beam/2.3.0/apache-beam-2.3.0-source-release.zip   (with props)
release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.asc
release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.md5
release/beam/2.3.0/apache-beam-2.3.0-source-release.zip.sha1
release/beam/latest   (with props)
Removed:
release/beam/0.1.0-incubating/
release/beam/0.2.0-incubating/
release/beam/0.3.0-incubating/
release/beam/0.4.0/
release/beam/0.5.0/
release/beam/0.6.0/
release/beam/2.0.0/
release/beam/2.1.0/
release/beam/2.1.1/
release/beam/2.2.0/


I don't think this is the intended release process -- don't we want them up
forever?

Suggest partial revert of that SVN commit to restore the old releases.

On Tue, Jun 19, 2018 at 1:39 PM Chamikara Jayalath 
wrote:

> I noticed that links to source code of releases is broken in the following
> page for everything but the latest release.
>
> https://beam.apache.org/get-started/downloads/
>
> For example
> http://www-eu.apache.org/dist/beam/2.3.0/apache-beam-2.3.0-source-release.zip
>
> Anybody know how to fix this ?
>
> - Cham
>

Re: Broken links to releases

2018-06-19 Thread Lukasz Cwik 
I think we want to get the links from https://archive.apache.org/dist/beam/

On Tue, Jun 19, 2018 at 1:39 PM Chamikara Jayalath 
wrote:

> I noticed that links to source code of releases is broken in the following
> page for everything but the latest release.
>
> https://beam.apache.org/get-started/downloads/
>
> For example
> http://www-eu.apache.org/dist/beam/2.3.0/apache-beam-2.3.0-source-release.zip
>
> Anybody know how to fix this ?
>
> - Cham
>

Broken links to releases

2018-06-19 Thread Chamikara Jayalath 
I noticed that links to source code of releases is broken in the following
page for everything but the latest release.

https://beam.apache.org/get-started/downloads/

For example
http://www-eu.apache.org/dist/beam/2.3.0/apache-beam-2.3.0-source-release.zip

Anybody know how to fix this ?

- Cham

Re: [VOTE] Apache Beam, version 2.5.0, release candidate #2

2018-06-19 Thread Udi Meiri 
+1

On Mon, Jun 18, 2018 at 6:16 PM Boyuan Zhang  wrote:

> Follow up with pervious email( sorry for the inconvenience).
>
> Hey JB,
>
> If you haven't built python wheels yet, then you can use files in the
> previous email to move forward. If you have done all of them, then please
> ignore previous email.
>
> Thanks so much for your help.
>
> Boyuan
>
> On Mon, Jun 18, 2018 at 6:08 PM Boyuan Zhang  wrote:
>
>> Hey JB,
>>
>> I built python wheels and you can get them here:
>> https://storage.googleapis.com/python_wheels/python_wheels.zip. You
>> probably need to sign them and upload them into
>> https://dist.apache.org/repos/dist/dev/beam/2.5.0/ like what we did for
>> 2.4.0: https://dist.apache.org/repos/dist/dev/beam/2.4.0/ . After you
>> upload, we can do more validations on python wheels.
>>
>> Boyuan
>>
>> On Mon, Jun 18, 2018 at 4:49 PM Pablo Estrada  wrote:
>>
>>> +0.9
>>>
>>> I've run the Mobile Gaming examples in Java+Dataflow, and others have
>>> documented their tests in the spreadsheet[1]. All seems to work well.
>>> We will need the wheel files to be available, and run a few validations
>>> on them, but other than that, the release looks good to me.
>>>
>>> Best
>>> -P.
>>>
>>> [1]
>>> https://docs.google.com/spreadsheets/d/1qk-N5vjXvbcEk68GjbkSZTR8AGqyNUM-oLFo_ZXBpJw/edit#gid=152451807
>>>
>>> On Mon, Jun 18, 2018 at 10:08 AM Jean-Baptiste Onofré 
>>> wrote:
>>>
 Hi,

 I thought I did it, let me double check.

 Regards
 JB

 On 18/06/2018 19:01, Boyuan Zhang wrote:
 > Hey JB,
 >
 > Could you please also build python wheels for RC2 and stage them in
 > https://dist.apache.org/repos/dist/dev/beam/2.5.0/? There are some
 > instructions in this PR: https://github.com/apache/beam-site/pull/467
 .
 > Hope it can be helpful.
 >
 > Thanks.
 > Boyuan
 >
 > On Sun, Jun 17, 2018 at 9:22 PM Jean-Baptiste Onofré >>> > > wrote:
 >
 > s/add/had/g
 >
 > On 17/06/2018 19:32, Pablo Estrada wrote:
 > > Going to https://github.com/apache/beam/tree/v2.5.0-RC2 gives
 me a 404
 > > error. May need to push the label?
 > > Best
 > > -P.
 > >
 > > On Sun, Jun 17, 2018 at 1:15 AM Romain Manni-Bucau
 > > mailto:rmannibu...@gmail.com>
 > >>
 wrote:
 > >
 > > +1 (same tests than for take #1)
 > >
 > > Le dim. 17 juin 2018 07:18, Jean-Baptiste Onofré
 > mailto:j...@nanthrax.net>
 > > >> a
 écrit :
 > >
 > > Hi everyone,
 > >
 > > Please review and vote on the release candidate #2 for
 the
 > version
 > > 2.5.0, as follows:
 > >
 > > [ ] +1, Approve the release
 > > [ ] -1, Do not approve the release (please provide
 specific
 > > comments)
 > >
 > > NB: this is the first release using Gradle, so don't be
 too
 > > harsh ;) A
 > > PR about the release guide will follow thanks to this
 release.
 > >
 > > The complete staging area is available for your review,
 which
 > > includes:
 > > * JIRA release notes [1],
 > > * the official Apache source release to be deployed to
 > > dist.apache.org 
 > 
 > > [2], which is signed with the key with fingerprint
 > C8282E76 [3],
 > > * all artifacts to be deployed to the Maven Central
 > Repository [4],
 > > * source code tag "v2.5.0-RC2" [5],
 > > * website pull request listing the release and
 publishing
 > the API
 > > reference manual [6].
 > > * Java artifacts were built with Gradle 4.7 (wrapper)
 and
 > > OpenJDK/Oracle
 > > JDK 1.8.0_172 (Oracle Corporation 25.172-b11).
 > > * Python artifacts are deployed along with the source
 > release to the
 > > dist.apache.org 
 >  [2].
 > >
 > > The vote will be open for at least 72 hours. It is
 adopted by
 > > majority
 > > approval, with at least 3 PMC affirmative votes.
 > >
 > > Thanks,
 > > JB
 > >
 > > [1]
 > >
 >
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12342847
 > > [2] https://dist.apache.org/repos/dist/dev/beam/2.5.0/
 > > [3]

Re: [DISCUSS] [BEAM-4126] Deleting Maven build files (pom.xml) grace period?

2018-06-19 Thread Kenneth Knowles 
-user@ for misc. progress reporting

I took a look at it yesterday. Learned a lot, made some progress that turns
out to just be general goodness, and left comment on the JIRA.

On Mon, Jun 18, 2018 at 6:37 PM Lukasz Cwik  wrote:

> Any updates on BEAM-4512?
>
> On Mon, Jun 11, 2018 at 1:42 PM Lukasz Cwik  wrote:
>
>> Thanks all, it seems as though only Google needs the grace period. I'll
>> wait for the shorter of BEAM-4512 or two weeks before merging
>> https://github.com/apache/beam/pull/5571
>>
>>
>> On Wed, Jun 6, 2018 at 8:29 PM Kenneth Knowles  wrote:
>>
>>> +1
>>>
>>> Definitely a good opportunity to decouple your build tools from your
>>> dependencies' build tools.
>>>
>>> On Wed, Jun 6, 2018 at 2:42 PM Ted Yu  wrote:
>>>
 +1 on this effort

  Original message 
 From: Chamikara Jayalath 
 Date: 6/6/18 2:09 PM (GMT-08:00)
 To: dev@beam.apache.org, u...@beam.apache.org
 Subject: Re: [DISCUSS] [BEAM-4126] Deleting Maven build files (pom.xml)
 grace period?

 +1 for the overall effort. As Pablo mentioned, we need some time to
 migrate internal Dataflow build off of Maven build files. I created
 https://issues.apache.org/jira/browse/BEAM-4512 for this.

 Thanks,
 Cham

 On Wed, Jun 6, 2018 at 1:30 PM Eugene Kirpichov 
 wrote:

> Is it possible for Dataflow to just keep a copy of the pom.xmls and
> delete it as soon as Dataflow is migrated?
>
> Overall +1, I've been using Gradle without issues for a while and
> almost forgot pom.xml's still existed.
>
> On Wed, Jun 6, 2018, 1:13 PM Pablo Estrada  wrote:
>
>> I agree that we should delete the pom.xml files soon, as they create
>> a burden for maintainers.
>>
>> I'd like to be able to extend the grace period by a bit, to allow the
>> internal build systems at Google to move away from using the Beam poms.
>>
>> We use these pom files to build Dataflow workers, and thus it's
>> critical for us that they are available for a few more weeks while we set
>> up a gradle build. Perhaps 4 weeks?
>> (Calling out+Chamikara Jayalath  who has
>> recently worked on internal Dataflow tooling.)
>>
>> Best
>> -P.
>>
>> On Wed, Jun 6, 2018 at 1:05 PM Lukasz Cwik  wrote:
>>
>>> Note: Apache Beam will still provide pom.xml for each release it
>>> produces. This is only about people using Maven to build Apache Beam
>>> themselves and not relying on the released artifacts in Maven Central.
>>>
>>> With the first release using Gradle as the build system is underway,
>>> I wanted to start this thread to remind people that we are going to 
>>> delete
>>> the Maven pom.xml files after the 2.5.0 release is finalized plus a two
>>> week grace period.
>>>
>>> Are there others who would like a shorter/longer grace period?
>>>
>>> The PR to delete the pom.xml is here:
>>> https://github.com/apache/beam/pull/5571
>>>
>> --
>> Got feedback? go/pabloem-feedback
>> 
>>
>

Re: [DISCUSS] Releasing Beam in the presence of emergencies

2018-06-19 Thread Rafael Fernandez 
Thanks all for the discussion, what I gather is:

(1) Stating the obvious: our release practices and mechanisms match what I
suggested we wrote down in a page. (That's great!). I heard no objections
to affirming our adherence to ASF vulnerability policy.
(2) There are details to be discussed further - specifically, which
releases to fix (which suggests discussion on what a "current" release
could be), and details on who the release manager could be (in the presence
of multiple releases)

So let me propose the following: I'll send a PR to the website for (1), and
for (2), let's organize the discussion along the various axis -- I liked
how Chamikara broke down the space, so we should be able to work along
those lines.

Thanks,
r



On Sun, Jun 17, 2018 at 1:14 AM Romain Manni-Bucau 
wrote:

> Hi guys
>
> If you take back this thread [1], then beam will only support two branches
> and no others.
>
> On the shortcut release cycle it is not really desired at apache and not
> the normal process so the 3 days vote is required but not a blocker IMHO.
> If really an issue the merge can be done quickly on master and you can
> release on your own nexus in 1h. The side note for the asf release manager
> here will be to emphase the diff to let people review it faster. Less
> change there is, easier it is to vote.
>

​This is a good point to discuss. If we simply state this as valid for any
release, the mechanics are simpler. I suggested breaking u p the
discussions along various axis so we can make progress on each concern --
please bring this up on release mechanics.​



>
> My question on that process will be for dataflow guys wich often request a
> few more days. Is it ok because dataflow doesnt use multiple versions or
> does it need some work on that side to support a faster support of new beam
> versions?
>

​I'm not sure what part of the cycle you are referring to - I have seen RCs
taking longer to validate because we discover issues in the service. If
that's the case, I'm happy to share that the team is working on increasing
test coverage in Beam itself so we minimize this class of surprises.

Thanks Romain!



>
> [1]
> https://lists.apache.org/thread.html/1f6941f112d080de3633d11c574fb67bce5f6f81679ec3bdeaf807f0@
> 
>
> Le dim. 17 juin 2018 01:08, Ravi Prakash  a écrit :
>
>> Hi Beam-devs!
>>
>> There is also the consideration of which releases to fix. I am familiar
>> with Apache Hadoop and there may be a "stable", "beta" and "alpha" release.
>> Usually security fixes are ported to one or more of those branches.
>>
>> Cheers
>> Ravi
>>
>> On Sat, Jun 16, 2018 at 2:43 PM, Kenneth Knowles  wrote:
>>
>>> I like the idea of explicitly affirming our use of the ASF Vulnerability
>>> Policy as well as listing other classes of bugs that are particularly
>>> critical for a project like Beam. I think the most valuable thing is having
>>> a clear and concise policy for users to understand at a glance. Then we can
>>> have a verbose walkthrough for someone implementing the policy.
>>>
>>> Kenn
>>>
>>> On Sat, Jun 16, 2018 at 9:54 AM Chamikara Jayalath 
>>> wrote:
>>>
 While performing a patch release is the correct approach for a critical
 fixes I think there are still several points that we might want to discuss
 and formalize/document if needed.

 (1) What if there's an ongoing major/minor version release ?
 If think patch releases should be independent of any ongoing
 major/minor versions releases and should be handled by a separate release
 manager who can commit the the time to cut the release immediately.

 (2) What changes should go in ?
 Only the fix that addresses the critical issue should go into the patch
 release. Any other fixes should be held till the next minor version 
 release.

 (3) What to do with the faulty release ?
 We should clearly document the issue with the faulty (non-patched)
 release in https://beam.apache.org/get-started/downloads/. This can be
 complemented by other announcements (to user list, twitter, for example)
 and runner specific solutions (rejecting jobs, warnings for running jobs).

 (4) Voting period and validation.
 We are performing the patch on top of an already released branch. So
 can we expedite the release validation and voting process ?

 I'm sure there are other points.

 Thanks,
 Cham

 On Thu, Jun 14, 2018 at 10:29 PM Jean-Baptiste Onofré 
 wrote:

> Hi Rafael,
>
> It's a good point but I don't see nothing more to do on our side: if a
> emergency issue is detected, then we have to address it and release a
> fix release (x.y.z where z is the specific release fixing the issue).
> The commitment is a best effort as in all community: if an emergency
> issue is detected, qualified and accepted, then we do our best to
> provide a fix and do the fix release.
>
> So, for me, it's already handled.
>
> By