Re: [VOTE] Apache Beam, version 2.6.0, release candidate #2

2018-08-04 Thread Jean-Baptiste Onofré 
+1 (binding)

Tested with beam-samples, checksum and sig verified.

Regards
JB

On 04/08/2018 01:27, Pablo Estrada wrote:
> Hello everyone!
> 
> Extra, extra! The Apache Beam 2.6.0 release candidate #2 is out.
> 
> Please review and vote on the release candidate #2 for the version
> 2.6.0, as follows:
> 
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
> 
> The complete staged set of artifacts is available for your review, which
> includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org
>  [2], which is signed with the key with
> fingerprint 2F1FEDCDF6DD7990422F482F65224E0292DD8A51 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.6.0-RC2" [5],
> * website pull request listing the release and publishing the API
> reference manual [6]. This did not change from the previous RC.
> * Python artifacts are deployed along with the source release to
> the dist.apache.org  [2].
> 
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
> 
> Regards
> -Pablo.
> 
> [1] 
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12343392
> [2] https://dist.apache.org/repos/dist/dev/beam/2.6.0/
> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1045/ 
> [5] https://github.com/apache/beam/tree/v2.6.0-RC2 
> [6] https://github.com/apache/beam-site/pull/518
> -- 
> Got feedback? go/pabloem-feedback

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Removing documentation for old Beam versions

2018-08-04 Thread Thomas Weise 
Yes, I think the separation of generated code will need to occur prior to
completing the merge and switching the web site to the main repo.

There should be no reason to check generated documentation into either of
the repos/branches. Please see as an example how this was solved in Flink,
using the ASF buildbot  infrastructure.

Documentation per version/release, for example:

https://ci.apache.org/projects/flink/flink-docs-release-1.5/

The buildbot configuration is here (requires committer access):

https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/flink.conf

Thanks,
Thomas

On Thu, Aug 2, 2018 at 6:46 PM Mikhail Gryzykhin  wrote:

> Last time I talked with Scott I brought this idea in. I believe the plan
> was either to publish compiled site to website directly, or keep it in
> separate storage from apache/beam repo.
>
> One of the main reasons not to check in compiled version of website is
> that every developer will have to pull all the versions of website every
> time they clone repo, which is not that good of an idea to do.
>
> Regards,
> --Mikhail
>
> Have feedback ?
>
>
> On Thu, Aug 2, 2018 at 6:42 PM Udi Meiri  wrote:
>
>> Pablo, the docs are generated into versioned paths, e.g.,
>> https://beam.apache.org/documentation/sdks/javadoc/2.5.0/ so tags are
>> not necessary?
>> Also, once apache/beam-site is merged with apache/beam the release branch
>> should have the relevant docs (although perhaps it's better to put them in
>> a different repo or storage system).
>>
>> Thomas, I would very much like to not have javadoc/pydoc generation be
>> part of the website review process, as it takes up a lot of time when
>> changes are staged (10s of thousands of files), especially when a PR is
>> updated and existing staged files need to be deleted.
>>
>>
>> On Thu, Aug 2, 2018 at 1:15 PM Mikhail Gryzykhin 
>> wrote:
>>
>>> +1 For removing old documentation.
>>>
>>> @Thomas: Migration work is in backlog and will be picked up in near time.
>>>
>>> --Mikhail
>>>
>>> Have feedback ?
>>>
>>>
>>> On Thu, Aug 2, 2018 at 12:54 PM Thomas Weise  wrote:
>>>
 +1 for removing pre 2.0 documentation (as well as the entries from
 https://beam.apache.org/get-started/downloads/)

 Isn't it part of the beam-site changes that we will no longer check in
 generated documentation into the repository? Those can be generated and
 deployed independently (when a commit to a branch occurs), such as done in
 the Apex and Flink projects.

 I was told that Scott who was working in the beam-site changes is on
 leave now and the migration is still pending (see note at
 https://github.com/apache/beam/tree/master/website). Is anyone else
 going to pick it up?

 Thanks,
 Thomas


 On Thu, Aug 2, 2018 at 12:33 PM Pablo Estrada 
 wrote:

> Is it worth adding a tag / branch to the repositories every time we
> make a release, so that people are able to dive in and find the docs?
> Best
> -P.
>
> On Thu, Aug 2, 2018 at 12:09 PM Ahmet Altay  wrote:
>
>> I would guess that users are still using some of these old releases.
>> It is unclear from Beam website which releases are still supported or 
>> not.
>> It probably makes sense to drop documentation for releases < 2.0. (I 
>> would
>> suggest keeping docs for 2.0). For the future I can work on updating the
>> Beam website to clarify the state of each release.
>>
>> On Thu, Aug 2, 2018 at 12:06 PM, Udi Meiri  wrote:
>>
>>> The older docs are not directly linked to and are in Github commit
>>> history.
>>>
>>> If there are no objections I'm going to delete javadocs and pydocs
>>> for releases older than 1 year,
>>> meaning 2.0.0 and older (going by the dates here
>>> ).
>>>
>>> On Thu, Aug 2, 2018 at 11:51 AM Daniel Oliveira <
>>> danolive...@google.com> wrote:
>>>
 The older docs should be recorded in the commit history of the
 website repository, right? If they're not currently used in the 
 website and
 they're in the commit history then I don't see a reason to save them.

 On Tue, Jul 31, 2018 at 1:51 PM Udi Meiri  wrote:

> Hi all,
> I'm writing a PR for apache/beam-site and
> beam_PreCommit_Website_Stage is timing out after 100 minutes, because 
> it's
> trying to deletes 22k files and then copy 22k files (warning
> large file
> 
> ).
>
> It seems that we could save a lot of time by deleting the older
> javadoc and pydoc files for older versions. Is there a good reason to 
> keep
> around

Re: [VOTE] Apache Beam, version 2.6.0, release candidate #2

2018-08-04 Thread Thomas Weise 
+1 (binding)

- verified signatures and hashes
- built Quickstart with Java Maven archetype against staging repo
- verified for Apex runner (embedded + YARN)

Please include the acceptance table with future VOTE emails:
https://s.apache.org/beam-release-validation

Thanks!


On Fri, Aug 3, 2018 at 4:28 PM Pablo Estrada  wrote:

> Hello everyone!
>
> Extra, extra! The Apache Beam 2.6.0 release candidate #2 is out.
>
> Please review and vote on the release candidate #2 for the version 2.6.0,
> as follows:
>
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> The complete staged set of artifacts is available for your review, which
> includes:
> * JIRA release notes [1],
> * the official Apache source release to be deployed to dist.apache.org [2],
> which is signed with the key with fingerprint
> 2F1FEDCDF6DD7990422F482F65224E0292DD8A51 [3],
> * all artifacts to be deployed to the Maven Central Repository [4],
> * source code tag "v2.6.0-RC2" [5],
> * website pull request listing the release and publishing the API
> reference manual [6]. This did not change from the previous RC.
> * Python artifacts are deployed along with the source release to the
> dist.apache.org [2].
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Regards
> -Pablo.
>
> [1]
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527=12343392
> [2] https://dist.apache.org/repos/dist/dev/beam/2.6.0/
> [3] https://dist.apache.org/repos/dist/dev/beam/KEYS
> [4] https://repository.apache.org/content/repositories/orgapachebeam-1045/
>
> [5] https://github.com/apache/beam/tree/v2.6.0-RC2
> [6] https://github.com/apache/beam-site/pull/518
> --
> Got feedback? go/pabloem-feedback
>

Re: Apache Beam Python Wheels Repository

2018-08-04 Thread Davor Bonaci 
New repository is not a ticket, it is a self-serve thing.

That said, you probably want to develop the proposal a bit further,
understanding/educating others about the benefits of what you are
proposing, any alternatives, why a repository is needed, why the sample
repository has Travis CI when everything else is on Jenkins, how this fits
into other decisions about repository management, and so on. Anything can
be done, of course, but I'd suggest developing (or communicating, or
educating) a bit more.

(I'm fine with any approach.)

On Fri, Aug 3, 2018 at 3:29 PM, Ahmet Altay  wrote:

> This LGTM, also greatly simplifies the creation of wheel files for
> multiple platforms.
>
> I can file an INFRA ticket to create a new repo to host wheel setup. Does
> anybody have experience with setting up a new repo similar to this?
>
> Ahmet
>
> On Fri, Aug 3, 2018 at 1:16 PM, Boyuan Zhang  wrote:
>
>> Hey all,
>>
>> I'm Boyuan Zhang from Google Dataflow Team, currently helping Release
>> Manager(Pablo Estrada) with 2.6.0 release. Since Beam decided to release
>> python wheels since 2.5.0, we need to create a wrapper repository(sample
>> repo ) under apache to
>> build and stage released python wheels for each release. Anyone can help to
>> create this repository?
>>
>> Thanks for all your help! Happy Friday~
>>
>> Boyuan Zhang
>>
>
>

Build failed in Jenkins: beam_Release_Gradle_NightlySnapshot #131

2018-08-04 Thread Apache Jenkins Server 
See 


Changes:

[yifanzou] [BEAM-4963] update org.apache.httpcomponents

[amaliujia] BEAM-4808: add e2e test for BeamSqlLine.

[garrettjonesgoogle] Switching to correct datastore proto package

[yifanzou] [BEAM-4906] upgrade org.apache.derby

[relax] Add Select and Filter transforms for schema types.

[relax] Add missing Experimental flag.

[chamikara] [Beam-4687]Automate JIRA for Dependency check (#6060)

[aaltay] Automate "Build a release candidate" step (#6059)

[apilloud] [BEAM-4804] Use beam-perf label for local perf tests

[gleb] [BEAM-5056] [SQL] Fix nullability in output schema

[valentyn] Use the same language semantics as defined by version of the 
interpreter

[relax] Address code-review comments.

[aaltay] [BEAM-4006] Futurize transforms subpackage (#5729)

[apilloud] [BEAM-4761] Run Nexmark in thread pool

[aaltay] Automate "Run validation tests" step (#6081)

[apilloud] [BEAM-4761] Handle skipped tests in parallel Nexmark

--
[...truncated 18.13 MB...]
Task ':beam-sdks-python-container:prepare' is not up-to-date because:
  Task has not declared any outputs despite executing actions.
Use project GOPATH: 

:beam-sdks-python-container:prepare (Thread[Task worker for ':' Thread 
10,5,main]) completed. Took 0.001 secs.
:beam-sdks-python-container:resolveBuildDependencies (Thread[Task worker for 
':' Thread 10,5,main]) started.

> Task :beam-sdks-python-container:resolveBuildDependencies
Build cache key for task ':beam-sdks-python-container:resolveBuildDependencies' 
is 8b9b155163bbd49020994445c942671f
Caching disabled for task 
':beam-sdks-python-container:resolveBuildDependencies': Caching has not been 
enabled for the task
Task ':beam-sdks-python-container:resolveBuildDependencies' is not up-to-date 
because:
  No history is available.
Cache 

 not found, skip.
Cache 

 not found, skip.
Resolving 
./github.com/apache/beam/sdks/go@
:beam-sdks-python-container:resolveBuildDependencies (Thread[Task worker for 
':' Thread 10,5,main]) completed. Took 2.879 secs.
:beam-sdks-python-container:installDependencies (Thread[Task worker for ':' 
Thread 10,5,main]) started.

> Task :beam-sdks-python-container:installDependencies
Caching disabled for task ':beam-sdks-python-container:installDependencies': 
Caching has not been enabled for the task
Task ':beam-sdks-python-container:installDependencies' is not up-to-date 
because:
  Task has not declared any outputs despite executing actions.
Cache 

 not found, skip.
:beam-sdks-python-container:installDependencies (Thread[Task worker for ':' 
Thread 10,5,main]) completed. Took 0.571 secs.
:beam-sdks-python-container:buildLinuxAmd64 (Thread[Task worker for ':' Thread 
10,5,main]) started.

> Task :beam-sdks-python-container:buildLinuxAmd64
Build cache key for task ':beam-sdks-python-container:buildLinuxAmd64' is 
96a676bb2799e7ccf5f31a42c61a6b8a
Caching disabled for task ':beam-sdks-python-container:buildLinuxAmd64': 
Caching has not been enabled for the task
Task ':beam-sdks-python-container:buildLinuxAmd64' is not up-to-date because:
  No history is available.
:beam-sdks-python-container:buildLinuxAmd64 (Thread[Task worker for ':' Thread 
10,5,main]) completed. Took 2.719 secs.
:beam-sdks-python-container:build (Thread[Task worker for ':' Thread 
10,5,main]) started.

> Task :beam-sdks-python-container:build
Caching disabled for task ':beam-sdks-python-container:build': Caching has not 
been enabled for the task
Task ':beam-sdks-python-container:build' is not up-to-date because:
  Task has not declared any outputs despite executing actions.
:beam-sdks-python-container:build (Thread[Task worker for ':' Thread 
10,5,main]) completed. Took 0.002 secs.
:beam-vendor-sdks-java-extensions-protobuf:jar (Thread[Task worker for ':' 
Thread 10,5,main]) started.

> Task :beam-vendor-sdks-java-extensions-protobuf:jar
Build cache key for task ':beam-vendor-sdks-java-extensions-protobuf:jar' is 
318184f3acfe150a3a38d605826c992a
Caching disabled for task ':beam-vendor-sdks-java-extensions-protobuf:jar': 
Caching has not been enabled for the task
Task ':beam-vendor-sdks-java-extensions-protobuf:jar' is not up-to-date because:
  No history is available.

Re: Hello Beam Community!!!

2018-08-04 Thread Kai Jiang 
Welcome!

On Fri, Aug 3, 2018, 23:59 Rui Wang  wrote:

> Welcome!
>
> -Rui
>
> On Fri, Aug 3, 2018 at 11:29 PM Griselda Cuevas  wrote:
>
>> Welcome!
>>
>> On Fri, Aug 3, 2018, 9:44 PM Jean-Baptiste Onofré 
>> wrote:
>>
>>> Welcome aboard !
>>>
>>> Regards
>>> JB
>>>
>>> On 04/08/2018 04:39, Connell O'Callaghan wrote:
>>> > Hi All
>>> >
>>> > My name is Connell. I am relatively new to Beam (and Google where I
>>> work
>>> > as a technical PM). My philosophy is very much oriented around how can
>>> I
>>> > serve the team. Hence, I am extremely interested to learn how I can
>>> help
>>> > continue the great work of this community. One particular area I am
>>> > curious about is how this community triages and manages issues. I will
>>> > do some self study on this. But I would be very interested to engage
>>> > with others interested in this general topic. Please let me know if you
>>> > wish to connect on this or other topics!!!
>>> >
>>> > Thank you in advance!!!
>>> >
>>> > - Connell
>>>
>>> --
>>> Jean-Baptiste Onofré
>>> jbono...@apache.org
>>> http://blog.nanthrax.net
>>> Talend - http://www.talend.com
>>>
>>

Re: Hello Beam Community!!!

2018-08-04 Thread Rui Wang 
Welcome!

-Rui

On Fri, Aug 3, 2018 at 11:29 PM Griselda Cuevas  wrote:

> Welcome!
>
> On Fri, Aug 3, 2018, 9:44 PM Jean-Baptiste Onofré  wrote:
>
>> Welcome aboard !
>>
>> Regards
>> JB
>>
>> On 04/08/2018 04:39, Connell O'Callaghan wrote:
>> > Hi All
>> >
>> > My name is Connell. I am relatively new to Beam (and Google where I work
>> > as a technical PM). My philosophy is very much oriented around how can I
>> > serve the team. Hence, I am extremely interested to learn how I can help
>> > continue the great work of this community. One particular area I am
>> > curious about is how this community triages and manages issues. I will
>> > do some self study on this. But I would be very interested to engage
>> > with others interested in this general topic. Please let me know if you
>> > wish to connect on this or other topics!!!
>> >
>> > Thank you in advance!!!
>> >
>> > - Connell
>>
>> --
>> Jean-Baptiste Onofré
>> jbono...@apache.org
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
>>
>

Re: Hello Beam Community!!!

2018-08-04 Thread Griselda Cuevas 
Welcome!

On Fri, Aug 3, 2018, 9:44 PM Jean-Baptiste Onofré  wrote:

> Welcome aboard !
>
> Regards
> JB
>
> On 04/08/2018 04:39, Connell O'Callaghan wrote:
> > Hi All
> >
> > My name is Connell. I am relatively new to Beam (and Google where I work
> > as a technical PM). My philosophy is very much oriented around how can I
> > serve the team. Hence, I am extremely interested to learn how I can help
> > continue the great work of this community. One particular area I am
> > curious about is how this community triages and manages issues. I will
> > do some self study on this. But I would be very interested to engage
> > with others interested in this general topic. Please let me know if you
> > wish to connect on this or other topics!!!
> >
> > Thank you in advance!!!
> >
> > - Connell
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>