Re: [HEADS UP] Preparing Beam 2.3.0

[Jean-Baptiste Onofré]

Hi,

As BEAM-793 and BEAM-3551 are not release blocker, if I don't have update in the
PRs in the coming couple of hours, I will bump these Jiras to 2.4.0 and start
the release process.

Thanks !
Regards
JB

On 01/28/2018 08:29 PM, Jean-Baptiste Onofré wrote:
> Hi guys,
> 
> a new update about 2.3.0 release preparation. Only 2 Jira are pending (not
> blocker for the release but good to have):
> 
> - BEAM-793: I updated the PR (#4500), waiting Eugene's feedback.
> - BEAM-3551: I created a PR (#4516), waiting Luke's feedback.
> 
> Thanks to the timezone, I hope to have feedbacks soon.
> 
> Anyway, these Jiras are not release blocker, so I will start the release 
> process
> tomorrow morning my time.
> 
> By the way, I created 2.4.0 version in Jira.
> 
> Thanks all for your help and support !
> 
> Regards
> JB
> 
> On 27/01/2018 13:00, Jean-Baptiste Onofré wrote:
>> Hi guys,
>>
>> we still have 7 Jira targeted to 2.3.0.
>>
>> For most of them, Ismaël and I are doing the PRs/fixes and we have review in
>> progress.
>>
>> I'm a little bit concerned by BEAM-3392: it's flagged as blocker but it's
>> related to a specific branch. Can you please provide an update asap ?
>>
>> However, I didn't have any update for BEAM-3087 (related to the Flink 
>> runner).
>> Without update soon, I will bump to 2.4.0.
>>
>> I would need a review on PR for BEAM-793 (PR #4500). To avoid to break 
>> anything
>> for existing user, I set the backoff strategy optional, the user has to
>> explicitly set to use it.
>>
>> I'm waiting a little more before cutting the release (especially for 
>> BEAM-3392
>> and BEAM-3087). However, I would like to cut the release asap.
>>
>> Thanks,
>> Regards
>> JB
>>
>>
>> On 01/23/2018 10:39 AM, Jean-Baptiste Onofré wrote:
>>> Hi guys,
>>>
>>> Some days ago, I proposed to start Beam 2.3.0 around January 26th. So, we 
>>> are
>>> few days from this date.
>>>
>>> As a best effort, can you please in Jira flag the Jira with fix version 
>>> 2.3.0
>>> and blocker for the release. Then, I will know when I can start the release
>>> process.
>>>
>>> Thanks !
>>>
>>> Regards
>>> JB
>>>
>>

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: Schema-Aware PCollections revisited

[Jean-Baptiste Onofré]

Hi Reuven,

Thanks for the update ! As I'm working with you on this, I fully agree and great
doc gathering the ideas.

It's clearly something we have to add asap in Beam, because it would allow new
use cases for our users (in a simple way) and open new areas for the runners
(for instance dataframe support in the Spark runner).

By the way, while ago, I created BEAM-3437 to track the PoC/PR around this.

Thanks !

Regards
JB

On 01/29/2018 02:08 AM, Reuven Lax wrote:
> Previously I submitted a proposal for adding schemas as a first-class concept 
> on
> Beam PCollections. The proposal engendered quite a bit of discussion from the
> community - more discussion than I've seen from almost any of our proposals to
> date! 
> 
> Based on the feedback and comments, I reworked the proposal document quite a
> bit. It now talks more explicitly about the different between dynamic schemas
> (where the schema is not fully not know at graph-creation time), and static
> schemas (which are fully know at graph-creation time). Proposed APIs are more
> fleshed out now (again thanks to feedback from community members), and the
> document talks in more detail about evolving schemas in long-running streaming
> pipelines.
> 
> Please take a look. I think this will be very valuable to Beam, and welcome 
> any
> feedback.
> 
> https://docs.google.com/document/d/1tnG2DPHZYbsomvihIpXruUmQ12pHGK0QIvXS1FOTgRc/edit#
> 
> Reuven

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [HEADS UP] Preparing Beam 2.3.0

[Jean-Baptiste Onofré]

OK, thanks. I keep you posted ;)

Regards
JB

On 28/01/2018 15:06, Romain Manni-Bucau wrote:

Will be online this afternoon. Ping me if you need help.

Le 28 janv. 2018 11:36, "Jean-Baptiste Onofré" > a écrit :


By the way, I created BEAM-3551 for the tracking.

I will work on it today.

Regards
JB

On 01/28/2018 09:50 AM, Romain Manni-Bucau wrote:
 > Hi guys
 >
 > Out if curiosity, can -parameters (javac) be part of the 2.3 if
not already?
 >
 > Le 27 janv. 2018 18:39, "Jean-Baptiste Onofré" 
 > >> a écrit :
 >
 >     Hi Reuven,
 >
 >     I gonna bump 3392 and 3087 to 2.4.0. For the PR, yes Eugene
did a first round
 >     review, I will work on it now.
 >
 >     We will pretty close !
 >
 >     Thanks !
 >     Regards
 >     JB
 >
 >     On 01/27/2018 05:58 PM, Reuven Lax wrote:
 >     > Seems that 3392 is not a blocker, and neither is 3087.
Looks like Eugene is
 >     > already reviewing the PR form BEAM-793.
 >     >
 >     > Reuven
 >     >
 >     > On Sat, Jan 27, 2018 at 4:00 AM, Jean-Baptiste Onofré

 >     >
 >     > 
     >
 >     >     Hi guys,
 >     >
 >     >     we still have 7 Jira targeted to 2.3.0.
 >     >
 >     >     For most of them, Ismaël and I are doing the PRs/fixes
and we have
 >     review in
 >     >     progress.
 >     >
 >     >     I'm a little bit concerned by BEAM-3392: it's flagged
as blocker but it's
 >     >     related to a specific branch. Can you please provide an
update asap ?
 >     >
 >     >     However, I didn't have any update for BEAM-3087
(related to the Flink
 >     runner).
 >     >     Without update soon, I will bump to 2.4.0.
 >     >
 >     >     I would need a review on PR for BEAM-793 (PR #4500). To
avoid to break
 >     anything
 >     >     for existing user, I set the backoff strategy optional,
the user has to
 >     >     explicitly set to use it.
 >     >
 >     >     I'm waiting a little more before cutting the release
(especially for
 >     BEAM-3392
 >     >     and BEAM-3087). However, I would like to cut the
release asap.
 >     >
 >     >     Thanks,
 >     >     Regards
 >     >     JB
 >     >
 >     >
 >     >     On 01/23/2018 10:39 AM, Jean-Baptiste Onofré wrote:
 >     >     > Hi guys,
 >     >     >
 >     >     > Some days ago, I proposed to start Beam 2.3.0 around
January 26th.
 >     So, we are
 >     >     > few days from this date.
 >     >     >
 >     >     > As a best effort, can you please in Jira flag the
Jira with fix
 >     version 2.3.0
 >     >     > and blocker for the release. Then, I will know when I
can start the
 >     >     release process.
 >     >     >
 >     >     > Thanks !
 >     >     >
 >     >     > Regards
 >     >     > JB
 >     >     >
 >     >
 >     >     --
 >     >     Jean-Baptiste Onofré
 >     > jbono...@apache.org 
>
 >     
>>
 >     > http://blog.nanthrax.net
 >     >     Talend - http://www.talend.com
 >     >
 >     >
 >
 >     --
 >     Jean-Baptiste Onofré
 > jbono...@apache.org 
>
 > http://blog.nanthrax.net
 >     Talend - http://www.talend.com
 >

--
Jean-Baptiste Onofré
jbono...@apache.org 
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [HEADS UP] Preparing Beam 2.3.0

[Romain Manni-Bucau]

Will be online this afternoon. Ping me if you need help.

Le 28 janv. 2018 11:36, "Jean-Baptiste Onofré"  a écrit :

> By the way, I created BEAM-3551 for the tracking.
>
> I will work on it today.
>
> Regards
> JB
>
> On 01/28/2018 09:50 AM, Romain Manni-Bucau wrote:
> > Hi guys
> >
> > Out if curiosity, can -parameters (javac) be part of the 2.3 if not
> already?
> >
> > Le 27 janv. 2018 18:39, "Jean-Baptiste Onofré"  > > a écrit :
> >
> > Hi Reuven,
> >
> > I gonna bump 3392 and 3087 to 2.4.0. For the PR, yes Eugene did a
> first round
> > review, I will work on it now.
> >
> > We will pretty close !
> >
> > Thanks !
> > Regards
> > JB
> >
> > On 01/27/2018 05:58 PM, Reuven Lax wrote:
> > > Seems that 3392 is not a blocker, and neither is 3087. Looks like
> Eugene is
> > > already reviewing the PR form BEAM-793.
> > >
> > > Reuven
> > >
> > > On Sat, Jan 27, 2018 at 4:00 AM, Jean-Baptiste Onofré <
> j...@nanthrax.net
> > 
> > > >> wrote:
> > >
> > > Hi guys,
> > >
> > > we still have 7 Jira targeted to 2.3.0.
> > >
> > > For most of them, Ismaël and I are doing the PRs/fixes and we
> have
> > review in
> > > progress.
> > >
> > > I'm a little bit concerned by BEAM-3392: it's flagged as
> blocker but it's
> > > related to a specific branch. Can you please provide an update
> asap ?
> > >
> > > However, I didn't have any update for BEAM-3087 (related to
> the Flink
> > runner).
> > > Without update soon, I will bump to 2.4.0.
> > >
> > > I would need a review on PR for BEAM-793 (PR #4500). To avoid
> to break
> > anything
> > > for existing user, I set the backoff strategy optional, the
> user has to
> > > explicitly set to use it.
> > >
> > > I'm waiting a little more before cutting the release
> (especially for
> > BEAM-3392
> > > and BEAM-3087). However, I would like to cut the release asap.
> > >
> > > Thanks,
> > > Regards
> > > JB
> > >
> > >
> > > On 01/23/2018 10:39 AM, Jean-Baptiste Onofré wrote:
> > > > Hi guys,
> > > >
> > > > Some days ago, I proposed to start Beam 2.3.0 around January
> 26th.
> > So, we are
> > > > few days from this date.
> > > >
> > > > As a best effort, can you please in Jira flag the Jira with
> fix
> > version 2.3.0
> > > > and blocker for the release. Then, I will know when I can
> start the
> > > release process.
> > > >
> > > > Thanks !
> > > >
> > > > Regards
> > > > JB
> > > >
> > >
> > > --
> > > Jean-Baptiste Onofré
> > > jbono...@apache.org 
> > >
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> > >
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org 
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Build failed in Jenkins: beam_PostRelease_NightlySnapshot #9

[Apache Jenkins Server]

See 


Changes:

[iemejia] Refactor code into idiomatic Java 8 style

[iemejia] Fix missing gearpump module activated only on Java 8 profile

[iemejia] Add missing amazon-web-services module from javadoc

[jbonofre] [BEAM-2271] Exclude files not intended to be in the releases source

--
Started by timer
[EnvInject] - Loading node environment variables.
Building remotely on beam4 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision e59c19d2e3f4684f1ceabbfc816c2e4b3b54a6a2 (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f e59c19d2e3f4684f1ceabbfc816c2e4b3b54a6a2
Commit message: "Merge pull request #4498 from jbonofre/BEAM-2271"
 > git rev-list b2e92836202ca32e90e9da3e1fd0fa09a0c781a7 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[beam_PostRelease_NightlySnapshot] $ /bin/bash -xe 
/tmp/jenkins8076369279855512185.sh
+ cd src/release
+ groovy quickstart-java-direct.groovy
/tmp/jenkins8076369279855512185.sh: line 2: groovy: command not found
Build step 'Execute shell' marked build as failure
Not sending mail to unregistered user ke...@google.com
Not sending mail to unregistered user z...@giggles.nyc.corp.google.com
Not sending mail to unregistered user xuming...@users.noreply.github.com
Not sending mail to unregistered user kirpic...@google.com

Re: [HEADS UP] Preparing Beam 2.3.0

[Jean-Baptiste Onofré]

By the way, I created BEAM-3551 for the tracking.

I will work on it today.

Regards
JB

On 01/28/2018 09:50 AM, Romain Manni-Bucau wrote:
> Hi guys
> 
> Out if curiosity, can -parameters (javac) be part of the 2.3 if not already?
> 
> Le 27 janv. 2018 18:39, "Jean-Baptiste Onofré"  > a écrit :
> 
> Hi Reuven,
> 
> I gonna bump 3392 and 3087 to 2.4.0. For the PR, yes Eugene did a first 
> round
> review, I will work on it now.
> 
> We will pretty close !
> 
> Thanks !
> Regards
> JB
> 
> On 01/27/2018 05:58 PM, Reuven Lax wrote:
> > Seems that 3392 is not a blocker, and neither is 3087. Looks like 
> Eugene is
> > already reviewing the PR form BEAM-793.
> >
> > Reuven
> >
> > On Sat, Jan 27, 2018 at 4:00 AM, Jean-Baptiste Onofré  
> > >> wrote:
> >
> >     Hi guys,
> >
> >     we still have 7 Jira targeted to 2.3.0.
> >
> >     For most of them, Ismaël and I are doing the PRs/fixes and we have
> review in
> >     progress.
> >
> >     I'm a little bit concerned by BEAM-3392: it's flagged as blocker 
> but it's
> >     related to a specific branch. Can you please provide an update asap 
> ?
> >
> >     However, I didn't have any update for BEAM-3087 (related to the 
> Flink
> runner).
> >     Without update soon, I will bump to 2.4.0.
> >
> >     I would need a review on PR for BEAM-793 (PR #4500). To avoid to 
> break
> anything
> >     for existing user, I set the backoff strategy optional, the user 
> has to
> >     explicitly set to use it.
> >
> >     I'm waiting a little more before cutting the release (especially for
> BEAM-3392
> >     and BEAM-3087). However, I would like to cut the release asap.
> >
> >     Thanks,
> >     Regards
> >     JB
> >
> >
> >     On 01/23/2018 10:39 AM, Jean-Baptiste Onofré wrote:
> >     > Hi guys,
> >     >
> >     > Some days ago, I proposed to start Beam 2.3.0 around January 26th.
> So, we are
> >     > few days from this date.
> >     >
> >     > As a best effort, can you please in Jira flag the Jira with fix
> version 2.3.0
> >     > and blocker for the release. Then, I will know when I can start 
> the
> >     release process.
> >     >
> >     > Thanks !
> >     >
> >     > Regards
> >     > JB
> >     >
> >
> >     --
> >     Jean-Baptiste Onofré
> >     jbono...@apache.org 
> >
> >     http://blog.nanthrax.net
> >     Talend - http://www.talend.com
> >
> >
> 
> --
> Jean-Baptiste Onofré
> jbono...@apache.org 
> http://blog.nanthrax.net
> Talend - http://www.talend.com
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [HEADS UP] Preparing Beam 2.3.0

[Jean-Baptiste Onofré]

Hi,

it's not yet done. Let me do a PR to test that.

Regards
JB

On 01/28/2018 09:50 AM, Romain Manni-Bucau wrote:
> Hi guys
> 
> Out if curiosity, can -parameters (javac) be part of the 2.3 if not already?
> 
> Le 27 janv. 2018 18:39, "Jean-Baptiste Onofré"  > a écrit :
> 
> Hi Reuven,
> 
> I gonna bump 3392 and 3087 to 2.4.0. For the PR, yes Eugene did a first 
> round
> review, I will work on it now.
> 
> We will pretty close !
> 
> Thanks !
> Regards
> JB
> 
> On 01/27/2018 05:58 PM, Reuven Lax wrote:
> > Seems that 3392 is not a blocker, and neither is 3087. Looks like 
> Eugene is
> > already reviewing the PR form BEAM-793.
> >
> > Reuven
> >
> > On Sat, Jan 27, 2018 at 4:00 AM, Jean-Baptiste Onofré  
> > >> wrote:
> >
> >     Hi guys,
> >
> >     we still have 7 Jira targeted to 2.3.0.
> >
> >     For most of them, Ismaël and I are doing the PRs/fixes and we have
> review in
> >     progress.
> >
> >     I'm a little bit concerned by BEAM-3392: it's flagged as blocker 
> but it's
> >     related to a specific branch. Can you please provide an update asap 
> ?
> >
> >     However, I didn't have any update for BEAM-3087 (related to the 
> Flink
> runner).
> >     Without update soon, I will bump to 2.4.0.
> >
> >     I would need a review on PR for BEAM-793 (PR #4500). To avoid to 
> break
> anything
> >     for existing user, I set the backoff strategy optional, the user 
> has to
> >     explicitly set to use it.
> >
> >     I'm waiting a little more before cutting the release (especially for
> BEAM-3392
> >     and BEAM-3087). However, I would like to cut the release asap.
> >
> >     Thanks,
> >     Regards
> >     JB
> >
> >
> >     On 01/23/2018 10:39 AM, Jean-Baptiste Onofré wrote:
> >     > Hi guys,
> >     >
> >     > Some days ago, I proposed to start Beam 2.3.0 around January 26th.
> So, we are
> >     > few days from this date.
> >     >
> >     > As a best effort, can you please in Jira flag the Jira with fix
> version 2.3.0
> >     > and blocker for the release. Then, I will know when I can start 
> the
> >     release process.
> >     >
> >     > Thanks !
> >     >
> >     > Regards
> >     > JB
> >     >
> >
> >     --
> >     Jean-Baptiste Onofré
> >     jbono...@apache.org 
> >
> >     http://blog.nanthrax.net
> >     Talend - http://www.talend.com
> >
> >
> 
> --
> Jean-Baptiste Onofré
> jbono...@apache.org 
> http://blog.nanthrax.net
> Talend - http://www.talend.com
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [HEADS UP] Preparing Beam 2.3.0

[Romain Manni-Bucau]

Hi guys

Out if curiosity, can -parameters (javac) be part of the 2.3 if not already?

Le 27 janv. 2018 18:39, "Jean-Baptiste Onofré"  a écrit :

> Hi Reuven,
>
> I gonna bump 3392 and 3087 to 2.4.0. For the PR, yes Eugene did a first
> round
> review, I will work on it now.
>
> We will pretty close !
>
> Thanks !
> Regards
> JB
>
> On 01/27/2018 05:58 PM, Reuven Lax wrote:
> > Seems that 3392 is not a blocker, and neither is 3087. Looks like Eugene
> is
> > already reviewing the PR form BEAM-793.
> >
> > Reuven
> >
> > On Sat, Jan 27, 2018 at 4:00 AM, Jean-Baptiste Onofré  > > wrote:
> >
> > Hi guys,
> >
> > we still have 7 Jira targeted to 2.3.0.
> >
> > For most of them, Ismaël and I are doing the PRs/fixes and we have
> review in
> > progress.
> >
> > I'm a little bit concerned by BEAM-3392: it's flagged as blocker but
> it's
> > related to a specific branch. Can you please provide an update asap ?
> >
> > However, I didn't have any update for BEAM-3087 (related to the
> Flink runner).
> > Without update soon, I will bump to 2.4.0.
> >
> > I would need a review on PR for BEAM-793 (PR #4500). To avoid to
> break anything
> > for existing user, I set the backoff strategy optional, the user has
> to
> > explicitly set to use it.
> >
> > I'm waiting a little more before cutting the release (especially for
> BEAM-3392
> > and BEAM-3087). However, I would like to cut the release asap.
> >
> > Thanks,
> > Regards
> > JB
> >
> >
> > On 01/23/2018 10:39 AM, Jean-Baptiste Onofré wrote:
> > > Hi guys,
> > >
> > > Some days ago, I proposed to start Beam 2.3.0 around January 26th.
> So, we are
> > > few days from this date.
> > >
> > > As a best effort, can you please in Jira flag the Jira with fix
> version 2.3.0
> > > and blocker for the release. Then, I will know when I can start the
> > release process.
> > >
> > > Thanks !
> > >
> > > Regards
> > > JB
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org 
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
> >
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>