Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Jean-Baptiste Onofré]

Thanks to Kenn, I'm able to move forward on RC2.

I will start the RC2 release process in the coming hour.

Regards
JB

On 02/06/2018 11:17 PM, Kenneth Knowles wrote:
> I've cherry-picked JB's commit to https://github.com/apache/beam/pull/4621
> 
> On master it seems that HBase tests are failing reliably, but actually mostly
> isolated to this PR. Hopefully the changes on the release branch will go 
> green.
> 
> On Tue, Feb 6, 2018 at 2:12 PM, Kenneth Knowles  > wrote:
> 
> Changes are cherrypicked to the release-2.3.0 branch, or applied directly 
> there.
> 
> On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov  > wrote:
> 
> Possibly stupid question: are new RCs created from master, or by
> cherrypicks of desired changes on top of previous RCs?
> (it affects whether or not my recently merged change on "master" will 
> be
> in 2.3.0; I'd like it to be :) )
> 
> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik  > wrote:
> 
> +1 for rollback as well.
> 
> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax  > wrote:
> 
> +1 for the rollback.
> 
> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles 
>  > wrote:
> 
> +1 for option 1. It is almost certainly repeated proto
> parsing. That is fixed for ParDo via generalized caching 
> but
> not for other transforms. Flink also has a proto round 
> trip,
> but the design might avoid the problem anyhow. We should
> investigate before releasing, or just preemptively roll it
> back too. The proto round trips are helpful for maturing
> portability and for preventing hacks from returning but
> otherwise don't matter so much yet.
> 
> Kenn
> 
> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré
> mailto:j...@nanthrax.net>> wrote:
> 
> Hi all,
> 
> As you might have seen, I found the cause of 
> performance
> degradation on the
> direct runner (BEAM-3617).
> 
> We have basically three options for RC2:
> 
> 1. We revert the change for 2.3.0. I think it's the
> fastest and more secure way.
> I created a PR for that
> (https://github.com/apache/beam/pull/4609
> ). It gives 
> us
> time to investigate and provide a better fix in Runner
> API for 2.4.0.
> 
> 2. If the fix is easy and with limited impact in 
> Runner
> API, we do that.
> 
> 3. We leave as it is, adding a note that direct runner
> has worse performance in
> 2.3.0 compared to 2.2.0.
> 
> In any case, I would like to cut RC2 later tonight or
> tomorrow morning (my time).
> 
> Thoughts ?
> 
> Regards
> JB
> 
> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> > Hi Reuven,
> >
> > it's what I'm suspecting. git bisect should give us
> more information (still in
> > progress, 3 more steps to complete ;)). I keep you 
> posted.
> >
> > Regards
> > JB
> >
> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
> >> Could this be related to any of the portability 
> changes?
> >>
> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste 
> Onofré
> mailto:j...@nanthrax.net>
> >>  >> wrote:
> >>
> >>     Created:
> >>
> >>     https://issues.apache.org/jira/browse/BEAM-3617
> 
> >>     
>  >
> >>
> >>     Regards
> >>     JB
> >>
> >>     On 02/05/2018 

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Kenneth Knowles]

I've cherry-picked JB's commit to https://github.com/apache/beam/pull/4621

On master it seems that HBase tests are failing reliably, but actually
mostly isolated to this PR. Hopefully the changes on the release branch
will go green.

On Tue, Feb 6, 2018 at 2:12 PM, Kenneth Knowles  wrote:

> Changes are cherrypicked to the release-2.3.0 branch, or applied directly
> there.
>
> On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov 
> wrote:
>
>> Possibly stupid question: are new RCs created from master, or by
>> cherrypicks of desired changes on top of previous RCs?
>> (it affects whether or not my recently merged change on "master" will be
>> in 2.3.0; I'd like it to be :) )
>>
>> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik  wrote:
>>
>>> +1 for rollback as well.
>>>
>>> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax  wrote:
>>>
 +1 for the rollback.

 On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles  wrote:

> +1 for option 1. It is almost certainly repeated proto parsing. That
> is fixed for ParDo via generalized caching but not for other transforms.
> Flink also has a proto round trip, but the design might avoid the problem
> anyhow. We should investigate before releasing, or just preemptively roll
> it back too. The proto round trips are helpful for maturing portability 
> and
> for preventing hacks from returning but otherwise don't matter so much 
> yet.
>
> Kenn
>
> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré 
> wrote:
>
>> Hi all,
>>
>> As you might have seen, I found the cause of performance degradation
>> on the
>> direct runner (BEAM-3617).
>>
>> We have basically three options for RC2:
>>
>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>> secure way.
>> I created a PR for that (https://github.com/apache/beam/pull/4609).
>> It gives us
>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>
>> 2. If the fix is easy and with limited impact in Runner API, we do
>> that.
>>
>> 3. We leave as it is, adding a note that direct runner has worse
>> performance in
>> 2.3.0 compared to 2.2.0.
>>
>> In any case, I would like to cut RC2 later tonight or tomorrow
>> morning (my time).
>>
>> Thoughts ?
>>
>> Regards
>> JB
>>
>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>> > Hi Reuven,
>> >
>> > it's what I'm suspecting. git bisect should give us more
>> information (still in
>> > progress, 3 more steps to complete ;)). I keep you posted.
>> >
>> > Regards
>> > JB
>> >
>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>> >> Could this be related to any of the portability changes?
>> >>
>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
>> j...@nanthrax.net
>> >> > wrote:
>> >>
>> >> Created:
>> >>
>> >> https://issues.apache.org/jira/browse/BEAM-3617
>> >> 
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> >> > What is the Jira for direct runner perf?
>> >> >
>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>> j...@nanthrax.net 
>> >> > >> wrote:
>> >> >
>> >> > Thanks !
>> >> >
>> >> > I cherry-pick on release-2.3.0 branch.
>> >> >
>> >> > I'm on the direct runner perf test in the mean time.
>> >> >
>> >> > Thanks again !
>> >> >
>> >> > Regards
>> >> > JB
>> >> >
>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> >> > > I merged fixes for:
>> >> > >  - https://issues.apache.org/jira/browse/BEAM-3186
>> >> 
>> >> > > >> >
>> >> > >  - https://issues.apache.org/jira/browse/BEAM-3589
>> >> 
>> >> > > >> >
>> >> > >
>> >> > > @JB I didn't yet merge them on the 2.3.0 branch,
>> though, but I can or you can go
>> >> > > ahead.
>> >> > >
>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>> j...@nanthrax.net  > >> >
>> >> > >> 
>> >> 

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Kenneth Knowles]

Changes are cherrypicked to the release-2.3.0 branch, or applied directly
there.

On Tue, Feb 6, 2018 at 2:08 PM, Eugene Kirpichov 
wrote:

> Possibly stupid question: are new RCs created from master, or by
> cherrypicks of desired changes on top of previous RCs?
> (it affects whether or not my recently merged change on "master" will be
> in 2.3.0; I'd like it to be :) )
>
> On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik  wrote:
>
>> +1 for rollback as well.
>>
>> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax  wrote:
>>
>>> +1 for the rollback.
>>>
>>> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles  wrote:
>>>
 +1 for option 1. It is almost certainly repeated proto parsing. That is
 fixed for ParDo via generalized caching but not for other transforms. Flink
 also has a proto round trip, but the design might avoid the problem anyhow.
 We should investigate before releasing, or just preemptively roll it back
 too. The proto round trips are helpful for maturing portability and for
 preventing hacks from returning but otherwise don't matter so much yet.

 Kenn

 On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré 
 wrote:

> Hi all,
>
> As you might have seen, I found the cause of performance degradation
> on the
> direct runner (BEAM-3617).
>
> We have basically three options for RC2:
>
> 1. We revert the change for 2.3.0. I think it's the fastest and more
> secure way.
> I created a PR for that (https://github.com/apache/beam/pull/4609).
> It gives us
> time to investigate and provide a better fix in Runner API for 2.4.0.
>
> 2. If the fix is easy and with limited impact in Runner API, we do
> that.
>
> 3. We leave as it is, adding a note that direct runner has worse
> performance in
> 2.3.0 compared to 2.2.0.
>
> In any case, I would like to cut RC2 later tonight or tomorrow morning
> (my time).
>
> Thoughts ?
>
> Regards
> JB
>
> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> > Hi Reuven,
> >
> > it's what I'm suspecting. git bisect should give us more information
> (still in
> > progress, 3 more steps to complete ;)). I keep you posted.
> >
> > Regards
> > JB
> >
> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
> >> Could this be related to any of the portability changes?
> >>
> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
> j...@nanthrax.net
> >> > wrote:
> >>
> >> Created:
> >>
> >> https://issues.apache.org/jira/browse/BEAM-3617
> >> 
> >>
> >> Regards
> >> JB
> >>
> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> >> > What is the Jira for direct runner perf?
> >> >
> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
> j...@nanthrax.net 
> >> > >> wrote:
> >> >
> >> > Thanks !
> >> >
> >> > I cherry-pick on release-2.3.0 branch.
> >> >
> >> > I'm on the direct runner perf test in the mean time.
> >> >
> >> > Thanks again !
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> >> > > I merged fixes for:
> >> > >  - https://issues.apache.org/jira/browse/BEAM-3186
> >> 
> >> >  >> >
> >> > >  - https://issues.apache.org/jira/browse/BEAM-3589
> >> 
> >> >  >> >
> >> > >
> >> > > @JB I didn't yet merge them on the 2.3.0 branch,
> though, but I can or you can go
> >> > > ahead.
> >> > >
> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
> j...@nanthrax.net   >> >
> >> > >> 
> >>  >> > >>
> >> > >> Hi guys,
> >> > >>
> >> > >> Quick update on the RC2 preparation:
> >> > >>
> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
> custom build with
> >> > Gradle (not
> >> > >> using artifacts created by Maven). Anyway, I will take
> a look today.
> >> >

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Eugene Kirpichov]

Possibly stupid question: are new RCs created from master, or by
cherrypicks of desired changes on top of previous RCs?
(it affects whether or not my recently merged change on "master" will be in
2.3.0; I'd like it to be :) )

On Tue, Feb 6, 2018 at 1:49 PM Lukasz Cwik  wrote:

> +1 for rollback as well.
>
> On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax  wrote:
>
>> +1 for the rollback.
>>
>> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles  wrote:
>>
>>> +1 for option 1. It is almost certainly repeated proto parsing. That is
>>> fixed for ParDo via generalized caching but not for other transforms. Flink
>>> also has a proto round trip, but the design might avoid the problem anyhow.
>>> We should investigate before releasing, or just preemptively roll it back
>>> too. The proto round trips are helpful for maturing portability and for
>>> preventing hacks from returning but otherwise don't matter so much yet.
>>>
>>> Kenn
>>>
>>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré 
>>> wrote:
>>>
 Hi all,

 As you might have seen, I found the cause of performance degradation on
 the
 direct runner (BEAM-3617).

 We have basically three options for RC2:

 1. We revert the change for 2.3.0. I think it's the fastest and more
 secure way.
 I created a PR for that (https://github.com/apache/beam/pull/4609). It
 gives us
 time to investigate and provide a better fix in Runner API for 2.4.0.

 2. If the fix is easy and with limited impact in Runner API, we do that.

 3. We leave as it is, adding a note that direct runner has worse
 performance in
 2.3.0 compared to 2.2.0.

 In any case, I would like to cut RC2 later tonight or tomorrow morning
 (my time).

 Thoughts ?

 Regards
 JB

 On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
 > Hi Reuven,
 >
 > it's what I'm suspecting. git bisect should give us more information
 (still in
 > progress, 3 more steps to complete ;)). I keep you posted.
 >
 > Regards
 > JB
 >
 > On 02/06/2018 08:36 AM, Reuven Lax wrote:
 >> Could this be related to any of the portability changes?
 >>
 >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré <
 j...@nanthrax.net
 >> > wrote:
 >>
 >> Created:
 >>
 >> https://issues.apache.org/jira/browse/BEAM-3617
 >> 
 >>
 >> Regards
 >> JB
 >>
 >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
 >> > What is the Jira for direct runner perf?
 >> >
 >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
 j...@nanthrax.net 
 >> > >> wrote:
 >> >
 >> > Thanks !
 >> >
 >> > I cherry-pick on release-2.3.0 branch.
 >> >
 >> > I'm on the direct runner perf test in the mean time.
 >> >
 >> > Thanks again !
 >> >
 >> > Regards
 >> > JB
 >> >
 >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
 >> > > I merged fixes for:
 >> > >  - https://issues.apache.org/jira/browse/BEAM-3186
 >> 
 >> > > >
 >> > >  - https://issues.apache.org/jira/browse/BEAM-3589
 >> 
 >> > > >
 >> > >
 >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
 but I can or you can go
 >> > > ahead.
 >> > >
 >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
 j...@nanthrax.net  > >
 >> > >> 
 >> > > >>
 >> > >> Hi guys,
 >> > >>
 >> > >> Quick update on the RC2 preparation:
 >> > >>
 >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
 custom build with
 >> > Gradle (not
 >> > >> using artifacts created by Maven). Anyway, I will take
 a look today.
 >> > >> * BEAM-3186 has a PR. Aljoscha will do the review
 pretty soon.
 >> > >> * I'm also taking a look on the direct runner
 performance change.
 >> > >>
 >> > >> The goal is to cut RC2 tomorrow or the day after.
 >> > >>

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Lukasz Cwik]

+1 for rollback as well.

On Tue, Feb 6, 2018 at 12:37 PM, Reuven Lax  wrote:

> +1 for the rollback.
>
> On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles  wrote:
>
>> +1 for option 1. It is almost certainly repeated proto parsing. That is
>> fixed for ParDo via generalized caching but not for other transforms. Flink
>> also has a proto round trip, but the design might avoid the problem anyhow.
>> We should investigate before releasing, or just preemptively roll it back
>> too. The proto round trips are helpful for maturing portability and for
>> preventing hacks from returning but otherwise don't matter so much yet.
>>
>> Kenn
>>
>> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré 
>> wrote:
>>
>>> Hi all,
>>>
>>> As you might have seen, I found the cause of performance degradation on
>>> the
>>> direct runner (BEAM-3617).
>>>
>>> We have basically three options for RC2:
>>>
>>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>>> secure way.
>>> I created a PR for that (https://github.com/apache/beam/pull/4609). It
>>> gives us
>>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>>
>>> 2. If the fix is easy and with limited impact in Runner API, we do that.
>>>
>>> 3. We leave as it is, adding a note that direct runner has worse
>>> performance in
>>> 2.3.0 compared to 2.2.0.
>>>
>>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>>> (my time).
>>>
>>> Thoughts ?
>>>
>>> Regards
>>> JB
>>>
>>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>>> > Hi Reuven,
>>> >
>>> > it's what I'm suspecting. git bisect should give us more information
>>> (still in
>>> > progress, 3 more steps to complete ;)). I keep you posted.
>>> >
>>> > Regards
>>> > JB
>>> >
>>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>>> >> Could this be related to any of the portability changes?
>>> >>
>>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré >> >> > wrote:
>>> >>
>>> >> Created:
>>> >>
>>> >> https://issues.apache.org/jira/browse/BEAM-3617
>>> >> 
>>> >>
>>> >> Regards
>>> >> JB
>>> >>
>>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>>> >> > What is the Jira for direct runner perf?
>>> >> >
>>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>>> j...@nanthrax.net 
>>> >> > >> wrote:
>>> >> >
>>> >> > Thanks !
>>> >> >
>>> >> > I cherry-pick on release-2.3.0 branch.
>>> >> >
>>> >> > I'm on the direct runner perf test in the mean time.
>>> >> >
>>> >> > Thanks again !
>>> >> >
>>> >> > Regards
>>> >> > JB
>>> >> >
>>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>>> >> > > I merged fixes for:
>>> >> > >  - https://issues.apache.org/jira/browse/BEAM-3186
>>> >> 
>>> >> > >> >> >
>>> >> > >  - https://issues.apache.org/jira/browse/BEAM-3589
>>> >> 
>>> >> > >> >> >
>>> >> > >
>>> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
>>> but I can or you can go
>>> >> > > ahead.
>>> >> > >
>>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>>> j...@nanthrax.net  >> >> >
>>> >> > >> 
>>> >> >> >> > >>
>>> >> > >> Hi guys,
>>> >> > >>
>>> >> > >> Quick update on the RC2 preparation:
>>> >> > >>
>>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a
>>> custom build with
>>> >> > Gradle (not
>>> >> > >> using artifacts created by Maven). Anyway, I will take a
>>> look today.
>>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
>>> soon.
>>> >> > >> * I'm also taking a look on the direct runner
>>> performance change.
>>> >> > >>
>>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>>> >> > >>
>>> >> > >> Regards
>>> >> > >> JB
>>> >> > >>
>>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>> >> > >>> Hi guys,
>>> >> > >>>
>>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
>>> cancel RC1.
>>> >> > >>>
>>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>>> >> > >>>
>>> >> > >>> I'm updating

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Reuven Lax]

+1 for the rollback.

On Tue, Feb 6, 2018 at 9:24 AM, Kenneth Knowles  wrote:

> +1 for option 1. It is almost certainly repeated proto parsing. That is
> fixed for ParDo via generalized caching but not for other transforms. Flink
> also has a proto round trip, but the design might avoid the problem anyhow.
> We should investigate before releasing, or just preemptively roll it back
> too. The proto round trips are helpful for maturing portability and for
> preventing hacks from returning but otherwise don't matter so much yet.
>
> Kenn
>
> On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré 
> wrote:
>
>> Hi all,
>>
>> As you might have seen, I found the cause of performance degradation on
>> the
>> direct runner (BEAM-3617).
>>
>> We have basically three options for RC2:
>>
>> 1. We revert the change for 2.3.0. I think it's the fastest and more
>> secure way.
>> I created a PR for that (https://github.com/apache/beam/pull/4609). It
>> gives us
>> time to investigate and provide a better fix in Runner API for 2.4.0.
>>
>> 2. If the fix is easy and with limited impact in Runner API, we do that.
>>
>> 3. We leave as it is, adding a note that direct runner has worse
>> performance in
>> 2.3.0 compared to 2.2.0.
>>
>> In any case, I would like to cut RC2 later tonight or tomorrow morning
>> (my time).
>>
>> Thoughts ?
>>
>> Regards
>> JB
>>
>> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
>> > Hi Reuven,
>> >
>> > it's what I'm suspecting. git bisect should give us more information
>> (still in
>> > progress, 3 more steps to complete ;)). I keep you posted.
>> >
>> > Regards
>> > JB
>> >
>> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
>> >> Could this be related to any of the portability changes?
>> >>
>> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré > >> > wrote:
>> >>
>> >> Created:
>> >>
>> >> https://issues.apache.org/jira/browse/BEAM-3617
>> >> 
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> >> > What is the Jira for direct runner perf?
>> >> >
>> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
>> j...@nanthrax.net 
>> >> > >> wrote:
>> >> >
>> >> > Thanks !
>> >> >
>> >> > I cherry-pick on release-2.3.0 branch.
>> >> >
>> >> > I'm on the direct runner perf test in the mean time.
>> >> >
>> >> > Thanks again !
>> >> >
>> >> > Regards
>> >> > JB
>> >> >
>> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> >> > > I merged fixes for:
>> >> > >  - https://issues.apache.org/jira/browse/BEAM-3186
>> >> 
>> >> > > >> >
>> >> > >  - https://issues.apache.org/jira/browse/BEAM-3589
>> >> 
>> >> > > >> >
>> >> > >
>> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
>> but I can or you can go
>> >> > > ahead.
>> >> > >
>> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
>> j...@nanthrax.net  > >> >
>> >> > >> 
>> >> > >> > >>
>> >> > >> Hi guys,
>> >> > >>
>> >> > >> Quick update on the RC2 preparation:
>> >> > >>
>> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom
>> build with
>> >> > Gradle (not
>> >> > >> using artifacts created by Maven). Anyway, I will take a
>> look today.
>> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
>> soon.
>> >> > >> * I'm also taking a look on the direct runner performance
>> change.
>> >> > >>
>> >> > >> The goal is to cut RC2 tomorrow or the day after.
>> >> > >>
>> >> > >> Regards
>> >> > >> JB
>> >> > >>
>> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> >> > >>> Hi guys,
>> >> > >>>
>> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
>> cancel RC1.
>> >> > >>>
>> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
>> >> > >>>
>> >> > >>> I'm updating Jira right now. When the fixes will be
>> cherry-picked, I
>> >> > will submit
>> >> > >>> a RC2 to vote.
>> >> > >>>
>> >> > >>> Thanks !
>> >> > >>> Regards
>> >> > 

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Kenneth Knowles]

+1 for option 1. It is almost certainly repeated proto parsing. That is
fixed for ParDo via generalized caching but not for other transforms. Flink
also has a proto round trip, but the design might avoid the problem anyhow.
We should investigate before releasing, or just preemptively roll it back
too. The proto round trips are helpful for maturing portability and for
preventing hacks from returning but otherwise don't matter so much yet.

Kenn

On Tue, Feb 6, 2018 at 8:54 AM, Jean-Baptiste Onofré 
wrote:

> Hi all,
>
> As you might have seen, I found the cause of performance degradation on the
> direct runner (BEAM-3617).
>
> We have basically three options for RC2:
>
> 1. We revert the change for 2.3.0. I think it's the fastest and more
> secure way.
> I created a PR for that (https://github.com/apache/beam/pull/4609). It
> gives us
> time to investigate and provide a better fix in Runner API for 2.4.0.
>
> 2. If the fix is easy and with limited impact in Runner API, we do that.
>
> 3. We leave as it is, adding a note that direct runner has worse
> performance in
> 2.3.0 compared to 2.2.0.
>
> In any case, I would like to cut RC2 later tonight or tomorrow morning (my
> time).
>
> Thoughts ?
>
> Regards
> JB
>
> On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> > Hi Reuven,
> >
> > it's what I'm suspecting. git bisect should give us more information
> (still in
> > progress, 3 more steps to complete ;)). I keep you posted.
> >
> > Regards
> > JB
> >
> > On 02/06/2018 08:36 AM, Reuven Lax wrote:
> >> Could this be related to any of the portability changes?
> >>
> >> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré  >> > wrote:
> >>
> >> Created:
> >>
> >> https://issues.apache.org/jira/browse/BEAM-3617
> >> 
> >>
> >> Regards
> >> JB
> >>
> >> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> >> > What is the Jira for direct runner perf?
> >> >
> >> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré <
> j...@nanthrax.net 
> >> > >> wrote:
> >> >
> >> > Thanks !
> >> >
> >> > I cherry-pick on release-2.3.0 branch.
> >> >
> >> > I'm on the direct runner perf test in the mean time.
> >> >
> >> > Thanks again !
> >> >
> >> > Regards
> >> > JB
> >> >
> >> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> >> > > I merged fixes for:
> >> > >  - https://issues.apache.org/jira/browse/BEAM-3186
> >> 
> >> >  >> >
> >> > >  - https://issues.apache.org/jira/browse/BEAM-3589
> >> 
> >> >  >> >
> >> > >
> >> > > @JB I didn't yet merge them on the 2.3.0 branch, though,
> but I can or you can go
> >> > > ahead.
> >> > >
> >> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré <
> j...@nanthrax.net   >> >
> >> > >> 
> >>  >> > >>
> >> > >> Hi guys,
> >> > >>
> >> > >> Quick update on the RC2 preparation:
> >> > >>
> >> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom
> build with
> >> > Gradle (not
> >> > >> using artifacts created by Maven). Anyway, I will take a
> look today.
> >> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty
> soon.
> >> > >> * I'm also taking a look on the direct runner performance
> change.
> >> > >>
> >> > >> The goal is to cut RC2 tomorrow or the day after.
> >> > >>
> >> > >> Regards
> >> > >> JB
> >> > >>
> >> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >> > >>> Hi guys,
> >> > >>>
> >> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I
> cancel RC1.
> >> > >>>
> >> > >>> We will cherry-pick fixes on release-2.3.0 branch.
> >> > >>>
> >> > >>> I'm updating Jira right now. When the fixes will be
> cherry-picked, I
> >> > will submit
> >> > >>> a RC2 to vote.
> >> > >>>
> >> > >>> Thanks !
> >> > >>> Regards
> >> > >>> JB
> >> > >>>
> >> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >> >  Hi everyone,
> >> > 
> >> >  Please review and vote on the release candidate #1 for
> t

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Jean-Baptiste Onofré]

Hi all,

As you might have seen, I found the cause of performance degradation on the
direct runner (BEAM-3617).

We have basically three options for RC2:

1. We revert the change for 2.3.0. I think it's the fastest and more secure way.
I created a PR for that (https://github.com/apache/beam/pull/4609). It gives us
time to investigate and provide a better fix in Runner API for 2.4.0.

2. If the fix is easy and with limited impact in Runner API, we do that.

3. We leave as it is, adding a note that direct runner has worse performance in
2.3.0 compared to 2.2.0.

In any case, I would like to cut RC2 later tonight or tomorrow morning (my 
time).

Thoughts ?

Regards
JB

On 02/06/2018 09:42 AM, Jean-Baptiste Onofré wrote:
> Hi Reuven,
> 
> it's what I'm suspecting. git bisect should give us more information (still in
> progress, 3 more steps to complete ;)). I keep you posted.
> 
> Regards
> JB
> 
> On 02/06/2018 08:36 AM, Reuven Lax wrote:
>> Could this be related to any of the portability changes?
>>
>> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré > > wrote:
>>
>> Created:
>>
>> https://issues.apache.org/jira/browse/BEAM-3617
>> 
>>
>> Regards
>> JB
>>
>> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> > What is the Jira for direct runner perf?
>> >
>> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré 
>> mailto:j...@nanthrax.net>
>> > >> wrote:
>> >
>> >     Thanks !
>> >
>> >     I cherry-pick on release-2.3.0 branch.
>> >
>> >     I'm on the direct runner perf test in the mean time.
>> >
>> >     Thanks again !
>> >
>> >     Regards
>> >     JB
>> >
>> >     On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> >     > I merged fixes for:
>> >     >  - https://issues.apache.org/jira/browse/BEAM-3186
>> 
>> >     > >
>> >     >  - https://issues.apache.org/jira/browse/BEAM-3589
>> 
>> >     > >
>> >     >
>> >     > @JB I didn't yet merge them on the 2.3.0 branch, though, but I 
>> can or you can go
>> >     > ahead.
>> >     >
>> >     >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré 
>> mailto:j...@nanthrax.net> > >
>> >     >> 
>> > >     >>
>> >     >> Hi guys,
>> >     >>
>> >     >> Quick update on the RC2 preparation:
>> >     >>
>> >     >> * BEAM-3587 (TextIO with Flink) seems related to a custom build 
>> with
>> >     Gradle (not
>> >     >> using artifacts created by Maven). Anyway, I will take a look 
>> today.
>> >     >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> >     >> * I'm also taking a look on the direct runner performance 
>> change.
>> >     >>
>> >     >> The goal is to cut RC2 tomorrow or the day after.
>> >     >>
>> >     >> Regards
>> >     >> JB
>> >     >>
>> >     >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> >     >>> Hi guys,
>> >     >>>
>> >     >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel 
>> RC1.
>> >     >>>
>> >     >>> We will cherry-pick fixes on release-2.3.0 branch.
>> >     >>>
>> >     >>> I'm updating Jira right now. When the fixes will be 
>> cherry-picked, I
>> >     will submit
>> >     >>> a RC2 to vote.
>> >     >>>
>> >     >>> Thanks !
>> >     >>> Regards
>> >     >>> JB
>> >     >>>
>> >     >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> >      Hi everyone,
>> >     
>> >      Please review and vote on the release candidate #1 for the 
>> version
>> >     2.3.0, as
>> >      follows:
>> >     
>> >      [ ] +1, Approve the release
>> >      [ ] -1, Do not approve the release (please provide specific 
>> comments)
>> >     
>> >     
>> >      The complete staging area is available for your review, which
>> includes:
>> >      * JIRA release notes [1],
>> >      * the official Apache source release to be deployed to
>> dist.apache.org 
>> >     
>> >       [2],
>> >      which is signed with the key with fingerprint C8282E76 [3],
>> >      * all artifacts to be deployed to the Maven Central 
>> Repository [4

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Jean-Baptiste Onofré]

Hi Reuven,

it's what I'm suspecting. git bisect should give us more information (still in
progress, 3 more steps to complete ;)). I keep you posted.

Regards
JB

On 02/06/2018 08:36 AM, Reuven Lax wrote:
> Could this be related to any of the portability changes?
> 
> On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré  > wrote:
> 
> Created:
> 
> https://issues.apache.org/jira/browse/BEAM-3617
> 
> 
> Regards
> JB
> 
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> > What is the Jira for direct runner perf?
> >
> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré  
> > >> wrote:
> >
> >     Thanks !
> >
> >     I cherry-pick on release-2.3.0 branch.
> >
> >     I'm on the direct runner perf test in the mean time.
> >
> >     Thanks again !
> >
> >     Regards
> >     JB
> >
> >     On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> >     > I merged fixes for:
> >     >  - https://issues.apache.org/jira/browse/BEAM-3186
> 
> >      >
> >     >  - https://issues.apache.org/jira/browse/BEAM-3589
> 
> >      >
> >     >
> >     > @JB I didn't yet merge them on the 2.3.0 branch, though, but I 
> can or you can go
> >     > ahead.
> >     >
> >     >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré 
> mailto:j...@nanthrax.net>  >
> >     >> 
>  >     >>
> >     >> Hi guys,
> >     >>
> >     >> Quick update on the RC2 preparation:
> >     >>
> >     >> * BEAM-3587 (TextIO with Flink) seems related to a custom build 
> with
> >     Gradle (not
> >     >> using artifacts created by Maven). Anyway, I will take a look 
> today.
> >     >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >     >> * I'm also taking a look on the direct runner performance change.
> >     >>
> >     >> The goal is to cut RC2 tomorrow or the day after.
> >     >>
> >     >> Regards
> >     >> JB
> >     >>
> >     >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >     >>> Hi guys,
> >     >>>
> >     >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel 
> RC1.
> >     >>>
> >     >>> We will cherry-pick fixes on release-2.3.0 branch.
> >     >>>
> >     >>> I'm updating Jira right now. When the fixes will be 
> cherry-picked, I
> >     will submit
> >     >>> a RC2 to vote.
> >     >>>
> >     >>> Thanks !
> >     >>> Regards
> >     >>> JB
> >     >>>
> >     >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >      Hi everyone,
> >     
> >      Please review and vote on the release candidate #1 for the 
> version
> >     2.3.0, as
> >      follows:
> >     
> >      [ ] +1, Approve the release
> >      [ ] -1, Do not approve the release (please provide specific 
> comments)
> >     
> >     
> >      The complete staging area is available for your review, which
> includes:
> >      * JIRA release notes [1],
> >      * the official Apache source release to be deployed to
> dist.apache.org 
> >     
> >       [2],
> >      which is signed with the key with fingerprint C8282E76 [3],
> >      * all artifacts to be deployed to the Maven Central Repository 
> [4],
> >      * source code tag "v2.3.0-RC1" [5],
> >      * website pull request listing the release and publishing the 
> API
> reference
> >      manual [6].
> >      * Java artifacts were built with Maven 3.3.9 and Oracle JDK
> 1.8.0_111.
> >      * Python artifacts are deployed along with the source release 
> to the
> >      dist.apache.org  
> 
>  [2].
> >     
> >      The vote will be open for at least 72 hours. It is adopted by 
> majority approval,
> >      with at least 3 PMC affirmative votes.
> >     
> >      Thanks,
> >      JB
> >     
> >      [1]
> >      
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=123

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Reuven Lax]

Could this be related to any of the portability changes?

On Mon, Feb 5, 2018 at 7:51 AM, Jean-Baptiste Onofré 
wrote:

> Created:
>
> https://issues.apache.org/jira/browse/BEAM-3617
>
> Regards
> JB
>
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> > What is the Jira for direct runner perf?
> >
> > On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré  > > wrote:
> >
> > Thanks !
> >
> > I cherry-pick on release-2.3.0 branch.
> >
> > I'm on the direct runner perf test in the mean time.
> >
> > Thanks again !
> >
> > Regards
> > JB
> >
> > On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > > I merged fixes for:
> > >  - https://issues.apache.org/jira/browse/BEAM-3186
> > 
> > >  - https://issues.apache.org/jira/browse/BEAM-3589
> > 
> > >
> > > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can
> or you can go
> > > ahead.
> > >
> > >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré  
> > >> >> wrote:
> > >>
> > >> Hi guys,
> > >>
> > >> Quick update on the RC2 preparation:
> > >>
> > >> * BEAM-3587 (TextIO with Flink) seems related to a custom build
> with
> > Gradle (not
> > >> using artifacts created by Maven). Anyway, I will take a look
> today.
> > >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> > >> * I'm also taking a look on the direct runner performance change.
> > >>
> > >> The goal is to cut RC2 tomorrow or the day after.
> > >>
> > >> Regards
> > >> JB
> > >>
> > >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> > >>> Hi guys,
> > >>>
> > >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel
> RC1.
> > >>>
> > >>> We will cherry-pick fixes on release-2.3.0 branch.
> > >>>
> > >>> I'm updating Jira right now. When the fixes will be
> cherry-picked, I
> > will submit
> > >>> a RC2 to vote.
> > >>>
> > >>> Thanks !
> > >>> Regards
> > >>> JB
> > >>>
> > >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
> >  Hi everyone,
> > 
> >  Please review and vote on the release candidate #1 for the
> version
> > 2.3.0, as
> >  follows:
> > 
> >  [ ] +1, Approve the release
> >  [ ] -1, Do not approve the release (please provide specific
> comments)
> > 
> > 
> >  The complete staging area is available for your review, which
> includes:
> >  * JIRA release notes [1],
> >  * the official Apache source release to be deployed to
> dist.apache.org
> > 
> >   [2],
> >  which is signed with the key with fingerprint C8282E76 [3],
> >  * all artifacts to be deployed to the Maven Central Repository
> [4],
> >  * source code tag "v2.3.0-RC1" [5],
> >  * website pull request listing the release and publishing the
> API reference
> >  manual [6].
> >  * Java artifacts were built with Maven 3.3.9 and Oracle JDK
> 1.8.0_111.
> >  * Python artifacts are deployed along with the source release
> to the
> >  dist.apache.org  <
> http://dist.apache.org> [2].
> > 
> >  The vote will be open for at least 72 hours. It is adopted by
> majority approval,
> >  with at least 3 PMC affirmative votes.
> > 
> >  Thanks,
> >  JB
> > 
> >  [1]
> >  https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
> >  projectId=12319527&version=12341608>
> >  [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> > 
> >  [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> > 
> >  [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
> >  orgapachebeam-1026/>
> >  [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> > 
> >  [6] https://github.com/apache/beam-site/pull/381
> > 
> > 
> > >>>
> > >>
> > >> --
> > >> Jean-Baptiste Onofré
> > >> jbono...@apache.org 
> > >
> > >> http://blog.nanthrax.net
> > >> Talend - http://www.talend.com
> > >
> >
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org 

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Jean-Baptiste Onofré]

Hi,

quick update about RC2: BEAM-3617 is the only Jira pending for the release.

I'm doing a git bisect to identify the commit who caused the performance
degradation.

Depending of the result, if it's a easy fix than we will try to do it for RC2,
else I will start the RC2 as now.

I will keep you posted.

Regards
JB

On 02/05/2018 04:51 PM, Jean-Baptiste Onofré wrote:
> Created:
> 
> https://issues.apache.org/jira/browse/BEAM-3617
> 
> Regards
> JB
> 
> On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
>> What is the Jira for direct runner perf?
>>
>> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré > > wrote:
>>
>> Thanks !
>>
>> I cherry-pick on release-2.3.0 branch.
>>
>> I'm on the direct runner perf test in the mean time.
>>
>> Thanks again !
>>
>> Regards
>> JB
>>
>> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
>> > I merged fixes for:
>> >  - https://issues.apache.org/jira/browse/BEAM-3186
>> 
>> >  - https://issues.apache.org/jira/browse/BEAM-3589
>> 
>> >
>> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or 
>> you can go
>> > ahead.
>> >
>> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré > 
>> >> >> wrote:
>> >>
>> >> Hi guys,
>> >>
>> >> Quick update on the RC2 preparation:
>> >>
>> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
>> Gradle (not
>> >> using artifacts created by Maven). Anyway, I will take a look today.
>> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> >> * I'm also taking a look on the direct runner performance change.
>> >>
>> >> The goal is to cut RC2 tomorrow or the day after.
>> >>
>> >> Regards
>> >> JB
>> >>
>> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> >>> Hi guys,
>> >>>
>> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>> >>>
>> >>> We will cherry-pick fixes on release-2.3.0 branch.
>> >>>
>> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
>> will submit
>> >>> a RC2 to vote.
>> >>>
>> >>> Thanks !
>> >>> Regards
>> >>> JB
>> >>>
>> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>  Hi everyone,
>> 
>>  Please review and vote on the release candidate #1 for the version
>> 2.3.0, as
>>  follows:
>> 
>>  [ ] +1, Approve the release
>>  [ ] -1, Do not approve the release (please provide specific 
>> comments)
>> 
>> 
>>  The complete staging area is available for your review, which 
>> includes:
>>  * JIRA release notes [1],
>>  * the official Apache source release to be deployed to 
>> dist.apache.org
>> 
>>   [2],
>>  which is signed with the key with fingerprint C8282E76 [3],
>>  * all artifacts to be deployed to the Maven Central Repository [4],
>>  * source code tag "v2.3.0-RC1" [5],
>>  * website pull request listing the release and publishing the API 
>> reference
>>  manual [6].
>>  * Java artifacts were built with Maven 3.3.9 and Oracle JDK 
>> 1.8.0_111.
>>  * Python artifacts are deployed along with the source release to the
>>  dist.apache.org   
>> [2].
>> 
>>  The vote will be open for at least 72 hours. It is adopted by 
>> majority approval,
>>  with at least 3 PMC affirmative votes.
>> 
>>  Thanks,
>>  JB
>> 
>>  [1]
>>  
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> 
>> 
>>  [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> 
>>  [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> 
>>  [4] 
>> https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> 
>>  [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> 
>>  [6] https://github.com/apache/beam-site/pull/381
>> 
>> 
>> >>>
>> >>
>> >> --
>> >> Jean-Baptiste Onofré
>> >> jbono...@apache.org 
>> >
>> >> http://

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Jean-Baptiste Onofré]

Created:

https://issues.apache.org/jira/browse/BEAM-3617

Regards
JB

On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> What is the Jira for direct runner perf?
> 
> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré  > wrote:
> 
> Thanks !
> 
> I cherry-pick on release-2.3.0 branch.
> 
> I'm on the direct runner perf test in the mean time.
> 
> Thanks again !
> 
> Regards
> JB
> 
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> >  - https://issues.apache.org/jira/browse/BEAM-3186
> 
> >  - https://issues.apache.org/jira/browse/BEAM-3589
> 
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or 
> you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré  
> >> >> wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>  Hi everyone,
> 
>  Please review and vote on the release candidate #1 for the version
> 2.3.0, as
>  follows:
> 
>  [ ] +1, Approve the release
>  [ ] -1, Do not approve the release (please provide specific comments)
> 
> 
>  The complete staging area is available for your review, which 
> includes:
>  * JIRA release notes [1],
>  * the official Apache source release to be deployed to 
> dist.apache.org
> 
>   [2],
>  which is signed with the key with fingerprint C8282E76 [3],
>  * all artifacts to be deployed to the Maven Central Repository [4],
>  * source code tag "v2.3.0-RC1" [5],
>  * website pull request listing the release and publishing the API 
> reference
>  manual [6].
>  * Java artifacts were built with Maven 3.3.9 and Oracle JDK 
> 1.8.0_111.
>  * Python artifacts are deployed along with the source release to the
>  dist.apache.org   
> [2].
> 
>  The vote will be open for at least 72 hours. It is adopted by 
> majority approval,
>  with at least 3 PMC affirmative votes.
> 
>  Thanks,
>  JB
> 
>  [1]
>  
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> 
> 
>  [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> 
>  [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> 
>  [4] 
> https://repository.apache.org/content/repositories/orgapachebeam-1026/
> 
>  [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> 
>  [6] https://github.com/apache/beam-site/pull/381
> 
> 
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbono...@apache.org 
> >
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
> 
> --
> Jean-Baptiste Onofré
> jbono...@apache.org 
> http://blog.nanthrax.net
> Talend - http://www.talend.com
> 
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Jean-Baptiste Onofré]

Hi Kenn,

my bad, I didn't create one yet (I was busy on the TextIO with flink runner, now
identify \o/ ;)).

Let me create it right now.

Thanks !
Regards
JB

On 02/05/2018 04:42 PM, Kenneth Knowles wrote:
> What is the Jira for direct runner perf?
> 
> On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré  > wrote:
> 
> Thanks !
> 
> I cherry-pick on release-2.3.0 branch.
> 
> I'm on the direct runner perf test in the mean time.
> 
> Thanks again !
> 
> Regards
> JB
> 
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> >  - https://issues.apache.org/jira/browse/BEAM-3186
> 
> >  - https://issues.apache.org/jira/browse/BEAM-3589
> 
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or 
> you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré  
> >> >> wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>  Hi everyone,
> 
>  Please review and vote on the release candidate #1 for the version
> 2.3.0, as
>  follows:
> 
>  [ ] +1, Approve the release
>  [ ] -1, Do not approve the release (please provide specific comments)
> 
> 
>  The complete staging area is available for your review, which 
> includes:
>  * JIRA release notes [1],
>  * the official Apache source release to be deployed to 
> dist.apache.org
> 
>   [2],
>  which is signed with the key with fingerprint C8282E76 [3],
>  * all artifacts to be deployed to the Maven Central Repository [4],
>  * source code tag "v2.3.0-RC1" [5],
>  * website pull request listing the release and publishing the API 
> reference
>  manual [6].
>  * Java artifacts were built with Maven 3.3.9 and Oracle JDK 
> 1.8.0_111.
>  * Python artifacts are deployed along with the source release to the
>  dist.apache.org   
> [2].
> 
>  The vote will be open for at least 72 hours. It is adopted by 
> majority approval,
>  with at least 3 PMC affirmative votes.
> 
>  Thanks,
>  JB
> 
>  [1]
>  
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
> 
> 
>  [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
> 
>  [3] https://dist.apache.org/repos/dist/release/beam/KEYS
> 
>  [4] 
> https://repository.apache.org/content/repositories/orgapachebeam-1026/
> 
>  [5] https://github.com/apache/beam/tree/v2.3.0-RC1
> 
>  [6] https://github.com/apache/beam-site/pull/381
> 
> 
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbono...@apache.org 
> >
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
> 
> --
> Jean-Baptiste Onofré
> jbono...@apache.org 
> http://blog.nanthrax.net
> Talend - http://www.talend.com
> 
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Kenneth Knowles]

What is the Jira for direct runner perf?

On Mon, Feb 5, 2018 at 4:35 AM, Jean-Baptiste Onofré 
wrote:

> Thanks !
>
> I cherry-pick on release-2.3.0 branch.
>
> I'm on the direct runner perf test in the mean time.
>
> Thanks again !
>
> Regards
> JB
>
> On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> > I merged fixes for:
> >  - https://issues.apache.org/jira/browse/BEAM-3186
> >  - https://issues.apache.org/jira/browse/BEAM-3589
> >
> > @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or
> you can go
> > ahead.
> >
> >> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré  >> > wrote:
> >>
> >> Hi guys,
> >>
> >> Quick update on the RC2 preparation:
> >>
> >> * BEAM-3587 (TextIO with Flink) seems related to a custom build with
> Gradle (not
> >> using artifacts created by Maven). Anyway, I will take a look today.
> >> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> >> * I'm also taking a look on the direct runner performance change.
> >>
> >> The goal is to cut RC2 tomorrow or the day after.
> >>
> >> Regards
> >> JB
> >>
> >> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> >>> Hi guys,
> >>>
> >>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> >>>
> >>> We will cherry-pick fixes on release-2.3.0 branch.
> >>>
> >>> I'm updating Jira right now. When the fixes will be cherry-picked, I
> will submit
> >>> a RC2 to vote.
> >>>
> >>> Thanks !
> >>> Regards
> >>> JB
> >>>
> >>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>  Hi everyone,
> 
>  Please review and vote on the release candidate #1 for the version
> 2.3.0, as
>  follows:
> 
>  [ ] +1, Approve the release
>  [ ] -1, Do not approve the release (please provide specific comments)
> 
> 
>  The complete staging area is available for your review, which
> includes:
>  * JIRA release notes [1],
>  * the official Apache source release to be deployed to
> dist.apache.org
>   [2],
>  which is signed with the key with fingerprint C8282E76 [3],
>  * all artifacts to be deployed to the Maven Central Repository [4],
>  * source code tag "v2.3.0-RC1" [5],
>  * website pull request listing the release and publishing the API
> reference
>  manual [6].
>  * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>  * Python artifacts are deployed along with the source release to the
>  dist.apache.org  [2].
> 
>  The vote will be open for at least 72 hours. It is adopted by
> majority approval,
>  with at least 3 PMC affirmative votes.
> 
>  Thanks,
>  JB
> 
>  [1]
>  https://issues.apache.org/jira/secure/ReleaseNote.jspa?
> projectId=12319527&version=12341608
>  [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>  [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>  [4] https://repository.apache.org/content/repositories/
> orgapachebeam-1026/
>  [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>  [6] https://github.com/apache/beam-site/pull/381
> 
> >>>
> >>
> >> --
> >> Jean-Baptiste Onofré
> >> jbono...@apache.org 
> >> http://blog.nanthrax.net
> >> Talend - http://www.talend.com
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Jean-Baptiste Onofré]

Thanks !

I cherry-pick on release-2.3.0 branch.

I'm on the direct runner perf test in the mean time.

Thanks again !

Regards
JB

On 02/05/2018 12:06 PM, Aljoscha Krettek wrote:
> I merged fixes for:
>  - https://issues.apache.org/jira/browse/BEAM-3186
>  - https://issues.apache.org/jira/browse/BEAM-3589
> 
> @JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can 
> go
> ahead.
> 
>> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré > > wrote:
>>
>> Hi guys,
>>
>> Quick update on the RC2 preparation:
>>
>> * BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle 
>> (not
>> using artifacts created by Maven). Anyway, I will take a look today.
>> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
>> * I'm also taking a look on the direct runner performance change.
>>
>> The goal is to cut RC2 tomorrow or the day after.
>>
>> Regards
>> JB
>>
>> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>>> Hi guys,
>>>
>>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>>>
>>> We will cherry-pick fixes on release-2.3.0 branch.
>>>
>>> I'm updating Jira right now. When the fixes will be cherry-picked, I will 
>>> submit
>>> a RC2 to vote.
>>>
>>> Thanks !
>>> Regards
>>> JB
>>>
>>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
 Hi everyone,

 Please review and vote on the release candidate #1 for the version 2.3.0, 
 as
 follows:

 [ ] +1, Approve the release
 [ ] -1, Do not approve the release (please provide specific comments)


 The complete staging area is available for your review, which includes:
 * JIRA release notes [1],
 * the official Apache source release to be deployed to dist.apache.org
  [2],
 which is signed with the key with fingerprint C8282E76 [3],
 * all artifacts to be deployed to the Maven Central Repository [4],
 * source code tag "v2.3.0-RC1" [5],
 * website pull request listing the release and publishing the API reference
 manual [6].
 * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
 * Python artifacts are deployed along with the source release to the
 dist.apache.org  [2].

 The vote will be open for at least 72 hours. It is adopted by majority 
 approval,
 with at least 3 PMC affirmative votes.

 Thanks,
 JB

 [1]
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
 [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
 [3] https://dist.apache.org/repos/dist/release/beam/KEYS
 [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
 [5] https://github.com/apache/beam/tree/v2.3.0-RC1
 [6] https://github.com/apache/beam-site/pull/381

>>>
>>
>> -- 
>> Jean-Baptiste Onofré
>> jbono...@apache.org 
>> http://blog.nanthrax.net
>> Talend - http://www.talend.com
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Aljoscha Krettek]

I merged fixes for:
 - https://issues.apache.org/jira/browse/BEAM-3186 

 - https://issues.apache.org/jira/browse/BEAM-3589 


@JB I didn't yet merge them on the 2.3.0 branch, though, but I can or you can 
go ahead.

> On 5. Feb 2018, at 06:30, Jean-Baptiste Onofré  wrote:
> 
> Hi guys,
> 
> Quick update on the RC2 preparation:
> 
> * BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle 
> (not
> using artifacts created by Maven). Anyway, I will take a look today.
> * BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
> * I'm also taking a look on the direct runner performance change.
> 
> The goal is to cut RC2 tomorrow or the day after.
> 
> Regards
> JB
> 
> On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
>> Hi guys,
>> 
>> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
>> 
>> We will cherry-pick fixes on release-2.3.0 branch.
>> 
>> I'm updating Jira right now. When the fixes will be cherry-picked, I will 
>> submit
>> a RC2 to vote.
>> 
>> Thanks !
>> Regards
>> JB
>> 
>> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>>> Hi everyone,
>>> 
>>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>>> follows:
>>> 
>>> [ ] +1, Approve the release
>>> [ ] -1, Do not approve the release (please provide specific comments)
>>> 
>>> 
>>> The complete staging area is available for your review, which includes:
>>> * JIRA release notes [1],
>>> * the official Apache source release to be deployed to dist.apache.org [2],
>>> which is signed with the key with fingerprint C8282E76 [3],
>>> * all artifacts to be deployed to the Maven Central Repository [4],
>>> * source code tag "v2.3.0-RC1" [5],
>>> * website pull request listing the release and publishing the API reference
>>> manual [6].
>>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>>> * Python artifacts are deployed along with the source release to the
>>> dist.apache.org [2].
>>> 
>>> The vote will be open for at least 72 hours. It is adopted by majority 
>>> approval,
>>> with at least 3 PMC affirmative votes.
>>> 
>>> Thanks,
>>> JB
>>> 
>>> [1]
>>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>>> [6] https://github.com/apache/beam-site/pull/381
>>> 
>> 
> 
> -- 
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com

Re: [CANCEL][VOTE] Release 2.3.0, release candidate #1

[Jean-Baptiste Onofré]

Hi guys,

Quick update on the RC2 preparation:

* BEAM-3587 (TextIO with Flink) seems related to a custom build with Gradle (not
using artifacts created by Maven). Anyway, I will take a look today.
* BEAM-3186 has a PR. Aljoscha will do the review pretty soon.
* I'm also taking a look on the direct runner performance change.

The goal is to cut RC2 tomorrow or the day after.

Regards
JB

On 02/01/2018 04:16 PM, Jean-Baptiste Onofré wrote:
> Hi guys,
> 
> Especially due to BEAM-3587 & BEAM-3186 regressions, I cancel RC1.
> 
> We will cherry-pick fixes on release-2.3.0 branch.
> 
> I'm updating Jira right now. When the fixes will be cherry-picked, I will 
> submit
> a RC2 to vote.
> 
> Thanks !
> Regards
> JB
> 
> On 01/30/2018 09:04 AM, Jean-Baptiste Onofré wrote:
>> Hi everyone,
>>
>> Please review and vote on the release candidate #1 for the version 2.3.0, as
>> follows:
>>
>> [ ] +1, Approve the release
>> [ ] -1, Do not approve the release (please provide specific comments)
>>
>>
>> The complete staging area is available for your review, which includes:
>> * JIRA release notes [1],
>> * the official Apache source release to be deployed to dist.apache.org [2],
>> which is signed with the key with fingerprint C8282E76 [3],
>> * all artifacts to be deployed to the Maven Central Repository [4],
>> * source code tag "v2.3.0-RC1" [5],
>> * website pull request listing the release and publishing the API reference
>> manual [6].
>> * Java artifacts were built with Maven 3.3.9 and Oracle JDK 1.8.0_111.
>> * Python artifacts are deployed along with the source release to the
>> dist.apache.org [2].
>>
>> The vote will be open for at least 72 hours. It is adopted by majority 
>> approval,
>> with at least 3 PMC affirmative votes.
>>
>> Thanks,
>> JB
>>
>> [1]
>> https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12319527&version=12341608
>> [2] https://dist.apache.org/repos/dist/dev/beam/2.3.0/
>> [3] https://dist.apache.org/repos/dist/release/beam/KEYS
>> [4] https://repository.apache.org/content/repositories/orgapachebeam-1026/
>> [5] https://github.com/apache/beam/tree/v2.3.0-RC1
>> [6] https://github.com/apache/beam-site/pull/381
>>
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com