Re: [DISCUSS] What to do about widespread KinesisIO breakage

[Kenneth Knowles]

Nice! Thanks both for taking this on.

Kenn

On Wed, Feb 21, 2018 at 7:00 AM, Alexey Romanenko 
wrote:

> Hi all,
>
> Just an intermediate update on this. All (except only one) flaky KinesisIO
> tests were fixed (thanks to Paweł Kaczmarczyk for help) and I didn’t see
> that they were failing anymore. Please, let me know if you did.
>
> That one, which was not fixed (BEAM-3317
> ), I can’t reproduce
> constantly. Even running it thousands times I’ve seen it only once. So,
> it’s still in progress.
>
> WBR,
> Alexey
>
>
> On 6 Feb 2018, at 11:40, Alexey Romanenko 
> wrote:
>
> Kenn, JB,
>
> Thank you for raising this question.
> I was working mostly on writing support for KinesisIO (PR is waiting to be
> merged) but I’ll take a look on flaky read tests as well.
>
> Regards,
> Alexey
>
> On 6 Feb 2018, at 06:27, Jean-Baptiste Onofré  wrote:
>
> Yes, I don't consider this as blocker, and I think it fails for a while ;)
>
> Regards
> JB
>
> On 02/06/2018 06:22 AM, Kenneth Knowles wrote:
>
> Thanks JB (and Alexey)! If you are confident the failures are not release
> blockers, then that's great.
>
> Kenn
>
> On Mon, Feb 5, 2018 at 9:17 PM, Jean-Baptiste Onofré  >> wrote:
>
>Hi Kenn,
>
>I have Alexey in my team who started to work on the write part of
> KinesisIO. I
>will ask him to work on those issues.
>
>I think it's OK to keep KinesisIO in the distribution and work on it in
> the mean
>time.
>
>Regards
>JB
>
>On 02/06/2018 04:20 AM, Kenneth Knowles wrote:
>
> The flaking of KinesisIO on both Maven and Gradle executions has become
> very
> bad. Multiple methods are flaky, and we've collected these
> Critical-severity
> tickets:
>
> (looks like https://issues.apache.org/jira/browse/BEAM-3228
>
> is fixed?)
>
> https://issues.apache.org/jira/browse/BEAM-3317
>
>
>
> https://issues.apache.org/jira/browse/BEAM-3598
>
>
>
> https://issues.apache.org/jira/browse/BEAM-3599
>
>
>
> https://issues.apache.org/jira/browse/BEAM-3605
>
>
>
>
> There don't appear to be immediate resources for addressing these so I have
> taken these so they can be sickbayed until there are resources to de-flake
>
>them
>
> and turn them on again.
>
> https://github.com/apache/beam/pull/4605
>
>
>
>
> If you are interested in KinesisIO, please volunteer and fix them properly.
>
> The trouble is that the flakes may indicate serious problems with
> KinesisIO.
> Should we stop shipping KinesisIO until this is fixed? I'm interested in
>
>hearing
>
> from folks on other ASF project what they have done.
>
> It also raises another issue, which is that I think we would benefit from
> granular Jira components for connectors so issues can be routed
> effectively.
> Connectors with no owner will likely rot and we'll need a plan for what to
> do
> when that happens.
>
> Kenn
>
>
>--
>Jean-Baptiste Onofré
>jbono...@apache.org >
>http://blog.nanthrax.net
>Talend - http://www.talend.com
>
>
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
>
>

Re: [DISCUSS] What to do about widespread KinesisIO breakage

[Alexey Romanenko]

Hi all, 

Just an intermediate update on this. All (except only one) flaky KinesisIO 
tests were fixed (thanks to Paweł Kaczmarczyk for help) and I didn’t see that 
they were failing anymore. Please, let me know if you did.  

That one, which was not fixed (BEAM-3317 
), I can’t reproduce 
constantly. Even running it thousands times I’ve seen it only once. So, it’s 
still in progress.

WBR,
Alexey

> On 6 Feb 2018, at 11:40, Alexey Romanenko  wrote:
> 
> Kenn, JB,
> 
> Thank you for raising this question. 
> I was working mostly on writing support for KinesisIO (PR is waiting to be 
> merged) but I’ll take a look on flaky read tests as well.
> 
> Regards,
> Alexey
> 
>> On 6 Feb 2018, at 06:27, Jean-Baptiste Onofré > > wrote:
>> 
>> Yes, I don't consider this as blocker, and I think it fails for a while ;)
>> 
>> Regards
>> JB
>> 
>> On 02/06/2018 06:22 AM, Kenneth Knowles wrote:
>>> Thanks JB (and Alexey)! If you are confident the failures are not release
>>> blockers, then that's great.
>>> 
>>> Kenn
>>> 
>>> On Mon, Feb 5, 2018 at 9:17 PM, Jean-Baptiste Onofré >> 
>>> >> wrote:
>>> 
>>>Hi Kenn,
>>> 
>>>I have Alexey in my team who started to work on the write part of 
>>> KinesisIO. I
>>>will ask him to work on those issues.
>>> 
>>>I think it's OK to keep KinesisIO in the distribution and work on it in 
>>> the mean
>>>time.
>>> 
>>>Regards
>>>JB
>>> 
>>>On 02/06/2018 04:20 AM, Kenneth Knowles wrote:
 The flaking of KinesisIO on both Maven and Gradle executions has become 
 very
 bad. Multiple methods are flaky, and we've collected these 
 Critical-severity
 tickets:
 
 (looks like https://issues.apache.org/jira/browse/BEAM-3228 
 
>>>>> > is fixed?)
 https://issues.apache.org/jira/browse/BEAM-3317 
 
>>>>> >
 https://issues.apache.org/jira/browse/BEAM-3598 
 
>>>>> >
 https://issues.apache.org/jira/browse/BEAM-3599 
 
>>>>> >
 https://issues.apache.org/jira/browse/BEAM-3605 
 
>>>>> >
 
 There don't appear to be immediate resources for addressing these so I have
 taken these so they can be sickbayed until there are resources to de-flake
>>>them
 and turn them on again.
 
 https://github.com/apache/beam/pull/4605 
 
>>>>> >
 
 If you are interested in KinesisIO, please volunteer and fix them properly.
 
 The trouble is that the flakes may indicate serious problems with 
 KinesisIO.
 Should we stop shipping KinesisIO until this is fixed? I'm interested in
>>>hearing
 from folks on other ASF project what they have done.
 
 It also raises another issue, which is that I think we would benefit from
 granular Jira components for connectors so issues can be routed 
 effectively.
 Connectors with no owner will likely rot and we'll need a plan for what to 
 do
 when that happens.
 
 Kenn
 
>>> 
>>>--
>>>Jean-Baptiste Onofré
>>>jbono...@apache.org  
>>> >
>>>http://blog.nanthrax.net 
>>>Talend - http://www.talend.com 
>>> 
>>> 
>> 
>> -- 
>> Jean-Baptiste Onofré
>> jbono...@apache.org 
>> http://blog.nanthrax.net 
>> Talend - http://www.talend.com 

Re: [DISCUSS] What to do about widespread KinesisIO breakage

[Alexey Romanenko]

Kenn, JB,

Thank you for raising this question. 
I was working mostly on writing support for KinesisIO (PR is waiting to be 
merged) but I’ll take a look on flaky read tests as well.

Regards,
Alexey

> On 6 Feb 2018, at 06:27, Jean-Baptiste Onofré  wrote:
> 
> Yes, I don't consider this as blocker, and I think it fails for a while ;)
> 
> Regards
> JB
> 
> On 02/06/2018 06:22 AM, Kenneth Knowles wrote:
>> Thanks JB (and Alexey)! If you are confident the failures are not release
>> blockers, then that's great.
>> 
>> Kenn
>> 
>> On Mon, Feb 5, 2018 at 9:17 PM, Jean-Baptiste Onofré > 
>> >> wrote:
>> 
>>Hi Kenn,
>> 
>>I have Alexey in my team who started to work on the write part of 
>> KinesisIO. I
>>will ask him to work on those issues.
>> 
>>I think it's OK to keep KinesisIO in the distribution and work on it in 
>> the mean
>>time.
>> 
>>Regards
>>JB
>> 
>>On 02/06/2018 04:20 AM, Kenneth Knowles wrote:
>>> The flaking of KinesisIO on both Maven and Gradle executions has become very
>>> bad. Multiple methods are flaky, and we've collected these Critical-severity
>>> tickets:
>>> 
>>> (looks like https://issues.apache.org/jira/browse/BEAM-3228 
>>> 
>>> > is fixed?)
>>> https://issues.apache.org/jira/browse/BEAM-3317 
>>> 
>>> >
>>> https://issues.apache.org/jira/browse/BEAM-3598 
>>> 
>>> >
>>> https://issues.apache.org/jira/browse/BEAM-3599 
>>> 
>>> >
>>> https://issues.apache.org/jira/browse/BEAM-3605 
>>> 
>>> >
>>> 
>>> There don't appear to be immediate resources for addressing these so I have
>>> taken these so they can be sickbayed until there are resources to de-flake
>>them
>>> and turn them on again.
>>> 
>>> https://github.com/apache/beam/pull/4605 
>>> 
>>
>>> 
>>> If you are interested in KinesisIO, please volunteer and fix them properly.
>>> 
>>> The trouble is that the flakes may indicate serious problems with KinesisIO.
>>> Should we stop shipping KinesisIO until this is fixed? I'm interested in
>>hearing
>>> from folks on other ASF project what they have done.
>>> 
>>> It also raises another issue, which is that I think we would benefit from
>>> granular Jira components for connectors so issues can be routed effectively.
>>> Connectors with no owner will likely rot and we'll need a plan for what to 
>>> do
>>> when that happens.
>>> 
>>> Kenn
>>> 
>> 
>>--
>>Jean-Baptiste Onofré
>>jbono...@apache.org  
>> >
>>http://blog.nanthrax.net 
>>Talend - http://www.talend.com 
>> 
>> 
> 
> -- 
> Jean-Baptiste Onofré
> jbono...@apache.org 
> http://blog.nanthrax.net 
> Talend - http://www.talend.com 

Re: [DISCUSS] What to do about widespread KinesisIO breakage

[Jean-Baptiste Onofré]

Yes, I don't consider this as blocker, and I think it fails for a while ;)

Regards
JB

On 02/06/2018 06:22 AM, Kenneth Knowles wrote:
> Thanks JB (and Alexey)! If you are confident the failures are not release
> blockers, then that's great.
> 
> Kenn
> 
> On Mon, Feb 5, 2018 at 9:17 PM, Jean-Baptiste Onofré  > wrote:
> 
> Hi Kenn,
> 
> I have Alexey in my team who started to work on the write part of 
> KinesisIO. I
> will ask him to work on those issues.
> 
> I think it's OK to keep KinesisIO in the distribution and work on it in 
> the mean
> time.
> 
> Regards
> JB
> 
> On 02/06/2018 04:20 AM, Kenneth Knowles wrote:
> > The flaking of KinesisIO on both Maven and Gradle executions has become 
> very
> > bad. Multiple methods are flaky, and we've collected these 
> Critical-severity
> > tickets:
> >
> >     (looks like https://issues.apache.org/jira/browse/BEAM-3228
>  is fixed?)
> >     https://issues.apache.org/jira/browse/BEAM-3317
> 
> >     https://issues.apache.org/jira/browse/BEAM-3598
> 
> >     https://issues.apache.org/jira/browse/BEAM-3599
> 
> >     https://issues.apache.org/jira/browse/BEAM-3605
> 
> >
> > There don't appear to be immediate resources for addressing these so I 
> have
> > taken these so they can be sickbayed until there are resources to 
> de-flake
> them
> > and turn them on again.
> >
> >     https://github.com/apache/beam/pull/4605
> 
> >
> > If you are interested in KinesisIO, please volunteer and fix them 
> properly.
> >
> > The trouble is that the flakes may indicate serious problems with 
> KinesisIO.
> > Should we stop shipping KinesisIO until this is fixed? I'm interested in
> hearing
> > from folks on other ASF project what they have done.
> >
> > It also raises another issue, which is that I think we would benefit 
> from
> > granular Jira components for connectors so issues can be routed 
> effectively.
> > Connectors with no owner will likely rot and we'll need a plan for what 
> to do
> > when that happens.
> >
> > Kenn
> >
> 
> --
> Jean-Baptiste Onofré
> jbono...@apache.org 
> http://blog.nanthrax.net
> Talend - http://www.talend.com
> 
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [DISCUSS] What to do about widespread KinesisIO breakage

[Kenneth Knowles]

Thanks JB (and Alexey)! If you are confident the failures are not release
blockers, then that's great.

Kenn

On Mon, Feb 5, 2018 at 9:17 PM, Jean-Baptiste Onofré 
wrote:

> Hi Kenn,
>
> I have Alexey in my team who started to work on the write part of
> KinesisIO. I
> will ask him to work on those issues.
>
> I think it's OK to keep KinesisIO in the distribution and work on it in
> the mean
> time.
>
> Regards
> JB
>
> On 02/06/2018 04:20 AM, Kenneth Knowles wrote:
> > The flaking of KinesisIO on both Maven and Gradle executions has become
> very
> > bad. Multiple methods are flaky, and we've collected these
> Critical-severity
> > tickets:
> >
> > (looks like https://issues.apache.org/jira/browse/BEAM-3228 is
> fixed?)
> > https://issues.apache.org/jira/browse/BEAM-3317
> > https://issues.apache.org/jira/browse/BEAM-3598
> > https://issues.apache.org/jira/browse/BEAM-3599
> > https://issues.apache.org/jira/browse/BEAM-3605
> >
> > There don't appear to be immediate resources for addressing these so I
> have
> > taken these so they can be sickbayed until there are resources to
> de-flake them
> > and turn them on again.
> >
> > https://github.com/apache/beam/pull/4605
> >
> > If you are interested in KinesisIO, please volunteer and fix them
> properly.
> >
> > The trouble is that the flakes may indicate serious problems with
> KinesisIO.
> > Should we stop shipping KinesisIO until this is fixed? I'm interested in
> hearing
> > from folks on other ASF project what they have done.
> >
> > It also raises another issue, which is that I think we would benefit from
> > granular Jira components for connectors so issues can be routed
> effectively.
> > Connectors with no owner will likely rot and we'll need a plan for what
> to do
> > when that happens.
> >
> > Kenn
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: [DISCUSS] What to do about widespread KinesisIO breakage

[Jean-Baptiste Onofré]

Hi Kenn,

I have Alexey in my team who started to work on the write part of KinesisIO. I
will ask him to work on those issues.

I think it's OK to keep KinesisIO in the distribution and work on it in the mean
time.

Regards
JB

On 02/06/2018 04:20 AM, Kenneth Knowles wrote:
> The flaking of KinesisIO on both Maven and Gradle executions has become very
> bad. Multiple methods are flaky, and we've collected these Critical-severity
> tickets:
> 
>     (looks like https://issues.apache.org/jira/browse/BEAM-3228 is fixed?)
>     https://issues.apache.org/jira/browse/BEAM-3317
>     https://issues.apache.org/jira/browse/BEAM-3598
>     https://issues.apache.org/jira/browse/BEAM-3599
>     https://issues.apache.org/jira/browse/BEAM-3605
> 
> There don't appear to be immediate resources for addressing these so I have
> taken these so they can be sickbayed until there are resources to de-flake 
> them
> and turn them on again.
> 
>     https://github.com/apache/beam/pull/4605
> 
> If you are interested in KinesisIO, please volunteer and fix them properly.
> 
> The trouble is that the flakes may indicate serious problems with KinesisIO.
> Should we stop shipping KinesisIO until this is fixed? I'm interested in 
> hearing
> from folks on other ASF project what they have done.
> 
> It also raises another issue, which is that I think we would benefit from
> granular Jira components for connectors so issues can be routed effectively.
> Connectors with no owner will likely rot and we'll need a plan for what to do
> when that happens.
> 
> Kenn
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: [DISCUSS] What to do about widespread KinesisIO breakage

[Kenneth Knowles]

I believe these are all unit tests running locally. The failures generally
look more like incorrect results than service problems.

On Mon, Feb 5, 2018 at 7:36 PM, Reuven Lax  wrote:

> Do these tests run locally, or are they contacting an actual Kinesis
> service?
>
> On Mon, Feb 5, 2018 at 7:20 PM, Kenneth Knowles  wrote:
>
>> The flaking of KinesisIO on both Maven and Gradle executions has become
>> very bad. Multiple methods are flaky, and we've collected these
>> Critical-severity tickets:
>>
>> (looks like https://issues.apache.org/jira/browse/BEAM-3228 is
>> fixed?)
>> https://issues.apache.org/jira/browse/BEAM-3317
>> https://issues.apache.org/jira/browse/BEAM-3598
>> https://issues.apache.org/jira/browse/BEAM-3599
>> https://issues.apache.org/jira/browse/BEAM-3605
>>
>> There don't appear to be immediate resources for addressing these so I
>> have taken these so they can be sickbayed until there are resources to
>> de-flake them and turn them on again.
>>
>> https://github.com/apache/beam/pull/4605
>>
>> If you are interested in KinesisIO, please volunteer and fix them
>> properly.
>>
>> The trouble is that the flakes may indicate serious problems with
>> KinesisIO. Should we stop shipping KinesisIO until this is fixed? I'm
>> interested in hearing from folks on other ASF project what they have done.
>>
>> It also raises another issue, which is that I think we would benefit from
>> granular Jira components for connectors so issues can be routed
>> effectively. Connectors with no owner will likely rot and we'll need a plan
>> for what to do when that happens.
>>
>> Kenn
>>
>>
>

Re: [DISCUSS] What to do about widespread KinesisIO breakage

[Reuven Lax]

Do these tests run locally, or are they contacting an actual Kinesis
service?

On Mon, Feb 5, 2018 at 7:20 PM, Kenneth Knowles  wrote:

> The flaking of KinesisIO on both Maven and Gradle executions has become
> very bad. Multiple methods are flaky, and we've collected these
> Critical-severity tickets:
>
> (looks like https://issues.apache.org/jira/browse/BEAM-3228 is fixed?)
> https://issues.apache.org/jira/browse/BEAM-3317
> https://issues.apache.org/jira/browse/BEAM-3598
> https://issues.apache.org/jira/browse/BEAM-3599
> https://issues.apache.org/jira/browse/BEAM-3605
>
> There don't appear to be immediate resources for addressing these so I
> have taken these so they can be sickbayed until there are resources to
> de-flake them and turn them on again.
>
> https://github.com/apache/beam/pull/4605
>
> If you are interested in KinesisIO, please volunteer and fix them properly.
>
> The trouble is that the flakes may indicate serious problems with
> KinesisIO. Should we stop shipping KinesisIO until this is fixed? I'm
> interested in hearing from folks on other ASF project what they have done.
>
> It also raises another issue, which is that I think we would benefit from
> granular Jira components for connectors so issues can be routed
> effectively. Connectors with no owner will likely rot and we'll need a plan
> for what to do when that happens.
>
> Kenn
>
>