Re: [VOTE] Apache Bahir 2.0.2 (RC2)
Thank you Ted for your repeated attempts at getting this to run!
I was able to reproduce similar test failures when adding additional stress
on CPU, memory and disk I/O using the 'stress' tool (
http://people.seas.harvard.edu/~apw/stress/). In each instance I was able
to bring the test case to succeed by increasing the stream write timeout
from 10 seconds to 15 or 20 seconds.
Best regards,
Christian Kadner
From: Ted Yu
To: dev@bahir.apache.org
Date: 01/12/2017 03:23 PM
Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC2)
A few more rounds of running the test (based on suggestion from Christian)
but it didn't pass.
I think this is not a blocker for release since other people cannot
reproduce it.
On Thu, Jan 12, 2017 at 1:55 AM, Ted Yu wrote:
> Running with PR #33, I got:
>
> ^[[31m- Send and receive 100 messages. *** FAILED ***^[[0m
> ^[[31m Buffer("MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is
a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is
a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is
a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is
a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is
a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is
a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is
a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is
a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.")
> had size 66 instead of expected size 100 (MQTTStreamSourceSuite.scala:
> 119)^[[0m
>
> Command line:
>
> mvn package
>
> On Thu, Jan 12, 2017 at 1:43 AM, Christian R Kadner
> wrote:
>
>> Hi Ted,
>>
>> your pastebin (http://pastebin.com/xRJaCa6H) shows that test case "Send
>> and receive 100 messages" failed. Could this be an intermittent problem?
>> I did several runs on my MacBook Pro and on a Ubuntu 16.4 test box
without
>> any test failures.
>>
>> I opened BAHIR-84 and created PR #33 to suppress the superfluous
>> "...INFO: org.apache.parquet.hadoop..." messages (
>> https://github.com/apache/bahir/pull/33). Could you take a look at that
>> PR and maybe give it a spin to see if that helps.
>>
>> To further reduce logging resources, you could set the root logger
>> threshold to ERROR in sql-streaming-mqtt/src/test/re
>> sources/log4j.properties.
>>
>> Thank you!
>> ~ Christian
>>
>> [image: Inactive hide details for Ted Yu ---01/11/2017 08:24:52 AM---ran
>> test suite and saw: http://pastebin.com/xRJaCa6H]Ted Yu ---01/11/2017
>> 08:24:52 AM---ran test suite and saw: http://pastebin.com/xRJaCa6H
>>
>> From: Ted Yu
>> To: dev@bahir.apache.org
>> Date: 01/11/2017 08:24 AM
>> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC2)
&
Re: [VOTE] Apache Bahir 2.0.2 (RC2)
A few more rounds of running the test (based on suggestion from Christian)
but it didn't pass.
I think this is not a blocker for release since other people cannot
reproduce it.
On Thu, Jan 12, 2017 at 1:55 AM, Ted Yu wrote:
> Running with PR #33, I got:
>
> ^[[31m- Send and receive 100 messages. *** FAILED ***^[[0m
> ^[[31m Buffer("MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.")
> had size 66 instead of expected size 100 (MQTTStreamSourceSuite.scala:
> 119)^[[0m
>
> Command line:
>
> mvn package
>
> On Thu, Jan 12, 2017 at 1:43 AM, Christian R Kadner
> wrote:
>
>> Hi Ted,
>>
>> your pastebin (http://pastebin.com/xRJaCa6H) shows that test case "Send
>> and receive 100 messages" failed. Could this be an intermittent problem?
>> I did several runs on my MacBook Pro and on a Ubuntu 16.4 test box without
>> any test failures.
>>
>> I opened BAHIR-84 and created PR #33 to suppress the superfluous
>> "...INFO: org.apache.parquet.hadoop..." messages (
>> https://github.com/apache/bahir/pull/33). Could you take a look at that
>> PR and maybe give it a spin to see if that helps.
>>
>> To further reduce logging resources, you could set the root logger
>> threshold to ERROR in sql-streaming-mqtt/src/test/re
>> sources/log4j.properties.
>>
>> Thank you!
>> ~ Christian
>>
>> [image: Inactive hide details for Ted Yu ---01/11/2017 08:24:52 AM---ran
>> test suite and saw: http://pastebin.com/xRJaCa6H]Ted Yu ---01/11/2017
>> 08:24:52 AM---ran test suite and saw: http://pastebin.com/xRJaCa6H
>>
>> From: Ted Yu
>> To: dev@bahir.apache.org
>> Date: 01/11/2017 08:24 AM
>> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC2)
>> --
>>
>>
>>
>> ran test suite and saw:
>>
>> http://pastebin.com/xRJaCa6H
>>
>> On Wed, Jan 11, 2017 at 3:33 AM, Christian Kadner
>> wrote:
>>
>> > Please vote to approve the second release candidate for Apache Bahir
>> > version 2.0.2 based on Apache Spark 2.0.2.
>> >
>> > The difference between 2.0.1 and 2.0.2 RC2 can be visualized at:
>> >
>> > https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc2
>> >
>> > The tag to be voted on is v2.0.2-rc2
>> > (dd82363878a6bb4a19aade8010c01ddfbf254aff)
>> >
>> > https://github.com/apache/bahir/tree/dd82363878a6bb4a19aade8010c01d
>> > dfbf254aff
>> >
>> > The release files, including signatures, digests, etc. can be found at:
>> >
>> > https://repository.apache.org/content/repositories/orgapachebahir-1008
>> >
>> > The source distribution is also available at:
>> >
>> > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc2/
>> >
>> > The vote is open for at least 72 hours and passes if a majority of at
>> least
>> > 3 +1 PMC votes are cast.
>> >
>> > [ ] +1 Release this package as Apache Bahir 2.0.1
>> > [ ] -1 Do not release this package because ...
>> >
>> > Thanks for you vote!
>> >
>>
>>
>>
>>
>
Re: [VOTE] Apache Bahir 2.0.2 (RC2)
Hi Ted,
what kind of system are you running on? Do you see this error consistently
even on previous Bahir releases (2.0.0, 2.0.1)?
Best regards,
Christian Kadner
From: Ted Yu
To: dev@bahir.apache.org
Date: 01/12/2017 01:56 AM
Subject:Re: [VOTE] Apache Bahir 2.0.2 (RC2)
Running with PR #33, I got:
^[[31m- Send and receive 100 messages. *** FAILED ***^[[0m
^[[31m Buffer("MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.")
had size 66 instead of expected size 100
(MQTTStreamSourceSuite.scala:119)^[[0m
Command line:
mvn package
On Thu, Jan 12, 2017 at 1:43 AM, Christian R Kadner
wrote:
> Hi Ted,
>
> your pastebin (http://pastebin.com/xRJaCa6H) shows that test case "Send
> and receive 100 messages" failed. Could this be an intermittent problem?
> I did several runs on my MacBook Pro and on a Ubuntu 16.4 test box
without
> any test failures.
>
> I opened BAHIR-84 and created PR #33 to suppress the superfluous
"...INFO:
> org.apache.parquet.hadoop..." messages (https://github.com/apache/
> bahir/pull/33). Could you take a look at that PR and maybe give it a spin
> to see if that helps.
>
> To further reduce logging resources, you could set the root logger
> threshold to ERROR in sql-streaming-mqtt/src/test/
> resources/log4j.properties.
>
> Thank you!
> ~ Christian
>
> [image: Inactive hide details for Ted Yu ---01/11/2017 08:24:52 AM---ran
> test suite and saw: http://pastebin.com/xRJaCa6H]Ted Yu ---01/11/2017
> 08:24:52 AM---ran test suite and saw: http://pastebin.com/xRJaCa6H
>
> From: Ted Yu
> To: dev@bahir.apache.org
> Date: 01/11/2017 08:24 AM
> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC2)
> --
>
>
>
> ran test suite and saw:
>
> http://pastebin.com/xRJaCa6H
>
> On Wed, Jan 11, 2017 at 3:33 AM, Christian Kadner
> wrote:
>
> > Please vote to approve the second release candidate for Apache Bahir
> > version 2.0.2 based on Apache Spark 2.0.2.
> >
> > The difference between 2.0.1 and 2.0.2 RC2 can be visualized at:
> >
> > https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc2
> >
> > The tag to be voted on is v2.0.2-rc2
> > (dd82363878a6bb4a19aade8010c01ddfbf254aff)
> >
> > https://github.com/apache/bahir/tree/dd82363878a6bb4a19aade8010c01d
> > dfbf254aff
> >
> > The release files, including signatures, digests, etc. can be found at:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1008
> >
> > The source distribution is also available at:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc2/
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Bahir 2.0.1
> > [ ] -1 Do not release this package because ...
> >
> > Thanks for you vote!
> >
>
>
>
>
Re: [VOTE] Apache Bahir 2.0.2 (RC2)
Running with PR #33, I got:
^[[31m- Send and receive 100 messages. *** FAILED ***^[[0m
^[[31m Buffer("MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.")
had size 66 instead of expected size 100
(MQTTStreamSourceSuite.scala:119)^[[0m
Command line:
mvn package
On Thu, Jan 12, 2017 at 1:43 AM, Christian R Kadner
wrote:
> Hi Ted,
>
> your pastebin (http://pastebin.com/xRJaCa6H) shows that test case "Send
> and receive 100 messages" failed. Could this be an intermittent problem?
> I did several runs on my MacBook Pro and on a Ubuntu 16.4 test box without
> any test failures.
>
> I opened BAHIR-84 and created PR #33 to suppress the superfluous "...INFO:
> org.apache.parquet.hadoop..." messages (https://github.com/apache/
> bahir/pull/33). Could you take a look at that PR and maybe give it a spin
> to see if that helps.
>
> To further reduce logging resources, you could set the root logger
> threshold to ERROR in sql-streaming-mqtt/src/test/
> resources/log4j.properties.
>
> Thank you!
> ~ Christian
>
> [image: Inactive hide details for Ted Yu ---01/11/2017 08:24:52 AM---ran
> test suite and saw: http://pastebin.com/xRJaCa6H]Ted Yu ---01/11/2017
> 08:24:52 AM---ran test suite and saw: http://pastebin.com/xRJaCa6H
>
> From: Ted Yu
> To: dev@bahir.apache.org
> Date: 01/11/2017 08:24 AM
> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC2)
> --
>
>
>
> ran test suite and saw:
>
> http://pastebin.com/xRJaCa6H
>
> On Wed, Jan 11, 2017 at 3:33 AM, Christian Kadner
> wrote:
>
> > Please vote to approve the second release candidate for Apache Bahir
> > version 2.0.2 based on Apache Spark 2.0.2.
> >
> > The difference between 2.0.1 and 2.0.2 RC2 can be visualized at:
> >
> > https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc2
> >
> > The tag to be voted on is v2.0.2-rc2
> > (dd82363878a6bb4a19aade8010c01ddfbf254aff)
> >
> > https://github.com/apache/bahir/tree/dd82363878a6bb4a19aade8010c01d
> > dfbf254aff
> >
> > The release files, including signatures, digests, etc. can be found at:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1008
> >
> > The source distribution is also available at:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc2/
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Bahir 2.0.1
> > [ ] -1 Do not release this package because ...
> >
> > Thanks for you vote!
> >
>
>
>
>
Re: [VOTE] Apache Bahir 2.0.2 (RC2)
Hi Ted, your pastebin (http://pastebin.com/xRJaCa6H) shows that test case "Send and receive 100 messages" failed. Could this be an intermittent problem? I did several runs on my MacBook Pro and on a Ubuntu 16.4 test box without any test failures. I opened BAHIR-84 and created PR #33 to suppress the superfluous "...INFO: org.apache.parquet.hadoop..." messages ( https://github.com/apache/bahir/pull/33). Could you take a look at that PR and maybe give it a spin to see if that helps. To further reduce logging resources, you could set the root logger threshold to ERROR in sql-streaming-mqtt/src/test/resources/log4j.properties. Thank you! ~ Christian From: Ted Yu To: dev@bahir.apache.org Date: 01/11/2017 08:24 AM Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC2) ran test suite and saw: http://pastebin.com/xRJaCa6H On Wed, Jan 11, 2017 at 3:33 AM, Christian Kadner wrote: > Please vote to approve the second release candidate for Apache Bahir > version 2.0.2 based on Apache Spark 2.0.2. > > The difference between 2.0.1 and 2.0.2 RC2 can be visualized at: > > https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc2 > > The tag to be voted on is v2.0.2-rc2 > (dd82363878a6bb4a19aade8010c01ddfbf254aff) > > https://github.com/apache/bahir/tree/dd82363878a6bb4a19aade8010c01d > dfbf254aff > > The release files, including signatures, digests, etc. can be found at: > > https://repository.apache.org/content/repositories/orgapachebahir-1008 > > The source distribution is also available at: > > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc2/ > > The vote is open for at least 72 hours and passes if a majority of at least > 3 +1 PMC votes are cast. > > [ ] +1 Release this package as Apache Bahir 2.0.1 > [ ] -1 Do not release this package because ... > > Thanks for you vote! >
Re: [VOTE] Apache Bahir 2.0.2 (RC2)
That's one way of making the test stable. On Wed, Jan 11, 2017 at 10:27 AM, Christian R Kadner wrote: > Yup, this is a noisy test suite. Are you advocating to limit the log > output? > > >
Re: [VOTE] Apache Bahir 2.0.2 (RC2)
Yup, this is a noisy test suite. Are you advocating to limit the log output?
Re: [VOTE] Apache Bahir 2.0.2 (RC2)
ran test suite and saw: http://pastebin.com/xRJaCa6H On Wed, Jan 11, 2017 at 3:33 AM, Christian Kadner wrote: > Please vote to approve the second release candidate for Apache Bahir > version 2.0.2 based on Apache Spark 2.0.2. > > The difference between 2.0.1 and 2.0.2 RC2 can be visualized at: > > https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc2 > > The tag to be voted on is v2.0.2-rc2 > (dd82363878a6bb4a19aade8010c01ddfbf254aff) > > https://github.com/apache/bahir/tree/dd82363878a6bb4a19aade8010c01d > dfbf254aff > > The release files, including signatures, digests, etc. can be found at: > > https://repository.apache.org/content/repositories/orgapachebahir-1008 > > The source distribution is also available at: > > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc2/ > > The vote is open for at least 72 hours and passes if a majority of at least > 3 +1 PMC votes are cast. > > [ ] +1 Release this package as Apache Bahir 2.0.1 > [ ] -1 Do not release this package because ... > > Thanks for you vote! >
