[jira] [Commented] (APEXMALHAR-2230) Intermittent test failure in Kafka module

[Siyuan Hua (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15526808#comment-15526808
 ] 

Siyuan Hua commented on APEXMALHAR-2230:


Another test failure that could be useful to diagnose
https://api.travis-ci.org/jobs/163154951/log.txt?deansi=true

> Intermittent test failure in Kafka module
> -
>
> Key: APEXMALHAR-2230
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2230
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Siyuan Hua
>
> Test fails intermittently in Travis CI. Could be a race condition in the test?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2230) Intermittent test failure in Kafka module

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493687#comment-15493687
 ] 

ASF GitHub Bot commented on APEXMALHAR-2230:


GitHub user siyuanh opened a pull request:

https://github.com/apache/apex-malhar/pull/413

APEXMALHAR-2230 simplify the kafka input operator test

OK, this change doesn't fix the intermittent error directly, but it 
simplifies the test to make it easier to debug in the future

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/siyuanh/apex-malhar APEXMALHAR-2230

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/apex-malhar/pull/413.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #413


commit 5909dfdc491fdca0cea7eca56fe72b8e1d32bcc3
Author: Siyuan Hua 
Date:   2016-09-15T15:31:34Z

APEXMALHAR-2230 simplify the kafka input operator test




> Intermittent test failure in Kafka module
> -
>
> Key: APEXMALHAR-2230
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2230
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Siyuan Hua
>
> Test fails intermittently in Travis CI. Could be a race condition in the test?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2230) Intermittent test failure in Kafka module

[Siyuan Hua (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491639#comment-15491639
 ] 

Siyuan Hua commented on APEXMALHAR-2230:


Good news is this time we can be sure this is the Kafka test issue. Because we 
don't see NullPointerException before this assertion error

> Intermittent test failure in Kafka module
> -
>
> Key: APEXMALHAR-2230
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2230
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Siyuan Hua
>
> Test fails intermittently in Travis CI. Could be a race condition in the test?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2230) Intermittent test failure in Kafka module

[Siyuan Hua (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491628#comment-15491628
 ] 

Siyuan Hua commented on APEXMALHAR-2230:


The code looks impossible to receive extra END_TUPLE but possible lose some 
END_TUPLE. [~brightchen] Could you also take a look since you change the test?

> Intermittent test failure in Kafka module
> -
>
> Key: APEXMALHAR-2230
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2230
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Siyuan Hua
>
> Test fails intermittently in Travis CI. Could be a race condition in the test?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2230) Intermittent test failure in Kafka module

[Thomas Weise (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491508#comment-15491508
 ] 

Thomas Weise commented on APEXMALHAR-2230:
--

https://s3.amazonaws.com/archive.travis-ci.org/jobs/159966486/log.txt

[~hsy541] any update?

> Intermittent test failure in Kafka module
> -
>
> Key: APEXMALHAR-2230
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2230
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Siyuan Hua
>
> Test fails intermittently in Travis CI. Could be a race condition in the test?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2230) Intermittent test failure in Kafka module

[Thomas Weise (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15488843#comment-15488843
 ] 

Thomas Weise commented on APEXMALHAR-2230:
--

Just hit it again:

https://s3.amazonaws.com/archive.travis-ci.org/jobs/159745501/log.txt


> Intermittent test failure in Kafka module
> -
>
> Key: APEXMALHAR-2230
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2230
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Siyuan Hua
>
> Test fails intermittently in Travis CI. Could be a race condition in the test?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2230) Intermittent test failure in Kafka module

[Siyuan Hua (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15474990#comment-15474990
 ] 

Siyuan Hua commented on APEXMALHAR-2230:


Not necessarily related to Kafka. This is caused by another exception 
2016-09-07 20:13:14,825 [6/Kafka 
inputtesttopic6.outputPort#unifier:DefaultUnifier] ERROR 
engine.StreamingContainer run - Operator set 
[OperatorDeployInfo.UnifierDeployInfo[id=6,name=Kafka 
inputtesttopic6.outputPort#unifier,type=UNIFIER,checkpoint={57d074d30001, 
0, 
0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=(1.outputPort),streamId=Kafka
 
messagetesttopic6,sourceNodeId=1,sourcePortName=outputPort,locality=,partitionMask=0,partitionKeys=],
 
OperatorDeployInfo.InputDeployInfo[portName=(2.outputPort),streamId=Kafka
 
messagetesttopic6,sourceNodeId=2,sourcePortName=outputPort,locality=,partitionMask=0,partitionKeys=],
 
OperatorDeployInfo.InputDeployInfo[portName=(3.outputPort),streamId=Kafka
 
messagetesttopic6,sourceNodeId=3,sourcePortName=outputPort,locality=,partitionMask=0,partitionKeys=],
 
OperatorDeployInfo.InputDeployInfo[portName=(4.outputPort),streamId=Kafka
 
messagetesttopic6,sourceNodeId=4,sourcePortName=outputPort,locality=,partitionMask=0,partitionKeys=]],outputs=[OperatorDeployInfo.OutputDeployInfo[portName=outputPort,streamId=Kafka
 
messagetesttopic6,bufferServer=testing-worker-linux-docker-c25f223a-3424-linux-7
 stopped running due to an exception.
java.lang.NullPointerException
at com.datatorrent.bufferserver.packet.Tuple.getTuple(Tuple.java:54)
at 
com.datatorrent.stram.stream.BufferServerSubscriber$BufferReservoir.sweep(BufferServerSubscriber.java:309)
at com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:259)
at 
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1407)
2016-09-07 20:13:14,826 [6/Kafka 
inputtesttopic6.outputPort#unifier:DefaultUnifier] INFO  
stram.StramLocalCluster log - container-1 msg: Stopped running due to an 
exception. java.lang.NullPointerException
at com.datatorrent.bufferserver.packet.Tuple.getTuple(Tuple.java:54)
at 
com.datatorrent.stram.stream.BufferServerSubscriber$BufferReservoir.sweep(BufferServerSubscriber.java:309)
at com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:259)
at 
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1407)This
 is related to some 

Basically on the bufferserver side it already received some unexpected data.  

> Intermittent test failure in Kafka module
> -
>
> Key: APEXMALHAR-2230
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2230
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>Assignee: Siyuan Hua
>
> Test fails intermittently in Travis CI. Could be a race condition in the test?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (APEXMALHAR-2230) Intermittent test failure in Kafka module

[Thomas Weise (JIRA)]

[ 
https://issues.apache.org/jira/browse/APEXMALHAR-2230?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15471928#comment-15471928
 ] 

Thomas Weise commented on APEXMALHAR-2230:
--

https://s3.amazonaws.com/archive.travis-ci.org/jobs/158274207/log.txt

{code}
2016-09-07 20:13:18,243 [5/TestMessageCollector:CollectorModule] ERROR 
engine.StreamingContainer run - Voluntary container termination due to an error 
in operator set 
[OperatorDeployInfo[id=5,name=TestMessageCollector,type=GENERIC,checkpoint={57d074d30001,
 0, 
0},inputs=[OperatorDeployInfo.InputDeployInfo[portName=inputPort,streamId=Kafka 
messagetesttopic6,sourceNodeId=6,sourcePortName=outputPort,locality=,partitionMask=0,partitionKeys=]],outputs=[]]].
java.lang.AssertionError: received END_TUPLES more than expected.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at 
org.apache.apex.malhar.kafka.KafkaInputOperatorTest$CollectorModule.endWindow(KafkaInputOperatorTest.java:238)
at 
com.datatorrent.stram.engine.GenericNode.processEndWindow(GenericNode.java:146)
at com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:357)
at 
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1407)
2016-09-07 20:13:18,244 [5/TestMessageCollector:CollectorModule] INFO  
stram.StramLocalCluster log - container-2 msg: Voluntary container termination 
due to an error. java.lang.AssertionError: received END_TUPLES more than 
expected.
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.assertTrue(Assert.java:41)
at 
org.apache.apex.malhar.kafka.KafkaInputOperatorTest$CollectorModule.endWindow(KafkaInputOperatorTest.java:238)
at 
com.datatorrent.stram.engine.GenericNode.processEndWindow(GenericNode.java:146)
at com.datatorrent.stram.engine.GenericNode.run(GenericNode.java:357)
at 
com.datatorrent.stram.engine.StreamingContainer$2.run(StreamingContainer.java:1407)
{code}

> Intermittent test failure in Kafka module
> -
>
> Key: APEXMALHAR-2230
> URL: https://issues.apache.org/jira/browse/APEXMALHAR-2230
> Project: Apache Apex Malhar
>  Issue Type: Bug
>Affects Versions: 3.5.0
>Reporter: Thomas Weise
>
> Test fails intermittently in Travis CI. Could be a race condition in the test?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)