[
https://issues.apache.org/jira/browse/BEAM-11314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17236545#comment-17236545
]
Yichi Zhang commented on BEAM-11314:
------------------------------------
at a glance this seems to be caused by the pubsub verifier pulling same message
from the output topic more than once, the metrics on the dataflow job page all
showing 500 so I assume the processing is correct. The output topic seems gone
so I'm not be able to verify my assumption, will try to manually run to see if
I can reproduce.
> StreamingWordCountIT is flaky
> -----------------------------
>
> Key: BEAM-11314
> URL: https://issues.apache.org/jira/browse/BEAM-11314
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Ahmet Altay
> Assignee: Yichi Zhang
> Priority: P1
> Labels: flaky-test
>
> Example:
> [https://ci-beam.apache.org/job/beam_PreCommit_Python_Commit/16401/consoleFull#gradle-task-235]
> (search for: py36:preCommitIT_streaming)
> job link:
> https://console.cloud.google.com/dataflow/jobs/us-central1/2020-11-19_18_25_20-8827839168269537580?project=apache-beam-testing
> Error:
>
> ======================================================================*18:16:37*
> FAIL: test_streaming_wordcount_it
> (apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT)*18:16:37*
>
> ----------------------------------------------------------------------*18:16:37*
> Traceback (most recent call last):*18:16:37* File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/apache_beam/examples/streaming_wordcount_it_test.py",
> line 107, in test_streaming_wordcount_it*18:16:37*
> save_main_session=False)*18:16:37* File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/apache_beam/examples/streaming_wordcount.py",
> line 108, in run*18:16:37* output |
> beam.io.WriteToPubSub(known_args.output_topic)*18:16:37* File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/apache_beam/pipeline.py",
> line 568, in __exit__*18:16:37* self.result = self.run()*18:16:37*
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/apache_beam/pipeline.py",
> line 547, in run*18:16:37* return self.runner.run_pipeline(self,
> self._options)*18:16:37* File
> "/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Python_Commit/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
> line 72, in run_pipeline*18:16:37* hc_assert_that(self.result,
> pickler.loads(on_success_matcher))*18:16:37* AssertionError: *18:16:37*
> Expected: (Test pipeline expected terminated in state: RUNNING and Expected
> 500 messages.)*18:16:37* but: Expected 500 messages. Got 523 messages.
> Diffs (item, count):*18:16:37* Expected but not in actual:
> dict_items([])*18:16:37* Unexpected: dict_items([(b'125: 1', 1), (b'399:
> 1', 1), (b'480: 1', 1), (b'495: 1', 1), (b'120: 1', 1), (b'115: 1', 1),
> (b'203: 1', 1), (b'236: 1', 1), (b'243: 1', 1), (b'322: 1', 1), (b'268: 1',
> 1), (b'173: 1', 1), (b'425: 1', 1), (b'363: 1', 1), (b'182: 1', 1), (b'445:
> 1', 1), (b'44: 1', 1), (b'437: 1', 1), (b'199: 1', 1), (b'298: 1', 1),
> (b'261: 1', 1), (b'380: 1', 1), (b'473: 1', 1)])*18:16:37*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
