[
https://issues.apache.org/jira/browse/BEAM-5911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733731#comment-16733731
]
Ahmet Altay commented on BEAM-5911:
-----------------------------------
It looks like a race condition in the way the output is chosen here:
[https://github.com/apache/beam/blob/5716dba6d10f32dfa8ab807ffacc2e68f9267225/sdks/python/apache_beam/examples/wordcount_it_test.py#L57]
Using the seconds (time) may not be unique especially when tests are running in
parallel.
Robin, could you make the output location a bit more random for this and other
python IT tests. (Adding a random bit is probably enough, you could also change
time to use millisecond resolution.)
> Python Dataflow IT test fails even though job succeeded
> -------------------------------------------------------
>
> Key: BEAM-5911
> URL: https://issues.apache.org/jira/browse/BEAM-5911
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Henning Rohde
> Assignee: Ahmet Altay
> Priority: Major
>
> https://scans.gradle.com/s/rult2khtcwhvy/console-log?task=:beam-sdks-python:postCommitITTests#L185
> ======================================================================
> ERROR: test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount_it_test.py",
> line 47, in test_wordcount_it
> self._run_wordcount_it()
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount_it_test.py",
> line 75, in _run_wordcount_it
> wordcount.run(test_pipeline.get_full_options_as_args(**extra_opts))
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount.py",
> line 115, in run
> result = p.run()
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
> line 405, in run
> self._options).run(False)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
> line 418, in run
> return self.runner.run_pipeline(self)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
> line 67, in run_pipeline
> hc_assert_that(self.result, pickler.loads(on_success_matcher))
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/build/gradleenv/local/lib/python2.7/site-packages/hamcrest/core/assert_that.py",
> line 43, in assert_that
> _assert_match(actual=arg1, matcher=arg2, reason=arg3)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/build/gradleenv/local/lib/python2.7/site-packages/hamcrest/core/assert_that.py",
> line 49, in _assert_match
> if not matcher.matches(actual):
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/build/gradleenv/local/lib/pytho
> n2.7/site-packages/hamcrest/core/core/allof.py", line 16, in matches
> if not matcher.matches(item):
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/build/gradleenv/local/lib/python2.7/site-packages/hamcrest/core/base_matcher.py",
> line 28, in matches
> match_result = self._matches(item)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/testing/pipeline_verifiers.py",
> line 139, in _matches
> read_lines = self._read_with_retry()
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/utils/retry.py",
> line 197, in wrapper
> raise_with_traceback(exn, exn_traceback)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/utils/retry.py",
> line 184, in wrapper
> return fun(*args, **kwargs)
> File
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/testing/pipeline_verifiers.py",
> line 122, in _read_with_retry
> raise IOError('No such file or directory: %s' % self.file_path)
> IOError: No such file or directory:
> gs://temp-storage-for-end-to-end-tests/py-it-cloud/output/1540778275/results*-of-*
> -------------------- >> begin captured stdout << ---------------------
> Found:
> https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-28_18_58_04-427174494982975515?project=apache-beam-testing.
> --------------------- >> end captured stdout << ----------------------
> ----------------------------------------------------------------------
> XML:
> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/nosetests.xml
> ----------------------------------------------------------------------
> Ran 18 tests in 3196.711s
> The job succeeded. Race condition or are we checking the wrong location?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
