See
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/362/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-12524] Ensure that failed BundleProcessor objects are not
re-added
------------------------------------------
[...truncated 1015.36 KB...]
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingPython
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest >
combineGloballyTest FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest >
partitionTest FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest >
coGroupByKeyTest FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest >
groupByKeyTest FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest >
combinePerKeyTest FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest > flattenTest
FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest >
singleInputOutputTest FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest >
multiInputOutputWithSideInputTest FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
8 tests completed, 8 failed
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingPython
> FAILED
> Task
> :runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingPythonOnly
org.apache.beam.runners.core.construction.ValidateRunnerXlangTest >
pythonDependenciesTest FAILED
java.lang.RuntimeException at ValidateRunnerXlangTest.java:126
1 test completed, 1 failed
> Task
> :runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingPythonOnly
> FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:38205 --environment_cache_millis=10000
>>> --experiments=beam_fn_api
>>> pytest options:
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4803 items / 4795 deselected / 8 selected
apache_beam/io/external/generate_sequence_test.py . [ 12%]
apache_beam/transforms/validate_runner_xlang_test.py F...... [100%]
=================================== FAILURES ===================================
_________________ ValidateRunnerXlangTest.test_cogroup_by_key __________________
self =
<apache_beam.transforms.validate_runner_xlang_test.ValidateRunnerXlangTest
testMethod=test_cogroup_by_key>
test_pipeline = None
@pytest.mark.xlang_transforms
def test_cogroup_by_key(self, test_pipeline=None):
CrossLanguageTestPipelines().run_cogroup_by_key(
> test_pipeline or self.create_pipeline())
apache_beam/transforms/validate_runner_xlang_test.py:269:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
apache_beam/transforms/validate_runner_xlang_test.py:164: in run_cogroup_by_key
assert_that(res, equal_to(['0:1,2,4', '1:3,5,6']))
apache_beam/pipeline.py:585: in __exit__
self.result = self.run()
apache_beam/testing/test_pipeline.py:116: in run
state = result.wait_until_finish()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <apache_beam.runners.portability.portable_runner.PipelineResult object
at 0x7f43da174080>
duration = None
def wait_until_finish(self, duration=None):
"""
:param duration: The maximum time in milliseconds to wait for the result
of
the execution. If None or zero, will wait until the pipeline finishes.
:return: The result of the pipeline, i.e. PipelineResult.
"""
def read_messages():
# type: () -> None
previous_state = -1
for message in self._message_stream:
if message.HasField('message_response'):
logging.log(
MESSAGE_LOG_LEVELS[message.message_response.importance],
"%s",
message.message_response.message_text)
else:
current_state = message.state_response.state
if current_state != previous_state:
_LOGGER.info(
"Job state changed to %s",
self._runner_api_state_to_pipeline_state(current_state))
previous_state = current_state
self._messages.append(message)
message_thread = threading.Thread(
target=read_messages, name='wait_until_finish_read')
message_thread.daemon = True
message_thread.start()
if duration:
state_thread = threading.Thread(
target=functools.partial(self._observe_state, message_thread),
name='wait_until_finish_state_observer')
state_thread.daemon = True
state_thread.start()
start_time = time.time()
duration_secs = duration / 1000
while (time.time() - start_time < duration_secs and
state_thread.is_alive()):
time.sleep(1)
else:
self._observe_state(message_thread)
if self._runtime_exception:
> raise self._runtime_exception
E RuntimeError: Pipeline
BeamApp-jenkins-0624225811-1c9f0cd9_32ff8c21-c89e-4645-9f21-75befd998979 failed
in state FAILED: java.util.concurrent.TimeoutException: Timed out while waiting
for command 'docker run -d --mount
type=bind,src=/home/jenkins/.config/gcloud,dst=/root/.config/gcloud
--network=host --env=DOCKER_MAC_CONTAINER=null
apache/beam_python3.6_sdk:2.32.0.dev --id=19-1
--provision_endpoint=localhost:38805'
apache_beam/runners/portability/portable_runner.py:600: RuntimeError
----------------------------- Captured stderr call -----------------------------
WARNING:root:Make sure that locally built Python SDK docker image has Python
3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python
3.6 interpreter.
ERROR:root:java.util.concurrent.TimeoutException: Timed out while waiting for
command 'docker run -d --mount
type=bind,src=/home/jenkins/.config/gcloud,dst=/root/.config/gcloud
--network=host --env=DOCKER_MAC_CONTAINER=null
apache/beam_python3.6_sdk:2.32.0.dev --id=19-1
--provision_endpoint=localhost:38805'
------------------------------ Captured log call -------------------------------
WARNING root:environments.py:374 Make sure that locally built Python SDK
docker image has Python 3.6 interpreter.
WARNING root:environments.py:374 Make sure that locally built Python SDK
docker image has Python 3.6 interpreter.
ERROR root:portable_runner.py:570 java.util.concurrent.TimeoutException:
Timed out while waiting for command 'docker run -d --mount
type=bind,src=/home/jenkins/.config/gcloud,dst=/root/.config/gcloud
--network=host --env=DOCKER_MAC_CONTAINER=null
apache/beam_python3.6_sdk:2.32.0.dev --id=19-1
--provision_endpoint=localhost:38805'
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54:
DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint:
disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62:
DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), #
pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/pytest_xlangValidateRunner.xml>
-
====== 1 failed, 7 passed, 4795 deselected, 2 warnings in 906.06 seconds =======
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava
> FAILED
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:38205 --environment_cache_millis=10000
>>> --experiments=beam_fn_api
>>> pytest options:
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4803 items / 4795 deselected / 8 selected
apache_beam/io/external/generate_sequence_test.py . [ 12%]
apache_beam/transforms/validate_runner_xlang_test.py ....... [100%]
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/pytest_xlangValidateRunner.xml>
-
================= 8 passed, 4795 deselected in 244.27 seconds ==================
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:38205 --environment_cache_millis=10000
>>> --experiments=beam_fn_api
>>> pytest options:
>>> collect markers: -m=xlang_sql_expansion_service
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 4803 items / 4794 deselected / 9 selected
apache_beam/transforms/sql_test.py ......... [100%]
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/pytest_xlangSqlValidateRunner.xml>
-
================= 9 passed, 4794 deselected in 277.84 seconds ==================
> Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 22987.
Stopping expansion service pid: 22990.
> Task :runners:spark:3:job-server:sparkJobServerCleanup
Stopping job server pid: 21802.
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingJava'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/runners/spark/3/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingJava/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingPython'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/runners/spark/3/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPython/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingPythonOnly'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/runners/spark/3/job-server/build/reports/tests/validatesCrossLanguageRunnerJavaUsingPythonOnly/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
4: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 32m 53s
181 actionable tasks: 133 executed, 44 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/fd7ysk6sl7yr6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
