See
<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/63/display/redirect>
------------------------------------------
[...truncated 1.05 MB...]
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
... 3 more
19/06/09 18:04:06 INFO BeamFileSystemArtifactRetrievalService: Manifest at
/tmp/sparktestN94Rj9/job_838cf4ac-ee6e-4d7a-a34c-3ae81838d7b6/MANIFEST has 0
artifact locations
ERROR:root:java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/11/shuffle_43_0_0.data.2d253e48-f868-4e01-9530-b51ae8f79b37
(No such file or directory)
19/06/09 18:04:06 INFO BeamFileSystemArtifactStagingService: Removed dir
/tmp/sparktestN94Rj9/job_838cf4ac-ee6e-4d7a-a34c-3ae81838d7b6/
INFO:root:Job state changed to FAILED
E
======================================================================
ERROR: test_flatten (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 543, in
test_flatten
assert_that(res, equal_to(['a', 'b', 'c'] + additional))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_flatten_1560103392.97_28ccc6f3-7144-4317-ae2a-5d30bdf5a7a3 failed in state
FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/23/temp_shuffle_b68073ad-a413-49aa-a210-0e9a4c62eaeb
(No such file or directory)
======================================================================
ERROR: test_flattened_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/runners/portability/spark_runner_test.py";,>
line 144, in test_flattened_side_input
with_transcoding=False)
File "apache_beam/runners/portability/fn_api_runner_test.py", line 234, in
test_flattened_side_input
label='CheckFlattenOfSideInput')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_flattened_side_input_1560103395.45_862fdf73-432a-42d5-b534-230f826ee985
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/26/temp_shuffle_2d746e81-27c0-46d1-8139-f3d0be300380
(No such file or directory)
======================================================================
ERROR: test_gbk_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 242, in
test_gbk_side_input
equal_to([(None, {'a': [1]})]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_gbk_side_input_1560103400.72_d5154a14-733c-43bd-b157-2eec1faf3f41 failed
in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/11/shuffle_12_0_0.data.5fa6f7da-52c4-4a05-94f4-2279b78ecc2c
(No such file or directory)
======================================================================
ERROR: test_group_by_key (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 525, in
test_group_by_key
assert_that(res, equal_to([('a', [1, 2]), ('b', [3])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_group_by_key_1560103403.45_d55b2703-d11e-4fb5-80b1-2f91c8797c7d failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/36/temp_shuffle_043600f5-178a-4e79-8d17-f1c5a259e34e
(No such file or directory)
======================================================================
ERROR: test_large_elements (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 592, in
test_large_elements
assert_that(gbk_res, equal_to(['a', 'b']), label='gbk')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_large_elements_1560103405.22_3d0533a6-9995-46b8-bce2-cf0af7b039b0 failed
in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/0e/shuffle_17_0_0.index.fb2e09a0-1c96-468e-b971-013d29d96814
(No such file or directory)
======================================================================
ERROR: test_multimap_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 253, in
test_multimap_side_input
equal_to([('a', [1, 3]), ('b', [2])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_multimap_side_input_1560103410.74_bb170362-dae6-4139-a772-9b0c1e42ec04
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/36/shuffle_20_0_0.index.5e7754cd-733d-4577-b2c0-46ee9d639f89
(No such file or directory)
======================================================================
ERROR: test_multimap_side_input_type_coercion (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 266, in
test_multimap_side_input_type_coercion
equal_to([('a', [1, 3]), ('b', [2])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_multimap_side_input_type_coercion_1560103414.35_e4b65fbc-fa20-41af-914f-84a933f43b2e
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/1f/temp_shuffle_e4e2163b-203a-4215-8155-4fdd10cef796
(No such file or directory)
======================================================================
ERROR: test_no_subtransform_composite (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 642, in
test_no_subtransform_composite
assert_that((pcoll_a, pcoll_b) | First(), equal_to(['a']))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_no_subtransform_composite_1560103415.75_61238439-f272-4914-b84b-f61489a7d10e
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/36/temp_shuffle_ebd83389-63d6-4f38-b94a-eb823fab1b3f
(No such file or directory)
======================================================================
ERROR: test_pardo_side_and_main_outputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 175, in
test_pardo_side_and_main_outputs
assert_that(unnamed.odd, equal_to([1, 3]), label='unnamed.odd')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_and_main_outputs_1560103423.38_c4991f65-4678-4bf1-8123-e68b541706df
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/0b/shuffle_27_0_0.data.6774a857-0d41-46dc-9f62-1314bcdf6d9a
(No such file or directory)
======================================================================
ERROR: test_pardo_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 186, in
test_pardo_side_inputs
('a', 'y'), ('b', 'y'), ('c', 'y')]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_inputs_1560103426.21_67d72fea-b52d-4fd8-8ac2-63933532f502
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/27/temp_shuffle_6677da53-5e30-459a-91f7-e8e0a519ba34
(No such file or directory)
======================================================================
ERROR: test_pardo_side_outputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 157, in
test_pardo_side_outputs
assert_that(xy.y, equal_to(['y', 'xy']), label='y')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_outputs_1560103427.76_1e5ccbd2-b8fe-48e9-8073-024f63447f1b
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/24/temp_shuffle_011685b9-77b1-4dbd-acf8-c53494a90e29
(No such file or directory)
======================================================================
ERROR: test_pardo_unfusable_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 276, in
test_pardo_unfusable_side_inputs
equal_to([('a', 'a'), ('a', 'b'), ('b', 'a'), ('b', 'b')]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_unfusable_side_inputs_1560103431.17_c9e85962-9ea3-4394-8323-35fbedca9d24
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/21/temp_shuffle_7c0f13be-dacd-44ef-bb66-68ffa99f7ed5
(No such file or directory)
======================================================================
ERROR: test_pardo_windowed_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 213, in
test_pardo_windowed_side_inputs
label='windowed')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_windowed_side_inputs_1560103432.45_88cda60f-1afb-4abf-b73b-2072484f7872
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/2f/shuffle_33_1_0.data.01041665-b812-4bf1-87de-393d8c67dd26
(No such file or directory)
======================================================================
ERROR: test_read (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 561, in
test_read
equal_to(['a', 'b', 'c']))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_read_1560103436.6_87d4fb8a-de02-4f6f-a8c7-eadf32ab3666 failed in state
FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/0b/temp_shuffle_526d10ef-2e63-4865-8a8c-d45a1bb10691
(No such file or directory)
======================================================================
ERROR: test_reshuffle (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 531, in
test_reshuffle
equal_to([1, 2, 3]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_reshuffle_1560103437.91_fd573e8f-ec8e-4bb4-9183-e693b47d2d33 failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/13/shuffle_36_0_0.data.577a4e0e-1b9c-48e7-8972-87a5e650fc01
(No such file or directory)
======================================================================
ERROR: test_sdf_wrap_range_source (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 517, in
test_sdf_wrap_range_source
self._run_sdf_wrapper_pipeline(RangeSource(0, 4), [0, 1, 2, 3])
File "apache_beam/runners/portability/fn_api_runner_test.py", line 503, in
_run_sdf_wrapper_pipeline
p | beam.io.Read(source)
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_sdf_wrap_range_source_1560103441.37_254ba761-efd1-4df1-aede-33eb766aa9ab
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/3d/temp_shuffle_8c74e654-fa6e-4c7e-b92c-96a64ba0c1a3
(No such file or directory)
======================================================================
ERROR: test_windowing (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 573, in
test_windowing
assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_windowing_1560103445.19_68569f9f-1dae-4334-934f-e61c2f382a3b failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-098f9d48-5c49-46ac-9a34-c0d1c7cac5c8/11/shuffle_43_0_0.data.2d253e48-f868-4e01-9530-b51ae8f79b37
(No such file or directory)
----------------------------------------------------------------------
Ran 37 tests in 78.270s
FAILED (errors=17, skipped=13)
> Task :sdks:python:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/build.gradle'>
line: 284
* What went wrong:
Execution failed for task ':sdks:python:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 50s
60 actionable tasks: 47 executed, 13 from cache
Publishing build scan...
https://gradle.com/s/x7p6pmveeeku6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
