See
<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/57/display/redirect>
------------------------------------------
[...truncated 1.06 MB...]
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
... 3 more
19/06/08 12:04:04 INFO BeamFileSystemArtifactRetrievalService: Manifest at
/tmp/sparktestqc9DpV/job_96700f2e-517c-43d1-850e-8e1c7b619a25/MANIFEST has 0
artifact locations
19/06/08 12:04:04 INFO BeamFileSystemArtifactStagingService: Removed dir
/tmp/sparktestqc9DpV/job_96700f2e-517c-43d1-850e-8e1c7b619a25/
ERROR:root:java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/11/shuffle_43_0_0.data.b73e368a-9cba-4398-9137-311b86237d54
(No such file or directory)
INFO:root:Job state changed to FAILED
E
======================================================================
ERROR: test_flattened_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/runners/portability/spark_runner_test.py";,>
line 144, in test_flattened_side_input
with_transcoding=False)
File "apache_beam/runners/portability/fn_api_runner_test.py", line 234, in
test_flattened_side_input
label='CheckFlattenOfSideInput')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_flattened_side_input_1559995396.81_bb6844f6-0790-44f1-ad36-751162fad7ce
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/11/shuffle_10_2_0.data.0e427d7d-0cb6-43fd-843d-2dd7ec2abe43
(No such file or directory)
======================================================================
ERROR: test_gbk_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 242, in
test_gbk_side_input
equal_to([(None, {'a': [1]})]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_gbk_side_input_1559995403.1_9c2f35f8-4436-4136-aefd-2e95182be956 failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/11/shuffle_12_0_0.data.131ce79e-cbf5-4f02-8153-dbbee030abe5
(No such file or directory)
======================================================================
ERROR: test_group_by_key (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 525, in
test_group_by_key
assert_that(res, equal_to([('a', [1, 2]), ('b', [3])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_group_by_key_1559995405.87_ae020faa-b3aa-41d4-b12a-ed3a33ad368a failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/21/temp_shuffle_6d14c01a-8164-470c-a639-1af70ba8c854
(No such file or directory)
======================================================================
ERROR: test_large_elements (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 592, in
test_large_elements
assert_that(gbk_res, equal_to(['a', 'b']), label='gbk')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_large_elements_1559995407.68_18168a94-f85f-4007-a813-2e6c700aacd7 failed
in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/22/temp_shuffle_e53535b9-facc-43f9-a890-2845903612b9
(No such file or directory)
======================================================================
ERROR: test_multimap_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 253, in
test_multimap_side_input
equal_to([('a', [1, 3]), ('b', [2])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_multimap_side_input_1559995409.17_64bae254-d856-470d-bd5f-0c6e10b85f52
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/0e/shuffle_17_0_0.index.4c713350-ffb3-4faa-99d7-5056bb5022af
(No such file or directory)
======================================================================
ERROR: test_multimap_side_input_type_coercion (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 266, in
test_multimap_side_input_type_coercion
equal_to([('a', [1, 3]), ('b', [2])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_multimap_side_input_type_coercion_1559995410.57_d9152959-3543-4429-89dc-3696be1b4645
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/36/shuffle_20_0_0.index.fd5ac60a-5827-4683-9ef3-1fe11c7d500a
(No such file or directory)
======================================================================
ERROR: test_no_subtransform_composite (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 642, in
test_no_subtransform_composite
assert_that((pcoll_a, pcoll_b) | First(), equal_to(['a']))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_no_subtransform_composite_1559995414.06_ccfe632c-ab32-42b7-b8d7-b47677a3fc26
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/11/shuffle_21_0_0.data.e265f161-0c9a-45e2-bdb3-b8c2ce471d38
(No such file or directory)
======================================================================
ERROR: test_pardo (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 103, in
test_pardo
assert_that(res, equal_to(['aax', 'bcbcx']))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_1559995416.26_c12efc0a-7a6a-41ec-8aae-dcbe950e26aa failed in state
FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/34/shuffle_22_0_0.index.3ad4aa8c-d393-46f9-9e5d-237a21631bc8
(No such file or directory)
======================================================================
ERROR: test_pardo_side_and_main_outputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 175, in
test_pardo_side_and_main_outputs
assert_that(unnamed.odd, equal_to([1, 3]), label='unnamed.odd')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_and_main_outputs_1559995422.7_b87a34eb-6e30-4189-9c29-20319b1e73d0
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/27/temp_shuffle_4d2fc14b-89e8-4cc9-9c79-65471f3be6e1
(No such file or directory)
======================================================================
ERROR: test_pardo_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 186, in
test_pardo_side_inputs
('a', 'y'), ('b', 'y'), ('c', 'y')]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_inputs_1559995424.94_f2c8b310-44a8-45d4-8017-03f3d3978892
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/11/temp_shuffle_316fa351-1c89-426a-9322-d304f178910a
(No such file or directory)
======================================================================
ERROR: test_pardo_side_outputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 157, in
test_pardo_side_outputs
assert_that(xy.y, equal_to(['y', 'xy']), label='y')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_outputs_1559995426.3_d1a0fc7b-68a3-4f00-9237-2081dda30e91
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/18/temp_shuffle_7d860e0d-1c8f-47fc-b620-fecb8beba11a
(No such file or directory)
======================================================================
ERROR: test_pardo_unfusable_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 276, in
test_pardo_unfusable_side_inputs
equal_to([('a', 'a'), ('a', 'b'), ('b', 'a'), ('b', 'b')]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_unfusable_side_inputs_1559995427.73_d3d02f26-8c03-4ee8-b5af-b38f59807956
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/24/temp_shuffle_9a67308d-49dd-4d78-9d0b-6a221269e3c8
(No such file or directory)
======================================================================
ERROR: test_pardo_windowed_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 213, in
test_pardo_windowed_side_inputs
label='windowed')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_windowed_side_inputs_1559995431.75_a03d7b9a-b6a9-47ee-98a9-cf9af18440d4
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/21/temp_shuffle_5ba5ce0c-ccc8-45ad-a269-6ceeab1e3792
(No such file or directory)
======================================================================
ERROR: test_read (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 561, in
test_read
equal_to(['a', 'b', 'c']))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_read_1559995436.29_f656b0a7-ed01-4919-b1db-25b3c3f273c6 failed in state
FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/12/shuffle_35_0_0.index.348e1e78-61e9-4192-9567-4e4e5522dffd
(No such file or directory)
======================================================================
ERROR: test_reshuffle (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 531, in
test_reshuffle
equal_to([1, 2, 3]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_reshuffle_1559995437.59_d8933675-4af8-43db-946c-64125ef09669 failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/34/shuffle_37_0_0.data.926f77e2-75f0-4d17-a4fd-7e4067480a8e
(No such file or directory)
======================================================================
ERROR: test_sdf_wrap_range_source (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 517, in
test_sdf_wrap_range_source
self._run_sdf_wrapper_pipeline(RangeSource(0, 4), [0, 1, 2, 3])
File "apache_beam/runners/portability/fn_api_runner_test.py", line 503, in
_run_sdf_wrapper_pipeline
p | beam.io.Read(source)
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_sdf_wrap_range_source_1559995439.86_fef41aad-65f3-4e22-b33b-f08225703438
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/36/shuffle_39_0_0.data.29046efe-3ef6-485d-9300-db5c341d720c
(No such file or directory)
======================================================================
ERROR: test_windowing (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 573, in
test_windowing
assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_windowing_1559995442.96_04e2c824-4fb2-4701-8bfd-f9e0cd9572a4 failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-dd638e28-7cce-45c6-944f-7dc187b9d687/11/shuffle_43_0_0.data.b73e368a-9cba-4398-9137-311b86237d54
(No such file or directory)
----------------------------------------------------------------------
Ran 37 tests in 77.780s
FAILED (errors=17, skipped=13)
> Task :sdks:python:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/build.gradle'>
line: 284
* What went wrong:
Execution failed for task ':sdks:python:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 49s
60 actionable tasks: 47 executed, 13 from cache
Publishing build scan...
https://gradle.com/s/f4bfcczova2wm
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
