See
<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/18/display/redirect>
------------------------------------------
[...truncated 1.01 MB...]
at
org.apache.beam.vendor.guava.v20_0.com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:75)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/13/shuffle_41_0_0.data.4fb1ebd9-b834-4861-ad7c-11593eee2937
(No such file or directory)
at java.io.FileOutputStream.open0(Native Method)
at java.io.FileOutputStream.open(FileOutputStream.java:270)
at java.io.FileOutputStream.<init>(FileOutputStream.java:213)
at
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.writePartitionedFile(BypassMergeSortShuffleWriter.java:191)
at
org.apache.spark.shuffle.sort.BypassMergeSortShuffleWriter.write(BypassMergeSortShuffleWriter.java:163)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:99)
at
org.apache.spark.scheduler.ShuffleMapTask.runTask(ShuffleMapTask.scala:55)
at org.apache.spark.scheduler.Task.run(Task.scala:121)
at
org.apache.spark.executor.Executor$TaskRunner$$anonfun$10.apply(Executor.scala:408)
at org.apache.spark.util.Utils$.tryWithSafeFinally(Utils.scala:1360)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:414)
... 3 more
19/06/06 06:04:15 INFO BeamFileSystemArtifactRetrievalService: Manifest at
/tmp/sparktestY3HvWI/job_a6cf39d3-09a1-4479-926d-1bdbf60edd21/MANIFEST has 0
artifact locations
19/06/06 06:04:15 INFO BeamFileSystemArtifactStagingService: Removed dir
/tmp/sparktestY3HvWI/job_a6cf39d3-09a1-4479-926d-1bdbf60edd21/
ERROR:root:java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/13/shuffle_41_0_0.data.4fb1ebd9-b834-4861-ad7c-11593eee2937
(No such file or directory)
INFO:root:Job state changed to FAILED
E
======================================================================
ERROR: test_create (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 95, in
test_create
assert_that(p | beam.Create(['a', 'b']), equal_to(['a', 'b']))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_create_1559800999.28_344f26e1-4b5d-48e6-82bc-d8ae6edb122d failed in state
FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/19/shuffle_5_0_0.data.ec393e2f-e0f9-4ccc-9c9b-ec23c8cdbab5
(No such file or directory)
======================================================================
ERROR: test_flatten (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 543, in
test_flatten
assert_that(res, equal_to(['a', 'b', 'c'] + additional))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_flatten_1559801006.49_736a87ef-9fc9-4568-851e-252395cb73cb failed in state
FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/1f/shuffle_8_3_0.data.65f363a6-c974-43b2-9af4-3fd33c25f59c
(No such file or directory)
======================================================================
ERROR: test_flattened_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/runners/portability/spark_runner_test.py";,>
line 144, in test_flattened_side_input
with_transcoding=False)
File "apache_beam/runners/portability/fn_api_runner_test.py", line 234, in
test_flattened_side_input
label='CheckFlattenOfSideInput')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_flattened_side_input_1559801008.94_47277943-19a3-47f3-bd37-4aa687fcc05a
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/11/shuffle_10_2_0.data.c88d339d-3041-4c96-9255-05fb15045ac2
(No such file or directory)
======================================================================
ERROR: test_gbk_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 242, in
test_gbk_side_input
equal_to([(None, {'a': [1]})]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_gbk_side_input_1559801013.99_71715c7c-9e46-4797-a3f4-212797c1d302 failed
in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/1f/temp_shuffle_7e114591-f63d-40ba-b0eb-3d358e2ec6d6
(No such file or directory)
======================================================================
ERROR: test_group_by_key (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 525, in
test_group_by_key
assert_that(res, equal_to([('a', [1, 2]), ('b', [3])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_group_by_key_1559801015.86_288518c6-d75e-43ca-adf4-c9543ec9945d failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/13/shuffle_14_0_0.data.1271801b-e641-4f2f-8f76-6e70e8e52127
(No such file or directory)
======================================================================
ERROR: test_large_elements (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 592, in
test_large_elements
assert_that(gbk_res, equal_to(['a', 'b']), label='gbk')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_large_elements_1559801017.21_afcdc3a4-9dc7-41b9-855c-4bc55c28ccfb failed
in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/13/temp_shuffle_17b1deef-ed95-418c-9e61-43ad14944ba8
(No such file or directory)
======================================================================
ERROR: test_multimap_side_input (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 253, in
test_multimap_side_input
equal_to([('a', [1, 3]), ('b', [2])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_multimap_side_input_1559801018.78_2c494981-3d1c-4f4d-853c-322147744de8
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/27/temp_shuffle_50731d8a-14f9-47fc-b97b-5a0594e94908
(No such file or directory)
======================================================================
ERROR: test_multimap_side_input_type_coercion (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 266, in
test_multimap_side_input_type_coercion
equal_to([('a', [1, 3]), ('b', [2])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_multimap_side_input_type_coercion_1559801020.25_e6bdcc60-1e02-469e-9710-2db837918bc1
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/0e/shuffle_17_0_0.index.dfa64c17-306b-463f-a19e-efc93c7a5a07
(No such file or directory)
======================================================================
ERROR: test_pardo (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 103, in
test_pardo
assert_that(res, equal_to(['aax', 'bcbcx']))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_1559801027.29_2f61c63a-ec5b-45f4-9184-d65078281450 failed in state
FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/36/shuffle_20_0_0.index.d2c2424d-6826-49ab-9266-8d2b4133550c
(No such file or directory)
======================================================================
ERROR: test_pardo_side_and_main_outputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 175, in
test_pardo_side_and_main_outputs
assert_that(unnamed.odd, equal_to([1, 3]), label='unnamed.odd')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_and_main_outputs_1559801028.85_2a9b7d1d-04e7-43ed-857e-701c5a840b82
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/11/temp_shuffle_7743fffa-170e-4065-b915-165c3e704423
(No such file or directory)
======================================================================
ERROR: test_pardo_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 186, in
test_pardo_side_inputs
('a', 'y'), ('b', 'y'), ('c', 'y')]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_inputs_1559801031.11_65726761-21c0-44e8-ba07-07282d5966ab
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/19/temp_shuffle_0df1d973-5ce9-4c4d-ae0b-dc385f394ff8
(No such file or directory)
======================================================================
ERROR: test_pardo_side_outputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 157, in
test_pardo_side_outputs
assert_that(xy.y, equal_to(['y', 'xy']), label='y')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_side_outputs_1559801034.58_4b1bb517-6268-47c5-b637-5d3c5d7adc6b
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/2a/shuffle_28_0_0.data.d6f7045f-8c89-4c98-b2b0-2d4fec5ef4bc
(No such file or directory)
======================================================================
ERROR: test_pardo_unfusable_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 276, in
test_pardo_unfusable_side_inputs
equal_to([('a', 'a'), ('a', 'b'), ('b', 'a'), ('b', 'b')]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_unfusable_side_inputs_1559801036.44_eb92209b-1c11-4635-a59d-c49689bdeec4
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/2a/shuffle_30_1_0.index.8975ad11-6ca0-4507-9d38-08be00cec985
(No such file or directory)
======================================================================
ERROR: test_pardo_windowed_side_inputs (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 213, in
test_pardo_windowed_side_inputs
label='windowed')
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_pardo_windowed_side_inputs_1559801041.39_4d91f412-1702-4832-a95e-c7e270d9b537
failed in state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/2a/shuffle_31_0_0.index.fb2a1b3a-6a29-46b5-9822-6e4675499887
(No such file or directory)
======================================================================
ERROR: test_reshuffle (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 531, in
test_reshuffle
equal_to([1, 2, 3]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_reshuffle_1559801048.44_7587c5ad-305d-40df-89a5-dc0193e424cd failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/13/shuffle_36_0_0.data.733d8d9c-e532-4a12-bed3-5e0c29fafc94
(No such file or directory)
======================================================================
ERROR: test_windowing (__main__.SparkRunnerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "apache_beam/runners/portability/fn_api_runner_test.py", line 573, in
test_windowing
assert_that(res, equal_to([('k', [1, 2]), ('k', [100, 101, 102])]))
File "apache_beam/pipeline.py", line 426, in __exit__
self.run().wait_until_finish()
File "apache_beam/runners/portability/portable_runner.py", line 436, in
wait_until_finish
self._job_id, self._state, self._last_error_message()))
RuntimeError: Pipeline
test_windowing_1559801054.03_e8a8eb0d-6c93-42d8-94f5-dd0b6ef24652 failed in
state FAILED: java.io.FileNotFoundException:
/tmp/blockmgr-167d2d2d-d766-4cc2-aadb-66f3f971286e/13/shuffle_41_0_0.data.4fb1ebd9-b834-4861-ad7c-11593eee2937
(No such file or directory)
----------------------------------------------------------------------
Ran 37 tests in 76.293s
FAILED (errors=16, skipped=13)
> Task :sdks:python:sparkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/build.gradle'>
line: 284
* What went wrong:
Execution failed for task ':sdks:python:sparkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 58s
60 actionable tasks: 47 executed, 13 from cache
Publishing build scan...
https://gradle.com/s/mn6pju5pupbhq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
