See
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/9910/display/redirect?page=changes>
Changes:
[vitaly.terentyev] Add sparkreceiver:2 module.
[vitaly.terentyev] Fix sparkreceiver dependencies
[noreply] Print diff and scope to state path
[noreply] Correctly print diff and swallow empty commits for the moment
[noreply] Remove quiet flag on debug
[noreply] Use git diff instead of git diff-index to avoid file timestamp changes
[noreply] Make `documentation/io/connectors/` canonical (#23877)
[noreply] [Tour of Beam] Learning content for "Introduction" module (#23085)
------------------------------------------
[...truncated 1.85 MB...]
at
org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at
org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at
org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
FAILED
apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_local_docker
- apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException:
Dataflow pipeline failed. State: FAILED, Error:
java.lang.RuntimeException: java.lang.NoSuchMethodError:
java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108)
at
org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128)
at
org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at
org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142)
Caused by: java.lang.NoSuchMethodError:
java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at
org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197)
at
org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180)
at
org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119)
at
org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126)
at
org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37)
at
org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
at
org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
at
org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
at
org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at
org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at
org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
FAILED
apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py::ExerciseMetricsPipelineTest::test_metrics_fnapi_it
- apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException:
Dataflow pipeline failed. State: FAILED, Error:
java.lang.RuntimeException: java.lang.NoSuchMethodError:
java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108)
at
org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128)
at
org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83)
at
org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
at
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239)
at
org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142)
Caused by: java.lang.NoSuchMethodError:
java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
at
org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197)
at
org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180)
at
org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119)
at
org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126)
at
org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59)
at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73)
at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37)
at
org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607)
at
org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598)
at
org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558)
at
org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340)
at
org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85)
at
org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206)
at
org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
at
org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185)
... 17 more
============ 4 failed, 5 skipped, 26 warnings in 2357.35s (0:39:17) ============
cleanup_container
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221107-165603168330528
Untagged:
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:a49b83baae682219574ca1fdd82ec53f3bc08eb63d77cebf13b211b99dcc03b4
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:bfd568e6-5348-4fa3-9f6f-0d411d9838d1
Untagged:
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:28fddec9ee7d140f11cd94de72248ab9e070f8fabdcaabdc33199c8d9a0c33d0
Deleted: sha256:f72b8fab155be2f31d47ee25b9a377b6d4ac0d4e6071534c04d4b606822bf554
Deleted: sha256:119faff3f83b55f840b8f61bb737873c85b248f777574f0da5603a3bfbef7cce
Deleted: sha256:199f79669f56b1e76919abf574d659dc69cd412500a757a1c32920b092e72f02
Deleted: sha256:32cec8e369e40763bd667881dc947b3dac290b189aec8a68513f9b8d995c5002
Deleted: sha256:b0bb05a6249d5f4ba85398b298bd93797eaf872a082f6aa52a3b6cf1b9d538e0
Deleted: sha256:0ec96c1dd6194a98179a7d8f79359006e64b16928cbecf6305825694115f6cb1
Deleted: sha256:d170d5fd4698c3b88041672bf4f9cdbca24276c64140dac8caa3d8e8a6202ee7
Deleted: sha256:c7b5b54422ed4d40f2b576cc1eda38e9aab5d8d98436ad1fb67399c3ed3061d5
Deleted: sha256:8e294b8074121e5b045a0b12dea0319b49d2b0431c415bedb6cbc60d6f7faa88
Deleted: sha256:b8820d0f6e52f87db76bb4313b7695b32cb84a7932014caae8ef1962cfefc3e4
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:a49b83baae682219574ca1fdd82ec53f3bc08eb63d77cebf13b211b99dcc03b4
Associated tags:
- 20221107-165603168330528
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221107-165603168330528
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221107-165603168330528].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:a49b83baae682219574ca1fdd82ec53f3bc08eb63d77cebf13b211b99dcc03b4].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:28fddec9ee7d140f11cd94de72248ab9e070f8fabdcaabdc33199c8d9a0c33d0
Associated tags:
- bfd568e6-5348-4fa3-9f6f-0d411d9838d1
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:bfd568e6-5348-4fa3-9f6f-0d411d9838d1].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:28fddec9ee7d140f11cd94de72248ab9e070f8fabdcaabdc33199c8d9a0c33d0].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:3c67c4a22d72f5cbfbcf0321e3abd150923232787d70482d1870886ac098a641
Associated tags:
- 5340253f-5647-4c7a-a9a6-68a4d90624f7
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:5340253f-5647-4c7a-a9a6-68a4d90624f7].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:3c67c4a22d72f5cbfbcf0321e3abd150923232787d70482d1870886ac098a641].
Removed the container
> Task :sdks:python:test-suites:dataflow:py310:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:dda3f1c72cf5692201f09b059091e4b0eb12f790bcd87604717a41e848fca231
Associated tags:
- 20221107-165603398807878
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221107-165603398807878
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221107-165603398807878].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:dda3f1c72cf5692201f09b059091e4b0eb12f790bcd87604717a41e848fca231].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:87eefab68169cdb96d015855dcbd4f33a4d38d7cf8296a72e4a8b1113d2d2d18
Associated tags:
- a2f83f27-8e02-48c8-82ff-b81bfe2d71b2
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:a2f83f27-8e02-48c8-82ff-b81bfe2d71b2].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:87eefab68169cdb96d015855dcbd4f33a4d38d7cf8296a72e4a8b1113d2d2d18].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:a33bc2cea938b1431233d3390cef58188d1fa3e59f450b631fd0da4c34edbae2
Associated tags:
- 0349a043-ddaa-42a6-b167-8245a57b5533
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:0349a043-ddaa-42a6-b167-8245a57b5533].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:a33bc2cea938b1431233d3390cef58188d1fa3e59f450b631fd0da4c34edbae2].
> Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer
cleanup_container
WARNING: Successfully resolved tag to sha256, but it is recommended to use
sha256 directly.
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:f290d50f049641da12cf2dd0dd6090b7e5498a6eda6b9854cd212cbd64da6169
Associated tags:
- 20221107-165603391347467
Tags:
-
us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221107-165603391347467
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221107-165603391347467].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:f290d50f049641da12cf2dd0dd6090b7e5498a6eda6b9854cd212cbd64da6169].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:4cd917863b6ae5569c7e002c7a70bef7f6b9bbc83b40ac64e8f4afd065ff25d0
Associated tags:
- 827cbc2d-ebd9-43d2-a899-d8783dc9d2ca
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:827cbc2d-ebd9-43d2-a899-d8783dc9d2ca].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:4cd917863b6ae5569c7e002c7a70bef7f6b9bbc83b40ac64e8f4afd065ff25d0].
Digests:
-
us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:714bc3ad85628ceb31fdc0720b185594454cb847f4969bffc4578c31d552939c
Associated tags:
- cedbc05e-db0a-461d-818c-d4a80fa6bd41
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:cedbc05e-db0a-461d-818c-d4a80fa6bd41].
Deleted
[us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:714bc3ad85628ceb31fdc0720b185594454cb847f4969bffc4578c31d552939c].
> Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED
FAILURE: Build completed with 4 failures.
1: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 363
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py37:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 363
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py310:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
3: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 363
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py39:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
4: Task failed with an exception.
-----------
* Where:
Script
'<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'>
line: 363
* What went wrong:
Execution failed for task
':sdks:python:test-suites:dataflow:py38:validatesContainer'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 13s
62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/sqkz3hqosav3q
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
