See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/9891/display/redirect?page=changes>
Changes: [noreply] [Go] Ensure iterated and emitted types are registered. (#23890) [chamikaramj] Updates Multi-language Java examples documentation [noreply] [Python SDK] Re-enable PipelineOptionsTest.test_display_data (#23787) [noreply] Unify PerformanceTest metric dashboard naming and series (#23914) ------------------------------------------ [...truncated 1.95 MB...] E apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: E java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer; E at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187) E at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108) E at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128) E at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56) E at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) E at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) E at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) E at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) E at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) E at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218) E at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169) E at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83) E at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127) E at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) E at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) E at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) E at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239) E at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142) E Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer; E at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197) E at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180) E at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119) E at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126) E at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59) E at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136) E at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73) E at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37) E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607) E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598) E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558) E at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340) E at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85) E at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206) E at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) E at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185) E ... 17 more apache_beam/runners/dataflow/dataflow_runner.py:1592: DataflowRuntimeException =============================== warnings summary =============================== ../../build/gradleenv/2050596098/lib/python3.10/site-packages/hdfs/config.py:15 ../../build/gradleenv/2050596098/lib/python3.10/site-packages/hdfs/config.py:15 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/hdfs/config.py>:15: DeprecationWarning: the imp module is deprecated in favour of importlib and slated for removal in Python 3.12; see the module's documentation for alternative uses from imp import load_source ../../build/gradleenv/2050596098/lib/python3.10/site-packages/google/api_core/operations_v1/abstract_operations_client.py:17 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/google/api_core/operations_v1/abstract_operations_client.py>:17: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives from distutils import util ../../build/gradleenv/2050596098/lib/python3.10/site-packages/tenacity/_asyncio.py:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead def call(self, fn, *args, **kwargs): apache_beam/typehints/pandas_type_compatibility_test.py:66 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:66: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead. }).set_index(pd.Int64Index(range(123, 223), name='an_index')), apache_beam/typehints/pandas_type_compatibility_test.py:89 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:89: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead. pd.Int64Index(range(123, 223), name='an_index'), apache_beam/typehints/pandas_type_compatibility_test.py:90 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead. pd.Int64Index(range(475, 575), name='another_index'), apache_beam/examples/wordcount_it_test.py: 23 warnings apache_beam/runners/dataflow/dataflow_exercise_metrics_pipeline_test.py: 2 warnings <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/2050596098/lib/python3.10/site-packages/httplib2/__init__.py>:147: DeprecationWarning: ssl.PROTOCOL_TLS is deprecated context = ssl.SSLContext(DEFAULT_TLS_VERSION) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.10_sdk.xml> - =========================== short test summary info ============================ FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_it_with_prebuilt_sdk_container_cloud_build - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer; at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169) at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83) at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239) at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142) Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer; at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197) at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180) at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119) at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126) at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59) at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136) at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73) at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37) at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607) at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598) at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558) at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340) at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85) at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185) ... 17 more ======= 1 failed, 3 passed, 5 skipped, 32 warnings in 2242.17s (0:37:22) ======= cleanup_container Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221102-225449097829502 Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:cc6a929e0a637212d27c77724b26926cbc63b5b6d8f8124c3dfe1a42e4225209 Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:6f31d771-a8ae-45d7-ae2e-c6b902e78f0c Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:49f7a99f85916dc523f554057d0af34efcc93137a9b0ff56031e7c6bd28e0b49 Deleted: sha256:112802a740550dc7f80c2c6595277e83d755092366e1cbecae8661089220df3a Deleted: sha256:ce2cdebf9ac1c582954665a252da84229e57c95be860b55e4dd4c57caf186676 Deleted: sha256:3e0c4e85ed147cd31bb7ff8a098371083f391f594eb861202badfaf22b1d1a8c Deleted: sha256:0a75725a808dc93a61a9ec1a390f20d66203633cc56cd6fcd291d7aaaf84d307 Deleted: sha256:94841318d038b874735bb924c637abfc20678cf341c4fb81a4ac50010a3de033 Deleted: sha256:29326fd6276d5611b2e613eebc355e92a35a3963e8aee19f672fbac806f63829 WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:cc6a929e0a637212d27c77724b26926cbc63b5b6d8f8124c3dfe1a42e4225209 Associated tags: - 20221102-225449097829502 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221102-225449097829502 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221102-225449097829502]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:cc6a929e0a637212d27c77724b26926cbc63b5b6d8f8124c3dfe1a42e4225209]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:49f7a99f85916dc523f554057d0af34efcc93137a9b0ff56031e7c6bd28e0b49 Associated tags: - 6f31d771-a8ae-45d7-ae2e-c6b902e78f0c Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:6f31d771-a8ae-45d7-ae2e-c6b902e78f0c]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:49f7a99f85916dc523f554057d0af34efcc93137a9b0ff56031e7c6bd28e0b49]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:b8362cc196aa32586a8e0c628fff9407fa89618d0ef6b89b4736f1c78f6c3639 Associated tags: - 04b4e11c-e11a-42a6-a607-3bd44298ba4a Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:04b4e11c-e11a-42a6-a607-3bd44298ba4a]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:b8362cc196aa32586a8e0c628fff9407fa89618d0ef6b89b4736f1c78f6c3639]. Removed the container > Task :sdks:python:test-suites:dataflow:py310:validatesContainer FAILED > Task :sdks:python:test-suites:dataflow:py39:validatesContainer cleanup_container WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:8a5c56e4049d5970ae35038285d3b8cc39cd0cd452f7207678ddd97555c06cc7 Associated tags: - 20221102-225448999233849 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221102-225448999233849 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221102-225448999233849]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:8a5c56e4049d5970ae35038285d3b8cc39cd0cd452f7207678ddd97555c06cc7]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:e028b4c983a3c13086e0aa0c17c69c8a97bfbd1a23f844c1d9c0d5394e2a51f3 Associated tags: - a20a9cc2-d435-4130-b552-a506bb62e8aa Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:a20a9cc2-d435-4130-b552-a506bb62e8aa]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:e028b4c983a3c13086e0aa0c17c69c8a97bfbd1a23f844c1d9c0d5394e2a51f3]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:8530e1ae89db0e0c0ddd3eb40bce0359135d89923defaea380a9c0e97568cb8a Associated tags: - c2c28316-a3bc-42e2-a24f-fdd82cc1f3ac Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:c2c28316-a3bc-42e2-a24f-fdd82cc1f3ac]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:8530e1ae89db0e0c0ddd3eb40bce0359135d89923defaea380a9c0e97568cb8a]. > Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED FAILURE: Build completed with 4 failures. 1: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== 2: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py37:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== 3: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py310:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== 4: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py39:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 57m 11s 62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date Publishing build scan... https://gradle.com/s/5buxvenn7dnvi Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
