See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/9887/display/redirect?page=changes>
Changes: [noreply] Add WriteParquetBatched (#23030) [noreply] Validate if user exists for author (#23761) [noreply] Add notebook for doing remote inference in Beam (#23887) [noreply] Fix python examples tests not running in Dataflow (#23546) [noreply] Add support for converting to/from pyarrow Arrays (#23894) ------------------------------------------ [...truncated 1.96 MB...] E at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) E at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) E at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) E at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239) E at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142) E Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer; E at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197) E at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180) E at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119) E at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126) E at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59) E at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136) E at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73) E at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37) E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607) E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598) E at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558) E at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340) E at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85) E at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206) E at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) E at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185) E ... 17 more apache_beam/runners/dataflow/dataflow_runner.py:1592: DataflowRuntimeException =============================== warnings summary =============================== ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py:15 ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py:15 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/hdfs/config.py>:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses from imp import load_source ../../build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py:42 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/build/gradleenv/-1734967050/lib/python3.9/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead def call(self, fn, *args, **kwargs): apache_beam/typehints/pandas_type_compatibility_test.py:66 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:66: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead. }).set_index(pd.Int64Index(range(123, 223), name='an_index')), apache_beam/typehints/pandas_type_compatibility_test.py:89 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:89: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead. pd.Int64Index(range(123, 223), name='an_index'), apache_beam/typehints/pandas_type_compatibility_test.py:90 <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/typehints/pandas_type_compatibility_test.py>:90: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead. pd.Int64Index(range(475, 575), name='another_index'), -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/pytest-beam_python3.9_sdk.xml> - =========================== short test summary info ============================ FAILED apache_beam/examples/wordcount_it_test.py::WordCountIT::test_wordcount_fnapi_it - apache_beam.runners.dataflow.dataflow_runner.DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error: java.lang.RuntimeException: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer; at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:187) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128) at org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) at org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218) at org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169) at org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83) at org.apache.beam.runners.dataflow.worker.fn.control.BeamFnMapTaskExecutor.execute(BeamFnMapTaskExecutor.java:127) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) at org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.start(DataflowRunnerHarness.java:239) at org.apache.beam.runners.dataflow.worker.DataflowRunnerHarness.main(DataflowRunnerHarness.java:142) Caused by: java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer; at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.outputBuffer(BufferedElementCountingOutputStream.java:197) at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.flush(BufferedElementCountingOutputStream.java:180) at org.apache.beam.sdk.util.BufferedElementCountingOutputStream.finish(BufferedElementCountingOutputStream.java:119) at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:126) at org.apache.beam.sdk.coders.IterableLikeCoder.encode(IterableLikeCoder.java:59) at org.apache.beam.sdk.coders.Coder.encode(Coder.java:136) at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:73) at org.apache.beam.sdk.coders.KvCoder.encode(KvCoder.java:37) at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:607) at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:598) at org.apache.beam.sdk.util.WindowedValue$FullWindowedValueCoder.encode(WindowedValue.java:558) at org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator$Receiver.accept(BeamFnDataOutboundAggregator.java:340) at org.apache.beam.sdk.fn.data.BeamFnDataOutboundObserver.accept(BeamFnDataOutboundObserver.java:85) at org.apache.beam.runners.dataflow.worker.fn.data.RemoteGrpcPortWriteOperation.process(RemoteGrpcPortWriteOperation.java:206) at org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) at org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185) ... 17 more ======= 1 failed, 3 passed, 5 skipped, 6 warnings in 2200.77s (0:36:40) ======== cleanup_container Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221101-225439387512193 Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:2c73e68db8e6a0542020a242d4fef1388d96e483518a98c75d8744361c68a82a Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:13ff1576-e375-421d-91ee-455b6000fd01 Untagged: us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:238c13436dcb26d1049320ccc24da7381b425c775581e9934c1a36e2ab9b1b65 Deleted: sha256:88164f70d69375b262d49cb768976b1fbdb1cf7783e8c5e17aedd30c33794e46 Deleted: sha256:e70c5f90dd294908739c5b80b27d4f1fe915568b289ea0c90a515b6c08cadeff Deleted: sha256:910247a9e88681f9a27d708fb047633f8bcc075c939dfe32994189ef08b8baaf Deleted: sha256:f19c6eaee50ba4b7cbbea060147cbf06bd6546bde9bbe29a3b114a0111e6366a Deleted: sha256:a478b7d3484941f9dc4313c2efa1d6fc19c67bf7f5aed6dba1b84e274df3d4ed Deleted: sha256:abb714cf4298fddd974a4c7b8e81323da9ae5159c0648599ac7680268668b604 WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:2c73e68db8e6a0542020a242d4fef1388d96e483518a98c75d8744361c68a82a Associated tags: - 20221101-225439387512193 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221101-225439387512193 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk:20221101-225439387512193]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.9_sdk@sha256:2c73e68db8e6a0542020a242d4fef1388d96e483518a98c75d8744361c68a82a]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:238c13436dcb26d1049320ccc24da7381b425c775581e9934c1a36e2ab9b1b65 Associated tags: - 13ff1576-e375-421d-91ee-455b6000fd01 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:13ff1576-e375-421d-91ee-455b6000fd01]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:238c13436dcb26d1049320ccc24da7381b425c775581e9934c1a36e2ab9b1b65]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:bba34ccb03b1b54db26f72cf5a9c006ded7a31bf0ed14cdbbd2c0ced79d0d420 Associated tags: - 4dd2afae-947d-4df0-a9fd-ec107b698ec1 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk:4dd2afae-947d-4df0-a9fd-ec107b698ec1]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python39_sdk/beam_python_prebuilt_sdk@sha256:bba34ccb03b1b54db26f72cf5a9c006ded7a31bf0ed14cdbbd2c0ced79d0d420]. Removed the container > Task :sdks:python:test-suites:dataflow:py39:validatesContainer FAILED > Task :sdks:python:test-suites:dataflow:py38:validatesContainer cleanup_container WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:0446518a36bc02c30a5bac60ad18924bba39049f09e518bda670204320e8325d Associated tags: - 20221101-225439387357673 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221101-225439387357673 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk:20221101-225439387357673]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.8_sdk@sha256:0446518a36bc02c30a5bac60ad18924bba39049f09e518bda670204320e8325d]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c0cd3f05c9f0d85ed4e63b55c64c0274f1d7eccb88514d46ff86d85e08e7a06d Associated tags: - 62d5b68e-3e0d-4e12-addf-d24e325b6079 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:62d5b68e-3e0d-4e12-addf-d24e325b6079]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:c0cd3f05c9f0d85ed4e63b55c64c0274f1d7eccb88514d46ff86d85e08e7a06d]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:225905004a069143f78dcb85c82edc2abe9b3d3e703288bbc83d00650699e4c0 Associated tags: - 36638577-f52e-4000-bcf8-d73206ba3c27 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk:36638577-f52e-4000-bcf8-d73206ba3c27]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python38_sdk/beam_python_prebuilt_sdk@sha256:225905004a069143f78dcb85c82edc2abe9b3d3e703288bbc83d00650699e4c0]. > Task :sdks:python:test-suites:dataflow:py38:validatesContainer FAILED > Task :sdks:python:test-suites:dataflow:py310:validatesContainer echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:c2b3d9c8ba7470ff06b51521bea8f716221a0752d50768eb4a549be694560d2c Associated tags: - 20221101-225437454790639 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221101-225437454790639 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk:20221101-225437454790639]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.10_sdk@sha256:c2b3d9c8ba7470ff06b51521bea8f716221a0752d50768eb4a549be694560d2c]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:29b6695fdb055674827f0dc9e396bd2ffd7728212dbde7d0cd0b0682dedb1904 Associated tags: - c0687965-8011-4581-bb37-7a3c5b7802fc Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:c0687965-8011-4581-bb37-7a3c5b7802fc]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:29b6695fdb055674827f0dc9e396bd2ffd7728212dbde7d0cd0b0682dedb1904]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:fbb93d02ae98232d4d3c36bc66d57ee7985bd0e953e5a39331e67a8d90cf59d4 Associated tags: - 9b85e27d-b89a-49cf-8fee-c56126d3a3c0 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk:9b85e27d-b89a-49cf-8fee-c56126d3a3c0]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python310_sdk/beam_python_prebuilt_sdk@sha256:fbb93d02ae98232d4d3c36bc66d57ee7985bd0e953e5a39331e67a8d90cf59d4]. > Task :sdks:python:test-suites:dataflow:py37:validatesContainer echo ">>> SUCCESS DATAFLOW RUNNER VALIDATESCONTAINER TEST" cleanup_container WARNING: Successfully resolved tag to sha256, but it is recommended to use sha256 directly. Digests: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:ee94b3354e034f320d8b0ad23e6d71e705983ac950a9540beedc44d720205353 Associated tags: - 20221101-225441747216648 Tags: - us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221101-225441747216648 Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk:20221101-225441747216648]. Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.7_sdk@sha256:ee94b3354e034f320d8b0ad23e6d71e705983ac950a9540beedc44d720205353]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:ad03de3c1d0990ce7d8fc1d07f2830b91a55549fb30023f7df91b843f2c5e7f7 Associated tags: - 03c1d1a4-98eb-4b4a-9d24-f0d2b2b9b9b2 Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:03c1d1a4-98eb-4b4a-9d24-f0d2b2b9b9b2]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:ad03de3c1d0990ce7d8fc1d07f2830b91a55549fb30023f7df91b843f2c5e7f7]. Digests: - us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:35c5b1d935dd1d9753edb6a8df75ec359129787d92dbfb1022e87c2df668937e Associated tags: - f5b336c5-3d5d-4b23-8d31-f39848d35dca Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk:f5b336c5-3d5d-4b23-8d31-f39848d35dca]. Deleted [us.gcr.io/apache-beam-testing/jenkins/prebuild_python37_sdk/beam_python_prebuilt_sdk@sha256:35c5b1d935dd1d9753edb6a8df75ec359129787d92dbfb1022e87c2df668937e]. FAILURE: Build completed with 2 failures. 1: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py39:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== 2: Task failed with an exception. ----------- * Where: Script '<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/test-suites/dataflow/common.gradle'> line: 363 * What went wrong: Execution failed for task ':sdks:python:test-suites:dataflow:py38:validatesContainer'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. ============================================================================== * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 55m 48s 62 actionable tasks: 51 executed, 4 from cache, 7 up-to-date Publishing build scan... https://gradle.com/s/qft2mw3uexexi Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
