See
<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/284/display/redirect?page=changes>
Changes:
[vachan] Update display data to include BQ information.
[noreply] Revert "[BEAM-14084] iterable_input_value_types changed from list to
[chamikaramj] Convert URLs to local jars when constructing filesToStage
[Valentyn Tymofieiev] Ensure the removed option
prebuild_sdk_container_base_image not used on
[noreply] [BEAM-13314]Revise recommendations to manage Python pipeline
[noreply] Merge pull request #17202 from [BEAM-14194]: Disallow autoscaling for
[noreply] Merge pull request #17080 from [BEAM-13880] [Playground] Increase test
[noreply] Merge pull request #17050 from [BEAM-13877] [Playground] Increase test
[noreply] [BEAM-14200] Improve SamzaJobInvoker extensibility (#17212)
[noreply] Merge pull request #17148 from [BEAM-14042] [playground] Scroll
imports
[noreply] [BEAM-13918] Increase datastoreio go sdk unit test coverage (#17173)
[noreply] Merge pull request #16819: [BEAM-13806] Adding test suite for Go
x-lang
------------------------------------------
[...truncated 821.31 KB...]
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/windmill_cache_access.cc:40
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/client/streaming_rpc_client.cc:706
==>
dist_proc/dax/workflow/****/streaming/merge_windows_fn.cc:222
generic::internal: The work item requesting state read is no longer valid on
the backend. The work has already completed or will be retried. This is
expected during autoscaling events.
passed through:
==>
dist_proc/windmill/cli
Mar 30, 2022 4:00:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-30T16:00:42.671Z: Cancel request is committed for workflow job:
2022-03-30_05_46_38-9178594846917040139.
Mar 30, 2022 4:00:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-30T16:00:42.732Z: Cleaning up.
Mar 30, 2022 4:00:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-30T16:00:42.814Z: Stopping **** pool...
Mar 30, 2022 4:00:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-30T16:00:42.859Z: Stopping **** pool...
Mar 30, 2022 4:03:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-30T16:03:16.459Z: Autoscaling: Reduced the number of ****s to 0
based on low average **** CPU utilization, and the pipeline having sufficiently
low backlog and keeping up with input rate.
Mar 30, 2022 4:03:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-03-30T16:03:16.512Z: Worker pool stopped.
Mar 30, 2022 4:03:22 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
logTerminalState
INFO: Job 2022-03-30_05_46_38-9178594846917040139 finished with status
CANCELLED.
Load test results for test (ID): d131cef7-ba52-45e5-8af3-00e4172f9140 and
timestamp: 2022-03-30T12:46:22.016000000Z:
Metric: Value:
dataflow_v2_java11_runtime_sec 11473.976
dataflow_v2_java11_total_bytes_count 3.19157773E10
Exception in thread "main" java.lang.RuntimeException: Invalid job state:
CANCELLED.
at
org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at
org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.run(CoGroupByKeyLoadTest.java:62)
at
org.apache.beam.sdk.loadtests.CoGroupByKeyLoadTest.main(CoGroupByKeyLoadTest.java:157)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220330124343
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef6576384c88cf5981db22de19518b8f3e5374815b59df7cbe9a5484bd4ceec8
Deleted: sha256:b59d92435feb90d57c7f3761059e6254af98f044659624a5d8411a8ac260d6ff
Deleted: sha256:832cefbcca2675047fb3e718ac4652e1947794378fff732ee2c703442a7ecbb3
Deleted: sha256:379dcedff93d4bccbd6a780129d54db08a50145269a96f4baba8bedfb0095e6c
Deleted: sha256:0e506b1f340493748b011150c0be9133e2fe0528e918dfa3172f766067111c26
Deleted: sha256:cf82fc4972bc84d08cbdffdc74d6552896bd03357f7dbd473cdbf3a00a0c5fb7
Deleted: sha256:34b758711162e8d6d62d960b45ac865f00cfb4ccb1caeb82a567c2a9d30bd2bd
Deleted: sha256:0e2d81a169ecc59f886783cd6a198a698bc339fcbf08e7912a9963d788f5a656
Deleted: sha256:895866a738d26143e8f784247ae8785e825144f75dcc5bc6ab7eda520378c560
Deleted: sha256:e48db740ae37e8cc53278eb8ae63986874a920a9ac4c3d093ab0a4c72ad05044
Deleted: sha256:2d8b359827c8910bfa0672679db725d38e39dca433182a4b07ef8ee3b3d7fc64
Deleted: sha256:9c9c907c2f2a0db5e275fc99383c2ec24d5886adca993bbcec6a4466cfa0100a
Deleted: sha256:7a893255b1bd330b16a642f1a5acc32a377c863f2ac6d3404cb076a18dc6ed76
Deleted: sha256:116250576388b95d8c1d538bc571be2a421e1bc8f49cb12131998b3c5f42dc38
Deleted: sha256:dc3dbbf0fe40f82a2b44bd9228070305357b84702ceeb26f00fbbdc547204ba7
Deleted: sha256:c551afcdd1c89688461644d553375fc137986db949e556282ce47773d4dc3726
Deleted: sha256:ff2eaf6f109ace3e75263045485f3b63e0385409135414a56b6e76c989d06f2a
Deleted: sha256:bb4a67f0ab8b98fb8c2dc55b540f932414a9d1d8166ff7b32dd423c32074ad39
Deleted: sha256:c0e3f37222352507e090a10c77bf0fedc0b4e9a387fd1bb061b41ab52484d022
Deleted: sha256:28a1b122c796a093a1e8a309a4e537be521c1ba73e8d5fef6b7ff476814c0b4c
Deleted: sha256:71685098caccd538111ee4b338d58d91e1beb424a2a63181393d14917551d6e8
Deleted: sha256:20ad25f0e21848edf3cebbc107976c03852821559393f969f42c90f2dac6634d
Deleted: sha256:0fdfd038be7d6e21f2bd6026dbcc136849619fbfd6c550edbb24d6c8d287db93
Deleted: sha256:91a9d1885fd4e8c229a05cdf64a1854730a851abb1433d6604c5295534e8a23f
Deleted: sha256:5f9ab137519dbf6d007438362c3e873a8ca49fc1b800a4e9e7a3cd8382c2504f
Deleted: sha256:9e35fc3474d3b8189968d8114faf606fd53f6e0977eb7d1fb665d0874133ee2a
Deleted: sha256:dfeabd97d9d3afe33a7678038208e0390fa58d6a416cff0b0ff05d317c2ed20f
Deleted: sha256:0accfedad215a51ceaed31ed0e5ae0684990137d35ca95d83e86c15b31afe494
Deleted: sha256:61dfc7b0bad16bcb99a33336f2f4f69c05aff80849b942eb12bc07604599b9b9
Deleted: sha256:ce9242559fe6bbc228d458b419d02d647577909a6699ad52552685c8fc50d63a
Deleted: sha256:a5c898284925a7eca78f173d19f9742a0f2c64cda17b1c44b652c031b008c425
Deleted: sha256:80488eff4fc5e32d8ab5d4a39d406012d3cad13eadbf9af4cb749cbd2af39503
Deleted: sha256:233acb9cc03a05bd97906f6de130f0d11f5f60c3a28345743103de30ee190570
Deleted: sha256:dff26e1ee1973caa4b67a0d9081ae065ebe3f18ac28af6958a5a1d24747ddd48
Deleted: sha256:d66f3c5646e0e50526a798e870b15f588e7f4af2de8bf7674f29acd7b25817d0
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220330124343]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef6576384c88cf5981db22de19518b8f3e5374815b59df7cbe9a5484bd4ceec8]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220330124343]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef6576384c88cf5981db22de19518b8f3e5374815b59df7cbe9a5484bd4ceec8])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef6576384c88cf5981db22de19518b8f3e5374815b59df7cbe9a5484bd4ceec8
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef6576384c88cf5981db22de19518b8f3e5374815b59df7cbe9a5484bd4ceec8
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:ef6576384c88cf5981db22de19518b8f3e5374815b59df7cbe9a5484bd4ceec8].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:13907352a71480629641f00abc705583ef645ae17d526a6741011f9942ec9435
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:13907352a71480629641f00abc705583ef645ae17d526a6741011f9942ec9435
ERROR: (gcloud.container.images.delete) Not found: response:
{'docker-distribution-api-version': 'registry/2.0', 'content-type':
'application/json', 'date': 'Wed, 30 Mar 2022 16:03:31 GMT', 'server': 'Docker
Registry', 'cache-control': 'private', 'x-xss-protection': '0',
'x-frame-options': 'SAMEORIGIN', 'transfer-encoding': 'chunked', 'status':
'404', 'content-length': '168', '-content-encoding': 'gzip'}
Failed to compute blob liveness for manifest:
'sha256:13907352a71480629641f00abc705583ef645ae17d526a6741011f9942ec9435': None
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* Where:
Build file
'<https://ci-beam.apache.org/job/beam_LoadTests_Java_CoGBK_Dataflow_V2_Streaming_Java11/ws/src/runners/google-cloud-dataflow-java/build.gradle'>
line: 297
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:cleanUpDockerJavaImages'.
> Process 'command './scripts/cleanup_untagged_gcr_images.sh'' finished with
> non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 20m 10s
109 actionable tasks: 72 executed, 33 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/zypmqrlxpgwek
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
