See
<https://ci-beam.apache.org/job/beam_LoadTests_Java_GBK_Dataflow_V2_Streaming_Java17/128/display/redirect?page=changes>
Changes:
[msbukal] FhirIO: use .search() or .searchType instead of .setResourceType()
[nick.caballero] [BEAM-14363] Fixes WatermarkParameters builder for Kinesis
[noreply] Remove unnecessary decorator from RunInference interface (#17463)
[noreply] [BEAM-13590] Minor deprecated warning fix (#17453)
[noreply] [BEAM-12164]: fix the negative throughput issue (#17461)
[noreply] Updated goldens for the screen diff integration tests (#17467)
[noreply] fixes copy by value error for bytes.Buffer in Error (#17469)
[noreply] Merge pull request #17354 from [BEAM-14170] - Create a test that runs
[noreply] Merge pull request #17447 from [BEAM-14357] Fix
[noreply] [BEAM-14324, BEAM-14325] Staticcheck cleanup in test files (#17393)
[noreply] BEAM-14187 Fix NPE (#17454)
[noreply] [BEAM-11105] Stateful watermark estimation (#17374)
[noreply] [BEAM-14304] implement parquetio to read/write parquet files (#17347)
------------------------------------------
[...truncated 73.42 KB...]
SEVERE: 2022-04-27T14:23:14.811Z: generic::aborted: SDK harness sdk-0-0
disconnected.
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
Apr 27, 2022 2:28:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:28:15.707Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:29:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:29:28.119Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:31:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:31:21.257Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:36:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:36:29.917Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:43:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:43:33.248Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:44:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:44:36.075Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:45:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:45:24.804Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:47:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:47:37.351Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:51:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:51:40.169Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:52:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:52:32.494Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:53:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:53:44.999Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:54:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:54:43.652Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 2:55:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T14:55:46.407Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:01:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:01:49.259Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:02:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:02:41.537Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:04:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:04:43.509Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:09:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:09:43.633Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:10:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:10:49.159Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:11:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:11:51.600Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:16:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:16:24.675Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:19:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:19:07.623Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:25:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:25:06.207Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:26:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:26:08.793Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:28:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:28:53.461Z: generic::cancelled: Data channel closed,
unable to send additional data to SDK sdk-0-0
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
generic::cancelled: Data channel closed, unable to send additional data to SDK
sdk-0-0
Apr 27, 2022 3:43:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:43:16.683Z: generic::aborted: SDK harness sdk-0-0
disconnected.
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
Apr 27, 2022 3:43:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:43:23.688Z: generic::aborted: SDK harness sdk-0-0
disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
generic::aborted: SDK harness sdk-0-0 disconnected.
Apr 27, 2022 3:48:11 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T15:48:09.769Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 3:48:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:48:29.612Z: generic::aborted: SDK harness sdk-0-0
disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
generic::aborted: SDK harness sdk-0-0 disconnected.
Apr 27, 2022 3:49:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:49:39.601Z: generic::cancelled: Data channel closed,
unable to send additional data to SDK sdk-0-0
generic::cancelled: Data channel closed, unable to send additional data to SDK
sdk-0-0
passed through:
==>
dist_proc/dax/workflow/****/monitoring_info_view_creator.cc:68
generic::cancelled: Data channel closed, unable to send additional data to SDK
sdk-0-0
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
Apr 27, 2022 3:49:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:49:49.952Z: generic::aborted: SDK harness sdk-0-0
disconnected.
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
Apr 27, 2022 3:50:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:50:13.862Z: generic::aborted: SDK harness sdk-0-0
disconnected.
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/monitoring_info_view_creator.cc:68
Apr 27, 2022 3:51:17 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:51:15.332Z: generic::aborted: SDK harness sdk-0-0
disconnected.
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
generic::cancelled: Data channel closed, unable to send additional data to SDK
sdk-0-0
passed through:
==>
dist_proc/dax/workflow/****/monitoring_info_view_creator.cc:68
Apr 27, 2022 3:52:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:52:19.272Z: generic::aborted: SDK harness sdk-0-0
disconnected.
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
Apr 27, 2022 3:55:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:55:07.106Z: generic::cancelled: Data channel closed,
unable to send additional data to SDK sdk-0-0
passed through:
==>
dist_proc/dax/workflow/****/monitoring_info_view_creator.cc:68
generic::cancelled: Data channel closed, unable to send additional data to SDK
sdk-0-0
generic::cancelled: Data channel closed, unable to send additional data to SDK
sdk-0-0
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
Apr 27, 2022 3:55:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:55:12.327Z: generic::cancelled: Data channel closed,
unable to send additional data to SDK sdk-0-0
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
generic::cancelled: Data channel closed, unable to send additional data to SDK
sdk-0-0
Apr 27, 2022 3:57:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
SEVERE: 2022-04-27T15:57:13.615Z: generic::aborted: SDK harness sdk-0-0
disconnected.
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/fnapi_service_impl.cc:922
generic::aborted: SDK harness sdk-0-0 disconnected.
passed through:
==>
dist_proc/dax/workflow/****/monitoring_info_view_creator.cc:68
Apr 27, 2022 4:00:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:00:03.520Z: Autoscaling: Raised the number of ****s to 5 so
that the pipeline can catch up with its backlog and keep up with its input rate.
Apr 27, 2022 4:00:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:00:41.938Z: Cancel request is committed for workflow job:
2022-04-27_06_00_13-9741245232342243086.
Apr 27, 2022 4:00:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:00:41.971Z: Cleaning up.
Apr 27, 2022 4:00:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:00:42.053Z: Stopping **** pool...
Apr 27, 2022 4:00:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:00:42.095Z: Stopping **** pool...
Apr 27, 2022 4:01:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:01:24.315Z: Autoscaling: Reduced the number of ****s to 0
based on low average **** CPU utilization, and the pipeline having sufficiently
low backlog and keeping up with input rate.
Apr 27, 2022 4:01:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2022-04-27T16:01:24.350Z: Worker pool stopped.
Apr 27, 2022 4:01:30 PM org.apache.beam.runners.dataflow.DataflowPipelineJob
logTerminalState
INFO: Job 2022-04-27_06_00_13-9741245232342243086 finished with status
CANCELLED.
Load test results for test (ID): bbd0dce1-648b-4322-a55a-26c3b4d8664b and
timestamp: 2022-04-27T13:00:05.514000000Z:
Metric: Value:
dataflow_v2_java17_runtime_sec 10100.739
dataflow_v2_java17_total_bytes_count 9.71135279E9
Exception in thread "main" java.lang.RuntimeException: Invalid job state:
CANCELLED.
at
org.apache.beam.sdk.loadtests.JobFailure.handleFailure(JobFailure.java:51)
at org.apache.beam.sdk.loadtests.LoadTest.run(LoadTest.java:139)
at
org.apache.beam.sdk.loadtests.GroupByKeyLoadTest.run(GroupByKeyLoadTest.java:57)
at
org.apache.beam.sdk.loadtests.GroupByKeyLoadTest.main(GroupByKeyLoadTest.java:131)
> Task :sdks:java:testing:load-tests:run FAILED
> Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages
Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220427125750
Untagged:
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9fdfc7aec8d437df29967197b0fcb6e6e8ac785f6c83f74a350e13c8660b4b8b
Deleted: sha256:5cf04e145b7c71b1099dc5d67d4391c9cf84f36f6435ddb0945cd5b1d08b43e5
Deleted: sha256:28452a0d9030747db64029021fddeea2ef3d82d619826187616e0a7abc56231c
Deleted: sha256:3d55316d965904d4889e8fce1ee314ddff02f6053695fee70ceeb70b62cccbae
Deleted: sha256:d3729aa95c48b9347a34c84766cc0cebb9beed869f94b1f1dd28251d7e7fe08a
Deleted: sha256:111f3df6bf6d2ff9ac9a76408ebd1cd522588fffa2fe55879fd1d9151eef4f4e
Deleted: sha256:02c010cc81f0e9e697dcb4176441198a10e6e520611316a6225fdb2c51d7b0fb
Deleted: sha256:2a8ae9fa3245ecf19c117626280d0460289b4bdb50e14f4bd03a015c567fb141
Deleted: sha256:835793f67d32820a30ff6eff57636641190b658aa3547f4986e1089d5b54ec9a
Deleted: sha256:5a6218b393580edd8ae12dd4c656926f5ae91c8ac2d68edeb4efcb9d0b657241
Deleted: sha256:3a8d301f29849d6ac2aadef727073c041548332d0fff062930bc981b654a9afc
Deleted: sha256:10b958ba5c70cea9f2a83ba43dc799f741f5091a382909ee26a5d12e797d3977
Deleted: sha256:9b94cd80a9c0d88f10fb392dd1022504788f3617ab028f0c9a1f8acbc51a274e
Deleted: sha256:87a717e95b20cf52b74d890f6e0892353fc0080e92a38907d2eba7c4e1238b11
Deleted: sha256:30883f30afed1ce7d8263b5c8b3e3c55f75a75db609fe2f369f06353f99cbf95
Deleted: sha256:9c05cb02fc7158d52d96342188741ab8e9276aa28aa19ab29b06d49287630ab4
Deleted: sha256:0169c276de9774e47ebc1fa0a54675fe0516d054fb3e484c28da2bf8a9c8aed7
Deleted: sha256:bfc7da5c46b9d64604076e023fcc0b8c015f3e0d85c745d03f99372ddfb95d9c
Deleted: sha256:49138d7dbf79eee0c791c153a9fc1725f600104e88acc0de177b92ddc48df287
Deleted: sha256:0cfcf6343675ab00e101b2768822e2f450cdc913b05a83434f97a3015ecd4925
Deleted: sha256:b763a57e8127e56b6493c66de69bc08f9ef167ea44c0b73d14d6d47764e63c25
Deleted: sha256:75b7855f3b6cbc3fa33a0305bca1b9292f9039950d7f5b19c2e0ec93bfcffc78
Deleted: sha256:36087f09c6d771eaafa72367dd38fcb33abde333a49f9133ecf17f4492a31daa
Deleted: sha256:651b347fbbd16e37e7829aa30fa87427ecd63f79cecc170afe413e6de7e71e18
Deleted: sha256:3547723351be5ee9c5ca066e6452e47fe4b604fb4ebc6ef462a035c7fe2f887a
Deleted: sha256:3889990e4e662f964b9cbb4f0b5f70d3bf1b98f410afdabbdb9f667a4dd5783d
Deleted: sha256:576dc9b1307172e532830d32ffe2baa518835e2ae8cf76127b2911bf6e6cc8c1
Deleted: sha256:f21e31e5557d54229319980def3d92083523c702789ecea15d953f1dcdb55e8a
Deleted: sha256:b9550d18a2ff13770f37d875deecfb819ada6a8862b15a957f8afafb1ed75b70
Deleted: sha256:073e6f5cc5adbf621ab5bfa62c54a4b1572db3106b6a27ef786363afb4e77221
Deleted: sha256:b2c047ea70372495271a55d91b96e2165b31cf13bddaeeca620c75b7d462769d
Deleted: sha256:ae78e0e98b1017b54f11f366d62dc945c13bca89947ec3b7b6c3254d75334d0b
Deleted: sha256:7df382e60d60050bf17befaf73fbdf0e37f0d90098ff0ebe9bf24ae77576345d
Deleted: sha256:ff6b7b868a66981b8e7f93e0cffad92318d2ca2dec960533ace655df1bcca07c
Deleted: sha256:d5bd08e80ef6ef791716485a1d740929b22ba179122f7a80ed6ddbfb59ff5ee0
Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220427125750]
- referencing digest:
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9fdfc7aec8d437df29967197b0fcb6e6e8ac785f6c83f74a350e13c8660b4b8b]
Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220427125750]
(referencing
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9fdfc7aec8d437df29967197b0fcb6e6e8ac785f6c83f74a350e13c8660b4b8b])].
Removing untagged image
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9fdfc7aec8d437df29967197b0fcb6e6e8ac785f6c83f74a350e13c8660b4b8b
Digests:
-
us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9fdfc7aec8d437df29967197b0fcb6e6e8ac785f6c83f74a350e13c8660b4b8b
Deleted
[us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:9fdfc7aec8d437df29967197b0fcb6e6e8ac785f6c83f74a350e13c8660b4b8b].
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':sdks:java:testing:load-tests:run'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 3h 4m 13s
110 actionable tasks: 75 executed, 31 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/hni4mpdwtcp44
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
