See <https://ci-beam.apache.org/job/beam_PostCommit_Java_Nexmark_Dataflow/10674/display/redirect?page=changes>
Changes: [noreply] Add Bigtable xlang bug to release notes for versions 2.49.0 and 2.50.0 ------------------------------------------ [...truncated 1.58 MB...] SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:57:10.683Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:57:10 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric 2023-09-25T20:57:10.735Z WARNING: streaming query appears to have been stuck for 57 min. 2023-09-25T20:57:10.735Z no activity SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:57:25.798Z RUNNING SqlQuery2 Sep 25, 2023 8:57:25 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.elements, from namespace SqlQuery2.Events Sep 25, 2023 8:57:25 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.bytes, from namespace SqlQuery2.Events Sep 25, 2023 8:57:26 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace SqlQuery2.Results Sep 25, 2023 8:57:26 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:57:26.412Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:57:26 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:57:26.509Z WARNING: streaming query appears to have been stuck for 58 min. 2023-09-25T20:57:26.509Z no activity 2023-09-25T20:57:41.555Z RUNNING SqlQuery2 Sep 25, 2023 8:57:41 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.elements, from namespace SqlQuery2.Events Sep 25, 2023 8:57:41 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.bytes, from namespace SqlQuery2.Events Sep 25, 2023 8:57:41 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace SqlQuery2.Results Sep 25, 2023 8:57:41 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:57:42.131Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:57:42 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric 2023-09-25T20:57:42.197Z WARNING: streaming query appears to have been stuck for 58 min. 2023-09-25T20:57:42.197Z no activity SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:57:57.255Z RUNNING SqlQuery2 Sep 25, 2023 8:57:57 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.elements, from namespace SqlQuery2.Events Sep 25, 2023 8:57:57 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.bytes, from namespace SqlQuery2.Events Sep 25, 2023 8:57:57 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace SqlQuery2.Results Sep 25, 2023 8:57:57 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:57:57.809Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:57:57 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric 2023-09-25T20:57:57.865Z WARNING: streaming query appears to have been stuck for 58 min. 2023-09-25T20:57:57.865Z no activity SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:58:12.895Z RUNNING SqlQuery2 Sep 25, 2023 8:58:12 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.elements, from namespace SqlQuery2.Events Sep 25, 2023 8:58:13 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.bytes, from namespace SqlQuery2.Events Sep 25, 2023 8:58:13 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace SqlQuery2.Results Sep 25, 2023 8:58:13 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:58:13.482Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:58:13 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric 2023-09-25T20:58:13.532Z WARNING: streaming query appears to have been stuck for 59 min. 2023-09-25T20:58:13.532Z no activity SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:58:28.581Z RUNNING SqlQuery2 Sep 25, 2023 8:58:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.elements, from namespace SqlQuery2.Events Sep 25, 2023 8:58:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.bytes, from namespace SqlQuery2.Events Sep 25, 2023 8:58:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace SqlQuery2.Results Sep 25, 2023 8:58:28 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:58:29.147Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:58:29 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:58:29.200Z WARNING: streaming query appears to have been stuck for 59 min. 2023-09-25T20:58:29.200Z no activity 2023-09-25T20:58:44.239Z RUNNING SqlQuery2 Sep 25, 2023 8:58:44 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.elements, from namespace SqlQuery2.Events Sep 25, 2023 8:58:44 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.bytes, from namespace SqlQuery2.Events Sep 25, 2023 8:58:44 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace SqlQuery2.Results Sep 25, 2023 8:58:44 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:58:44.842Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:58:44 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:58:44.899Z WARNING: streaming query appears to have been stuck for 59 min. 2023-09-25T20:58:44.899Z no activity 2023-09-25T20:58:59.943Z RUNNING SqlQuery2 Sep 25, 2023 8:58:59 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.elements, from namespace SqlQuery2.Events Sep 25, 2023 8:59:00 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.bytes, from namespace SqlQuery2.Events Sep 25, 2023 8:59:00 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace SqlQuery2.Results Sep 25, 2023 8:59:00 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:59:00.540Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:59:00 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:59:00.595Z WARNING: streaming query appears to have been stuck for 59 min. 2023-09-25T20:59:00.595Z no activity 2023-09-25T20:59:15.638Z RUNNING SqlQuery2 Sep 25, 2023 8:59:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.elements, from namespace SqlQuery2.Events Sep 25, 2023 8:59:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric event.bytes, from namespace SqlQuery2.Events Sep 25, 2023 8:59:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.elements, from namespace SqlQuery2.Results Sep 25, 2023 8:59:15 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric result.bytes, from namespace SqlQuery2.Results 2023-09-25T20:59:16.174Z 0 samples not enough to calculate steady-state event rate Sep 25, 2023 8:59:16 PM org.apache.beam.sdk.testutils.metrics.MetricsReader getCounterMetric SEVERE: Failed to get metric fatal, from namespace SqlQuery2 2023-09-25T20:59:16.230Z ERROR: streaming query appears to have been stuck for 60 minutes, cancelling job. Sep 25, 2023 8:59:16 PM org.apache.beam.sdk.extensions.gcp.util.RetryHttpRequestInitializer$LoggingHttpBackOffHandler handleResponse WARNING: Request failed with code 400, performed 0 retries due to IOExceptions, performed 0 retries due to unsuccessful status codes, HTTP framework says request can be retried, (caller responsible for retrying): https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2023-09-25_12_55_10-7565622937059565250. Sep 25, 2023 8:59:16 PM org.apache.beam.runners.dataflow.DataflowPipelineJob lambda$cancel$1 WARNING: Failed to cancel job in state RUNNING, please go to the Developers Console to cancel it manually: https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-25_12_55_10-7565622937059565250?project=apache-beam-testing ========================================================================================== Run started 2023-09-25T19:54:53.558Z and ran for PT3862.738S Default configuration: {"debug":true,"query":null,"sourceType":"DIRECT","generateEventFilePathPrefix":null,"sinkType":"DEVNULL","exportSummaryToBigQuery":false,"pubSubMode":"COMBINED","pubsubMessageSerializationMethod":"CODER","sideInputType":"DIRECT","sideInputRowCount":500,"sideInputNumShards":3,"sideInputUrl":null,"sessionGap":600000,"numEvents":100000,"numEventGenerators":100,"rateShape":"SINE","firstEventRate":10000,"nextEventRate":10000,"rateUnit":"PER_SECOND","ratePeriodSec":600,"preloadSeconds":0,"streamTimeout":240,"isRateLimited":false,"useWallclockEventTime":false,"avgPersonByteSize":200,"avgAuctionByteSize":500,"avgBidByteSize":100,"hotAuctionRatio":2,"hotSellersRatio":4,"hotBiddersRatio":4,"windowSizeSec":10,"windowPeriodSec":5,"watermarkHoldbackSec":0,"numInFlightAuctions":100,"numActivePeople":1000,"coderStrategy":"HAND","cpuDelayMs":0,"diskBusyBytes":0,"auctionSkip":123,"fanout":5,"maxAuctionsWaitingTime":600,"occasionalDelaySec":3,"probDelayedEvent":0.1,"maxLogEvents":100000,"usePubsubPublishTime":false,"outOfOrderGroupSize":1,"numKeyBuckets":20000,"pardoCPUFactor":1.0} Configurations: Conf Description 0000 query:PASSTHROUGH; exportSummaryToBigQuery:true; numEvents:10000000 0001 query:CURRENCY_CONVERSION; exportSummaryToBigQuery:true; numEvents:10000000 0002 query:SELECTION; exportSummaryToBigQuery:true; numEvents:10000000 0003 query:LOCAL_ITEM_SUGGESTION; exportSummaryToBigQuery:true; numEvents:10000000 0004 query:AVERAGE_PRICE_FOR_CATEGORY; exportSummaryToBigQuery:true; numEvents:1000000 0005 query:HOT_ITEMS; exportSummaryToBigQuery:true; numEvents:10000000 0006 query:AVERAGE_SELLING_PRICE_BY_SELLER; exportSummaryToBigQuery:true; numEvents:1000000 0007 query:HIGHEST_BID; exportSummaryToBigQuery:true; numEvents:10000000 0008 query:MONITOR_NEW_USERS; exportSummaryToBigQuery:true; numEvents:10000000 0009 query:WINNING_BIDS; exportSummaryToBigQuery:true; numEvents:1000000 0010 query:LOG_TO_SHARDED_FILES; exportSummaryToBigQuery:true; numEvents:10000000 0011 query:USER_SESSIONS; exportSummaryToBigQuery:true; numEvents:10000000 0012 query:PROCESSING_TIME_WINDOWS; exportSummaryToBigQuery:true; numEvents:10000000 0013 query:PORTABILITY_BATCH; exportSummaryToBigQuery:true; numEvents:10000000 0014 query:RESHUFFLE; exportSummaryToBigQuery:true; numEvents:10000000 0015 query:BOUNDED_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true; numEvents:10000000 0016 query:SESSION_SIDE_INPUT_JOIN; exportSummaryToBigQuery:true; numEvents:10000000 Performance: Conf Runtime(sec) (Baseline) Events(/sec) (Baseline) Results (Baseline) 0000 26.9 372411.7 9200000 0001 29.3 341541.7 9200000 0002 *** not run *** 0003 40.2 248558.4 9054 0004 *** not run *** 0005 *** not run *** 0006 *** not run *** 0007 926.8 14721.8 100 0008 *** not run *** 0009 *** not run *** 0010 *** not run *** 0011 *** not run *** 0012 *** not run *** 0013 *** not run *** 0014 *** not run *** 0015 49.4 202515.2 9200000 0016 *** not run *** ========================================================================================== Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to cancel main job: at org.apache.beam.sdk.nexmark.Main.runAll(Main.java:137) at org.apache.beam.sdk.nexmark.Main.main(Main.java:441) Caused by: java.lang.RuntimeException: Unable to cancel main job: at org.apache.beam.sdk.nexmark.NexmarkLauncher.monitor(NexmarkLauncher.java:513) at org.apache.beam.sdk.nexmark.NexmarkLauncher.run(NexmarkLauncher.java:1242) at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:99) at org.apache.beam.sdk.nexmark.Main$Run.call(Main.java:88) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: Failed to cancel job in state RUNNING, please go to the Developers Console to cancel it manually: https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-25_12_55_10-7565622937059565250?project=apache-beam-testing at org.apache.beam.runners.dataflow.DataflowPipelineJob.cancel(DataflowPipelineJob.java:497) at org.apache.beam.sdk.nexmark.NexmarkLauncher.monitor(NexmarkLauncher.java:511) ... 9 more Caused by: java.util.concurrent.ExecutionException: java.io.IOException: Failed to cancel job in state RUNNING, please go to the Developers Console to cancel it manually: https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-25_12_55_10-7565622937059565250?project=apache-beam-testing at java.util.concurrent.FutureTask.report(FutureTask.java:122) at java.util.concurrent.FutureTask.get(FutureTask.java:192) at org.apache.beam.runners.dataflow.DataflowPipelineJob.cancel(DataflowPipelineJob.java:495) ... 10 more Caused by: java.io.IOException: Failed to cancel job in state RUNNING, please go to the Developers Console to cancel it manually: https://console.cloud.google.com/dataflow/jobs/us-central1/2023-09-25_12_55_10-7565622937059565250?project=apache-beam-testing at org.apache.beam.runners.dataflow.DataflowPipelineJob.lambda$cancel$1(DataflowPipelineJob.java:485) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.beam.runners.dataflow.DataflowPipelineJob.cancel(DataflowPipelineJob.java:492) ... 10 more Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException: 400 Bad Request PUT https://dataflow.googleapis.com/v1b3/projects/apache-beam-testing/locations/us-central1/jobs/2023-09-25_12_55_10-7565622937059565250 { "code" : 400, "errors" : [ { "domain" : "global", "message" : "(967b4e94700aeec2): Workflow modification failed. Causes: (c6cc95ce76659683): Operation cancel not allowed for job 2023-09-25_12_55_10-7565622937059565250. Job is not yet ready for canceling. Please retry in a few minutes.", "reason" : "failedPrecondition" } ], "message" : "(967b4e94700aeec2): Workflow modification failed. Causes: (c6cc95ce76659683): Operation cancel not allowed for job 2023-09-25_12_55_10-7565622937059565250. Job is not yet ready for canceling. Please retry in a few minutes.", "status" : "FAILED_PRECONDITION" } at com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146) at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118) at com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37) at com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:439) at com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111) at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:525) at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:466) at com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:576) at org.apache.beam.runners.dataflow.DataflowClient.updateJob(DataflowClient.java:92) at org.apache.beam.runners.dataflow.DataflowPipelineJob.lambda$cancel$1(DataflowPipelineJob.java:456) ... 12 more > Task :sdks:java:testing:nexmark:run FAILED FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:java:testing:nexmark:run'. > Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with > non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.6.2/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 1h 4m 48s 131 actionable tasks: 2 executed, 129 up-to-date Publishing build scan... https://ge.apache.org/s/imksu74gbl2mq Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
