See
<https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1444/display/redirect?page=changes>
Changes:
[ajamato] [BEAM-11994] Update BigQueryStorageStreamSource and
BigQueryServicesImpl
[ajamato] [BEAM-12670] Fix regression, properly report API call metric properly
on
[noreply] [BEAM-9615] Recover from registration panics. (#15228)
[noreply] [BEAM-12609] Enable projection pushdown in SchemaIO. (#15216)
[noreply] [BEAM-12675] Fix issue with language switcher on hadoop page (#15243)
------------------------------------------
[...truncated 2.65 MB...]
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Jul 30, 2021 12:46:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:46:51.012Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:47:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:47:17.769Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:47:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:47:29.377Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:47:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:47:31.999Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:47:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:47:41.966Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_rb6d45c51812190f_0000017af774c6aa_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_rb6d45c51812190f_0000017af774c6aa_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Jul 30, 2021 12:47:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:47:53.235Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:48:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:48:06.483Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:48:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:48:20.424Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:48:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:48:23.108Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:48:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:48:45.535Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r6ef1d2e728346b45_0000017af775c17f_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r6ef1d2e728346b45_0000017af775c17f_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Jul 30, 2021 12:49:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:09.445Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
..................................................................................................................................................................................................................................................................................................................................................................Introducing
a parse error.
Jul 30, 2021 12:49:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:39.028Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:49:47 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:49:46.881Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r75550dec07cd19b1_0000017af776b7d7_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r75550dec07cd19b1_0000017af776b7d7_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Jul 30, 2021 12:50:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:50:13.529Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:50:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:50:28.737Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
.....................................................................................................................................................Introducing
a parse error.
Jul 30, 2021 12:50:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:50:49.277Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r5dfb6df21d1b0771_0000017af777aeda_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r5dfb6df21d1b0771_0000017af777aeda_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
.................................................Introducing a parse error.
Jul 30, 2021 12:51:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:51:03.878Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:51:12 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:51:11.718Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
...............................................DELAY(308000, 1)
{timestamp_ms=1627649180000}
late data for:
user6_AndroidGreenCassowary,AndroidGreenCassowary,8,1627649180000,2021-07-30
05:51:28.237
Jul 30, 2021 12:51:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:51:34.358Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:51:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:51:42.379Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r2bdbe2b6a55087aa_0000017af778a58a_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r2bdbe2b6a55087aa_0000017af778a58a_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Jul 30, 2021 12:52:11 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:52:09.124Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:52:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:52:36.982Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:52:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:52:50.809Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r472e80f7c9f1fd22_0000017af7799c52_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r472e80f7c9f1fd22_0000017af7799c52_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Jul 30, 2021 12:53:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:53:08.502Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:53:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:53:15.845Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:53:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:53:37.856Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:53:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:53:56.274Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
.............................................................................................................................................................................................................................................................................................................Introducing
a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r32b64b0119a67c0f_0000017af77a92ba_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r32b64b0119a67c0f_0000017af77a92ba_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Jul 30, 2021 12:54:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:54:08.824Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:54:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:54:14.524Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:54:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:54:27.269Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:54:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:54:36.067Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:54:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:54:46.101Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:55:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:55:00.537Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r3e6d0c2fe39623e8_0000017af77b8944_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r3e6d0c2fe39623e8_0000017af77b8944_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
...............................................................................................................................................Introducing
a parse error.
Jul 30, 2021 12:55:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:55:14.332Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:55:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:55:33.614Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:55:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:55:51.034Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Jul 30, 2021 12:56:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-07-30T12:56:00.329Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
RUNNING_JOB=`gcloud dataflow jobs list | grep
leaderboard-validation-1627648546234-257 | grep Running | cut -d' ' -f1`
if [ ! -z "${RUNNING_JOB}" ]
then
gcloud dataflow jobs cancel ${RUNNING_JOB}
else
echo "Job 'leaderboard-validation-1627648546234-257' is not running."
fi
Cancelled job [2021-07-30_05_36_56-6892540154501719596]
[ERROR] FAILED: Failed running LeaderBoard on DataflowRunner with Streaming
Engine
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 55m 53s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/75kbbko672tso
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
