See
<https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1459/display/redirect?page=changes>
Changes:
[noreply] [BEAM-12757]: Remove call for Json Dag in Portable PipelineRunner as
[noreply] fix confusing function def (#15120)
------------------------------------------
[...truncated 2.84 MB...]
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 14, 2021 12:43:15 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:43:14.771Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:43:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:43:27.215Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:43:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:43:54.218Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r127fe748961a6843_0000017b44b0d97d_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r127fe748961a6843_0000017b44b0d97d_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 14, 2021 12:44:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:44:15.448Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:44:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:44:42.288Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r1581e97e147c3b77_0000017b44b1d0da_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r1581e97e147c3b77_0000017b44b1d0da_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 14, 2021 12:45:12 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:45:11.787Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:45:26.950Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:45:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:45:43.392Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:45:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:45:51.730Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r73cb707d968a4c5b_0000017b44b2c840_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r73cb707d968a4c5b_0000017b44b2c840_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 14, 2021 12:46:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:46:17.205Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:46:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:46:20.680Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:46:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:46:48.427Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
.....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Introducing
a parse error.
Aug 14, 2021 12:47:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:47:09.796Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Aug 14, 2021 12:47:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:47:14.089Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Waiting on bqjob_r34fe01856f96008b_0000017b44b3bfe7_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r34fe01856f96008b_0000017b44b3bfe7_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
.............................................................Introducing a
parse error.
Aug 14, 2021 12:47:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:47:26.459Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:47:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:47:33.069Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
.....................DELAY(541522, 1)
{timestamp_ms=1628944713000}
late data for: user16_AmethystWombat,AmethystWombat,1,1628944713000,2021-08-14
05:47:35.452
Aug 14, 2021 12:47:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:47:55.014Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:48:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:48:01.917Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r2ba7801a93a263e_0000017b44b4b6a6_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r2ba7801a93a263e_0000017b44b4b6a6_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 14, 2021 12:48:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:48:31.273Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:48:53 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:48:52.544Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
....................................................................................................................................................................................Introducing
a parse error.
Aug 14, 2021 12:49:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:04.753Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r2e24602e6aa9e521_0000017b44b5ada7_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r2e24602e6aa9e521_0000017b44b5ada7_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 14, 2021 12:49:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:25.684Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:49:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:49:42.905Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:50:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:50:13.290Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:50:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:50:16.781Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r7d7ab81723e5277e_0000017b44b6a471_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r7d7ab81723e5277e_0000017b44b6a471_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DataflowRunner_user |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
bq query --batch "SELECT user FROM
[apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user]
LIMIT 10"
Waiting on bqjob_r7cadc975aca30732_0000017b44b6b0ac_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r7cadc975aca30732_0000017b44b6b0ac_1 ... (0s) Current status:
DONE
Cannot find [Magenta, AliceBlue, Almond, Amaranth, Amber, Amethyst,
AndroidGreen, AntiqueBrass, Fuchsia, Ruby, AppleGreen, Apricot, Aqua,
ArmyGreen, Asparagus, Auburn, Azure, Banana, Beige, Bisque, BarnRed,
BattleshipGrey] in text
Waiting for pipeline to produce more results...
Aug 14, 2021 12:50:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:50:34.825Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:50:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:50:43.598Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:50:54 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:50:52.492Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 14, 2021 12:51:12 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:51:10.241Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Aug 14, 2021 12:51:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-14T12:51:32.837Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Waiting on bqjob_r6fc2ca6352ac22bf_0000017b44b7a749_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r6fc2ca6352ac22bf_0000017b44b7a749_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DataflowRunner_user |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
bq query --batch "SELECT user FROM
[apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user]
LIMIT 10"
Waiting on bqjob_r3a98e5bf1841c8f8_0000017b44b7b40d_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r3a98e5bf1841c8f8_0000017b44b7b40d_1 ... (0s) Current status:
DONE
+-----------------------------+
| user |
+-----------------------------+
| user8_AlmondMarmot |
| user2_AppleGreenKangaroo |
| user1_RubyCaneToad |
| user5_AlmondMarmot |
| user5_RubyCaneToad |
| user4_MagentaMarmot |
| user15_MagentaMarmot |
| user6_BattleshipGreyPossum |
| user18_RubyWombat |
| user10_BattleshipGreyQuokka |
+-----------------------------+
Verified Magenta
RUNNING_JOB=`gcloud dataflow jobs list | grep
leaderboard-validation-1628944346383-866 | grep Running | cut -d' ' -f1`
if [ ! -z "${RUNNING_JOB}" ]
then
gcloud dataflow jobs cancel ${RUNNING_JOB}
else
echo "Job 'leaderboard-validation-1628944346383-866' is not running."
fi
Cancelled job [2021-08-14_05_32_44-8966452935119157767]
**************************************
* SUCCESS: LeaderBoard successfully run on DataflowRunner. with Streaming Engine
**************************************
[SUCCESS]
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 51m 21s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/qb2d4uf5stkxi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
