See
<https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1455/display/redirect?page=changes>
Changes:
[heejong] [BEAM-12716] Update XLang Kafka taxi example to use BigQuery sink
[noreply] Merge pull request #15298 from [BEAM-12729] Suppress Avro Runtime
[heejong] add comment
[noreply] Refactor _RemoveDuplicates to use counter state (#15242)
------------------------------------------
[...truncated 2.91 MB...]
Aug 10, 2021 12:42:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:42:17.537Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r22044b13612b276e_0000017b3015d876_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r22044b13612b276e_0000017b3015d876_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 10, 2021 12:42:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:42:29.120Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:42:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:42:33.084Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:42:54 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:42:51.687Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:43:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:43:07.371Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r2b53c7efd6f66389_0000017b3016cef0_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r2b53c7efd6f66389_0000017b3016cef0_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 10, 2021 12:43:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:43:36.456Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:43:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:43:43.537Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
.....................................................................................................................................................................................................................................Introducing
a parse error.
Aug 10, 2021 12:44:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:44:04.658Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
............................................................Introducing a parse
error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r50ddca522268f749_0000017b3017c58a_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r50ddca522268f749_0000017b3017c58a_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 10, 2021 12:44:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:44:35.562Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:44:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:44:42.859Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:44:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:44:47.917Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:45:10 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:45:09.857Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r97a2a0f542fc09a_0000017b3018bc14_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r97a2a0f542fc09a_0000017b3018bc14_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 10, 2021 12:45:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:45:39.362Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:46:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:46:07.039Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Aug 10, 2021 12:46:36 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:46:36.358Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Waiting on bqjob_r4b743ff298b3a107_0000017b3019b2d6_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r4b743ff298b3a107_0000017b3019b2d6_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
......................................................................................................................................................................................................................................................................................................................DELAY(395213,
1)
{timestamp_ms=1628599218000}
late data for: user12_RubyCassowary,RubyCassowary,2,1628599218000,2021-08-10
05:46:54.177
Aug 10, 2021 12:46:57 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:46:56.963Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:47:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:47:02.706Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:47:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:47:13.876Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:47:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:47:30.435Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4000338b6a95f148_0000017b301aaa53_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r4000338b6a95f148_0000017b301aaa53_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 10, 2021 12:47:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:47:53.906Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:48:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:23.723Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4b5d37e94f27aec4_0000017b301ba10f_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r4b5d37e94f27aec4_0000017b301ba10f_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 10, 2021 12:48:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:48:52.869Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:49:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:49:17.798Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
......................................................................................................................................................................................................................................................................................................................Introducing
a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r54c65646a12c17a_0000017b301c9782_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r54c65646a12c17a_0000017b301c9782_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_user |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 10, 2021 12:49:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:49:49.056Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:50:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:50:16.144Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 10, 2021 12:50:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-10T12:50:27.626Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4378e27eb8d0c801_0000017b301d8e10_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r4378e27eb8d0c801_0000017b301d8e10_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DataflowRunner_user |
| leaderboard_DirectRunner_team |
+-----------------------------------+
bq query --batch "SELECT user FROM
[apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user]
LIMIT 10"
Waiting on bqjob_r35b5f865008c78c2_0000017b301d9a46_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r35b5f865008c78c2_0000017b301d9a46_1 ... (0s) Current status:
DONE
+---------------------------------+
| user |
+---------------------------------+
| user4_FuchsiaKookaburra |
| user16_BattleshipGreyAntechinus |
| user3_FuchsiaCaneToad |
| user12_BisqueQuokka |
| user9_AntiqueBrassCassowary |
| user1_AntiqueBrassCassowary |
| user10_AntiqueBrassCassowary |
| user2_AquaWallaby |
| user12_AliceBluePossum |
| user2_AzureNumbat |
+---------------------------------+
Verified AliceBlue
RUNNING_JOB=`gcloud dataflow jobs list | grep
leaderboard-validation-1628598706733-365 | grep Running | cut -d' ' -f1`
if [ ! -z "${RUNNING_JOB}" ]
then
gcloud dataflow jobs cancel ${RUNNING_JOB}
else
echo "Job 'leaderboard-validation-1628598706733-365' is not running."
fi
Job 'leaderboard-validation-1628598706733-365' is not running.
**************************************
* SUCCESS: LeaderBoard successfully run on DataflowRunner. with Streaming Engine
**************************************
[SUCCESS]
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 50m 37s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/ky6lbbpk2ke3a
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
