See
<https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1451/display/redirect?page=changes>
Changes:
[emilyye] bump FnAPI container
[noreply] [BEAM-6516] Fixes race condition in RabbitMqIO causing duplicate acks
[noreply] [BEAM-12601] Add append-only option (#15257)
[noreply] Revert "[BEAM-11934] Remove Dataflow override of streaming WriteFiles
[andyxu] Add google cloud heap profiling support to beam java sdk container
[Ismaël Mejía] [BEAM-12628] Add Avro reflect-based Coder option
------------------------------------------
[...truncated 2.91 MB...]
INFO: 2021-08-06T12:43:01.988Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r15e3e2b17305feeb_0000017b1b7d2c57_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r15e3e2b17305feeb_0000017b1b7d2c57_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 06, 2021 12:43:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:43:24.846Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:43:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:43:32.752Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:43:48 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:43:46.412Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:44:12 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:44:11.251Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r772a679caca49b70_0000017b1b7e23fc_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r772a679caca49b70_0000017b1b7e23fc_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 06, 2021 12:44:29 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:44:29.714Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:44:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:44:32.900Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:44:47 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:44:47.588Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:44:56 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:44:56.611Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4865c025360d8929_0000017b1b7f1a90_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r4865c025360d8929_0000017b1b7f1a90_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 06, 2021 12:45:24 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:45:21.632Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:45:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:45:38.573Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:46:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:45:58.429Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:46:03 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:46:02.890Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:46:16 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:46:15.256Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r1b1b7a98e9399ca7_0000017b1b8010fc_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r1b1b7a98e9399ca7_0000017b1b8010fc_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 06, 2021 12:46:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:46:38.081Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DELAY(396060,
1)
{timestamp_ms=1628253605000}
late data for: user5_RubyKookaburra,RubyKookaburra,18,1628253605000,2021-08-06
05:46:41.568
Aug 06, 2021 12:47:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:47:07.455Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
.....................................................Introducing a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r1c9f3ed3df3c7db3_0000017b1b8106e9_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r1c9f3ed3df3c7db3_0000017b1b8106e9_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 06, 2021 12:47:33 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:47:31.888Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:47:59 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:47:58.256Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:48:09 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:48:07.594Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Aug 06, 2021 12:48:30 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:48:29.892Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Waiting on bqjob_rf00297a0f080fd5_0000017b1b81fd1e_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_rf00297a0f080fd5_0000017b1b81fd1e_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 06, 2021 12:48:38 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:48:37.256Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:48:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:48:54.939Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:49:04 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:49:02.881Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:49:11 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:49:10.813Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:49:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:49:22.952Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r64292e8c7bc29aa6_0000017b1b82f340_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r64292e8c7bc29aa6_0000017b1b82f340_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 06, 2021 12:49:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:49:42.129Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:49:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:49:50.453Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:49:55 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:49:55.491Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:50:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:50:01.573Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:50:27 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:50:26.845Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r2fc4e69b439bac1a_0000017b1b83e978_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r2fc4e69b439bac1a_0000017b1b83e978_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DataflowRunner_user |
| leaderboard_DirectRunner_team |
+-----------------------------------+
bq query --batch "SELECT user FROM
[apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user]
LIMIT 10"
Waiting on bqjob_r420edd8968c86588_0000017b1b83f687_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r420edd8968c86588_0000017b1b83f687_1 ... (0s) Current status:
DONE
Cannot find [Magenta, AliceBlue, Almond, Amaranth, Amber, Amethyst,
AndroidGreen, AntiqueBrass, Fuchsia, Ruby, AppleGreen, Apricot, Aqua,
ArmyGreen, Asparagus, Auburn, Azure, Banana, Beige, Bisque, BarnRed,
BattleshipGrey] in text
Waiting for pipeline to produce more results...
Aug 06, 2021 12:50:43 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:50:41.334Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:50:46 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:50:45.443Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:51:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:51:11.885Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:51:22 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:51:21.677Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 06, 2021 12:51:39 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-06T12:51:39.233Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
team (AsparagusKangaroo, num members: 18, starting at: 1628253099816, expires
in: 20, robot: Robot-17) is too old; replacing.
start time: 1628253099816, end time: 1628254299816, current time:1628254300309
[-(AsparagusKangaroo, num members: 18, starting at: 1628253099816, expires in:
20, robot: Robot-17)]
[+(AmberMarmot, num members: 13, starting at: 1628254300310, expires in: 34,
robot: null)]
RUNNING_JOB=`gcloud dataflow jobs list | grep
leaderboard-validation-1628253094523-624 | grep Running | cut -d' ' -f1`
if [ ! -z "${RUNNING_JOB}" ]
then
gcloud dataflow jobs cancel ${RUNNING_JOB}
else
echo "Job 'leaderboard-validation-1628253094523-624' is not running."
fi
Job 'leaderboard-validation-1628253094523-624' is not running.
[ERROR] FAILED: Failed running LeaderBoard on DataflowRunner with Streaming
Engine
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 3 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
3: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 51m 23s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/btxu4pqn5aflc
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
