See
<https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1456/display/redirect?page=changes>
Changes:
[andyxu] Add support for python sdk container to turn on google cloud profiler
[andyxu] format file
[marwant] Update Google Cloud Healthcare API version from v1beta1 to GA
[chamikaramj] Fix typo
[zyichi] Update beam dataflow python container versions.
[noreply] [BEAM-12403][BEAM-12348] Add User State Support for Portable
SamzaRunner
------------------------------------------
[...truncated 2.84 MB...]
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 11, 2021 12:44:20 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:44:20.368Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
..................................................................................................................................................................................................................................Introducing
a parse error.
Aug 11, 2021 12:44:25 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:44:23.608Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:44:31 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:44:30.735Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:44:37 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:44:35.274Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:44:40 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:44:38.960Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
...................................................................Introducing
a parse error.
Aug 11, 2021 12:45:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:45:00.620Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r68396ba8ecdd2c02_0000017b353eae49_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r68396ba8ecdd2c02_0000017b353eae49_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 11, 2021 12:45:21 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:45:21.192Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:45:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:45:32.719Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:45:52 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:45:51.184Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r1833a64b5d30b6dc_0000017b353fa480_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r1833a64b5d30b6dc_0000017b353fa480_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 11, 2021 12:46:20 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:46:20.209Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:46:32 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:46:31.514Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:47:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:46:57.892Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:47:08 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:47:06.969Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r5d247944b31c18de_0000017b35409ad0_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r5d247944b31c18de_0000017b35409ad0_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
..................................................................................................................................................................................................................................................................................................................DELAY(505390,
1)
{timestamp_ms=1628685544000}
late data for: user3_RubyAntechinus,RubyAntechinus,19,1628685544000,2021-08-11
05:47:29.434
Aug 11, 2021 12:47:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:47:33.655Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:48:05 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:48:04.852Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r78031a7151023c9e_0000017b35419174_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r78031a7151023c9e_0000017b35419174_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 11, 2021 12:48:35 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:48:33.826Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:48:50 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:48:48.840Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:49:14 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:12.956Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r17aa3890b7e9798e_0000017b354287c8_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r17aa3890b7e9798e_0000017b354287c8_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 11, 2021 12:49:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:22.081Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:49:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:27.490Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:49:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:49:44.405Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:50:02 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:50:00.145Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r6764cc2ac52c5e6b_0000017b35437df6_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r6764cc2ac52c5e6b_0000017b35437df6_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 11, 2021 12:50:31 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:50:30.621Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:50:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:50:39.810Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:51:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:50:59.647Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:51:13 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:51:12.100Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:51:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:51:21.038Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r3527d4a421389624_0000017b35447486_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r3527d4a421389624_0000017b35447486_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 11, 2021 12:51:49 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:51:49.110Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:52:00 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:52:00.311Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Introducing
a parse error.
Aug 11, 2021 12:52:19 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:52:19.467Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:52:23 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:52:23.632Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
.................................DELAY(325212, 1)
{timestamp_ms=1628686024000}
late data for:
user9_ArmyGreenCockatoo,ArmyGreenCockatoo,18,1628686024000,2021-08-11
05:52:29.776
Waiting on bqjob_r63469efbfcd83d43_0000017b35456e9b_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r63469efbfcd83d43_0000017b35456e9b_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
Aug 11, 2021 12:52:34 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:52:34.593Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:52:41 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:52:40.123Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
Aug 11, 2021 12:53:07 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2021-08-11T12:53:06.997Z: Pub/Sub resources set up for topic
'projects/apache-beam-testing/topics/java_mobile_gaming_topic'.
......................................................................................................................
team (ArmyGreenCockatoo, num members: 17, starting at: 1628685148706, expires
in: 21, robot: null) is too old; replacing.
start time: 1628685148706, end time: 1628686408706, current time:1628686408822
[-(ArmyGreenCockatoo, num members: 17, starting at: 1628685148706, expires in:
21, robot: null)]
[+(BattleshipGreyKookaburra, num members: 9, starting at: 1628686408823,
expires in: 21, robot: Robot-7)]
RUNNING_JOB=`gcloud dataflow jobs list | grep
leaderboard-validation-1628685143769-80 | grep Running | cut -d' ' -f1`
if [ ! -z "${RUNNING_JOB}" ]
then
gcloud dataflow jobs cancel ${RUNNING_JOB}
else
echo "Job 'leaderboard-validation-1628685143769-80' is not running."
fi
Cancelled job [2021-08-11_05_32_37-8557587871923488106]
[ERROR] FAILED: Failed running LeaderBoard on DataflowRunner with Streaming
Engine
> Task :runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflow'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':runners:google-cloud-dataflow-java:runMobileGamingJavaDataflowBom'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 53m 14s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/actsfktbf3s32
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
