See
<https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1384/display/redirect?page=changes>
Changes:
[Ismaël Mejía] [BEAM-12384] Refine generic types on Read.Bounded internals
[Ismaël Mejía] [BEAM-12384] Set output typeDescriptor explictly in Read.Bounded
[Ismaël Mejía] [BEAM-12384] Infer typeDescriptor from coder if typeDescriptor
is not
[noreply] [BEAM-12093] Overhaul ElasticsearchIO.Write (#14347)
[zhoufek] [BEAM-9487] Add trigger safety check to GroupByKey
[Ismaël Mejía] [BEAM-12415] Update Spark 3 version to 3.1.2
[Boyuan Zhang] Populate use_sdf_read in ExpansionService.
[noreply] [BEAM-12370] Support side input in Samza portable runner (#14883)
[noreply] [BEAM-12158][BEAM-12341] Pubsub fixes. (#14896)
------------------------------------------
[...truncated 2.44 MB...]
"reason" : "notFound"
} ],
"message" : "Table
844138762903:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_team is
deleted.",
"status" : "NOT_FOUND"
}
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:981)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.insertAll(BigQueryServicesImpl.java:1021)
org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.flushRows(BatchedStreamingWrite.java:360)
org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite.access$800(BatchedStreamingWrite.java:68)
org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite$BatchAndInsertElements.finishBundle(BatchedStreamingWrite.java:251)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException:
404 Not Found
POST
https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/beam_postrelease_mobile_gaming/tables/leaderboard_DataflowRunner_team/insertAll?prettyPrint=false
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Table
844138762903:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_team is
deleted.",
"reason" : "notFound"
} ],
"message" : "Table
844138762903:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_team is
deleted.",
"status" : "NOT_FOUND"
}
com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl.lambda$insertAll$1(BigQueryServicesImpl.java:905)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call(BigQueryServicesImpl.java:1484)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r1fd3b2b0d1f8b338_00000179b2ed5b4c_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r1fd3b2b0d1f8b338_00000179b2ed5b4c_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................Introducing
a parse error.
.................................................................................................DELAY(488714,
1)
{timestamp_ms=1622204135000}
late data for:
user12_AliceBlueMarmot,AliceBlueMarmot,9,1622204135000,2021-05-28 05:23:44.006
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r402c9bd1f1a3fb43_00000179b2ee52a6_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r402c9bd1f1a3fb43_00000179b2ee52a6_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
......................................................................Introducing
a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r6e10687a810435a_00000179b2ef4926_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r6e10687a810435a_00000179b2ef4926_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
............................................................................................................................Introducing
a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r49f1ac09241b074b_00000179b2f04028_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r49f1ac09241b074b_00000179b2f04028_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r680e0c2d74a71f76_00000179b2f1392d_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r680e0c2d74a71f76_00000179b2f1392d_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r93286641ae83eab_00000179b2f23122_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r93286641ae83eab_00000179b2f23122_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
.....................................................................................................................................................................................................................................................................................................................................................................................................................DELAY(360902,
1)
{timestamp_ms=1622204564000}
late data for:
user0_MagentaCaneToad,MagentaCaneToad,14,1622204564000,2021-05-28 05:28:45.345
..................Introducing a parse error.
...........Introducing a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r6840b9ca53abb178_00000179b2f32896_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r6840b9ca53abb178_00000179b2f32896_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_rd2ddaa733973b0e_00000179b2f4202d_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_rd2ddaa733973b0e_00000179b2f4202d_1 ... (1s) Current status:
RUNNING
Waiting on bqjob_rd2ddaa733973b0e_00000179b2f4202d_1 ... (1s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
.........................................................................................................................................................................................................................................................Introducing
a parse error.
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r2c7d59997443f5ff_00000179b2f51b91_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r2c7d59997443f5ff_00000179b2f51b91_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r217bbe6874871d8e_00000179b2f612c6_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r217bbe6874871d8e_00000179b2f612c6_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DataflowRunner_user |
| leaderboard_DirectRunner_team |
+-----------------------------------+
bq query --batch "SELECT user FROM
[apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user]
LIMIT 10"
Waiting on bqjob_r67c2e66edfc8363a_00000179b2f61f5f_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r67c2e66edfc8363a_00000179b2f61f5f_1 ... (0s) Current status:
DONE
+-----------------------------+
| user |
+-----------------------------+
| user4_MagentaCaneToad |
| user7_BeigeCockatoo |
| user4_FuchsiaWombat |
| user9_AuburnPlatypus |
| user7_BananaKangaroo |
| user16_AliceBlueBandicoot |
| user9_ArmyGreenBandicoot |
| user3_RubyAntechinus |
| user10_MagentaMarmot |
| user8_AntiqueBrassBandicoot |
+-----------------------------+
Verified Magenta
RUNNING_JOB=`gcloud dataflow jobs list | grep
leaderboard-validation-1622204312623-816 | grep Running | cut -d' ' -f1`
if [ ! -z "${RUNNING_JOB}" ]
then
gcloud dataflow jobs cancel ${RUNNING_JOB}
else
echo "Job 'leaderboard-validation-1622204312623-816' is not running."
fi
Job 'leaderboard-validation-1622204312623-816' is not running.
**************************************
* SUCCESS: LeaderBoard successfully run on DataflowRunner. with Streaming Engine
**************************************
[SUCCESS]
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:direct-java:runMobileGamingJavaDirect'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 32m 9s
9 actionable tasks: 8 executed, 1 from cache
Publishing build scan...
https://gradle.com/s/otgeeypw4vuqw
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
