See
<https://ci-beam.apache.org/job/beam_PostRelease_NightlySnapshot/1700/display/redirect?page=changes>
Changes:
[benjamin.gonzalez] [BEAM-8970] Add docs to run wordcount example on portable
Spark Runner
[Kiley Sok] Update python container version
[benjamin.gonzalez] [BEAM-8970] Add period to end of sentence
[Kyle Weaver] Add self-descriptive message for expected errors.
[noreply] Add --dataflowServiceOptions=enable_prime to useUnifiedWorker
conditions
[noreply] [BEAM-10529] nullable xlang coder (#16923)
[noreply] Fix go fmt break in core/typex/special.go (#17266)
[noreply] [BEAM-5436] Add doc page on Go cross compilation. (#17256)
[noreply] Pr-bot Don't count all reviews as approvals (#17269)
[noreply] Fix postcommits (#17263)
[noreply] [BEAM-14241] Address staticcheck warnings in boot.go (#17264)
[noreply] [BEAM-14157] GrpcWindmillServer: Use stream specific boolean to do
[noreply] [BEAM-10582] Allow (and test) pyarrow 7 (#17229)
[noreply] [BEAM-13519] Solve race issues when the server responds with an error
------------------------------------------
[...truncated 1019.81 KB...]
org.apache.beam.sdk.io.gcp.bigquery.BatchedStreamingWrite$BatchAndInsertElements.finishBundle(BatchedStreamingWrite.java:263)
Caused by: com.google.api.client.googleapis.json.GoogleJsonResponseException:
404 Not Found
POST
https://bigquery.googleapis.com/bigquery/v2/projects/apache-beam-testing/datasets/beam_postrelease_mobile_gaming/tables/leaderboard_DataflowRunner_team/insertAll?prettyPrint=false
{
"code" : 404,
"errors" : [ {
"domain" : "global",
"message" : "Table
844138762903:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_team not
found.",
"reason" : "notFound"
} ],
"message" : "Table
844138762903:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_team not
found.",
"status" : "NOT_FOUND"
}
com.google.api.client.googleapis.json.GoogleJsonResponseException.from(GoogleJsonResponseException.java:146)
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:118)
com.google.api.client.googleapis.services.json.AbstractGoogleJsonClientRequest.newExceptionOnError(AbstractGoogleJsonClientRequest.java:37)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest$1.interceptResponse(AbstractGoogleClientRequest.java:428)
com.google.api.client.http.HttpRequest.execute(HttpRequest.java:1111)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:514)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.executeUnparsed(AbstractGoogleClientRequest.java:455)
com.google.api.client.googleapis.services.AbstractGoogleClientRequest.execute(AbstractGoogleClientRequest.java:565)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl$InsertBatchofRowsCallable.call(BigQueryServicesImpl.java:868)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$DatasetServiceImpl$InsertBatchofRowsCallable.call(BigQueryServicesImpl.java:819)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$BoundedExecutorService$SemaphoreCallable.call(BigQueryServicesImpl.java:1699)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4421876b4e642574_0000017ff86cce4c_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r4421876b4e642574_0000017ff86cce4c_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r25b843597628781b_0000017ff86dc6d9_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r25b843597628781b_0000017ff86dc6d9_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
.....................................................................................................................................................................................................................................................................................................................................................................DELAY(574932,
1)
{timestamp_ms=1649139890000}
late data for:
user3_ApricotBandicoot,ApricotBandicoot,16,1649139890000,2022-04-04 23:34:25.310
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r1373df80b348934b_0000017ff86ec01e_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r1373df80b348934b_0000017ff86ec01e_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r19580884d550d059_0000017ff86fb8e2_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r19580884d550d059_0000017ff86fb8e2_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r7b60ea7040f5d445_0000017ff870b213_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r7b60ea7040f5d445_0000017ff870b213_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r21e717a5cd026d04_0000017ff871aa2d_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r21e717a5cd026d04_0000017ff871aa2d_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_rbc279a91109953b_0000017ff872a283_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_rbc279a91109953b_0000017ff872a283_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................DELAY(465102,
1)
{timestamp_ms=1649140301000}
late data for:
user8_AppleGreenCassowary,AppleGreenCassowary,0,1649140301000,2022-04-04
23:39:26.360
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r259a1af2f74006ce_0000017ff8739a65_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r259a1af2f74006ce_0000017ff8739a65_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r75b2a62c2a640e7_0000017ff87490e2_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r75b2a62c2a640e7_0000017ff87490e2_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
Waiting for pipeline to produce more results...
bq query SELECT table_id FROM beam_postrelease_mobile_gaming.__TABLES_SUMMARY__
Waiting on bqjob_r4f0ed09fba8c07e8_0000017ff8758863_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r4f0ed09fba8c07e8_0000017ff8758863_1 ... (0s) Current status:
DONE
+-----------------------------------+
| table_id |
+-----------------------------------+
| hourly_team_score_python_dataflow |
| hourly_team_score_python_direct |
| leaderboard_DataflowRunner_team |
| leaderboard_DataflowRunner_user |
| leaderboard_DirectRunner_team |
| leaderboard_DirectRunner_user |
+-----------------------------------+
bq query --batch "SELECT user FROM
[apache-beam-testing:beam_postrelease_mobile_gaming.leaderboard_DataflowRunner_user]
LIMIT 10"
Waiting on bqjob_r381de8c821dd7dfe_0000017ff87596e2_1 ... (0s) Current status:
RUNNING
Waiting on bqjob_r381de8c821dd7dfe_0000017ff87596e2_1 ... (0s) Current status:
DONE
+-----------------------------+
| user |
+-----------------------------+
| user2_MagentaWallaby |
| user0_BattleshipGreyEchidna |
| user4_AppleGreenCassowary |
| user9_AmaranthBandicoot |
| user0_FuchsiaNumbat |
| user0_MagentaKangaroo |
| user5_AppleGreenWombat |
| user10_AmaranthCassowary |
| user14_AmaranthWombat |
| user3_ApricotCassowary |
+-----------------------------+
Verified Magenta
RUNNING_JOB=`gcloud dataflow jobs list | grep
leaderboard-validation-1649140155825-453 | grep Running | cut -d' ' -f1`
if [ ! -z "${RUNNING_JOB}" ]
then
gcloud dataflow jobs cancel ${RUNNING_JOB}
else
echo "Job 'leaderboard-validation-1649140155825-453' is not running."
fi
`--region` not set; getting jobs from all available regions. Some jobs may be
missing in the event of an outage.
https://cloud.google.com/dataflow/docs/concepts/regional-endpoints
WARNING: `--region` not set; defaulting to 'us-central1'. In an upcoming
release, users must specify a region explicitly. See
https://cloud.google.com/dataflow/docs/concepts/regional-endpoints for
additional details.
Cancelled job [2022-04-04_23_29_37-11648099530772863361]
**************************************
* SUCCESS: LeaderBoard successfully run on DataflowRunner. with Streaming Engine
**************************************
[SUCCESS]
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:flink:1.13:runQuickstartJavaFlinkLocal'.
> Process 'command '/usr/lib/jvm/java-8-openjdk-amd64/bin/java'' finished with
> non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 1h 35m 55s
19 actionable tasks: 12 executed, 5 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/ew64hsh3mikqo
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
