See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/804/display/redirect?page=changes>
Changes:
[bulat.safiullin] [Website] update python-dependencies.md link #23478
[bulat.safiullin] [Website] update styles of iframe with video #23499
[bulat.safiullin] [Website] add version.html to shortcodes, update jet.md 22985
[noreply] Support named databases in Firestore connector. Fix and enable
Firestore
[noreply] [fixes #23000] Update the Python SDK harness state cache to be a
loading
[noreply] Fix permission for Build python wheel branch_repo_nightly step
(#23563)
------------------------------------------
[...truncated 7.71 MB...]
Oct 10, 2022 9:53:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099/10049a8f-9a9f-45ad-8873-cee675293039.
Oct 10, 2022 9:53:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099/7fde26f4-eb93-49fb-949b-3a7183072bf0.
Oct 10, 2022 9:53:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099/70f3f021-9fda-4b84-b9a4-a5410850b755.
Oct 10, 2022 9:53:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099/9ebff889-e8ca-4d39-ae04-c7dc0ed823f2.
Oct 10, 2022 9:53:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099/6be1b74e-2210-404a-a9d1-1d6e701af6eb.
Oct 10, 2022 9:53:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099/506c40aa-c536-4334-adb2-48de276c26b2.
Oct 10, 2022 9:53:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099/c6099982-73cc-45ba-9cb6-49b2bdf8f37c.
Oct 10, 2022 9:53:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099/42efb7dc-5cd7-4b60-a7a1-22c549a47ea2.
Oct 10, 2022 9:54:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:54:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:54:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:54:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
pending. retrying.
Oct 10, 2022 9:54:32 PM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 8045 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=combine_per_key_examples1665438505394,
projectId=apache-beam-testing, tableId=combine_per_key_examples_table}} using
job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 10, 2022 9:54:32 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:54:32 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
started
Oct 10, 2022 9:54:33 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:54:34 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:54:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:54:37 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:54:41 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:54:46 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:54:57 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:55:07 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:55:28 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:56:01 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
org.apache.beam.examples.snippets.transforms.io.gcp.bigquery.BigQuerySamplesIT
> testTableIO STANDARD_ERROR
Oct 10, 2022 9:56:10 PM org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$4
getTempFilePrefix
INFO: Writing BigQuery temporary files to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7/
before loading them.
Oct 10, 2022 9:56:10 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7/adf50259-8acf-4344-ad92-bdbfb5385d5d.
Oct 10, 2022 9:56:10 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:56:10 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:56:10 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:56:10 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0
pending. retrying.
Oct 10, 2022 9:56:12 PM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 1 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=beam_bigquery_samples_1665438969286_7,
projectId=apache-beam-testing, tableId=testTableIO}} using job
GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 10, 2022 9:56:12 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0
Oct 10, 2022 9:56:12 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0
started
Oct 10, 2022 9:56:13 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0
Oct 10, 2022 9:56:13 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0
Oct 10, 2022 9:56:14 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 10, 2022 9:56:14 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0
succeeded. Statistics:
{"creationTime":"1665438972731","endTime":"1665438974269","load":{"badRecords":"0","inputFileBytes":"484","inputFiles":"1","outputBytes":"239","outputRows":"1","timeline":[{"elapsedMs":"1334","totalSlotMs":"163","pendingUnits":"0","completedUnits":"1","estimatedRunnableUnits":"0"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"163"}],"reservation_id":"default-pipeline","startTime":"1665438972908","totalSlotMs":"163","finalExecutionDurationMs":"653"}
Oct 10, 2022 9:56:14 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins10102156103de85445_7ef4985363e44ad6bb18509400079de7_b494890bdde9c1a1f1bf0f2f9a0d5c80_00001_00000-0
completed successfully.
Oct 10, 2022 9:56:16 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: Starting BigQuery extract job:
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins1010215616114c3d99_1a53b4e2ecca4b61a3d1eaab5a100b00
Oct 10, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins1010215616114c3d99_1a53b4e2ecca4b61a3d1eaab5a100b00,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins1010215616114c3d99_1a53b4e2ecca4b61a3d1eaab5a100b00
Oct 10, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins1010215616114c3d99_1a53b4e2ecca4b61a3d1eaab5a100b00,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins1010215616114c3d99_1a53b4e2ecca4b61a3d1eaab5a100b00
Oct 10, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins1010215616114c3d99_1a53b4e2ecca4b61a3d1eaab5a100b00,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 10, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: BigQuery extract job completed:
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins1010215616114c3d99_1a53b4e2ecca4b61a3d1eaab5a100b00
Oct 10, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase split
INFO: Extract job produced 1 files
Oct 10, 2022 9:56:17 PM org.apache.beam.sdk.io.FileBasedSource createReader
INFO: Matched 1 files for pattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/1a53b4e2ecca4b61a3d1eaab5a100b00/000000000000.avro
org.apache.beam.examples.snippets.transforms.io.gcp.bigquery.BigQuerySamplesIT
> testTableIO FAILED
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.VerifyException
at Verify.java:124
***********************************************************
***********************************************************
**********************Set Up Pubsub************************
The Pub/Sub topic has been set up for this example:
projects/apache-beam-testing/topics/testpipeline-jenkins-1010214634-f26b75e4
The Pub/Sub subscription has been set up for this example:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-1010214634-f26b75e4
*************************Tear Down*************************
The Pub/Sub topic has been deleted:
projects/apache-beam-testing/topics/testpipeline-jenkins-1010214634-f26b75e4
The Pub/Sub subscription has been deleted:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-1010214634-f26b75e4
***********************************************************
***********************************************************
org.apache.beam.examples.cookbook.CombinePerKeyExamplesIT >
testE2ECombinePerKey STANDARD_ERROR
Oct 10, 2022 9:56:39 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
Oct 10, 2022 9:57:20 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 10, 2022 9:57:20 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
succeeded. Statistics:
{"completionRatio":1.0,"creationTime":"1665438872418","endTime":"1665439016807","load":{"badRecords":"0","inputFileBytes":"586637","inputFiles":"8045","outputBytes":"401602","outputRows":"8045","timeline":[{"elapsedMs":"141337","totalSlotMs":"323244","pendingUnits":"3078","completedUnits":"4967","activeUnits":"1000","estimatedRunnableUnits":"0"},{"elapsedMs":"141847","totalSlotMs":"510447","pendingUnits":"501","completedUnits":"8045","activeUnits":"1000","estimatedRunnableUnits":"1000"},{"elapsedMs":"142982","totalSlotMs":"630976","pendingUnits":"0","completedUnits":"8546","activeUnits":"3","estimatedRunnableUnits":"1000"},{"elapsedMs":"143759","totalSlotMs":"630976","pendingUnits":"0","completedUnits":"8546","activeUnits":"0","estimatedRunnableUnits":"1000"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"630976"}],"reservation_id":"default-pipeline","startTime":"1665438872845","totalSlotMs":"630976","finalExecutionDurationMs":"2546"}
Oct 10, 2022 9:57:20 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins101021482871275192_bb5e6a23c28b4df2a6439d527328f099_c7d0c780080403a0a2aeffe8b137d0ff_00001_00000-0
completed successfully.
Oct 10, 2022 9:57:26 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteTable
INFO: Successfully deleted table: combine_per_key_examples_table
Oct 10, 2022 9:57:26 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteDataset
INFO: Successfully deleted dataset: combine_per_key_examples1665438505394
org.apache.beam.examples.cookbook.MaxPerKeyExamplesIT >
testE2EMaxPerKeyExamples STANDARD_ERROR
Oct 10, 2022 9:57:26 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
createNewDataset
INFO: Successfully created new dataset :
apache-beam-testing:max_per_key_examples1665439046360
Oct 10, 2022 9:57:28 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: Starting BigQuery extract job:
beam_bq_job_EXPORT_testpipelinejenkins10102157272a200a56_1cad66240cc04b3ab4e2ca261e945402
Oct 10, 2022 9:57:28 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_EXPORT_testpipelinejenkins10102157272a200a56_1cad66240cc04b3ab4e2ca261e945402,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_testpipelinejenkins10102157272a200a56_1cad66240cc04b3ab4e2ca261e945402
Oct 10, 2022 9:57:28 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_EXPORT_testpipelinejenkins10102157272a200a56_1cad66240cc04b3ab4e2ca261e945402,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_testpipelinejenkins10102157272a200a56_1cad66240cc04b3ab4e2ca261e945402
Oct 10, 2022 9:57:29 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_EXPORT_testpipelinejenkins10102157272a200a56_1cad66240cc04b3ab4e2ca261e945402,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 10, 2022 9:57:29 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: BigQuery extract job completed:
beam_bq_job_EXPORT_testpipelinejenkins10102157272a200a56_1cad66240cc04b3ab4e2ca261e945402
Oct 10, 2022 9:57:29 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase split
INFO: Extract job produced 1 files
Oct 10, 2022 9:57:29 PM org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$4
getTempFilePrefix
INFO: Writing BigQuery temporary files to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/
before loading them.
Oct 10, 2022 9:57:29 PM org.apache.beam.sdk.io.FileBasedSource createReader
INFO: Matched 1 files for pattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/1cad66240cc04b3ab4e2ca261e945402/000000000000.avro
Oct 10, 2022 9:57:29 PM org.apache.beam.sdk.io.FileBasedSource
getEstimatedSizeBytes
INFO: Filepattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/1cad66240cc04b3ab4e2ca261e945402/000000000000.avro
matched 1 files with total size 119755
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/a0cc0d34-a846-4135-b770-42c22853e422.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/19c3ef10-eb04-467e-b337-8fc5087970f4.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/6748108a-529c-4c84-85ba-9541a05ab799.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/9ec3b565-be3c-4747-a92f-16de717b1be5.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/4f0ac956-108b-4c62-be32-a6019a6829ff.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/a3f1b80f-3648-4be0-8b63-17ddf2451e03.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/e0584e4c-e2a5-4a6e-bf46-4483a29e9a73.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/9c86c845-f113-4e71-946b-cb455f66c272.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/443281c9-a972-4841-85dc-d7e3a1e1d568.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/c0fda080-f78a-4b1f-b359-e8941428eb8c.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/31a5ac62-432e-4269-ac5e-1f17422a1f32.
Oct 10, 2022 9:57:30 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6/89ed2afe-093c-4788-a670-3fd35b82517d.
Oct 10, 2022 9:57:31 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:57:31 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:57:31 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 10, 2022 9:57:31 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0
pending. retrying.
Oct 10, 2022 9:57:32 PM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 12 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=max_per_key_examples1665439046360,
projectId=apache-beam-testing, tableId=max_per_key_examples_table}} using job
GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 10, 2022 9:57:32 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0
Oct 10, 2022 9:57:32 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0
started
Oct 10, 2022 9:57:33 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0
Oct 10, 2022 9:57:34 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0
Oct 10, 2022 9:57:36 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 10, 2022 9:57:36 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0
succeeded. Statistics:
{"completionRatio":1.0,"creationTime":"1665439052717","endTime":"1665439054950","load":{"badRecords":"0","inputFileBytes":"522","inputFiles":"12","outputBytes":"192","outputRows":"12","timeline":[{"elapsedMs":"1136","totalSlotMs":"98","pendingUnits":"1","completedUnits":"0","activeUnits":"1","estimatedRunnableUnits":"0"},{"elapsedMs":"2035","totalSlotMs":"406","pendingUnits":"0","completedUnits":"1","activeUnits":"1","estimatedRunnableUnits":"0"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"406"}],"reservation_id":"default-pipeline","startTime":"1665439052887","totalSlotMs":"406","finalExecutionDurationMs":"1149"}
Oct 10, 2022 9:57:36 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins10102157272a200a56_ecade46248224d0e8a9f0bec5bd1a1e6_2e71eac6f912c102919a52b722052b0b_00001_00000-0
completed successfully.
Oct 10, 2022 9:57:37 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteTable
INFO: Successfully deleted table: max_per_key_examples_table
Oct 10, 2022 9:57:38 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteDataset
INFO: Successfully deleted dataset: max_per_key_examples1665439046360
17 tests completed, 1 failed
> Task :runners:direct-java:examplesIntegrationTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:direct-java:examplesIntegrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/ws/src/runners/direct-java/build/reports/tests/examplesIntegrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 12s
89 actionable tasks: 56 executed, 31 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/vyyssztztyqeg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
