See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/789/display/redirect?page=changes>
Changes:
[noreply] Fix bug where `astype(CategoricalDtype)` is rejected (#23513)
------------------------------------------
[...truncated 7.52 MB...]
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec/220e1976-edee-463a-99db-e8ad6ee60bab.
Oct 07, 2022 3:51:39 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec/bbeede40-26a9-4254-ad95-731947ff9e7c.
Oct 07, 2022 3:51:39 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec/e1c6b41d-6794-4683-a03d-68fb466512fb.
Oct 07, 2022 3:51:39 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec/0002cd49-e469-4c5f-a43d-2121af3186d8.
Oct 07, 2022 3:51:39 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec/fb438847-18d3-4d07-a857-f50ca2e08d5a.
Oct 07, 2022 3:51:39 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec/5239fd93-f762-448b-b756-b38654e7e8fa.
Oct 07, 2022 3:51:39 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec/d347c44b-d670-43c4-9fb0-bc54e24cf55b.
Oct 07, 2022 3:52:55 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:52:55 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:52:55 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:52:55 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
pending. retrying.
Oct 07, 2022 3:52:56 AM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 8045 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=combine_per_key_examples1665114440676,
projectId=apache-beam-testing, tableId=combine_per_key_examples_table}} using
job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 07, 2022 3:52:58 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:52:58 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
started
Oct 07, 2022 3:52:58 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:52:59 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:53:00 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:53:02 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:53:07 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:53:13 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:53:22 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:53:28 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:53:43 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:54:10 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:54:40 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
org.apache.beam.examples.snippets.transforms.io.gcp.bigquery.BigQuerySamplesIT
> testTableIO STANDARD_ERROR
Oct 07, 2022 3:54:43 AM org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$4
getTempFilePrefix
INFO: Writing BigQuery temporary files to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea/
before loading them.
Oct 07, 2022 3:54:43 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea/c07e7fae-4500-421c-be80-c52128ccb0b1.
Oct 07, 2022 3:54:43 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:54:43 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:54:43 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:54:43 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0
pending. retrying.
Oct 07, 2022 3:54:44 AM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 1 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=beam_bigquery_samples_1665114882233_4,
projectId=apache-beam-testing, tableId=testTableIO}} using job
GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 07, 2022 3:54:45 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0
Oct 07, 2022 3:54:45 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0
started
Oct 07, 2022 3:54:45 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0
Oct 07, 2022 3:54:46 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0
Oct 07, 2022 3:54:47 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 07, 2022 3:54:47 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0
succeeded. Statistics:
{"completionRatio":1.0,"creationTime":"1665114884813","endTime":"1665114886405","load":{"badRecords":"0","inputFileBytes":"484","inputFiles":"1","outputBytes":"239","outputRows":"1","timeline":[{"elapsedMs":"927","totalSlotMs":"194","pendingUnits":"0","completedUnits":"1","activeUnits":"0"},{"elapsedMs":"1400","totalSlotMs":"194","pendingUnits":"0","completedUnits":"1","activeUnits":"0","estimatedRunnableUnits":"0"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"194"}],"reservation_id":"default-pipeline","startTime":"1665114884977","totalSlotMs":"194","finalExecutionDurationMs":"706"}
Oct 07, 2022 3:54:47 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins1007035443dc4580a5_6c9af882aa34423391fa4e264184bcea_6dd17a0d5216051181f08d9b0b2c7bc4_00001_00000-0
completed successfully.
Oct 07, 2022 3:54:49 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: Starting BigQuery extract job:
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10070354497172abb0_3401083ac7cd47b0affac6003d5de8ba
Oct 07, 2022 3:54:49 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10070354497172abb0_3401083ac7cd47b0affac6003d5de8ba,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10070354497172abb0_3401083ac7cd47b0affac6003d5de8ba
Oct 07, 2022 3:54:49 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10070354497172abb0_3401083ac7cd47b0affac6003d5de8ba,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10070354497172abb0_3401083ac7cd47b0affac6003d5de8ba
Oct 07, 2022 3:54:51 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10070354497172abb0_3401083ac7cd47b0affac6003d5de8ba,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 07, 2022 3:54:51 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: BigQuery extract job completed:
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10070354497172abb0_3401083ac7cd47b0affac6003d5de8ba
Oct 07, 2022 3:54:51 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase split
INFO: Extract job produced 1 files
Oct 07, 2022 3:54:51 AM org.apache.beam.sdk.io.FileBasedSource createReader
INFO: Matched 1 files for pattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/3401083ac7cd47b0affac6003d5de8ba/000000000000.avro
org.apache.beam.examples.snippets.transforms.io.gcp.bigquery.BigQuerySamplesIT
> testTableIO FAILED
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.VerifyException
at Verify.java:124
***********************************************************
***********************************************************
**********************Set Up Pubsub************************
The Pub/Sub topic has been set up for this example:
projects/apache-beam-testing/topics/testpipeline-jenkins-1007034515-7f530daf
The Pub/Sub subscription has been set up for this example:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-1007034515-7f530daf
*************************Tear Down*************************
The Pub/Sub topic has been deleted:
projects/apache-beam-testing/topics/testpipeline-jenkins-1007034515-7f530daf
The Pub/Sub subscription has been deleted:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-1007034515-7f530daf
***********************************************************
***********************************************************
org.apache.beam.examples.cookbook.CombinePerKeyExamplesIT >
testE2ECombinePerKey STANDARD_ERROR
Oct 07, 2022 3:55:10 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
Oct 07, 2022 3:55:41 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 07, 2022 3:55:41 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
succeeded. Statistics:
{"completionRatio":1.0,"creationTime":"1665114777224","endTime":"1665114936135","load":{"badRecords":"0","inputFileBytes":"586637","inputFiles":"8045","outputBytes":"401602","outputRows":"8045","timeline":[{"elapsedMs":"155341","totalSlotMs":"481530","pendingUnits":"3181","completedUnits":"4864","activeUnits":"1000","estimatedRunnableUnits":"0"},{"elapsedMs":"155848","totalSlotMs":"785306","pendingUnits":"501","completedUnits":"8045","activeUnits":"1000","estimatedRunnableUnits":"0"},{"elapsedMs":"156936","totalSlotMs":"944380","pendingUnits":"0","completedUnits":"8546","activeUnits":"5","estimatedRunnableUnits":"0"},{"elapsedMs":"157984","totalSlotMs":"944380","pendingUnits":"0","completedUnits":"8546","activeUnits":"0","estimatedRunnableUnits":"0"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"944380"}],"reservation_id":"default-pipeline","startTime":"1665114777976","totalSlotMs":"944380","finalExecutionDurationMs":"2958"}
Oct 07, 2022 3:55:41 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins10070347237a46ceb0_3ef532a3f9234fa6904a9ca9b0eec0ec_6022d60de7d40e142b0250f258b7db0e_00001_00000-0
completed successfully.
Oct 07, 2022 3:55:46 AM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteTable
INFO: Successfully deleted table: combine_per_key_examples_table
Oct 07, 2022 3:55:46 AM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteDataset
INFO: Successfully deleted dataset: combine_per_key_examples1665114440676
org.apache.beam.examples.cookbook.MaxPerKeyExamplesIT >
testE2EMaxPerKeyExamples STANDARD_ERROR
Oct 07, 2022 3:55:47 AM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
createNewDataset
INFO: Successfully created new dataset :
apache-beam-testing:max_per_key_examples1665114946951
Oct 07, 2022 3:55:48 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: Starting BigQuery extract job:
beam_bq_job_EXPORT_testpipelinejenkins1007035548f9f5022b_11b57ac46ac745bd8781934b2829a7e4
Oct 07, 2022 3:55:49 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_EXPORT_testpipelinejenkins1007035548f9f5022b_11b57ac46ac745bd8781934b2829a7e4,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_testpipelinejenkins1007035548f9f5022b_11b57ac46ac745bd8781934b2829a7e4
Oct 07, 2022 3:55:49 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_EXPORT_testpipelinejenkins1007035548f9f5022b_11b57ac46ac745bd8781934b2829a7e4,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_testpipelinejenkins1007035548f9f5022b_11b57ac46ac745bd8781934b2829a7e4
Oct 07, 2022 3:55:49 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_EXPORT_testpipelinejenkins1007035548f9f5022b_11b57ac46ac745bd8781934b2829a7e4,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 07, 2022 3:55:49 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: BigQuery extract job completed:
beam_bq_job_EXPORT_testpipelinejenkins1007035548f9f5022b_11b57ac46ac745bd8781934b2829a7e4
Oct 07, 2022 3:55:49 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase split
INFO: Extract job produced 1 files
Oct 07, 2022 3:55:49 AM org.apache.beam.sdk.io.FileBasedSource createReader
INFO: Matched 1 files for pattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/11b57ac46ac745bd8781934b2829a7e4/000000000000.avro
Oct 07, 2022 3:55:49 AM org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$4
getTempFilePrefix
INFO: Writing BigQuery temporary files to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/
before loading them.
Oct 07, 2022 3:55:49 AM org.apache.beam.sdk.io.FileBasedSource
getEstimatedSizeBytes
INFO: Filepattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/11b57ac46ac745bd8781934b2829a7e4/000000000000.avro
matched 1 files with total size 119755
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/4213ea23-7007-406e-85e4-0bbf8a9b0c8c.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/419a12c6-0d65-4f54-b395-5b571a8d1235.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/cecbd5ba-08e3-431f-9ef3-cb1458c041ac.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/5421e8e1-08ae-4d61-b76f-281d012da568.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/7ba0d8cc-c89f-47d5-9662-408b84633e16.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/25d560ef-eeb8-4c76-a2d8-e6fccb2aa589.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/887637df-f912-468f-b559-ddec9d295543.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/9359caa9-bf28-4133-9524-1c7445a5d661.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/1f85fef7-6a82-4c95-b92f-3608da29a42e.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/096b5a26-fd44-4f66-a5ec-63cf2f298b83.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/00af8101-3deb-4f47-bfd8-7ea0782d9e55.
Oct 07, 2022 3:55:50 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab/e66fa046-c0f3-4c2c-b2b8-609d356a362b.
Oct 07, 2022 3:55:51 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:55:51 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:55:51 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 07, 2022 3:55:51 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0
pending. retrying.
Oct 07, 2022 3:55:52 AM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 12 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=max_per_key_examples1665114946951,
projectId=apache-beam-testing, tableId=max_per_key_examples_table}} using job
GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 07, 2022 3:55:53 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0
Oct 07, 2022 3:55:53 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0
started
Oct 07, 2022 3:55:53 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0
Oct 07, 2022 3:55:54 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0
Oct 07, 2022 3:55:56 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 07, 2022 3:55:56 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0
succeeded. Statistics:
{"completionRatio":1.0,"creationTime":"1665114952820","endTime":"1665114956356","load":{"badRecords":"0","inputFileBytes":"522","inputFiles":"12","outputBytes":"192","outputRows":"12","timeline":[{"elapsedMs":"1027","totalSlotMs":"248","pendingUnits":"1","completedUnits":"0","activeUnits":"1","estimatedRunnableUnits":"0"},{"elapsedMs":"1835","totalSlotMs":"623","pendingUnits":"1","completedUnits":"0","activeUnits":"1","estimatedRunnableUnits":"0"},{"elapsedMs":"3345","totalSlotMs":"1418","pendingUnits":"0","completedUnits":"1","activeUnits":"0","estimatedRunnableUnits":"0"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"1418"}],"reservation_id":"default-pipeline","startTime":"1665114952981","totalSlotMs":"1418","finalExecutionDurationMs":"2523"}
Oct 07, 2022 3:55:56 AM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins1007035548f9f5022b_a881aa67dd4448148b8a9004224fd7ab_42a555e81160638b69ddbc50486acaed_00001_00000-0
completed successfully.
Oct 07, 2022 3:55:57 AM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteTable
INFO: Successfully deleted table: max_per_key_examples_table
Oct 07, 2022 3:55:58 AM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteDataset
INFO: Successfully deleted dataset: max_per_key_examples1665114946951
17 tests completed, 1 failed
> Task :runners:direct-java:examplesIntegrationTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:direct-java:examplesIntegrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/ws/src/runners/direct-java/build/reports/tests/examplesIntegrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 29s
89 actionable tasks: 54 executed, 33 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/ovg7deo33wdqk
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
