See
<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/788/display/redirect?page=changes>
Changes:
[noreply] [Website] update links to https (#23523)
[noreply] Support custom avro DatumReader when reading from BigQuery (#22718)
[noreply] Rename 'clean' Gradle task that required Flutter and has been breaking
[noreply] Model handler unit test (#23506)
[noreply] Content/multi model pipelines (#23498)
[noreply] [Tour of Beam][Frontend] Content Tree and SDK models (#23316) (#23417)
------------------------------------------
[...truncated 7.76 MB...]
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/e203b7ad-4f33-4754-b14f-66cb5c2e1479.
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/718f6243-789b-4f19-97c2-aa47197b53dc.
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/2ca218d2-9df8-4043-b7b5-0ddb280b1932.
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/20f4d10c-4d6f-4147-9f18-37bf46cdab58.
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/1d3c8ded-16cd-4787-acd4-9a6aded28155.
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/790b62d0-5e06-4dd0-b87d-cc2867a9030d.
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/dcef90a0-df9c-4dca-bfb9-7b4e71f939e3.
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/4a25ba98-5c77-477a-af11-776ebe77330b.
Oct 06, 2022 9:51:35 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45/d66773ec-4d04-4217-8ef2-eb34742ec778.
Oct 06, 2022 9:52:52 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:52:52 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:52:53 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:52:53 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
pending. retrying.
Oct 06, 2022 9:52:54 PM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 8045 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=combine_per_key_examples1665092822309,
projectId=apache-beam-testing, tableId=combine_per_key_examples_table}} using
job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 06, 2022 9:52:55 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:52:55 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
started
Oct 06, 2022 9:52:55 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:52:56 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:52:58 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:52:59 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:53:04 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:53:08 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:53:20 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:53:31 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:53:47 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:54:11 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:54:44 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
Oct 06, 2022 9:55:52 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 06, 2022 9:55:52 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
succeeded. Statistics:
{"completionRatio":1.0,"creationTime":"1665093174358","endTime":"1665093331170","load":{"badRecords":"0","inputFileBytes":"586637","inputFiles":"8045","outputBytes":"401602","outputRows":"8045","timeline":[{"elapsedMs":"152371","totalSlotMs":"271771","pendingUnits":"4080","completedUnits":"3965","activeUnits":"1000","estimatedRunnableUnits":"0"},{"elapsedMs":"152875","totalSlotMs":"590847","pendingUnits":"500","completedUnits":"8045","activeUnits":"1000","estimatedRunnableUnits":"0"},{"elapsedMs":"154372","totalSlotMs":"667854","pendingUnits":"1","completedUnits":"8545","activeUnits":"1","estimatedRunnableUnits":"1000"},{"elapsedMs":"155918","totalSlotMs":"668008","pendingUnits":"0","completedUnits":"8546","activeUnits":"0","estimatedRunnableUnits":"1000"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"668008"}],"reservation_id":"default-pipeline","startTime":"1665093175069","totalSlotMs":"668008","finalExecutionDurationMs":"3937"}
Oct 06, 2022 9:55:52 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins1006214706869060d4_4acf4ec56f7d49b4a866829db622fe45_21af183cc15e21f90d6fcc5974f673d7_00001_00000-0
completed successfully.
Oct 06, 2022 9:55:58 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteTable
INFO: Successfully deleted table: combine_per_key_examples_table
Oct 06, 2022 9:55:58 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteDataset
INFO: Successfully deleted dataset: combine_per_key_examples1665092822309
org.apache.beam.examples.cookbook.MaxPerKeyExamplesIT >
testE2EMaxPerKeyExamples STANDARD_ERROR
Oct 06, 2022 9:55:58 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
createNewDataset
INFO: Successfully created new dataset :
apache-beam-testing:max_per_key_examples1665093358353
Oct 06, 2022 9:56:00 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: Starting BigQuery extract job:
beam_bq_job_EXPORT_testpipelinejenkins1006215559a6132bfc_c9051568cb4d4386a097dd365a6c7252
Oct 06, 2022 9:56:00 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_EXPORT_testpipelinejenkins1006215559a6132bfc_c9051568cb4d4386a097dd365a6c7252,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_testpipelinejenkins1006215559a6132bfc_c9051568cb4d4386a097dd365a6c7252
Oct 06, 2022 9:56:00 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_EXPORT_testpipelinejenkins1006215559a6132bfc_c9051568cb4d4386a097dd365a6c7252,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_testpipelinejenkins1006215559a6132bfc_c9051568cb4d4386a097dd365a6c7252
Oct 06, 2022 9:56:01 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_EXPORT_testpipelinejenkins1006215559a6132bfc_c9051568cb4d4386a097dd365a6c7252,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 06, 2022 9:56:01 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: BigQuery extract job completed:
beam_bq_job_EXPORT_testpipelinejenkins1006215559a6132bfc_c9051568cb4d4386a097dd365a6c7252
Oct 06, 2022 9:56:01 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase split
INFO: Extract job produced 1 files
Oct 06, 2022 9:56:01 PM org.apache.beam.sdk.io.FileBasedSource createReader
INFO: Matched 1 files for pattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/c9051568cb4d4386a097dd365a6c7252/000000000000.avro
Oct 06, 2022 9:56:01 PM org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$4
getTempFilePrefix
INFO: Writing BigQuery temporary files to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/
before loading them.
Oct 06, 2022 9:56:01 PM org.apache.beam.sdk.io.FileBasedSource
getEstimatedSizeBytes
INFO: Filepattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/c9051568cb4d4386a097dd365a6c7252/000000000000.avro
matched 1 files with total size 119755
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/8ca686b1-0afa-4982-a87b-703542bec93a.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/a2b86fa5-00ed-4476-8c7a-525ed97a1796.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/2a6a396c-2bda-49b8-9bec-35a08c2f2d63.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/6d1eaf8f-5f5e-46ba-be06-7854143ee958.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/e414cf9a-929b-4a76-b1ae-27ff3eeb650c.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/87d1268e-037f-4c8c-adf6-13c96b3365e1.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/90f4b669-d02a-4528-abd8-a5bf848743f0.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/b049119b-c013-4c8a-b57d-92e8476edc69.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/4b463735-ded6-4e5b-8950-095baadea634.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/187650fb-bd83-4787-b68d-959c5d6f4680.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/faf06e85-f976-4dcf-b2fc-bfcf29d84b3b.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739/d66fbb6e-b22a-49f6-a1e2-676a297b1d2d.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:56:02 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:56:03 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:56:03 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0
pending. retrying.
Oct 06, 2022 9:56:03 PM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 12 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=max_per_key_examples1665093358353,
projectId=apache-beam-testing, tableId=max_per_key_examples_table}} using job
GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 06, 2022 9:56:04 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0
Oct 06, 2022 9:56:04 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0
started
Oct 06, 2022 9:56:04 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0
Oct 06, 2022 9:56:05 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0
Oct 06, 2022 9:56:07 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 06, 2022 9:56:07 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0
succeeded. Statistics:
{"completionRatio":1.0,"creationTime":"1665093364324","endTime":"1665093366686","load":{"badRecords":"0","inputFileBytes":"522","inputFiles":"12","outputBytes":"192","outputRows":"12","timeline":[{"elapsedMs":"1122","totalSlotMs":"0","pendingUnits":"1","completedUnits":"0","activeUnits":"1","estimatedRunnableUnits":"0"},{"elapsedMs":"2209","totalSlotMs":"1040","pendingUnits":"0","completedUnits":"1","activeUnits":"1","estimatedRunnableUnits":"0"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"1040"}],"reservation_id":"default-pipeline","startTime":"1665093364452","totalSlotMs":"1040","finalExecutionDurationMs":"1344"}
Oct 06, 2022 9:56:07 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_testpipelinejenkins1006215559a6132bfc_933f3a9094f747ce93a15bd20e182739_29d741729a03df25992519888f5a6562_00001_00000-0
completed successfully.
Oct 06, 2022 9:56:08 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteTable
INFO: Successfully deleted table: max_per_key_examples_table
Oct 06, 2022 9:56:09 PM org.apache.beam.sdk.io.gcp.testing.BigqueryClient
deleteDataset
INFO: Successfully deleted dataset: max_per_key_examples1665093358353
org.apache.beam.examples.snippets.transforms.io.gcp.bigquery.BigQuerySamplesIT
> testTableIO STANDARD_ERROR
Oct 06, 2022 9:56:15 PM org.apache.beam.sdk.io.gcp.bigquery.BatchLoads$4
getTempFilePrefix
INFO: Writing BigQuery temporary files to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455/
before loading them.
Oct 06, 2022 9:56:15 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryRowWriter <init>
INFO: Opening TableRowWriter to
gs://temp-storage-for-end-to-end-tests/BigQueryWriteTemp/beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455/73c07dbe-2e74-414b-8e9e-78d55d2fa30a.
Oct 06, 2022 9:56:15 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:56:15 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:56:15 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Waiting for jobs to complete.
Oct 06, 2022 9:56:15 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0
pending. retrying.
Oct 06, 2022 9:56:16 PM org.apache.beam.sdk.io.gcp.bigquery.WriteTables
lambda$startLoad$79287d2b$1
INFO: Loading 1 files into GenericData{classInfo=[datasetId, projectId,
tableId], {datasetId=beam_bigquery_samples_1665093374077_9,
projectId=apache-beam-testing, tableId=testTableIO}} using job
GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0,
location=US, projectId=apache-beam-testing}}, job id iteration 0
Oct 06, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0
Oct 06, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob runJob
INFO: job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0
started
Oct 06, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0
Oct 06, 2022 9:56:17 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0
Oct 06, 2022 9:56:19 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 06, 2022 9:56:19 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJob pollJob
INFO: Load job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0
succeeded. Statistics:
{"completionRatio":1.0,"creationTime":"1665093377025","endTime":"1665093379209","load":{"badRecords":"0","inputFileBytes":"484","inputFiles":"1","outputBytes":"239","outputRows":"1","timeline":[{"elapsedMs":"916","totalSlotMs":"299","pendingUnits":"1","completedUnits":"0","activeUnits":"1","estimatedRunnableUnits":"0"},{"elapsedMs":"1993","totalSlotMs":"820","pendingUnits":"0","completedUnits":"1","activeUnits":"1","estimatedRunnableUnits":"0"}]},"reservationUsage":[{"name":"default-pipeline","slotMs":"820"}],"reservation_id":"default-pipeline","startTime":"1665093377188","totalSlotMs":"820","finalExecutionDurationMs":"1277"}
Oct 06, 2022 9:56:19 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryHelpers$PendingJobManager
waitForDone
INFO: Job
beam_bq_job_LOAD_bigquerysamplesit0testtableiojenkins100621561477fe783a_9fb3c60f452a4a67b99e04b647324455_5bf3c13342d4acd9a96f474d0b69abde_00001_00000-0
completed successfully.
Oct 06, 2022 9:56:21 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: Starting BigQuery extract job:
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10062156217d85e081_13174085179e46d083372c5f50af5687
Oct 06, 2022 9:56:21 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl startJob
INFO: Started BigQuery job: GenericData{classInfo=[jobId, location,
projectId],
{jobId=beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10062156217d85e081_13174085179e46d083372c5f50af5687,
location=US, projectId=apache-beam-testing}}.
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10062156217d85e081_13174085179e46d083372c5f50af5687
Oct 06, 2022 9:56:21 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: Still waiting for BigQuery job
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10062156217d85e081_13174085179e46d083372c5f50af5687,
currently in status {"state":"RUNNING"}
bq show -j --format=prettyjson --project_id=apache-beam-testing
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10062156217d85e081_13174085179e46d083372c5f50af5687
Oct 06, 2022 9:56:22 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQueryServicesImpl$JobServiceImpl pollJob
INFO: BigQuery job GenericData{classInfo=[jobId, location, projectId],
{jobId=beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10062156217d85e081_13174085179e46d083372c5f50af5687,
location=US, projectId=apache-beam-testing}} completed in state DONE
Oct 06, 2022 9:56:22 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase executeExtract
INFO: BigQuery extract job completed:
beam_bq_job_EXPORT_bigquerysamplesit0testtableiojenkins10062156217d85e081_13174085179e46d083372c5f50af5687
Oct 06, 2022 9:56:22 PM
org.apache.beam.sdk.io.gcp.bigquery.BigQuerySourceBase split
INFO: Extract job produced 1 files
Oct 06, 2022 9:56:22 PM org.apache.beam.sdk.io.FileBasedSource createReader
INFO: Matched 1 files for pattern
gs://temp-storage-for-end-to-end-tests/BigQueryExtractTemp/13174085179e46d083372c5f50af5687/000000000000.avro
org.apache.beam.examples.snippets.transforms.io.gcp.bigquery.BigQuerySamplesIT
> testTableIO FAILED
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.VerifyException
at Verify.java:124
***********************************************************
***********************************************************
**********************Set Up Pubsub************************
The Pub/Sub topic has been set up for this example:
projects/apache-beam-testing/topics/testpipeline-jenkins-1006214514-514e69af
The Pub/Sub subscription has been set up for this example:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-1006214514-514e69af
*************************Tear Down*************************
The Pub/Sub topic has been deleted:
projects/apache-beam-testing/topics/testpipeline-jenkins-1006214514-514e69af
The Pub/Sub subscription has been deleted:
projects/apache-beam-testing/subscriptions/testpipeline-jenkins-1006214514-514e69af
***********************************************************
***********************************************************
17 tests completed, 1 failed
> Task :runners:direct-java:examplesIntegrationTest FAILED
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':runners:direct-java:examplesIntegrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_PostCommit_Java_Examples_Direct/ws/src/runners/direct-java/build/reports/tests/examplesIntegrationTest/index.html>
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 11m 56s
89 actionable tasks: 54 executed, 33 from cache, 2 up-to-date
Publishing build scan...
https://gradle.com/s/jhxvabvlcoqk4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
