See
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/1130/display/redirect>
Changes:
------------------------------------------
[...truncated 272.09 KB...]
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`,
`HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND
`HACKER_NEWS`.`score` > 2
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DEFAULT
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..13=[{inputs}], expr#14=['story':VARCHAR],
expr#15=[=($t8, $t14)], expr#16=['job':VARCHAR], expr#17=[=($t8, $t16)],
expr#18=[OR($t15, $t17)], expr#19=[2], expr#20=[>($t5, $t19)],
expr#21=[AND($t18, $t20)], author=[$t4], type=[$t8], title=[$t0], score=[$t5],
$condition=[$t21])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT
> readUsingDefaultMethod FAILED
java.lang.IllegalStateException: Unable to return a default Coder for
BeamIOSourceRel_95/ParDo(RowMonitor)/ParMultiDo(RowMonitor).output
[PCollection]. Correct one of the following root causes:
No Coder has been manually specified; you may do so using .setCoder().
Inferring a Coder from the CoderRegistry failed: Cannot provide a coder
for a Beam Row. Please provide a schema instead using PCollection.setRowSchema.
Using the default output Coder from the producing PTransform failed:
PTransform.getOutputCoder called.
at
org.apache.beam.vendor.guava.v26_0_jre.com.google.common.base.Preconditions.checkState(Preconditions.java:507)
at org.apache.beam.sdk.values.PCollection.getCoder(PCollection.java:277)
at
org.apache.beam.sdk.values.PCollection.finishSpecifying(PCollection.java:114)
at
org.apache.beam.sdk.runners.TransformHierarchy.finishSpecifyingInput(TransformHierarchy.java:191)
at org.apache.beam.sdk.Pipeline.applyInternal(Pipeline.java:541)
at org.apache.beam.sdk.Pipeline.applyTransform(Pipeline.java:493)
at
org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:69)
at
org.apache.beam.sdk.extensions.sql.impl.rel.BeamSqlRelUtils.toPCollection(BeamSqlRelUtils.java:39)
at
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT.readUsingDefaultMethod(BigQueryIOPushDownIT.java:176)
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT
> readUsingDirectReadMethodPushDown STANDARD_ERROR
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQL:
SELECT `HACKER_NEWS`.`by` AS `author`, `HACKER_NEWS`.`type`,
`HACKER_NEWS`.`title`, `HACKER_NEWS`.`score`
FROM `beam`.`HACKER_NEWS` AS `HACKER_NEWS`
WHERE (`HACKER_NEWS`.`type` = 'story' OR `HACKER_NEWS`.`type` = 'job') AND
`HACKER_NEWS`.`score` > 2
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery method is set to: DIRECT_READ
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable <init>
INFO: BigQuery writeDisposition is set to: WRITE_EMPTY
Oct 18, 2020 12:45:11 PM
org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: SQLPlan>
LogicalProject(author=[$4], type=[$8], title=[$0], score=[$5])
LogicalFilter(condition=[AND(OR(=($8, 'story'), =($8, 'job')), >($5, 2))])
BeamIOSourceRel(table=[[beam, HACKER_NEWS]])
Oct 18, 2020 12:45:12 PM
org.apache.beam.sdk.extensions.sql.impl.CalciteQueryPlanner convertToBeamRel
INFO: BEAMPlan>
BeamCalcRel(expr#0..3=[{inputs}], proj#0..3=[{exprs}])
BeamPushDownIOSourceRel(table=[[beam, HACKER_NEWS]], usedFields=[[by,
type, title, score]], BigQueryFilter=[[supported{OR(=($8, 'story'), =($8,
'job'))>($5, 2)}, unsupported{}]])
Oct 18, 2020 12:45:12 PM
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryTable
buildIOReader
INFO: Pushing down the following filter: (`type` = 'story' OR `type` =
'job') AND `score` > 2
Oct 18, 2020 12:45:13 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Executing pipeline on the Dataflow Service, which will have billing
implications related to Google Compute Engine usage and other Google Cloud
Services.
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Uploading 219 files from PipelineOptions.filesToStage to staging
location to prepare for execution.
Oct 18, 2020 12:45:15 PM
org.apache.beam.runners.dataflow.util.PackageUtil$PackageAttributes
forFileToStage
INFO: Staging custom dataflow-worker.jar as
beam-runners-google-cloud-dataflow-java-legacy-worker-2.26.0-SNAPSHOT-UIbwpUH1rKV9_81rKsEYEtBO9uAa9JwWq7wuwDAG5FI.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/expansion-service/build/libs/beam-sdks-java-expansion-service-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-expansion-service-2.26.0-SNAPSHOT-4SoKIpOsIvM-bgH4aMcDDxRGjgYzF3wCKVuq97cpgVU.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/worker/legacy-worker/build/libs/beam-runners-google-cloud-dataflow-java-legacy-worker-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-legacy-worker-2.26.0-SNAPSHOT-UIbwpUH1rKV9_81rKsEYEtBO9uAa9JwWq7wuwDAG5FI.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.26.0-SNAPSHOT-F6RSYO5Cd_QYlIImd70KpzMDyilZdi7-SmpUfldwsS8.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.26.0-SNAPSHOT-tests.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.26.0-SNAPSHOT-tests-L387xwFsN5IaIl4THfp3YeA0-EA4vTopNa_MseN50ZM.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.26.0-SNAPSHOT-TzEFH7pHNhdIWadARIklA2T89HnzePoTBA_O37WJt7k.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/java-fn-execution/build/libs/beam-runners-java-fn-execution-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-java-fn-execution-2.26.0-SNAPSHOT-FIc8GoBG1BPs1VYEpESdv0He30FM6X7NldqLSY6AOLo.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/protobuf/build/libs/beam-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT-kNZEenaKLrDt81J6_yhpRnW9KkitBPAeqYX2qB_rKtU.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.26.0-SNAPSHOT-IGmojWrBNPlFMy5aS_ohyLi-wZfmegcN8af7AW1BZ8E.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.26.0-SNAPSHOT-tests.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.26.0-SNAPSHOT-tests-q1BOo0jLJzXT0X-jFsnR9TFgZKmpyGmwuq5lGDvHv-g.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/direct-java/build/libs/beam-runners-direct-java-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-direct-java-2.26.0-SNAPSHOT-h0slC5bRM60mGu8zUuRvxW2BUW98APSPCb_5pGAwNg8.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/google-cloud-platform/build/libs/beam-sdks-java-io-google-cloud-platform-2.26.0-SNAPSHOT-tests.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-google-cloud-platform-2.26.0-SNAPSHOT-tests-wzqifWxmSwVeLjQ_76OlNcfzbjjEfOICFpo5MrrjbpU.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.26.0-SNAPSHOT-tests.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.26.0-SNAPSHOT-tests-sbUvpt4M4spNa8tmqhO4KUqkUqqlLliL7nN5x9-v96s.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/testing/test-utils/build/libs/beam-sdks-java-test-utils-2.26.0-SNAPSHOT-tests.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-test-utils-2.26.0-SNAPSHOT-tests-3qU2axmKqq9YgjeTj-hAfMQaohQ1HnKLU8qhi1tg8Zo.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/mongodb/build/libs/beam-sdks-java-io-mongodb-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-mongodb-2.26.0-SNAPSHOT-IEBGm3Y2GS7gq04jGNscsAz2_0dfkLjILp3mJhV0nTM.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/core-construction-java/build/libs/beam-runners-core-construction-java-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-core-construction-java-2.26.0-SNAPSHOT-CS4ntFUZ_r5dhDFd5Jm1akgRJlQOISbjbcvWI_0nh68.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.26.0-SNAPSHOT-unshaded.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.26.0-SNAPSHOT-unshaded-Y1emYkIr7ym53C-okSxMIfmLfWhDK2XcHiMpWu6DQCU.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.26.0-SNAPSHOT-rshDSjjth-BCXETHeeWx86iOk_K0FZaUCQ8w6IlDnek.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/build/libs/beam-sdks-java-extensions-sql-2.26.0-SNAPSHOT-tests.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-sql-2.26.0-SNAPSHOT-tests-EXUo01kUM0pLMo0wNhlc0_hnEtcIy2v4IvUB2pY25ds.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/fn-execution/build/libs/beam-sdks-java-fn-execution-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-fn-execution-2.26.0-SNAPSHOT-s6zKTZgz1gCoKi3o0JXUzinZn1O5Zb3fMG55CXRkR3I.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/core/build/libs/beam-sdks-java-core-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-core-2.26.0-SNAPSHOT-UGwsE0D-S6ZYkD1LrolcNQaM_U9AoPal3qAN1wEQCrM.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.26.0-SNAPSHOT-r3WWZ4HMmmDYjw3dqcwuy64q6XFYopZNVTD70kBrCs0.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading /tmp/test2245824606906015532.zip to
gs://temp-storage-for-perf-tests/loadtests/staging/test-qsg_Z090W8hT6yE8UgFY7J1VBIKrt5M2nO2ysxKWBcw.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/io/common/build/libs/beam-sdks-java-io-common-2.26.0-SNAPSHOT-tests.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-io-common-2.26.0-SNAPSHOT-tests-_INvjBBbc0B-WbyFiI4b8taCSv6y0F5NWc4Cl4922yE.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/runners/google-cloud-dataflow-java/build/libs/beam-runners-google-cloud-dataflow-java-2.26.0-SNAPSHOT-tests.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-runners-google-cloud-dataflow-java-2.26.0-SNAPSHOT-tests-w9taT2ksM9Q-RqZnPqocyAtJtwlWkRdQnvlVnzWGD2A.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/google-cloud-platform-core/build/libs/beam-sdks-java-extensions-google-cloud-platform-core-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-google-cloud-platform-core-2.26.0-SNAPSHOT-k82GT2RuYqmcMvPnBF2SReMgqSc7PkjBzf-q1gFSiE0.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/join-library/build/libs/beam-sdks-java-extensions-join-library-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-sdks-java-extensions-join-library-2.26.0-SNAPSHOT-HtGRCwaWbwlXS3RFTjJ2d9abunbafXIIqLvmAWRjGbk.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/vendor/sdks-java-extensions-protobuf/build/libs/beam-vendor-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-vendor-sdks-java-extensions-protobuf-2.26.0-SNAPSHOT-pKv-aAGLTFaMOopcaOfYvSDwFWt2xNp45ZnySc4wm6g.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/pipeline/build/libs/beam-model-pipeline-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-pipeline-2.26.0-SNAPSHOT-5hu5BdJ8hsHpPSDLwhYUlNRWlO7wZYNJuL3NCYPdkyE.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/job-management/build/libs/beam-model-job-management-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-job-management-2.26.0-SNAPSHOT-D-YP45Vv19xI3GM1V0ra1teKIQHjp2KNt3tQbGwE2bY.jar
Oct 18, 2020 12:45:15 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/model/fn-execution/build/libs/beam-model-fn-execution-2.26.0-SNAPSHOT.jar>
to
gs://temp-storage-for-perf-tests/loadtests/staging/beam-model-fn-execution-2.26.0-SNAPSHOT-LG40e98xeRXLCrMb64J4zv4HG2QuqIsvSH1wlH3IBJo.jar
Oct 18, 2020 12:45:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
stageClasspathElements
INFO: Staging files complete: 189 files cached, 30 files newly uploaded in
0 seconds
Oct 18, 2020 12:45:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/Read Input BQ Rows with
push-down/Read(BigQueryStorageTableSource) as step s1
Oct 18, 2020 12:45:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamPushDownIOSourceRel_229/ParDo(RowMonitor) as step s2
Oct 18, 2020 12:45:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding BeamCalcRel_285/ParDo(Calc) as step s3
Oct 18, 2020 12:45:16 PM
org.apache.beam.runners.dataflow.DataflowPipelineTranslator$Translator addStep
INFO: Adding ParDo(TimeMonitor) as step s4
Oct 18, 2020 12:45:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Staging pipeline description to
gs://temp-storage-for-perf-tests/loadtests/staging/
Oct 18, 2020 12:45:16 PM org.apache.beam.runners.dataflow.util.PackageUtil
tryStagePackage
INFO: Uploading <96019 bytes, hash
0bfb8a55169e9600da419140b50ac15b41217b88c3268f6a67733362108e92cc> to
gs://temp-storage-for-perf-tests/loadtests/staging/pipeline-C_uKVRaelgDaQZFAtQrBW0Ehe4jDJo9qZ3MzYhCOksw.pb
Oct 18, 2020 12:45:16 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Dataflow SDK version: 2.26.0-SNAPSHOT
Oct 18, 2020 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To access the Dataflow monitoring console, please navigate to
https://console.cloud.google.com/dataflow/jobs/us-central1/2020-10-18_05_45_16-4291198803537892704?project=apache-beam-testing
Oct 18, 2020 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: Submitted job: 2020-10-18_05_45_16-4291198803537892704
Oct 18, 2020 12:45:18 PM org.apache.beam.runners.dataflow.DataflowRunner run
INFO: To cancel the job using the 'gcloud' tool, run:
> gcloud dataflow jobs --project=apache-beam-testing cancel
--region=us-central1 2020-10-18_05_45_16-4291198803537892704
Oct 18, 2020 12:45:18 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
WARNING: 2020-10-18T12:45:16.928Z: The requested max number of workers (5)
is ignored as autoscaling is explicitly disabled (autoscalingAlgorithm=NONE).
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:25.705Z: Worker configuration: n1-standard-1 in
us-central1-f.
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:26.515Z: Expanding CoGroupByKey operations into
optimizable parts.
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:26.555Z: Expanding GroupByKey operations into
optimizable parts.
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:26.595Z: Lifting ValueCombiningMappingFns into
MergeBucketsMappingFns
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:26.669Z: Fusing adjacent ParDo, Read, Write, and
Flatten operations
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:26.697Z: Fusing consumer
BeamPushDownIOSourceRel_229/ParDo(RowMonitor) into
BeamPushDownIOSourceRel_229/Read Input BQ Rows with
push-down/Read(BigQueryStorageTableSource)
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:26.722Z: Fusing consumer BeamCalcRel_285/ParDo(Calc)
into BeamPushDownIOSourceRel_229/ParDo(RowMonitor)
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:26.750Z: Fusing consumer ParDo(TimeMonitor) into
BeamCalcRel_285/ParDo(Calc)
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:27.117Z: Executing operation
BeamPushDownIOSourceRel_229/Read Input BQ Rows with
push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Oct 18, 2020 12:45:28 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:27.199Z: Starting 5 workers in us-central1-f...
Oct 18, 2020 12:45:44 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:41.538Z: Your project already contains 100
Dataflow-created metric descriptors, so new user metrics of the form
custom.googleapis.com/* will not be created. However, all user metrics are also
available in the metric dataflow.googleapis.com/job/user_counter. If you rely
on the custom metrics, you can delete old / unused metric descriptors. See
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.list
and
https://developers.google.com/apis-explorer/#p/monitoring/v3/monitoring.projects.metricDescriptors.delete
Oct 18, 2020 12:45:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:50.297Z: Autoscaling: Raised the number of workers
to 3 based on the rate of progress in the currently running stage(s).
Oct 18, 2020 12:45:51 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:50.316Z: Resized worker pool to 3, though goal was
5. This could be a quota issue.
Oct 18, 2020 12:45:58 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:45:55.615Z: Autoscaling: Raised the number of workers
to 5 based on the rate of progress in the currently running stage(s).
Oct 18, 2020 12:46:12 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:46:12.808Z: Workers have started successfully.
Oct 18, 2020 12:46:12 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:46:12.836Z: Workers have started successfully.
Oct 18, 2020 12:46:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:46:42.956Z: Finished operation
BeamPushDownIOSourceRel_229/Read Input BQ Rows with
push-down/Read(BigQueryStorageTableSource)+BeamPushDownIOSourceRel_229/ParDo(RowMonitor)+BeamCalcRel_285/ParDo(Calc)+ParDo(TimeMonitor)
Oct 18, 2020 12:46:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:46:43.197Z: Cleaning up.
Oct 18, 2020 12:46:45 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:46:43.269Z: Stopping worker pool...
Oct 18, 2020 12:47:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:47:41.579Z: Autoscaling: Resized worker pool from 5 to
0.
Oct 18, 2020 12:47:42 PM
org.apache.beam.runners.dataflow.util.MonitoringUtil$LoggingHandler process
INFO: 2020-10-18T12:47:41.614Z: Worker pool stopped.
Oct 18, 2020 12:47:50 PM
org.apache.beam.runners.dataflow.DataflowPipelineJob logTerminalState
INFO: Job 2020-10-18_05_45_16-4291198803537892704 finished with status DONE.
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT
> readUsingDirectReadMethodPushDown STANDARD_OUT
Load test results for test (ID): ef08c5fe-eaa0-42f1-9e18-03a436f84bb5 and
timestamp: 2020-10-18T12:47:50.191000000Z:
Metric: Value:
read_time 12.34
fields_read 4375276.0
org.apache.beam.sdk.extensions.sql.meta.provider.bigquery.BigQueryIOPushDownIT
> readUsingDirectReadMethodPushDown STANDARD_ERROR
Oct 18, 2020 12:47:50 PM
org.apache.beam.sdk.testutils.publishing.InfluxDBPublisher publishWithCheck
WARNING: Missing property -- measurement/database. Metrics won't be
published.
Gradle Test Executor 1 finished executing tests.
> Task :sdks:java:extensions:sql:perf-tests:integrationTest FAILED
3 tests completed, 2 failed
Finished generating test XML results (0.02 secs) into:
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/test-results/integrationTest>
Generating HTML test report...
Finished generating test html results (0.027 secs) into:
<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest>
:sdks:java:extensions:sql:perf-tests:integrationTest (Thread[Execution worker
for ':' Thread 10,5,main]) completed. Took 2 mins 46.621 secs.
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':sdks:java:extensions:sql:perf-tests:integrationTest'.
> There were failing tests. See the report at:
> file://<https://ci-beam.apache.org/job/beam_SQLBigQueryIO_Batch_Performance_Test_Java/ws/src/sdks/java/extensions/sql/perf-tests/build/reports/tests/integrationTest/index.html>
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to
get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.6.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 34s
107 actionable tasks: 67 executed, 40 from cache
Publishing build scan...
https://gradle.com/s/pnu3uspqf7mbs
Build cache (/home/jenkins/.gradle/caches/build-cache-1) removing files not
accessed on or after Sun Oct 11 12:44:20 UTC 2020.
Build cache (/home/jenkins/.gradle/caches/build-cache-1) cleaned up in 0.149
secs.
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
No JDK named ‘JDK 1.8 (latest)’ found
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
