See
<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/5387/display/redirect?page=changes>
Changes:
[Luke Cwik] [BEAM-13015] Migrate bundle processing in the SDK harness to using
[Brian Hulette] [BEAM-13099] Copy vendored calcite build for 1.28.0
[Brian Hulette] [BEAM-13099] Modifications for vendor/calcite-1_28_0 build
[noreply] [BEAM-13001] DoFn metrics for Go SDK (#15657)
------------------------------------------
[...truncated 82.03 KB...]
Step 16/18 : RUN if [ "${pull_licenses}" = "false" ] ; then rm -rf
/opt/apache/beam/third_party_licenses ; fi
---> Running in beb91fb5980b
Removing intermediate container beb91fb5980b
---> a02ed457cded
Step 17/18 : COPY target/profiler/* /opt/google_cloud_profiler/
---> 6021995ac23a
Step 18/18 : ENTRYPOINT ["/opt/apache/beam/boot"]
---> Running in db39ef13cb19
Removing intermediate container db39ef13cb19
---> bbb460f6fcc9
Successfully built bbb460f6fcc9
Successfully tagged apache/beam_java8_sdk:2.35.0.dev
> Task :sdks:java:container:java11:docker
---> Running in 2e3f02964ff0
Removing intermediate container 2e3f02964ff0
---> bd0e22ed2b47
Step 4/18 : ARG pull_licenses
---> Running in ad08bc70262d
Removing intermediate container ad08bc70262d
---> 9787a3a90834
Step 5/18 : ADD target/slf4j-api.jar /opt/apache/beam/jars/
---> d5b1bf4055f1
Step 6/18 : ADD target/slf4j-jdk14.jar /opt/apache/beam/jars/
---> d9184acd465f
Step 7/18 : ADD target/beam-sdks-java-harness.jar /opt/apache/beam/jars/
---> dcc3a777a3db
Step 8/18 : ADD target/beam-sdks-java-io-kafka.jar /opt/apache/beam/jars/
---> fe9e1b355181
Step 9/18 : ADD target/kafka-clients.jar /opt/apache/beam/jars/
---> 6a61332ba99e
Step 10/18 : ADD target/linux_amd64/boot /opt/apache/beam/
---> 565a4a8264fe
Step 11/18 : COPY target/LICENSE /opt/apache/beam/
---> 83e57f45945c
Step 12/18 : COPY target/NOTICE /opt/apache/beam/
---> 75972bc75d65
Step 13/18 : ADD target/third_party_licenses
/opt/apache/beam/third_party_licenses/
---> e9eb8877720d
Step 14/18 : COPY target/LICENSE target/go-licenses/*
/opt/apache/beam/third_party_licenses/golang/
---> 5759dcb9ec6b
Step 15/18 : RUN rm /opt/apache/beam/third_party_licenses/golang/LICENSE
---> Running in 3ef79fbdcc19
Removing intermediate container 3ef79fbdcc19
---> 1f3b2c36fa6c
Step 16/18 : RUN if [ "${pull_licenses}" = "false" ] ; then rm -rf
/opt/apache/beam/third_party_licenses ; fi
---> Running in 6d2f359a88fa
Removing intermediate container 6d2f359a88fa
---> 3fa594b7a85e
Step 17/18 : COPY target/profiler/* /opt/google_cloud_profiler/
---> 698ba536b069
Step 18/18 : ENTRYPOINT ["/opt/apache/beam/boot"]
---> Running in 4ef6856f25b2
Removing intermediate container 4ef6856f25b2
---> a6b837f941e0
Successfully built a6b837f941e0
Successfully tagged apache/beam_java11_sdk:2.35.0.dev
> Task :runners:flink:1.13:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :runners:flink:1.13:classes
> Task :runners:flink:1.13:jar
> Task :runners:flink:1.13:job-server:compileJava NO-SOURCE
> Task :runners:flink:1.13:job-server:classes UP-TO-DATE
> Task :sdks:java:io:google-cloud-platform:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :sdks:java:io:google-cloud-platform:classes
> Task :sdks:java:io:google-cloud-platform:jar
> Task :sdks:java:extensions:sql:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :sdks:java:extensions:sql:classes
> Task :sdks:java:extensions:sql:jar
> Task :sdks:java:extensions:sql:zetasql:compileJava
Note: Some input files use or override a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
Note: Some input files use unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
> Task :sdks:java:extensions:sql:zetasql:classes
> Task :sdks:java:extensions:sql:zetasql:jar
> Task :sdks:java:extensions:sql:expansion-service:compileJava
Note:
<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/ws/src/sdks/java/extensions/sql/expansion-service/src/main/java/org/apache/beam/sdk/extensions/sql/expansion/ExternalSqlTransformRegistrar.java>
uses or overrides a deprecated API.
Note: Recompile with -Xlint:deprecation for details.
> Task :sdks:java:extensions:sql:expansion-service:classes
> Task :sdks:java:extensions:sql:expansion-service:jar
> Task :sdks:java:extensions:sql:expansion-service:shadowJar
> Task :runners:flink:1.13:job-server:shadowJar
> Task :sdks:go:test:flinkValidatesRunner
$ TESTS=./test/integration/... ./test/regression
$ RUNNER=portable
$ TIMEOUT=2h
$ SIMULTANEOUS=3
$ GCS_LOCATION=gs://temp-storage-for-end-to-end-tests
$ PROJECT=apache-beam-testing
$ DATAFLOW_PROJECT=apache-beam-testing
$ REGION=us-central1
$ trap exit_background_processes SIGINT SIGTERM EXIT
$ key=--runner
$ case --runner in
$ RUNNER=flink
$ key=--flink_job_server_jar
$ case --flink_job_server_jar in
$
FLINK_JOB_SERVER_JAR=<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/ws/src/runners/flink/1.13/job-server/build/libs/beam-runners-flink-1.13-job-server-2.35.0-SNAPSHOT.jar>
$ key=--test_expansion_jar
$ case --test_expansion_jar in
$
TEST_EXPANSION_JAR=<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.35.0-SNAPSHOT.jar>
$ cd <https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/ws/src>
$ test -d sdks/go/test
$ JOB_PORT=41211
$ ENDPOINT=localhost:42455
No endpoint specified; starting a new flink job server on localhost:42455
$ java -jar
<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/ws/src/runners/flink/1.13/job-server/build/libs/beam-runners-flink-1.13-job-server-2.35.0-SNAPSHOT.jar>
--flink-master [local] --job-port 42455 --expansion-port 0 --artifact-port 0
$ JOBSERVER_PID=8993
$ EXPANSION_PORT=34897
$ TEST_EXPANSION_ADDR=localhost:42653
No test expansion address specified; starting a new test expansion server on
localhost:42653
$ java -jar
<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/ws/src/sdks/java/testing/expansion-service/build/libs/beam-sdks-java-testing-expansion-service-testExpansionService-2.35.0-SNAPSHOT.jar>
42653
$ TEST_EXPANSION_PID=9050
$ SIMULTANEOUS=1
$ TAG=dev
$ ./gradlew :sdks:go:container:docker -Pdocker-tag=dev
Starting expansion service at localhost:42653
Starting a Gradle Daemon, 4 busy Daemons could not be reused, use --status for
details
Oct 30, 2021 12:28:47 AM org.apache.beam.sdk.expansion.service.ExpansionService
loadRegisteredTransforms
INFO: Registering external transforms: [beam:transforms:xlang:test:cgbk,
beam:transforms:xlang:test:flatten, beam:transforms:xlang:test:prefix,
beam:transforms:xlang:test:multi, beam:transforms:xlang:test:gbk,
beam:transforms:xlang:test:comgl, beam:transforms:xlang:test:compk,
beam:transforms:xlang:count, beam:transforms:xlang:filter_less_than_eq,
beam:transforms:xlang:test:partition, beam:transforms:xlang:test:parquet_write,
beam:transforms:xlang:parquet_read, beam:transforms:xlang:textio_read,
beam:external:java:generate_sequence:v1]
beam:transforms:xlang:test:cgbk:
org.apache.beam.sdk.testing.expansion.TestExpansionService$TestServiceRegistrar$TestCoGroupByKeyTransformProvider@1aafa419
beam:transforms:xlang:test:flatten:
org.apache.beam.sdk.testing.expansion.TestExpansionService$TestServiceRegistrar$TestFlattenTransformProvider@47eaca72
beam:transforms:xlang:test:prefix:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@55141def
beam:transforms:xlang:test:multi:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@55182842
beam:transforms:xlang:test:gbk:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@3270d194
beam:transforms:xlang:test:comgl:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@235834f2
beam:transforms:xlang:test:compk:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@5656be13
beam:transforms:xlang:count:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4218d6a3
beam:transforms:xlang:filter_less_than_eq:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@76505305
beam:transforms:xlang:test:partition:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@14cd1699
beam:transforms:xlang:test:parquet_write:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@77888435
beam:transforms:xlang:parquet_read:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@73a1e9a9
beam:transforms:xlang:textio_read:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@71d44a3
beam:external:java:generate_sequence:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@7b98f307
Oct 30, 2021 12:28:49 AM
org.apache.beam.sdk.io.aws2.s3.DefaultS3ClientBuilderFactory createBuilder
INFO: The AWS S3 Beam extension was included in this build, but the awsRegion
flag was not specified. If you don't plan to use S3, then ignore this message.
Oct 30, 2021 12:28:49 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createArtifactStagingService
INFO: ArtifactStagingService started on localhost:38935
Oct 30, 2021 12:28:50 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createExpansionService
INFO: Java ExpansionService started on localhost:40595
Oct 30, 2021 12:28:50 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createJobServer
INFO: JobService started on localhost:42455
Oct 30, 2021 12:28:50 AM org.apache.beam.runners.jobsubmission.JobServerDriver
run
INFO: Job server now running, terminate with Ctrl+C
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy UP-TO-DATE
> Task :buildSrc:pluginDescriptors UP-TO-DATE
> Task :buildSrc:processResources UP-TO-DATE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:spotlessGroovy UP-TO-DATE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:go:container:copyLicenses UP-TO-DATE
> Task :sdks:go:container:dockerClean
> Task :sdks:go:container:goPrepare
Use project GOPATH:
<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/ws/src/sdks/go/container/.gogradle/project_gopath>
> Task :sdks:go:container:resolveBuildDependencies SKIPPED
> Task :sdks:go:container:installDependencies SKIPPED
> Task :sdks:go:container:buildLinuxAmd64 UP-TO-DATE
> Task :sdks:go:container:goBuild
> Task :sdks:go:container:skipPullLicenses
> Task :sdks:go:container:dockerPrepare
> Task :sdks:go:container:docker
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD SUCCESSFUL in 54s
8 actionable tasks: 6 executed, 2 up-to-date
Publishing build scan...
https://gradle.com/s/zh4xts6tukoam
$ CONTAINER=apache/beam_go_sdk
$ cd sdks/go
>>> RUNNING flink integration tests with pipeline options: -p 1 -timeout 2h
>>> --runner=flink --project=apache-beam-testing --region=us-central1
>>> --environment_type=DOCKER --environment_config=apache/beam_go_sdk:dev
>>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test
>>>
>>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test
>>> --dataflow_worker_jar= --endpoint=localhost:42455
>>> --test_expansion_addr=localhost:42653
$
./run_validatesrunner_tests.sh: line 399: go: command not found
$ TEST_EXIT_CODE=0
$ cd ../..
$ exit 127
$ exit 127
> Task :sdks:go:test:flinkValidatesRunner FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/ws/src/sdks/go/test/build.gradle'>
line: 106
* What went wrong:
Execution failed for task ':sdks:go:test:flinkValidatesRunner'.
> Process 'command 'sh'' finished with non-zero exit value 127
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 29m 2s
164 actionable tasks: 157 executed, 2 from cache, 5 up-to-date
Publishing build scan...
https://gradle.com/s/for2jcdv5bxg4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
