See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_GoUsingJava_Dataflow/2/display/redirect?page=changes>
Changes: [Carl Yeksigian] Cache bucket matcher regex in GcsPath [rogelio.hernandez] [BEAM-13051] Pylint misplaced-bare-raise warning enabled [Kyle Weaver] [BEAM-13796] projection pushdown in BQ IO [Jeff Tapper] Update Java LTS roadmap info on website for Java 17 [Kyle Weaver] [BEAM-12712] Spark: Exclude looping timer tests. [akustov] fix name project id from secreton scio deploy action [Kyle Weaver] [BEAM-13919] Annotate PerKeyOrderingTest with UsesStatefulParDo. [Pablo Estrada] Simplify README for new users [alexander.zhuravlev] [BEAM-13775] Fixed bug with run button [ihr] [BEAM-13836] Fix the answers placeholders locations in the Python katas [noreply] [BEAM-4032]Support staging binary distributions of dependency packages [noreply] [BEAM-13834] Increase influxDB persistent storage. (#16817) [noreply] Minor: Fix link to nexmark benchmarks (#16803) [noreply] Regenerate python container base_image_requirements.txt (#16832) [noreply] Update 2.36.0 blog post to mention ARM64 support [randomstep] [BEAM-9195] Bump org.testcontainers to 1.16.3 [noreply] Merge pull request #16703 from [BEAM-13804][Playground][Bugfix] Add [noreply] Merge pull request #16611 from [BEAM-13712][Playground] Add graph for [noreply] Merge pull request #16757 from [BEAM-13655] [Playground] Persist the [stranniknm] [BEAM-13785] playground - enable scio sdk [noreply] Minor: Disable checker framework in nightly snapshot (#16829) [artur.khanin] Updated example link [noreply] [BEAM-13860] Make `DoFn.infer_output_type` return element type (#16788) [noreply] [BEAM-13894] Unit test utilities in the ptest package (#16830) [Kenneth Knowles] Add test for processing time continuation trigger [Kyle Weaver] [BEAM-13106] Support Flink 1.14. [Kyle Weaver] [BEAM-13106] Reuse executor instead of shutting it down mid-test. [Kyle Weaver] [BEAM-13106] Prevent infinite wait in Flink savepoint test. [noreply] [BEAM-13922] [Coverage] Make boot.go more testable and add tests [noreply] Exclude SpannerChangeStream IT from Dataflow V1 postcommit (#16851) [noreply] [BEAM-13930] Address StateSpec consistency issue between Runner and Fn [Ismaël Mejía] [BEAM-13202] Fix typos on tests names for VarianceFnTest [mattcasters] [BEAM-13854] Document casting trick for Avro value serializer in KafkaIO [Kenneth Knowles] Disable AfterSynchronizedProcessingTime test on Dataflow [Ismaël Mejía] [BEAM-13202] Add Coder to CountIfFn.Accum [Ismaël Mejía] [BEAM-13202] Reuse Count transform code since CountIf is a specific case [noreply] Merge pull request #16838 from [BEAM-13931] - make sure large rows cause [Kenneth Knowles] Add test category UsesProcessingTimeTimers [Kenneth Knowles] Label tests that need UsesProcessingTimeTimers [Kenneth Knowles] Exclude UsesProcessingTimeTimers from SamzaRunner tests [noreply] Seznam Case Study (#16825) [noreply] [Website] Apache Hop Case Study (#16824) [noreply] [BEAM-13694] Force hadoop-hdfs-client in hadoopVersion tests for hdfs [noreply] [Website] Ricardo - added case study feedback (#16807) [noreply] Merge pull request #16735 from [BEAM-13827] - fix medium file size [Kyle Weaver] [BEAM-13106] A couple additional fixes to FlinkSavepointTest. [noreply] Merge pull request #16753 from [BEAM-13837] [Playground] show graph on [mmack] [adhoc] Migrate KinesisIOIT to use ITEnvironment for Localstack based IT [rogelio.hernandez] [BEAM-13051] Added descriptions to Kinesis and PortableRunner exceptions [noreply] [BEAM-13955] Fix pylint breakage from #16836 (#16867) [relax] Fix TableRow conversion for the case of fields named "f" [noreply] Bump dataflow.fnapi_container_version (#16874) [mmack] [BEAM-13563] Introducing common AWS ClientBuilderFactory to unify [laraschmidt] Fix final allowskew error to properly handle a large allowedSkew [noreply] Case studies page improvements (#16702) [noreply] [BEAM-13946] Add get_dummies(), a non-deferred column operation on [Kyle Weaver] [BEAM-13796] Move test to ReadTest class and correct javadoc for [noreply] [release-2.36.0] Fix pickler argument for 2.36 blog (#16774) [thiagotnunes] fix: fix bug when retrieving either string or json [noreply] [adhoc] Avoid using SerializablePipelineOptions for testing to minimize [noreply] [BEAM-13812] Integrate DataprocClusterManager into Interactive [Kyle Weaver] [BEAM-13796] Pushdown is not supported on TypedRead#fromQuery. [noreply] [BEAM-12572] Fix failing python examples tests in Dataflow runner [noreply] Remove build status from PR (#16902) [noreply] [BEAM-13738] Reenable ignored SQS test after bumping elasticmq for fixed [noreply] fix build status link (#16907) [noreply] Merge pull request #16549 from [BEAM-13681][Playground] Refactoring [noreply] Merge pull request #16732 from [BEAM-13825] [Playground] updated [noreply] Merge pull request #16683 from [BEAM-13713][Playground] Java graph [noreply] case study pages - improvements and fixes (#16896) [noreply] Palo Alto case study (#16915) [noreply] [BEAM-12645] Fix code-cov flakes due to monorepo. (#16925) [noreply] [BEAM-13969] Deprecate stringx package (#16884) [noreply] Add Go badge to ReadMe (#16897) [noreply] [BEAM-13980] Re-add method gone missing in af2f8ee6 (#16918) [benjamin.gonzalez] Revert PR#16253 due errors with plugin flaky-test-handler [noreply] [BEAM-13884] Improve mtime package (#16924) [noreply] Minor: Update Go API doc links (#16932) [noreply] [BEAM-13218] Re-enable [noreply] Merge pull request #16857 from [BEAM-13662] [Playground] Support [noreply] Merge pull request #16826 from [BEAM-13870] [Playground] Increase test [noreply] Fix BoundedQueueExecutor and StreamingDataflowWorker to actually limit [noreply] [BEAM-1857] Add Neo4jIO (#15916) [noreply] [BEAM-13767] Migrate serveral portable runner tasks to use configuration [noreply] [BEAM-13996] Removing 'No cluster_manager is associated with the [noreply] [BEAM-13906] Improve coverage of errors package (#16934) [noreply] [BEAM-13886] unit tests for trigger package (#16935) [noreply] [BEAM-4767] Remove beam- prefix from release script tags (#16899) [noreply] [BEAM-13866] Add small unit tests to errorx, make boolean assignment [noreply] [BEAM-13925] Add most of the supporting files for the pr management ------------------------------------------ [...truncated 484.29 KB...] Removing intermediate container fcd51d9f2467 ---> 8dc8665b1335 Step 23/31 : COPY target/license_scripts /tmp/license_scripts/ ---> 0af984519488 Step 24/31 : COPY target/LICENSE target/go-licenses/* /opt/apache/beam/third_party_licenses/golang/ ---> a69ef6c751d3 Step 25/31 : RUN rm /opt/apache/beam/third_party_licenses/golang/LICENSE ---> Running in 0f431ab9eb37 Removing intermediate container 0f431ab9eb37 ---> d39a27424a72 Step 26/31 : COPY target/license_scripts /tmp/license_scripts/ ---> 54c00132e2b2 Step 27/31 : RUN if [ "$pull_licenses" = "true" ] ; then pip install 'pip-licenses<3.0.0' pyyaml tenacity && python /tmp/license_scripts/pull_licenses_py.py ; fi ---> Running in e22ee332804b Collecting pip-licenses<3.0.0 Downloading pip_licenses-2.3.0-py3-none-any.whl (14 kB) Requirement already satisfied: pyyaml in /usr/local/lib/python3.8/site-packages (6.0) Requirement already satisfied: tenacity in /usr/local/lib/python3.8/site-packages (5.1.5) Collecting PTable Downloading PTable-0.9.2.tar.gz (31 kB) Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.8/site-packages (from tenacity) (1.16.0) Building wheels for collected packages: PTable Building wheel for PTable (setup.py): started Building wheel for PTable (setup.py): finished with status 'done' Created wheel for PTable: filename=PTable-0.9.2-py3-none-any.whl size=22925 sha256=6e144acd9e8844d9fb152ba31cdafe4e3ee811d697462a142ff41b2d7d72502f Stored in directory: /root/.cache/pip/wheels/1b/3a/02/8d8da2bca2223dda2f827949c88b2d82dc85dccbc2bb6265e5 Successfully built PTable Installing collected packages: PTable, pip-licenses Successfully installed PTable-0.9.2 pip-licenses-2.3.0 [91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [0m[91mWARNING: You are using pip version 21.2.4; however, version 22.0.3 is available. You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command. [0mSkip pulling license for bs4 [91mINFO:root:Successfully pulled licenses for 137 dependencies [0mRemoving intermediate container e22ee332804b ---> 4c36c1476ef5 Step 28/31 : FROM beam ---> 125bc455d1c1 Step 29/31 : ARG pull_licenses ---> Running in 2c8dc5fc4c50 Removing intermediate container 2c8dc5fc4c50 ---> e07897343e00 Step 30/31 : COPY --from=third_party_licenses /opt/apache/beam/third_party_licenses /opt/apache/beam/third_party_licenses ---> 31de58d422fb Step 31/31 : RUN if [ "$pull_licenses" != "true" ] ; then rm -rf /opt/apache/beam/third_party_licenses ; fi ---> Running in a078077f2483 Removing intermediate container a078077f2483 ---> c1242c41e91d Successfully built c1242c41e91d Successfully tagged apache/beam_python3.8_sdk:2.38.0.dev > Task :runners:google-cloud-dataflow-java:buildAndPushDockerPythonContainer WARNING: `gcloud docker` will not be supported for Docker client versions above 18.03. As an alternative, use `gcloud auth configure-docker` to configure `docker` to use `gcloud` as a credential helper, then use `docker` as you would for non-GCR registries, e.g. `docker pull gcr.io/project-id/my-image`. Add `--verbosity=error` to silence this warning: `gcloud docker --verbosity=error -- pull gcr.io/project-id/my-image`. See: https://cloud.google.com/container-registry/docs/support/deprecation-notices#gcloud-docker The push refers to repository [us.gcr.io/apache-beam-testing/java-postcommit-it/python] be50347189e7: Preparing f81bbf9ee954: Preparing 0822cb076801: Preparing 3e8860381462: Preparing ec98399874ac: Preparing 2fbf632f9f51: Preparing 42e5612af656: Preparing af9be7639813: Preparing d5681e211920: Preparing 507069cc9871: Preparing a0ab55928b9e: Preparing 4657cfb2a616: Preparing f79fead44a82: Preparing e0c36419967d: Preparing 60e32711b276: Preparing ae233fa4f2a3: Preparing e14403cd4d18: Preparing 8a8d6e9f7282: Preparing a3232401de62: Preparing 204e42b3d47b: Preparing 613ab28cf833: Preparing bed676ceab7a: Preparing 6398d5cccd2c: Preparing 0b0f2f2f5279: Preparing 42e5612af656: Waiting e0c36419967d: Waiting af9be7639813: Waiting 60e32711b276: Waiting ae233fa4f2a3: Waiting a0ab55928b9e: Waiting e14403cd4d18: Waiting d5681e211920: Waiting 4657cfb2a616: Waiting 8a8d6e9f7282: Waiting f79fead44a82: Waiting 2fbf632f9f51: Waiting bed676ceab7a: Waiting 507069cc9871: Waiting 204e42b3d47b: Waiting a3232401de62: Waiting 0b0f2f2f5279: Waiting 6398d5cccd2c: Waiting 0822cb076801: Pushed 3e8860381462: Pushed ec98399874ac: Pushed be50347189e7: Pushed 2fbf632f9f51: Pushed 42e5612af656: Pushed f81bbf9ee954: Pushed d5681e211920: Pushed 507069cc9871: Pushed a0ab55928b9e: Pushed e0c36419967d: Pushed ae233fa4f2a3: Layer already exists 60e32711b276: Pushed e14403cd4d18: Layer already exists a3232401de62: Layer already exists 8a8d6e9f7282: Layer already exists 204e42b3d47b: Layer already exists bed676ceab7a: Layer already exists 613ab28cf833: Layer already exists 6398d5cccd2c: Layer already exists 0b0f2f2f5279: Layer already exists af9be7639813: Pushed 4657cfb2a616: Pushed f79fead44a82: Pushed 20220225060255: digest: sha256:3574a164cf20f8149121104e7c2c2fc10314cef7ce4fead674ae1d864da7889c size: 5361 > Task :runners:google-cloud-dataflow-java:setupXVR > Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerSetup Launching Java expansion service @ 35813 Launching Python expansion service @ 35825 > Task > :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerGoUsingJava > FAILED $ TESTS=./test/integration/... ./test/regression $ RUNNER=portable $ TIMEOUT=2h $ SIMULTANEOUS=3 $ GCS_LOCATION=gs://temp-storage-for-end-to-end-tests $ PROJECT=apache-beam-testing $ DATAFLOW_PROJECT=apache-beam-testing $ REGION=us-central1 $ trap exit_background_processes SIGINT SIGTERM EXIT $ key=--io_expansion_jar $ case --io_expansion_jar in $ IO_EXPANSION_JAR=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_GoUsingJava_Dataflow/ws/src/sdks/java/io/expansion-service/build/libs/beam-sdks-java-io-expansion-service-2.38.0-SNAPSHOT.jar> $ key=--schemaio_expansion_jar $ case --schemaio_expansion_jar in $ SCHEMAIO_EXPANSION_JAR=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_GoUsingJava_Dataflow/ws/src/sdks/java/extensions/schemaio-expansion-service/build/libs/beam-sdks-java-extensions-schemaio-expansion-service-2.38.0-SNAPSHOT.jar> $ key=--debeziumio_expansion_jar $ case --debeziumio_expansion_jar in $ DEBEZIUMIO_EXPANSION_JAR=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_GoUsingJava_Dataflow/ws/src/sdks/java/io/debezium/expansion-service/build/libs/beam-sdks-java-io-debezium-expansion-service-2.38.0-SNAPSHOT.jar> $ key=--pipeline_opts $ case --pipeline_opts in $ PIPELINE_OPTS=--kafka_jar=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_GoUsingJava_Dataflow/ws/src/sdks/java/testing/kafka-service/build/libs/beam-sdks-java-testing-kafka-service-testKafkaService-2.38.0-SNAPSHOT.jar> $ key=--test_expansion_addr $ case --test_expansion_addr in $ TEST_EXPANSION_ADDR=localhost:35813 $ key=--runner $ case --runner in $ RUNNER=dataflow $ key=--project $ case --project in $ PROJECT=apache-beam-testing $ key=--dataflow_project $ case --dataflow_project in $ DATAFLOW_PROJECT=apache-beam-testing $ key=--region $ case --region in $ REGION=us-central1 $ key=--tests $ case --tests in $ TESTS=./test/integration/xlang $ key=--sdk_overrides $ case --sdk_overrides in $ SDK_OVERRIDES=.*java.*,us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220225060255 $ cd <https://ci-beam.apache.org/job/beam_PostCommit_XVR_GoUsingJava_Dataflow/ws/src> $ test -d sdks/go/test $ DATAFLOW_WORKER_JAR= find: ‘<https://ci-beam.apache.org/job/beam_PostCommit_XVR_GoUsingJava_Dataflow/ws/src/runners/google-cloud-dataflow-java/worker/build/libs/beam-runners-google-cloud-dataflow-java-fn-api-worker-*.jar%E2%80%99>: No such file or directory $ DATAFLOW_WORKER_JAR= > Task :runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerCleanup Stopping expansion service pid: 2204328. Stopping expansion service pid: 2204329. > Task :runners:google-cloud-dataflow-java:cleanupXVR UP-TO-DATE > Task :runners:google-cloud-dataflow-java:cleanUpDockerPythonImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/python:20220225060255 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3574a164cf20f8149121104e7c2c2fc10314cef7ce4fead674ae1d864da7889c Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/python:20220225060255] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3574a164cf20f8149121104e7c2c2fc10314cef7ce4fead674ae1d864da7889c] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/python:20220225060255] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3574a164cf20f8149121104e7c2c2fc10314cef7ce4fead674ae1d864da7889c])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3574a164cf20f8149121104e7c2c2fc10314cef7ce4fead674ae1d864da7889c Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3574a164cf20f8149121104e7c2c2fc10314cef7ce4fead674ae1d864da7889c Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/python@sha256:3574a164cf20f8149121104e7c2c2fc10314cef7ce4fead674ae1d864da7889c]. > Task :runners:google-cloud-dataflow-java:cleanUpDockerJavaImages Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220225060255 Untagged: us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f2bc88a43651fbc8d254dc5d421d477b8dc203a6b2916935f44afd8a6f6d506 Tag: [us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220225060255] - referencing digest: [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f2bc88a43651fbc8d254dc5d421d477b8dc203a6b2916935f44afd8a6f6d506] Deleted [[us.gcr.io/apache-beam-testing/java-postcommit-it/java:20220225060255] (referencing [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f2bc88a43651fbc8d254dc5d421d477b8dc203a6b2916935f44afd8a6f6d506])]. Removing untagged image us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f2bc88a43651fbc8d254dc5d421d477b8dc203a6b2916935f44afd8a6f6d506 Digests: - us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f2bc88a43651fbc8d254dc5d421d477b8dc203a6b2916935f44afd8a6f6d506 Deleted [us.gcr.io/apache-beam-testing/java-postcommit-it/java@sha256:4f2bc88a43651fbc8d254dc5d421d477b8dc203a6b2916935f44afd8a6f6d506]. FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_GoUsingJava_Dataflow/ws/src/sdks/go/test/build.gradle'> line: 176 * What went wrong: Execution failed for task ':runners:google-cloud-dataflow-java:validatesCrossLanguageRunnerGoUsingJava'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 13m 1s 176 actionable tasks: 128 executed, 39 from cache, 9 up-to-date Publishing build scan... https://gradle.com/s/vynlkribjk5cu Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
