See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/873/display/redirect?page=changes>
Changes: [noreply] Merge pull request #15753 from [BEAM-13080] Add option in Reshuffle to ------------------------------------------ [...truncated 494.78 KB...] tensorboard==2.7.0 tensorboard-data-server==0.6.1 tensorboard-plugin-wit==1.8.0 tensorflow==2.6.0 tensorflow-estimator==2.7.0 termcolor==1.1.0 threadpoolctl==3.0.0 tqdm==4.62.3 typing-extensions==3.7.4.3 typing-inspect==0.7.1 uritemplate==4.1.1 urllib3==1.25.11 wcwidth==0.2.5 Werkzeug==2.0.2 wheel==0.37.0 wrapt==1.12.1 zipp==3.6.0 Removing intermediate container b80de4236607 ---> 34952c73015b Step 22/27 : RUN pip check ---> Running in c705b6f6a5e5 No broken requirements found. Removing intermediate container c705b6f6a5e5 ---> a5f47f10b3f3 Step 23/27 : COPY target/LICENSE /opt/apache/beam/ ---> f1e36f57913f Step 24/27 : COPY target/LICENSE.python /opt/apache/beam/ ---> 2b532c370791 Step 25/27 : COPY target/NOTICE /opt/apache/beam/ ---> 1822cf2e5221 Step 26/27 : ADD target/launcher/linux_amd64/boot /opt/apache/beam/ ---> 35c98a760166 Step 27/27 : ENTRYPOINT ["/opt/apache/beam/boot"] ---> Running in 1f7beae1d80a Removing intermediate container 1f7beae1d80a ---> 24b96ac160af Successfully built 24b96ac160af Successfully tagged apache/beam_python3.6_sdk:2.35.0.dev > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerSetup Launching Java expansion service @ 37427 Launching Python expansion service @ 33981 > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerGoUsingJava $ TESTS=./test/integration/... ./test/regression $ RUNNER=portable $ TIMEOUT=2h $ SIMULTANEOUS=3 $ GCS_LOCATION=gs://temp-storage-for-end-to-end-tests $ PROJECT=apache-beam-testing $ DATAFLOW_PROJECT=apache-beam-testing $ REGION=us-central1 $ trap exit_background_processes SIGINT SIGTERM EXIT $ key=--io_expansion_jar $ case --io_expansion_jar in $ IO_EXPANSION_JAR=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/io/expansion-service/build/libs/beam-sdks-java-io-expansion-service-2.35.0-SNAPSHOT.jar> $ key=--pipeline_opts $ case --pipeline_opts in $ PIPELINE_OPTS=--kafka_jar=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/testing/kafka-service/build/libs/beam-sdks-java-testing-kafka-service-testKafkaService-2.35.0-SNAPSHOT.jar> $ key=--test_expansion_addr $ case --test_expansion_addr in $ TEST_EXPANSION_ADDR=localhost:37427 $ key=--runner $ case --runner in $ RUNNER=spark $ key=--tests $ case --tests in $ TESTS=./test/integration/xlang ./test/integration/io/xlang/... $ key=--endpoint $ case --endpoint in $ ENDPOINT=localhost:34473 $ cd <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src> $ test -d sdks/go/test $ EXPANSION_PORT=48105 $ IO_EXPANSION_ADDR=localhost:38079 No IO expansion address specified; starting a new IO expansion server on localhost:38079 $ java -jar <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/io/expansion-service/build/libs/beam-sdks-java-io-expansion-service-2.35.0-SNAPSHOT.jar> 38079 $ IO_EXPANSION_PID=8777 $ SIMULTANEOUS=1 $ TAG=dev $ ./gradlew :sdks:go:container:docker -Pdocker-tag=dev Starting expansion service at localhost:38079 Starting a Gradle Daemon, 3 busy Daemons could not be reused, use --status for details Oct 31, 2021 12:13:00 AM org.apache.beam.sdk.expansion.service.ExpansionService loadRegisteredTransforms INFO: Registering external transforms: [beam:transform:org.apache.beam:kafka_read_with_metadata:v1, beam:transform:org.apache.beam:kafka_read_without_metadata:v1, beam:transform:org.apache.beam:kafka_write:v1, beam:external:java:generate_sequence:v1] beam:transform:org.apache.beam:kafka_read_with_metadata:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@551bdc27 beam:transform:org.apache.beam:kafka_read_without_metadata:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@58fdd99 beam:transform:org.apache.beam:kafka_write:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@6b1274d2 beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@7bc1a03d > Task :buildSrc:compileJava NO-SOURCE > Task :buildSrc:compileGroovy UP-TO-DATE > Task :buildSrc:pluginDescriptors UP-TO-DATE > Task :buildSrc:processResources UP-TO-DATE > Task :buildSrc:classes UP-TO-DATE > Task :buildSrc:jar UP-TO-DATE > Task :buildSrc:assemble UP-TO-DATE > Task :buildSrc:spotlessGroovy UP-TO-DATE > Task :buildSrc:spotlessGroovyCheck UP-TO-DATE > Task :buildSrc:spotlessGroovyGradle UP-TO-DATE > Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE > Task :buildSrc:spotlessCheck UP-TO-DATE > Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:validatePlugins UP-TO-DATE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build UP-TO-DATE Configuration on demand is an incubating feature. > Task :sdks:go:container:copyLicenses UP-TO-DATE > Task :sdks:go:container:dockerClean > Task :sdks:go:container:goPrepare Use project GOPATH: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/go/container/.gogradle/project_gopath> > Task :sdks:go:container:resolveBuildDependencies SKIPPED > Task :sdks:go:container:installDependencies SKIPPED > Task :sdks:go:container:buildLinuxAmd64 UP-TO-DATE > Task :sdks:go:container:goBuild > Task :sdks:go:container:skipPullLicenses > Task :sdks:go:container:dockerPrepare > Task :sdks:go:container:docker Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD SUCCESSFUL in 25s 8 actionable tasks: 6 executed, 2 up-to-date Publishing build scan... https://gradle.com/s/pq22lew4uh6zi $ CONTAINER=apache/beam_go_sdk $ cd sdks/go >>> RUNNING spark integration tests with pipeline options: -p 1 -timeout 2h >>> --runner=spark --project=apache-beam-testing --region=us-central1 >>> --environment_type=DOCKER --environment_config=apache/beam_go_sdk:dev >>> --staging_location=gs://temp-storage-for-end-to-end-tests/staging-validatesrunner-test >>> >>> --temp_location=gs://temp-storage-for-end-to-end-tests/temp-validatesrunner-test >>> --dataflow_worker_jar= --endpoint=localhost:34473 >>> --test_expansion_addr=localhost:37427 --io_expansion_addr=localhost:38079 >>> --kafka_jar=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/testing/kafka-service/build/libs/beam-sdks-java-testing-kafka-service-testKafkaService-2.35.0-SNAPSHOT.jar> $ ./run_validatesrunner_tests.sh: line 399: go: command not found $ TEST_EXIT_CODE=0 $ cd ../.. $ exit 127 $ exit 127 > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerGoUsingJava > FAILED > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingJava > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingPython > Task > :runners:spark:3:job-server:validatesCrossLanguageRunnerJavaUsingPythonOnly > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingJava >>> RUNNING integration tests with pipeline options: --runner=PortableRunner >>> --job_endpoint=localhost:34473 --environment_cache_millis=10000 >>> --experiments=beam_fn_api >>> pytest options: >>> collect markers: -m=xlang_transforms ============================= test session starts ============================== platform linux -- Python 3.6.13, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 5110 items / 5100 deselected / 10 selected apache_beam/io/external/generate_sequence_test.py ... [ 30%] apache_beam/transforms/validate_runner_xlang_test.py ....... [100%] =============================== warnings summary =============================== apache_beam/io/filesystems_test.py:54 <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string apache_beam/io/filesystems_test.py:62 <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/pytest_xlangValidateRunner.xml> - =========== 10 passed, 5100 deselected, 2 warnings in 101.84 seconds =========== > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingPython >>> RUNNING integration tests with pipeline options: --runner=PortableRunner >>> --job_endpoint=localhost:34473 --environment_cache_millis=10000 >>> --experiments=beam_fn_api >>> pytest options: >>> collect markers: -m=xlang_transforms ============================= test session starts ============================== platform linux -- Python 3.6.13, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 5110 items / 5100 deselected / 10 selected apache_beam/io/external/generate_sequence_test.py .ss [ 30%] apache_beam/transforms/validate_runner_xlang_test.py ....... [100%] - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/pytest_xlangValidateRunner.xml> - ============ 8 passed, 2 skipped, 5100 deselected in 57.25 seconds ============= > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerPythonUsingSql >>> RUNNING integration tests with pipeline options: --runner=PortableRunner >>> --job_endpoint=localhost:34473 --environment_cache_millis=10000 >>> --experiments=beam_fn_api >>> pytest options: >>> collect markers: -m=xlang_sql_expansion_service ============================= test session starts ============================== platform linux -- Python 3.6.13, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 5110 items / 5101 deselected / 9 selected apache_beam/transforms/sql_test.py ......... [100%] - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/python/pytest_xlangSqlValidateRunner.xml> - ================= 9 passed, 5101 deselected in 178.84 seconds ================== > Task :runners:spark:3:job-server:validatesCrossLanguageRunnerCleanup Stopping expansion service pid: 8699. Stopping expansion service pid: 8702. > Task :runners:spark:3:job-server:sparkJobServerCleanup Stopping job server pid: 24700. FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/go/test/build.gradle'> line: 212 * What went wrong: Execution failed for task ':runners:spark:3:job-server:validatesCrossLanguageRunnerGoUsingJava'. > Process 'command 'sh'' finished with non-zero exit value 127 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 30m 29s 216 actionable tasks: 155 executed, 54 from cache, 7 up-to-date Publishing build scan... https://gradle.com/s/2rgkuqy54chya Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
