See <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/3591/display/redirect?page=changes>
Changes: [relax] handle changing schemas in Storage API sink [noreply] Fix a couple style issues (#17361) [noreply] [BEAM-14287] Clean up staticcheck warnings in graph/coder (#17337) [noreply] Improvements to dataflow job service for non-Python jobs. (#17338) ------------------------------------------ [...truncated 527.58 KB...] Building wheel for PTable (setup.py): started Building wheel for PTable (setup.py): finished with status 'done' Created wheel for PTable: filename=PTable-0.9.2-py3-none-any.whl size=22924 sha256=858d2aa2dce4eb3b09e550164d9cd701108fad36ed9803943fc52a74d3a4591f Stored in directory: /root/.cache/pip/wheels/f3/65/67/71f473ec87ea4554d1bdfb1b5128cfe1414c2b113b72a1ee8e Successfully built PTable Installing collected packages: PTable, pip-licenses Successfully installed PTable-0.9.2 pip-licenses-3.5.3 [91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv [0m[91mWARNING: You are using pip version 21.2.4; however, version 21.3.1 is available. You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command. [0mSkip pulling license for bs4 [91mINFO:root:Successfully pulled licenses for 136 dependencies [0mRemoving intermediate container b62a0ff6edc9 ---> 3942959c8842 Step 28/31 : FROM beam ---> 83173c0a17fa Step 29/31 : ARG pull_licenses ---> Running in 115923571b0d Removing intermediate container 115923571b0d ---> 2cbaeadf6c2c Step 30/31 : COPY --from=third_party_licenses /opt/apache/beam/third_party_licenses /opt/apache/beam/third_party_licenses ---> 58485980c8dd Step 31/31 : RUN if [ "$pull_licenses" != "true" ] ; then rm -rf /opt/apache/beam/third_party_licenses ; fi ---> Running in da61d9864970 Removing intermediate container da61d9864970 ---> b6f5210ba8a1 Successfully built b6f5210ba8a1 Successfully tagged apache/beam_python3.6_sdk:2.39.0.dev > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerSetup Launching Java expansion service @ 36113 Launching Python expansion service @ 44369 > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerGoUsingJava $ CURRENT_DIRECTORY=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/go/test> $ TESTS=./test/integration/... ./test/regression $ RUNNER=portable $ TIMEOUT=2h $ SIMULTANEOUS=3 $ GCS_LOCATION=gs://temp-storage-for-end-to-end-tests $ PROJECT=apache-beam-testing $ DATAFLOW_PROJECT=apache-beam-testing $ REGION=us-central1 $ trap exit_background_processes SIGINT SIGTERM EXIT $ key=--pipeline_opts $ case --pipeline_opts in $ PIPELINE_OPTS=--kafka_jar=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/testing/kafka-service/build/libs/beam-sdks-java-testing-kafka-service-testKafkaService-2.39.0-SNAPSHOT.jar> --expansion_jar=io:<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/io/expansion-service/build/libs/beam-sdks-java-io-expansion-service-2.39.0-SNAPSHOT.jar> --expansion_jar=schemaio:<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/extensions/schemaio-expansion-service/build/libs/beam-sdks-java-extensions-schemaio-expansion-service-2.39.0-SNAPSHOT.jar> --expansion_jar=debeziumio:<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/java/io/debezium/expansion-service/build/libs/beam-sdks-java-io-debezium-expansion-service-2.39.0-SNAPSHOT.jar> --expansion_addr=test:localhost:36113 $ key=--runner $ case --runner in $ RUNNER=spark $ key=--tests $ case --tests in $ TESTS=./test/integration/xlang ./test/integration/io/xlang/... $ key=--endpoint $ case --endpoint in $ ENDPOINT=localhost:42457 $ cd <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src> $ test -d sdks/go/test $ SIMULTANEOUS=1 $ TAG=dev $ ./gradlew :sdks:go:container:docker -Pdocker-tag=dev Configuration on demand is an incubating feature. > Task :buildSrc:compileJava NO-SOURCE > Task :buildSrc:compileGroovy UP-TO-DATE > Task :buildSrc:pluginDescriptors UP-TO-DATE > Task :buildSrc:processResources UP-TO-DATE > Task :buildSrc:classes UP-TO-DATE > Task :buildSrc:jar UP-TO-DATE > Task :buildSrc:assemble UP-TO-DATE > Task :buildSrc:spotlessGroovy UP-TO-DATE > Task :buildSrc:spotlessGroovyCheck UP-TO-DATE > Task :buildSrc:spotlessGroovyGradle UP-TO-DATE > Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE > Task :buildSrc:spotlessCheck UP-TO-DATE > Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE > Task :buildSrc:compileTestJava NO-SOURCE > Task :buildSrc:compileTestGroovy NO-SOURCE > Task :buildSrc:processTestResources NO-SOURCE > Task :buildSrc:testClasses UP-TO-DATE > Task :buildSrc:test NO-SOURCE > Task :buildSrc:validatePlugins UP-TO-DATE > Task :buildSrc:check UP-TO-DATE > Task :buildSrc:build UP-TO-DATE System Go installation: /snap/bin/go is go version go1.16.15 linux/amd64; Preparing to use /home/jenkins/go/bin/go1.16.12 GOCMD=/home/jenkins/go/bin/go1.16.12 go1.16.12: already downloaded in /home/jenkins/sdk/go1.16.12 > Task :sdks:go:container:copyLicenses UP-TO-DATE > Task :sdks:go:container:dockerClean > Task :sdks:go:container:goPrepare UP-TO-DATE > Task :sdks:go:container:goBuild /home/jenkins/go/bin/go1.16.12 build -o ./build/target/linux_amd64/boot boot.go boot_test.go > Task :sdks:go:container:skipPullLicenses > Task :sdks:go:container:dockerPrepare The message received from the daemon indicates that the daemon has disappeared. Build request sent: Build{id=e5624c80-5429-404f-bd42-d4dfc4281a47, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src}> Attempting to read last messages from the daemon log... Daemon pid: 3757864 log file: /home/jenkins/.gradle/daemon/7.3.2/daemon-3757864.out.log ----- Last 20 lines from daemon log file - daemon-3757864.out.log ----- 2022-04-13T18:10:54.081+0000 [DEBUG] [org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Marking busy by address: [0e996f59-4feb-4253-9c5e-cedf43b1de2a port:34085, addresses:[localhost/127.0.0.1]] 2022-04-13T18:10:54.082+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive lock on daemon addresses registry. 2022-04-13T18:10:54.082+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon addresses registry. 2022-04-13T18:10:54.083+0000 [DEBUG] [org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon addresses registry. 2022-04-13T18:10:54.083+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] resetting idle timer 2022-04-13T18:10:54.083+0000 [DEBUG] [org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon is running. Sleeping until state changes. 2022-04-13T18:10:54.085+0000 [INFO] [org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy] Daemon is about to start building Build{id=e5624c80-5429-404f-bd42-d4dfc4281a47, currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src}.> Dispatching build started information... 2022-04-13T18:10:54.085+0000 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 457: dispatching org.gradle.launcher.daemon.protocol.BuildStarted@50e3c5db 2022-04-13T18:10:54.086+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment] Configuring env variables: [PATH, RUN_DISPLAY_URL, HUDSON_HOME, RUN_CHANGES_DISPLAY_URL, JOB_URL, HUDSON_COOKIE, MOTD_SHOWN, DBUS_SESSION_BUS_ADDRESS, JENKINS_SERVER_COOKIE, LOGNAME, PWD, RUN_TESTS_DISPLAY_URL, JENKINS_URL, SHELL, BUILD_TAG, ROOT_BUILD_CAUSE, BUILD_CAUSE_TIMERTRIGGER, OLDPWD, GIT_CHECKOUT_DIR, JENKINS_HOME, sha1, CODECOV_TOKEN, NODE_NAME, BUILD_DISPLAY_NAME, JOB_DISPLAY_URL, GIT_BRANCH, SETUPTOOLS_USE_DISTUTILS, SHLVL, WORKSPACE_TMP, GIT_PREVIOUS_COMMIT, JAVA_HOME, BUILD_ID, LANG, XDG_SESSION_ID, XDG_SESSION_TYPE, JOB_NAME, SPARK_LOCAL_IP, BUILD_CAUSE, GIT_PREVIOUS_SUCCESSFUL_COMMIT, NODE_LABELS, HUDSON_URL, WORKSPACE, ROOT_BUILD_CAUSE_TIMERTRIGGER, XDG_SESSION_CLASS, _, GIT_COMMIT, COVERALLS_REPO_TOKEN, CI, EXECUTOR_NUMBER, HUDSON_SERVER_COOKIE, SSH_CLIENT, JOB_BASE_NAME, USER, SSH_CONNECTION, BUILD_NUMBER, BUILD_URL, RUN_ARTIFACTS_DISPLAY_URL, GIT_URL, XDG_RUNTIME_DIR, HOME] 2022-04-13T18:10:54.088+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.LogToClient] About to start relaying all logs to the client via the connection. 2022-04-13T18:10:54.088+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogToClient] The client will now receive all logging from the daemon (pid: 3757864). The daemon log file: /home/jenkins/.gradle/daemon/7.3.2/daemon-3757864.out.log 2022-04-13T18:10:54.091+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogAndCheckHealth] Starting 3rd build in daemon [uptime: 50 mins 51.172 secs, performance: 100%] 2022-04-13T18:10:54.092+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has started executing the build. 2022-04-13T18:10:54.092+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] Executing build with daemon context: DefaultDaemonContext[uid=e9ebc82c-ef11-4ff7-9a86-b95ab81b2d08,javaHome=/usr/lib/jvm/java-8-openjdk-amd64,daemonRegistryDir=/home/jenkins/.gradle/daemon,pid=3757864,idleTimeout=10800000,priority=NORMAL,daemonOpts=-Xss10240k,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant] Configuration on demand is an incubating feature. System Go installation: /snap/bin/go is go version go1.16.15 linux/amd64; Preparing to use /home/jenkins/go/bin/go1.16.12 go1.16.12: already downloaded in /home/jenkins/sdk/go1.16.12 GOCMD=/home/jenkins/go/bin/go1.16.12 /home/jenkins/go/bin/go1.16.12 build -o ./build/target/linux_amd64/boot boot.go boot_test.go Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt. ----- End of the daemon log ----- FAILURE: Build failed with an exception. * What went wrong: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed) * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org > Task :sdks:go:container:docker $ ./gradlew :sdks:go:container:docker -Pdocker-tag=dev > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerGoUsingJava > FAILED > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerJavaUsingJava > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerJavaUsingPython > Task > :runners:spark:2:job-server:validatesCrossLanguageRunnerJavaUsingPythonOnly > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerPythonUsingJava >>> RUNNING integration tests with pipeline options: --runner=PortableRunner >>> --job_endpoint=localhost:42457 --environment_cache_millis=10000 >>> --experiments=beam_fn_api >>> pytest options: >>> collect markers: -m=xlang_transforms ============================= test session starts ============================== platform linux -- Python 3.6.15, pytest-4.6.11, py-1.11.0, pluggy-0.13.1 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 5316 items / 5306 deselected / 10 selected apache_beam/io/external/generate_sequence_test.py ... [ 30%] apache_beam/transforms/validate_runner_xlang_test.py ....... [100%] =============================== warnings summary =============================== apache_beam/io/filesystems_test.py:54 <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54: DeprecationWarning: invalid escape sequence \c self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint: disable=anomalous-backslash-in-string apache_beam/io/filesystems_test.py:62 <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62: DeprecationWarning: invalid escape sequence \d self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), # pylint: disable=anomalous-backslash-in-string -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/pytest_xlangValidateRunner.xml> - =========== 10 passed, 5306 deselected, 2 warnings in 111.59 seconds =========== > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerPythonUsingPython >>> RUNNING integration tests with pipeline options: --runner=PortableRunner >>> --job_endpoint=localhost:42457 --environment_cache_millis=10000 >>> --experiments=beam_fn_api >>> pytest options: >>> collect markers: -m=xlang_transforms ============================= test session starts ============================== platform linux -- Python 3.6.15, pytest-4.6.11, py-1.11.0, pluggy-0.13.1 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 5316 items / 5306 deselected / 10 selected apache_beam/io/external/generate_sequence_test.py .ss [ 30%] apache_beam/transforms/validate_runner_xlang_test.py ....... [100%] - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/pytest_xlangValidateRunner.xml> - ============ 8 passed, 2 skipped, 5306 deselected in 66.55 seconds ============= > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerPythonUsingSql >>> RUNNING integration tests with pipeline options: --runner=PortableRunner >>> --job_endpoint=localhost:42457 --environment_cache_millis=10000 >>> --experiments=beam_fn_api >>> pytest options: >>> collect markers: -m=xlang_sql_expansion_service ============================= test session starts ============================== platform linux -- Python 3.6.15, pytest-4.6.11, py-1.11.0, pluggy-0.13.1 rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 5316 items / 5307 deselected / 9 selected apache_beam/transforms/sql_test.py ......... [100%] - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/python/pytest_xlangSqlValidateRunner.xml> - ================= 9 passed, 5307 deselected in 214.29 seconds ================== > Task :runners:spark:2:job-server:validatesCrossLanguageRunnerCleanup Stopping expansion service pid: 3895635. Stopping expansion service pid: 3895636. > Task :runners:spark:2:job-server:sparkJobServerCleanup Stopping job server pid: 3843779. FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark/ws/src/sdks/go/test/build.gradle'> line: 195 * What went wrong: Execution failed for task ':runners:spark:2:job-server:validatesCrossLanguageRunnerGoUsingJava'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: > Run with --stacktrace option to get the stack trace. > Run with --info or --debug option to get more log output. > Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0. You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins. See https://docs.gradle.org/7.3.2/userguide/command_line_interface.html#sec:command_line_warnings Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness. Please consult deprecation warnings for more details. BUILD FAILED in 33m 16s 250 actionable tasks: 172 executed, 67 from cache, 11 up-to-date Publishing build scan... https://gradle.com/s/qgmji25svowwg Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
