See
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/4485/display/redirect>
Changes:
------------------------------------------
[...truncated 507.04 KB...]
# Disable parallelism on runners that don't support it.
if [[ "$RUNNER" == "flink" || "$RUNNER" == "spark" || "$RUNNER" == "samza" ]];
then
SIMULTANEOUS=1
fi
if [[ "$RUNNER" == "dataflow" ]]; then
# Verify docker and gcloud commands exist
command -v docker
docker -v
command -v gcloud
gcloud --version
# ensure gcloud is version 186 or above
TMPDIR=$(mktemp -d)
gcloud_ver=$(gcloud -v | head -1 | awk '{print $4}')
if [[ "$gcloud_ver" < "186" ]]
then
pushd $TMPDIR
curl
https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-186.0.0-linux-x86_64.tar.gz
--output gcloud.tar.gz
tar xf gcloud.tar.gz
./google-cloud-sdk/install.sh --quiet
. ./google-cloud-sdk/path.bash.inc
popd
gcloud components update --quiet || echo 'gcloud components update failed'
gcloud -v
fi
# Build the container
TAG=$(date +%Y%m%d-%H%M%S)
CONTAINER=us.gcr.io/$PROJECT/$USER/beam_go_sdk
echo "Using container $CONTAINER"
./gradlew :sdks:go:container:docker
-Pdocker-repository-root=us.gcr.io/$PROJECT/$USER -Pdocker-tag=$TAG
# Verify it exists
docker images | grep $TAG
# Push the container
gcloud docker -- push $CONTAINER:$TAG
if [[ -n "$TEST_EXPANSION_ADDR" || -n "$IO_EXPANSION_ADDR" ]]; then
ARGS="$ARGS --experiments=use_portable_job_submission"
if [[ -z "$SDK_OVERRIDES" ]]; then
# Build the java container for cross-language
JAVA_TAG=$(date +%Y%m%d-%H%M%S)
JAVA_CONTAINER=us.gcr.io/$PROJECT/$USER/beam_java11_sdk
echo "Using container $JAVA_CONTAINER for cross-language java transforms"
./gradlew :sdks:java:container:java11:docker
-Pdocker-repository-root=us.gcr.io/$PROJECT/$USER -Pdocker-tag=$JAVA_TAG
# Verify it exists
docker images | grep $JAVA_TAG
# Push the container
gcloud docker -- push $JAVA_CONTAINER:$JAVA_TAG
SDK_OVERRIDES=".*java.*,$JAVA_CONTAINER:$JAVA_TAG"
fi
fi
else
TAG=dev
./gradlew :sdks:go:container:docker -Pdocker-tag=$TAG
CONTAINER=apache/beam_go_sdk
fi
Starting expansion service at localhost:44145
Sep 26, 2021 12:12:24 AM org.apache.beam.sdk.expansion.service.ExpansionService
loadRegisteredTransforms
INFO: Registering external transforms:
[beam:external:java:kafkaio:externalwithmetadata:v1,
beam:external:java:kafkaio:typedwithoutmetadata:v1,
beam:external:java:kafka:write:v1, beam:external:java:generate_sequence:v1]
beam:external:java:kafkaio:externalwithmetadata:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$51/477289012@3d36e4cd
beam:external:java:kafkaio:typedwithoutmetadata:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$51/477289012@6a472554
beam:external:java:kafka:write:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$51/477289012@7ff2a664
beam:external:java:generate_sequence:v1:
org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$$Lambda$51/477289012@525b461a
> Task :buildSrc:compileJava NO-SOURCE
> Task :buildSrc:compileGroovy UP-TO-DATE
> Task :buildSrc:pluginDescriptors UP-TO-DATE
> Task :buildSrc:processResources UP-TO-DATE
> Task :buildSrc:classes UP-TO-DATE
> Task :buildSrc:jar UP-TO-DATE
> Task :buildSrc:assemble UP-TO-DATE
> Task :buildSrc:spotlessGroovy UP-TO-DATE
> Task :buildSrc:spotlessGroovyCheck UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradle UP-TO-DATE
> Task :buildSrc:spotlessGroovyGradleCheck UP-TO-DATE
> Task :buildSrc:spotlessCheck UP-TO-DATE
> Task :buildSrc:pluginUnderTestMetadata UP-TO-DATE
> Task :buildSrc:compileTestJava NO-SOURCE
> Task :buildSrc:compileTestGroovy NO-SOURCE
> Task :buildSrc:processTestResources NO-SOURCE
> Task :buildSrc:testClasses UP-TO-DATE
> Task :buildSrc:test NO-SOURCE
> Task :buildSrc:validatePlugins UP-TO-DATE
> Task :buildSrc:check UP-TO-DATE
> Task :buildSrc:build UP-TO-DATE
Configuration on demand is an incubating feature.
> Task :sdks:go:container:copyLicenses UP-TO-DATE
> Task :sdks:go:container:dockerClean
> Task :sdks:go:container:goPrepare
Use project GOPATH:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/go/container/.gogradle/project_gopath>
> Task :sdks:go:container:resolveBuildDependencies SKIPPED
> Task :sdks:go:container:installDependencies SKIPPED
> Task :sdks:go:container:buildLinuxAmd64 UP-TO-DATE
> Task :sdks:go:container:goBuild
> Task :sdks:go:container:skipPullLicenses
> Task :sdks:go:container:dockerPrepare
> Task :sdks:go:container:docker FAILED
The message received from the daemon indicates that the daemon has disappeared.
Build request sent: Build{id=33069fae-c869-4091-9814-f0ea500919aa,
currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src}>
Attempting to read last messages from the daemon log...
Daemon pid: 6974
log file: /home/jenkins/.gradle/daemon/6.9.1/daemon-6974.out.log
----- Last 20 lines from daemon log file - daemon-6974.out.log -----
2021-09-26T00:12:24.741+0000 [DEBUG]
[org.gradle.launcher.daemon.server.DefaultIncomingConnectionHandler] Starting
executing command: Build{id=33069fae-c869-4091-9814-f0ea500919aa,
currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src}>
with connection: socket connection from /127.0.0.1:40959 to /127.0.0.1:46108.
2021-09-26T00:12:24.748+0000 [ERROR]
[org.gradle.launcher.daemon.server.DaemonStateCoordinator] Command execution:
started DaemonCommandExecution[command =
Build{id=33069fae-c869-4091-9814-f0ea500919aa,
currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src},>
connection = DefaultDaemonConnection: socket connection from /127.0.0.1:40959
to /127.0.0.1:46108] after 5.0 minutes of idle
2021-09-26T00:12:24.749+0000 [INFO]
[org.gradle.launcher.daemon.server.DaemonRegistryUpdater] Marking the daemon as
busy, address: [74d780ae-c6af-435d-8866-e73566475d59 port:40959,
addresses:[localhost/127.0.0.1]]
2021-09-26T00:12:24.749+0000 [DEBUG]
[org.gradle.launcher.daemon.registry.PersistentDaemonRegistry] Marking busy by
address: [74d780ae-c6af-435d-8866-e73566475d59 port:40959,
addresses:[localhost/127.0.0.1]]
2021-09-26T00:12:24.749+0000 [DEBUG]
[org.gradle.cache.internal.DefaultFileLockManager] Waiting to acquire exclusive
lock on daemon addresses registry.
2021-09-26T00:12:24.749+0000 [DEBUG]
[org.gradle.cache.internal.DefaultFileLockManager] Lock acquired on daemon
addresses registry.
2021-09-26T00:12:24.750+0000 [DEBUG]
[org.gradle.cache.internal.DefaultFileLockManager] Releasing lock on daemon
addresses registry.
2021-09-26T00:12:24.750+0000 [DEBUG]
[org.gradle.launcher.daemon.server.DaemonStateCoordinator] resetting idle timer
2021-09-26T00:12:24.751+0000 [DEBUG]
[org.gradle.launcher.daemon.server.DaemonStateCoordinator] daemon is running.
Sleeping until state changes.
2021-09-26T00:12:24.751+0000 [INFO]
[org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy] Daemon is
about to start building Build{id=33069fae-c869-4091-9814-f0ea500919aa,
currentDir=<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src}.>
Dispatching build started information...
2021-09-26T00:12:24.751+0000 [DEBUG]
[org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 252:
dispatching org.gradle.launcher.daemon.protocol.BuildStarted@6c654c3e
2021-09-26T00:12:24.752+0000 [DEBUG]
[org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment] Configuring
env variables: [PATH, RUN_DISPLAY_URL, HUDSON_HOME, RUN_CHANGES_DISPLAY_URL,
JOB_URL, HUDSON_COOKIE, MAIL, JENKINS_SERVER_COOKIE, LOGNAME, PWD,
RUN_TESTS_DISPLAY_URL, JENKINS_URL, SHELL, BUILD_TAG, ROOT_BUILD_CAUSE,
BUILD_CAUSE_TIMERTRIGGER, OLDPWD, GIT_CHECKOUT_DIR, JENKINS_HOME, sha1,
CODECOV_TOKEN, NODE_NAME, BUILD_DISPLAY_NAME, JOB_DISPLAY_URL, GIT_BRANCH,
SETUPTOOLS_USE_DISTUTILS, SHLVL, WORKSPACE_TMP, GIT_PREVIOUS_COMMIT, JAVA_HOME,
BUILD_ID, LANG, XDG_SESSION_ID, JOB_NAME, SPARK_LOCAL_IP, BUILD_CAUSE,
GIT_PREVIOUS_SUCCESSFUL_COMMIT, NODE_LABELS, HUDSON_URL, WORKSPACE,
ROOT_BUILD_CAUSE_TIMERTRIGGER, _, GIT_COMMIT, COVERALLS_REPO_TOKEN, CI,
EXECUTOR_NUMBER, HUDSON_SERVER_COOKIE, SSH_CLIENT, JOB_BASE_NAME, USER,
SSH_CONNECTION, BUILD_NUMBER, BUILD_URL, RUN_ARTIFACTS_DISPLAY_URL, GIT_URL,
XDG_RUNTIME_DIR, HOME]
2021-09-26T00:12:24.753+0000 [DEBUG]
[org.gradle.launcher.daemon.server.exec.LogToClient] About to start relaying
all logs to the client via the connection.
2021-09-26T00:12:24.753+0000 [INFO]
[org.gradle.launcher.daemon.server.exec.LogToClient] The client will now
receive all logging from the daemon (pid: 6974). The daemon log file:
/home/jenkins/.gradle/daemon/6.9.1/daemon-6974.out.log
2021-09-26T00:12:24.760+0000 [INFO]
[org.gradle.launcher.daemon.server.exec.LogAndCheckHealth] Starting 2nd build
in daemon [uptime: 6 mins 9.603 secs, performance: 98%]
2021-09-26T00:12:24.760+0000 [DEBUG]
[org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has started
executing the build.
2021-09-26T00:12:24.760+0000 [DEBUG]
[org.gradle.launcher.daemon.server.exec.ExecuteBuild] Executing build with
daemon context:
DefaultDaemonContext[uid=7be4888f-f6dc-4678-ac84-919c4c1365d8,javaHome=/usr/lib/jvm/java-8-openjdk-amd64,daemonRegistryDir=/home/jenkins/.gradle/daemon,pid=6974,idleTimeout=10800000,priority=NORMAL,daemonOpts=-Xss10240k,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]
Configuration on demand is an incubating feature.
Use project GOPATH:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/go/container/.gogradle/project_gopath>
Daemon vm is shutting down... The daemon has exited normally or was terminated
in response to a user interrupt.
----- End of the daemon log -----
FAILURE: Build failed with an exception.
* What went wrong:
Gradle build daemon disappeared unexpectedly (it may have been killed or may
have crashed)
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
exit_background_processes
> Task :runners:flink:1.13:job-server:validatesCrossLanguageRunnerGoUsingJava
> FAILED
> Task :runners:flink:1.13:job-server:validatesCrossLanguageRunnerJavaUsingJava
> Task
> :runners:flink:1.13:job-server:validatesCrossLanguageRunnerJavaUsingPython
> Task
> :runners:flink:1.13:job-server:validatesCrossLanguageRunnerJavaUsingPythonOnly
> Task
> :runners:flink:1.13:job-server:validatesCrossLanguageRunnerPythonUsingJava
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:40171 --environment_cache_millis=10000
>>> --experiments=beam_fn_api --parallelism=2
>>> pytest options:
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 5013 items / 5005 deselected / 8 selected
apache_beam/io/external/generate_sequence_test.py . [ 12%]
apache_beam/transforms/validate_runner_xlang_test.py ....... [100%]
=============================== warnings summary ===============================
apache_beam/io/filesystems_test.py:54
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:54:
DeprecationWarning: invalid escape sequence \c
self.assertIsNone(FileSystems.get_scheme('c:\\abc\cdf')) # pylint:
disable=anomalous-backslash-in-string
apache_beam/io/filesystems_test.py:62
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python/apache_beam/io/filesystems_test.py>:62:
DeprecationWarning: invalid escape sequence \d
self.assertTrue(isinstance(FileSystems.get_filesystem('c:\\abc\def'), #
pylint: disable=anomalous-backslash-in-string
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python/pytest_xlangValidateRunner.xml>
-
============ 8 passed, 5005 deselected, 2 warnings in 84.68 seconds ============
> Task
> :runners:flink:1.13:job-server:validatesCrossLanguageRunnerPythonUsingPython
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:40171 --environment_cache_millis=10000
>>> --experiments=beam_fn_api --parallelism=2
>>> pytest options:
>>> collect markers: -m=xlang_transforms
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 5013 items / 5005 deselected / 8 selected
apache_beam/io/external/generate_sequence_test.py . [ 12%]
apache_beam/transforms/validate_runner_xlang_test.py ....... [100%]
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python/pytest_xlangValidateRunner.xml>
-
================== 8 passed, 5005 deselected in 46.90 seconds ==================
> Task :runners:flink:1.13:job-server:validatesCrossLanguageRunnerPythonUsingSql
>>> RUNNING integration tests with pipeline options: --runner=PortableRunner
>>> --job_endpoint=localhost:40171 --environment_cache_millis=10000
>>> --experiments=beam_fn_api --parallelism=2
>>> pytest options:
>>> collect markers: -m=xlang_sql_expansion_service
============================= test session starts ==============================
platform linux -- Python 3.6.8, pytest-4.6.11, py-1.10.0, pluggy-0.13.1
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 5013 items / 5004 deselected / 9 selected
apache_beam/transforms/sql_test.py ......... [100%]
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/python/pytest_xlangSqlValidateRunner.xml>
-
================= 9 passed, 5004 deselected in 164.99 seconds ==================
> Task :runners:flink:1.13:job-server:validatesCrossLanguageRunnerCleanup
Stopping expansion service pid: 9037.
Stopping expansion service pid: 9040.
> Task :runners:flink:1.13:job-server:flinkJobServerCleanup
Stopping job server pid: 15636.
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Flink/ws/src/sdks/go/test/build.gradle'>
line: 198
* What went wrong:
Execution failed for task
':runners:flink:1.13:job-server:validatesCrossLanguageRunnerGoUsingJava'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 7.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/6.9.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 26m 58s
221 actionable tasks: 162 executed, 52 from cache, 7 up-to-date
Publishing build scan...
https://gradle.com/s/higbostohcgr6
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
