See <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/5366/display/redirect?page=changes>
Changes: [thiagotnunes] [BEAM-12164] Add Spanner Change Stream DAOs [noreply] [BEAM-13218] Sickbay [noreply] [BEAM-13399] Add infrastructure to start JARs from Go functions (#16214) ------------------------------------------ Started by timer Running as SYSTEM [EnvInject] - Loading node environment variables. Building remotely on apache-beam-jenkins-13 (beam) in workspace <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/> The recommended git tool is: NONE No credentials specified Wiping out workspace first. Cloning the remote Git repository Cloning repository https://github.com/apache/beam.git > git init > <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src> > # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git --version # timeout=10 > git --version # 'git version 2.7.4' > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # > timeout=10 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10 Fetching upstream changes from https://github.com/apache/beam.git > git fetch --tags --progress https://github.com/apache/beam.git > +refs/heads/*:refs/remotes/origin/* > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/* # > timeout=10 > git rev-parse origin/master^{commit} # timeout=10 Checking out Revision 759da419c2c43e5abce41806ae5543744f0b4eb1 (origin/master) > git config core.sparsecheckout # timeout=10 > git checkout -f 759da419c2c43e5abce41806ae5543744f0b4eb1 # timeout=10 Commit message: "Merge pull request #16124 from [BEAM-12164] Add Spanner Change Stream DAOs" > git rev-list --no-walk 7e27e0f5eca1441e0c7b653703cf86ec15cc2e87 # timeout=10 No emails were triggered. [EnvInject] - Executing scripts and injecting environment variables after the SCM step. [EnvInject] - Injecting as environment variables the properties content SPARK_LOCAL_IP=127.0.0.1 SETUPTOOLS_USE_DISTUTILS=stdlib [EnvInject] - Variables injected successfully. [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5890900302579471449.sh + cp /home/jenkins/.kube/config <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> [EnvInject] - Variables injected successfully. [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins5906178661822207559.sh + gcloud container clusters get-credentials io-datastores --zone=us-central1-a Fetching cluster endpoint and auth data. kubeconfig entry generated for io-datastores. [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins1934635023427463131.sh + <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/kubernetes.sh> createNamespace beam-performancetests-avroioit-hdfs-5366 + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> + KUBERNETES_NAMESPACE=default + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=default' + createNamespace beam-performancetests-avroioit-hdfs-5366 + eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> create namespace beam-performancetests-avroioit-hdfs-5366' ++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> create namespace beam-performancetests-avroioit-hdfs-5366 namespace/beam-performancetests-avroioit-hdfs-5366 created [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties content KUBERNETES_NAMESPACE=beam-performancetests-avroioit-hdfs-5366 [EnvInject] - Variables injected successfully. [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8383974550835036240.sh + <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/kubernetes.sh> apply <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml> + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> + KUBERNETES_NAMESPACE=beam-performancetests-avroioit-hdfs-5366 + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366' + apply <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml> + eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 apply -R -f <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml'> ++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 apply -R -f <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/hadoop/LargeITCluster/hdfs-multi-datanode-cluster.yml> service/hadoop created service/hadoop-datanodes created statefulset.apps/datanode created pod/namenode-0 created [beam_PerformanceTests_AvroIOIT_HDFS] $ /bin/bash -xe /tmp/jenkins8909233611034341313.sh + set -eo pipefail + sed 's/^/LOAD_BALANCER_IP=/' + eval <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP hadoop ++ <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP hadoop + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> + KUBERNETES_NAMESPACE=beam-performancetests-avroioit-hdfs-5366 + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366' + loadBalancerIP hadoop + local name=hadoop + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' + retry 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' 36 10 + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' + local max_retries=36 + local sleep_time=10 + (( i = 1 )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output= + local status=0 + [[ 0 == 0 ]] + [[ -n '' ]] + [[ 1 == \3\6 ]] + sleep 10 + (( i++ )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output= + local status=0 + [[ 0 == 0 ]] + [[ -n '' ]] + [[ 2 == \3\6 ]] + sleep 10 + (( i++ )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output= + local status=0 + [[ 0 == 0 ]] + [[ -n '' ]] + [[ 3 == \3\6 ]] + sleep 10 + (( i++ )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output= + local status=0 + [[ 0 == 0 ]] + [[ -n '' ]] + [[ 4 == \3\6 ]] + sleep 10 + (( i++ )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/config-beam-performancetests-avroioit-hdfs-5366> --namespace=beam-performancetests-avroioit-hdfs-5366 get svc hadoop '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output=35.202.127.160 + local status=0 + [[ 0 == 0 ]] + [[ -n 35.202.127.160 ]] + echo 35.202.127.160 + return 0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'job.properties' [EnvInject] - Variables injected successfully. [Gradle] - Launching build. [src] $ <https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/gradlew> --continue --max-****s=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -Dorg.gradle.vfs.watch=false -Pdocker-pull-licenses --info -DintegrationTestPipelineOptions=["--bigQueryDataset=beam_performance","--bigQueryTable=avroioit_hdfs_results","--influxMeasurement=avroioit_hdfs_results","--numberOfRecords=225000000","--expectedHash=2f9f5ca33ea464b25109c0297eb6aecb","--datasetSize=1089730000","--numWorkers=5","--autoscalingAlgorithm=NONE","--influxDatabase=beam_test_metrics","--influxHost=http://10.128.0.96:8086","--runner=DataflowRunner","--project=apache-beam-testing","--tempRoot=gs://temp-storage-for-perf-tests","--hdfsConfiguration=[{\"fs.defaultFS\":\"hdfs:35.202.127.160:9000\",\"dfs.replication\":1}]","--filenamePrefix=hdfs://35.202.127.160:9000/TEXTIO_IT_";] -Dfilesystem=hdfs -DintegrationTestRunner=dataflow :sdks:java:io:file-based-io-tests:integrationTest --tests org.apache.beam.sdk.io.avro.AvroIOIT Initialized native services in: /home/jenkins/.gradle/native Found daemon DaemonInfo{pid=28511, address=[6b9736cb-eb46-4907-8e03-b0032782ba38 port:45627, addresses:[/127.0.0.1]], state=Idle, lastBusy=1639635261640, context=DefaultDaemonContext[uid=aab58436-489a-436d-8b7a-5199fba9f895,javaHome=/usr/lib/jvm/java-8-openjdk-amd64,daemonRegistryDir=/home/jenkins/.gradle/daemon,pid=28511,idleTimeout=10800000,priority=NORMAL,daemonOpts=-Xss10240k,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant]} however its context does not match the desired criteria. At least one daemon option is different. Wanted: DefaultDaemonContext[uid=null,javaHome=/usr/lib/jvm/java-8-openjdk-amd64,daemonRegistryDir=/home/jenkins/.gradle/daemon,pid=23843,idleTimeout=null,priority=NORMAL,daemonOpts=-Xmx4g,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant] Actual: DefaultDaemonContext[uid=aab58436-489a-436d-8b7a-5199fba9f895,javaHome=/usr/lib/jvm/java-8-openjdk-amd64,daemonRegistryDir=/home/jenkins/.gradle/daemon,pid=28511,idleTimeout=10800000,priority=NORMAL,daemonOpts=-Xss10240k,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant] Looking for a different daemon... The client will now receive all logging from the daemon (pid: 32208). The daemon log file: /home/jenkins/.gradle/daemon/6.9.1/daemon-32208.out.log Starting 2nd build in daemon [uptime: 19 mins 59.803 secs, performance: 100%, GC rate: 0.05/s, heap usage: 1% of 2.6 GiB] Using 12 **** leases. Closing daemon's stdin at end of input. The daemon will no longer process any standard input. Watching the file system is disabled Starting Build Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.9.1/fileHashes/fileHashes.bin Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.9.1/fileHashes/resourceHashesCache.bin Settings evaluated using settings file '<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/settings.gradle.kts'.> The message received from the daemon indicates that the daemon has disappeared. Build request sent: Build{id=b357540f-e07c-4c20-b21b-961fd016dd1d, currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src}> Attempting to read last messages from the daemon log... Daemon pid: 32208 log file: /home/jenkins/.gradle/daemon/6.9.1/daemon-32208.out.log ----- Last 20 lines from daemon log file - daemon-32208.out.log ----- 2021-12-16T06:20:30.980+0000 [INFO] [org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy] Daemon is about to start building Build{id=b357540f-e07c-4c20-b21b-961fd016dd1d, currentDir=<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src}.> Dispatching build started information... 2021-12-16T06:20:30.981+0000 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 27: dispatching org.gradle.launcher.daemon.protocol.BuildStarted@6145ca14 2021-12-16T06:20:30.985+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment] Configuring env variables: [PATH, INFLUXDB_USER, LOAD_BALANCER_IP, RUN_DISPLAY_URL, KUBECONFIG, HUDSON_HOME, RUN_CHANGES_DISPLAY_URL, JOB_URL, HUDSON_COOKIE, MAIL, JENKINS_SERVER_COOKIE, LOGNAME, PWD, RUN_TESTS_DISPLAY_URL, JENKINS_URL, SHELL, BUILD_TAG, ROOT_BUILD_CAUSE, BUILD_CAUSE_TIMERTRIGGER, OLDPWD, GIT_CHECKOUT_DIR, JENKINS_HOME, sha1, NODE_NAME, BUILD_DISPLAY_NAME, JOB_DISPLAY_URL, GIT_BRANCH, SETUPTOOLS_USE_DISTUTILS, KUBERNETES_NAMESPACE, SHLVL, WORKSPACE_TMP, GIT_PREVIOUS_COMMIT, INFLUXDB_USER_PASSWORD, JAVA_HOME, BUILD_ID, LANG, XDG_SESSION_ID, JOB_NAME, SPARK_LOCAL_IP, BUILD_CAUSE, GIT_PREVIOUS_SUCCESSFUL_COMMIT, NODE_LABELS, HUDSON_URL, WORKSPACE, ROOT_BUILD_CAUSE_TIMERTRIGGER, _, GIT_COMMIT, CI, EXECUTOR_NUMBER, HUDSON_SERVER_COOKIE, SSH_CLIENT, JOB_BASE_NAME, USER, SSH_CONNECTION, BUILD_NUMBER, BUILD_URL, RUN_ARTIFACTS_DISPLAY_URL, GIT_URL, XDG_RUNTIME_DIR, HOME] 2021-12-16T06:20:30.986+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.LogToClient] About to start relaying all logs to the client via the connection. 2021-12-16T06:20:30.986+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogToClient] The client will now receive all logging from the daemon (pid: 32208). The daemon log file: /home/jenkins/.gradle/daemon/6.9.1/daemon-32208.out.log 2021-12-16T06:20:30.997+0000 [INFO] [org.gradle.launcher.daemon.server.exec.LogAndCheckHealth] Starting 2nd build in daemon [uptime: 19 mins 59.803 secs, performance: 100%, GC rate: 0.05/s, heap usage: 1% of 2.6 GiB] 2021-12-16T06:20:31.006+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] The daemon has started executing the build. 2021-12-16T06:20:31.007+0000 [DEBUG] [org.gradle.launcher.daemon.server.exec.ExecuteBuild] Executing build with daemon context: DefaultDaemonContext[uid=ed9da04c-c0fe-4c8f-9d9a-815cca427bd8,javaHome=/usr/lib/jvm/java-8-openjdk-amd64,daemonRegistryDir=/home/jenkins/.gradle/daemon,pid=32208,idleTimeout=10800000,priority=NORMAL,daemonOpts=-Xmx4g,-Dfile.encoding=UTF-8,-Duser.country=US,-Duser.language=en,-Duser.variant] 2021-12-16T06:20:31.016+0000 [DEBUG] [org.gradle.launcher.daemon.server.SynchronizedDispatchConnection] thread 278: received class org.gradle.launcher.daemon.protocol.CloseInput 2021-12-16T06:20:31.017+0000 [DEBUG] [org.gradle.launcher.daemon.server.DefaultDaemonConnection] thread 278: Received IO message from client: org.gradle.launcher.daemon.protocol.CloseInput@6bd898f1 Using 12 **** leases. Closing daemon's stdin at end of input. The daemon will no longer process any standard input. Watching the file system is disabled Starting Build Invalidating in-memory cache of /home/jenkins/.gradle/caches/journal-1/file-access.bin Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.9.1/fileHashes/fileHashes.bin Invalidating in-memory cache of /home/jenkins/.gradle/caches/6.9.1/fileHashes/resourceHashesCache.bin Daemon vm is shutting down... The daemon has exited normally or was terminated in response to a user interrupt. Settings evaluated using settings file '<https://ci-beam.apache.org/job/beam_PerformanceTests_AvroIOIT_HDFS/ws/src/settings.gradle.kts'.> ----- End of the daemon log ----- FAILURE: Build failed with an exception. * What went wrong: Gradle build daemon disappeared unexpectedly (it may have been killed or may have crashed) * Try: Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
