See <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/3203/display/redirect?page=changes>
Changes: [Moritz Mack] Minor improvements to the tpcds gradle build for Spark [Moritz Mack] Fix SparkSessionFactory to not fail when using Spark master local[*] [Moritz Mack] [Spark dataset runner] Add direct translation of Reshuffle and ------------------------------------------ [...truncated 9.44 KB...] + IFS= + read -r usedPort + '[' 32402 = 30658 ']' + IFS= + read -r usedPort + '[' 32402 = 31888 ']' + IFS= + read -r usedPort + '[' 32402 = 30146 ']' + IFS= + read -r usedPort + '[' 32402 = 30114 ']' + IFS= + read -r usedPort + '[' 32402 = 31977 ']' + IFS= + read -r usedPort + false + echo 32402 + return 0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'job.properties' [EnvInject] - Variables injected successfully. [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins5823471682213544666.sh + set -xo pipefail + eval <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> getAvailablePort 32402 32767 ++ <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> getAvailablePort 32402 32767 + sed 's/^/KAFKA_SERVICE_PORT_2=/' + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> + KUBERNETES_NAMESPACE=beam-performancetests-kafka-io-3203 + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203' + getAvailablePort 32402 32767 + local lowRangePort=32403 + local highRangePort=32767 + local used=false + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc --all-namespaces -o go-template='\''{{range .items}}{{range.spec.ports}}{{if .nodePort}}{{.nodePort}}{{"\n"}}{{end}}{{end}}{{end}}'\''' + local usedPorts ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc --all-namespaces -o go-template='\''{{range .items}}{{range.spec.ports}}{{if .nodePort}}{{.nodePort}}{{"\n"}}{{end}}{{end}}{{end}}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc --all-namespaces -o 'go-template={{range .items}}{{range.spec.ports}}{{if .nodePort}}{{.nodePort}}{{"\n"}}{{end}}{{end}}{{end}}' + usedPorts='31620 30382 30763 30658 31888 30146 30114 31977' + local availablePort=32403 ++ seq 32403 32767 + for i in $(seq $lowRangePort $highRangePort) + IFS= + read -r usedPort + '[' 32403 = 31620 ']' + IFS= + read -r usedPort + '[' 32403 = 30382 ']' + IFS= + read -r usedPort + '[' 32403 = 30763 ']' + IFS= + read -r usedPort + '[' 32403 = 30658 ']' + IFS= + read -r usedPort + '[' 32403 = 31888 ']' + IFS= + read -r usedPort + '[' 32403 = 30146 ']' + IFS= + read -r usedPort + '[' 32403 = 30114 ']' + IFS= + read -r usedPort + '[' 32403 = 31977 ']' + IFS= + read -r usedPort + false + echo 32403 + return 0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'job.properties' [EnvInject] - Variables injected successfully. [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins6300048641536491082.sh + sed -i -e s/32400/32401/ <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kafka-cluster/04-outside-services/outside-0.yml> [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins7998196093178349363.sh + sed -i -e s/32401/32402/ <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kafka-cluster/04-outside-services/outside-1.yml> [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins1852220025147084652.sh + sed -i -e s/32402/32403/ <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kafka-cluster/04-outside-services/outside-2.yml> [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins6458049539922874791.sh + <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> apply <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kafka-cluster> + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> + KUBERNETES_NAMESPACE=beam-performancetests-kafka-io-3203 + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203' + apply <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kafka-cluster> + eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 apply -R -f <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kafka-cluster'> ++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 apply -R -f <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kafka-cluster> storageclass.storage.k8s.io/kafka-broker unchanged storageclass.storage.k8s.io/kafka-zookeeper unchanged clusterrole.rbac.authorization.k8s.io/node-reader unchanged clusterrolebinding.rbac.authorization.k8s.io/kafka-node-reader configured role.rbac.authorization.k8s.io/pod-labler created rolebinding.rbac.authorization.k8s.io/kafka-pod-labler created configmap/zookeeper-config created service/pzoo created service/zookeeper created statefulset.apps/pzoo created service/outside-0 created service/outside-1 created service/outside-2 created configmap/broker-config created service/broker created service/bootstrap created statefulset.apps/kafka created configmap/kafka-config created job.batch/kafka-config-eff079ec created [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins5295686156657973394.sh + set -eo pipefail + sed 's/^/KAFKA_BROKER_0=/' + eval <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP outside-0 ++ <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP outside-0 + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> + KUBERNETES_NAMESPACE=beam-performancetests-kafka-io-3203 + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203' + loadBalancerIP outside-0 + local name=outside-0 + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' + retry 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' 36 10 + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' + local max_retries=36 + local sleep_time=10 + (( i = 1 )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output= + local status=0 + [[ 0 == 0 ]] + [[ -n '' ]] + [[ 1 == \3\6 ]] + sleep 10 + (( i++ )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output= + local status=0 + [[ 0 == 0 ]] + [[ -n '' ]] + [[ 2 == \3\6 ]] + sleep 10 + (( i++ )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output= + local status=0 + [[ 0 == 0 ]] + [[ -n '' ]] + [[ 3 == \3\6 ]] + sleep 10 + (( i++ )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-0 '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output=34.123.178.119 + local status=0 + [[ 0 == 0 ]] + [[ -n 34.123.178.119 ]] + echo 34.123.178.119 + return 0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'job.properties' [EnvInject] - Variables injected successfully. [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins622261877235563733.sh + set -eo pipefail + eval <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP outside-1 ++ <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP outside-1 + sed 's/^/KAFKA_BROKER_1=/' + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> + KUBERNETES_NAMESPACE=beam-performancetests-kafka-io-3203 + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203' + loadBalancerIP outside-1 + local name=outside-1 + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-1 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' + retry 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-1 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' 36 10 + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-1 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' + local max_retries=36 + local sleep_time=10 + (( i = 1 )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-1 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-1 '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output=34.171.71.28 + local status=0 + [[ 0 == 0 ]] + [[ -n 34.171.71.28 ]] + echo 34.171.71.28 + return 0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'job.properties' [EnvInject] - Variables injected successfully. [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins8635067715365866544.sh + set -eo pipefail + eval <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP outside-2 ++ <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> loadBalancerIP outside-2 + sed 's/^/KAFKA_BROKER_2=/' + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> + KUBERNETES_NAMESPACE=beam-performancetests-kafka-io-3203 + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203' + loadBalancerIP outside-2 + local name=outside-2 + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-2 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' + retry 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-2 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' 36 10 + local 'command=kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-2 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' + local max_retries=36 + local sleep_time=10 + (( i = 1 )) + (( i <= max_retries )) + local output ++ eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-2 -ojsonpath='\''{.status.loadBalancer.ingress[0].ip}'\''' +++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 get svc outside-2 '-ojsonpath={.status.loadBalancer.ingress[0].ip}' + output=35.232.195.239 + local status=0 + [[ 0 == 0 ]] + [[ -n 35.232.195.239 ]] + echo 35.232.195.239 + return 0 [EnvInject] - Injecting environment variables from a build step. [EnvInject] - Injecting as environment variables the properties file path 'job.properties' [EnvInject] - Variables injected successfully. [beam_PerformanceTests_Kafka_IO] $ /bin/bash -xe /tmp/jenkins1475685251868431296.sh + eval <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> waitForJob job.batch/kafka-config-eff079ec 40m ++ <https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/src/.test-infra/kubernetes/kubernetes.sh> waitForJob job.batch/kafka-config-eff079ec 40m + KUBECONFIG=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> + KUBERNETES_NAMESPACE=beam-performancetests-kafka-io-3203 + KUBECTL='kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203' + waitForJob job.batch/kafka-config-eff079ec 40m + echo 'Waiting for job completion...' Waiting for job completion... + jobName=job.batch/kafka-config-eff079ec + eval 'kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 wait --for=condition=complete --timeout=40m job.batch/kafka-config-eff079ec' ++ kubectl --kubeconfig=<https://ci-beam.apache.org/job/beam_PerformanceTests_Kafka_IO/ws/config-beam-performancetests-kafka-io-3203> --namespace=beam-performancetests-kafka-io-3203 wait --for=condition=complete --timeout=40m job.batch/kafka-config-eff079ec error: timed out waiting for the condition on jobs/kafka-config-eff079ec Build step 'Execute shell' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
