See
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/2295/display/redirect?page=changes>
Changes:
[noreply] Migrate examples and maven-archetypes (including Java Quickstart) to
[Moritz Mack] Update remaining pointers to Spark runner to Spark 3 module
(addresses
------------------------------------------
[...truncated 1.29 MB...]
Collecting mypy-protobuf==1.18
Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB)
Collecting distlib==0.3.1
Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Collecting numpy<1.23.0,>=1.14.3
Downloading
numpy-1.22.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8
MB)
Collecting pluggy>=0.12.0
Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting py>=1.4.17
Using cached py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting packaging>=14
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting
virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
Downloading virtualenv-20.13.2-py2.py3-none-any.whl (8.7 MB)
Collecting filelock>=3.0.0
Downloading filelock-3.4.2-py3-none-any.whl (9.9 kB)
Collecting toml>=0.9.4
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting six>=1.14.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting grpcio>=1.37.0
Downloading
grpcio-1.43.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.1
MB)
Collecting protobuf<4.0dev,>=3.5.0.post1
Downloading
protobuf-3.19.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.1
MB)
Collecting pyparsing!=3.0.5,>=2.0.2
Downloading pyparsing-3.0.6-py3-none-any.whl (97 kB)
Collecting platformdirs<3,>=2
Downloading platformdirs-2.4.1-py3-none-any.whl (14 kB)
Installing collected packages: six, pyparsing, platformdirs, filelock, distlib,
virtualenv, toml, setuptools, py, protobuf, pluggy, packaging, grpcio, wheel,
tox, numpy, mypy-protobuf, grpcio-tools
Attempting uninstall: setuptools
Found existing installation: setuptools 58.1.0
Uninstalling setuptools-58.1.0:
Successfully uninstalled setuptools-58.1.0
Successfully installed distlib-0.3.1 filelock-3.4.2 grpcio-1.43.0
grpcio-tools-1.37.0 mypy-protobuf-1.18 numpy-1.22.2 packaging-21.3
platformdirs-2.4.1 pluggy-1.0.0 protobuf-3.19.1 py-1.11.0 pyparsing-3.0.6
setuptools-60.9.3 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.13.2
wheel-0.37.1
WARNING: You are using pip version 21.3.1; however, version 22.3 is available.
You should consider upgrading via the
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/bin/python3.10>
-m pip install --upgrade pip' command.
> Task :sdks:python:sdist
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.10/site-packages/setuptools/dist.py>:505:
UserWarning: Normalizing '2.44.0.dev' to '2.44.0.dev0'
warnings.warn(tmpl.format(**locals()))
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
warning: no files found matching 'LICENSE.python'
warning: sdist: standard file not found: should have one of README, README.rst,
README.txt, README.md
> Task :sdks:java:container:pullLicenses
INFO:root:pull_licenses_java.py succeed. It took 12.250064 seconds with 16
threads.
Copying licenses from
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/container/build/target/java_third_party_licenses>
to
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/container/build/target/third_party_licenses.>
Finished license_scripts.sh
> Task :sdks:java:container:goPrepare UP-TO-DATE
> Task :sdks:python:container:py310:copyDockerfileDependencies
> Task :sdks:java:container:java8:copyJavaThirdPartyLicenses
> Task :sdks:python:container:py310:dockerPrepare
> Task :sdks:java:container:goBuild
/home/jenkins/go/bin/go1.18.1 build -o ./build/target/linux_amd64/boot boot.go
boot_test.go
> Task :sdks:java:container:java8:copySdkHarnessLauncher
Execution optimizations have been disabled for task
':sdks:java:container:java8:copySdkHarnessLauncher' to ensure correctness due
to the following reasons:
- Gradle detected a problem with the following location:
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/container/build/target'.>
Reason: Task ':sdks:java:container:java8:copySdkHarnessLauncher' uses this
output of task ':sdks:java:container:downloadCloudProfilerAgent' without
declaring an explicit or implicit dependency. This can lead to incorrect
results being produced, depending on what order the tasks are executed. Please
refer to
https://docs.gradle.org/7.5.1/userguide/validation_problems.html#implicit_dependency
for more details about this problem.
- Gradle detected a problem with the following location:
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/sdks/java/container/build/target'.>
Reason: Task ':sdks:java:container:java8:copySdkHarnessLauncher' uses this
output of task ':sdks:java:container:pullLicenses' without declaring an
explicit or implicit dependency. This can lead to incorrect results being
produced, depending on what order the tasks are executed. Please refer to
https://docs.gradle.org/7.5.1/userguide/validation_problems.html#implicit_dependency
for more details about this problem.
> Task :sdks:python:installGcpTest
Processing ./build/apache-beam.tar.gz
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Requirement already satisfied: protobuf<4,>3.12.2 in
<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/lib/python3.10/site-packages>
(from apache-beam==2.44.0.dev0) (3.19.1)
Collecting crcmod<2.0,>=1.7
Using cached crcmod-1.7-cp310-cp310-linux_x86_64.whl
> Task :sdks:java:container:java8:dockerPrepare
> Task :sdks:python:container:py310:docker
Sending build context to Docker daemon 16.78MB
Step 1/32 : ARG py_version
Step 2/32 : FROM python:"${py_version}"-bullseye as beam
---> f05c8762fe15
Step 3/32 : MAINTAINER "Apache Beam <[email protected]>"
> Task :sdks:python:installGcpTest FAILED
ERROR: Could not find a version that satisfies the requirement
cloudpickle~=2.2.0 (from apache-beam[aws,azure,dataframe,gcp,test]) (from
versions: 0.1.0, 0.1.1, 0.2.1, 0.2.2, 0.3.0, 0.3.1, 0.4.0, 0.4.1, 0.4.2, 0.4.3,
0.4.4, 0.5.0, 0.5.1, 0.5.2, 0.5.3, 0.5.4, 0.5.5, 0.5.6, 0.6.0, 0.6.1, 0.7.0,
0.8.0, 0.8.1, 1.0.0, 1.1.1, 1.2.0, 1.2.1, 1.2.2, 1.3.0, 1.4.0, 1.4.1, 1.5.0,
1.6.0, 2.0.0)
ERROR: No matching distribution found for cloudpickle~=2.2.0
WARNING: You are using pip version 21.3.1; however, version 22.3 is available.
You should consider upgrading via the
'<https://ci-beam.apache.org/job/beam_PostCommit_XVR_Spark3/ws/src/build/gradleenv/1922375555/bin/python3.10>
-m pip install --upgrade pip' command.
> Task :runners:spark:3:job-server:sparkJobServerCleanup
Stopping job server pid: 3959144.
> Task :sdks:java:container:java8:docker
Sending build context to Docker daemon 209MB
Step 1/21 : ARG java_version
Step 2/21 : FROM openjdk:${java_version}-bullseye
---> b273004037cc
Step 3/21 : MAINTAINER "Apache Beam <[email protected]>"
---> Running in 00d11960ebba
Removing intermediate container 00d11960ebba
---> 3f78c34da2e2
Step 4/21 : ARG pull_licenses
---> Running in b0f7400f5bef
Removing intermediate container b0f7400f5bef
---> fb3fff8b2715
Step 5/21 : ADD target/slf4j-api.jar /opt/apache/beam/jars/
---> cf015cd12726
Step 6/21 : ADD target/slf4j-jdk14.jar /opt/apache/beam/jars/
---> 59ee05aa91e8
Step 7/21 : ADD target/beam-sdks-java-harness.jar /opt/apache/beam/jars/
---> 6338adc9c3ab
Step 8/21 : ADD target/beam-sdks-java-io-kafka.jar /opt/apache/beam/jars/
---> 2b2369efdd20
Step 9/21 : ADD target/kafka-clients.jar /opt/apache/beam/jars/
---> c534295daf0d
Step 10/21 : COPY target/jamm.jar target/open-module-agent*.jar
/opt/apache/beam/jars/
---> 831a9afc463e
Step 11/21 : ADD target/linux_amd64/boot /opt/apache/beam/
---> 20fd5b80023e
Step 12/21 : COPY target/LICENSE /opt/apache/beam/
---> 0f21164a9c00
Step 13/21 : COPY target/NOTICE /opt/apache/beam/
---> e26c05f1fe30
Step 14/21 : ADD target/third_party_licenses
/opt/apache/beam/third_party_licenses/
---> a82321c1d1d0
Step 15/21 : COPY target/LICENSE target/options/* /opt/apache/beam/options/
---> f0c75bb5cfe9
Step 16/21 : RUN rm /opt/apache/beam/options/LICENSE
---> Running in eaa0ca017ca2
Removing intermediate container eaa0ca017ca2
---> a0ba9adc818c
Step 17/21 : COPY target/LICENSE target/go-licenses/*
/opt/apache/beam/third_party_licenses/golang/
---> fd793ee719de
Step 18/21 : RUN rm /opt/apache/beam/third_party_licenses/golang/LICENSE
---> Running in b95a2c29c951
Removing intermediate container b95a2c29c951
---> 171f42a3bc46
Step 19/21 : RUN if [ "${pull_licenses}" = "false" ] ; then rm -rf
/opt/apache/beam/third_party_licenses ; fi
---> Running in 6aac722b68e7
Removing intermediate container 6aac722b68e7
---> 75e0ca165b6e
Step 20/21 : COPY target/profiler/* /opt/google_cloud_profiler/
---> 5247ad5d2f35
Step 21/21 : ENTRYPOINT ["/opt/apache/beam/boot"]
---> Running in 61187a9c1137
Removing intermediate container 61187a9c1137
---> c349f402eb96
Successfully built c349f402eb96
Successfully tagged apache/beam_java8_sdk:2.44.0.dev
> Task :sdks:python:container:py310:docker
---> Running in 92fdb1784cc8
Removing intermediate container 92fdb1784cc8
---> 116677bf99a9
Step 4/32 : RUN apt-get update && apt-get install -y libsnappy-dev
libyaml-dev ccache && rm -rf /var/lib/apt/lists/*
---> Running in ce044b4aa693
Get:1 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:2 http://deb.debian.org/debian-security bullseye-security InRelease [48.4
kB]
Get:3 http://deb.debian.org/debian bullseye-updates InRelease [44.1 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 Packages [8184 kB]
Get:5 http://deb.debian.org/debian-security bullseye-security/main amd64
Packages [193 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [14.6
kB]
Fetched 8599 kB in 2s (4582 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
libyaml-dev is already the newest version (0.2.2-1).
Suggested packages:
distcc | icecc
The following NEW packages will be installed:
ccache libsnappy-dev libsnappy1v5
0 upgraded, 3 newly installed, 0 to remove and 7 not upgraded.
Need to get 495 kB of archives.
After this operation, 1425 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 ccache amd64 4.2-1 [446
kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 libsnappy1v5 amd64
1.1.8-1 [17.9 kB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 libsnappy-dev amd64
1.1.8-1 [31.1 kB]
[91mdebconf: delaying package configuration, since apt-utils is not installed
[0mFetched 495 kB in 0s (1737 kB/s)
Selecting previously unselected package ccache.
(Reading database ... (Reading database ... 5%(Reading database ... 10%(Reading
database ... 15%(Reading database ... 20%(Reading database ... 25%(Reading
database ... 30%(Reading database ... 35%(Reading database ... 40%(Reading
database ... 45%(Reading database ... 50%(Reading database ... 55%(Reading
database ... 60%(Reading database ... 65%(Reading database ... 70%(Reading
database ... 75%(Reading database ... 80%(Reading database ... 85%(Reading
database ... 90%(Reading database ... 95%(Reading database ... 100%(Reading
database ... 23422 files and directories currently installed.)
Preparing to unpack .../ccache_4.2-1_amd64.deb ...
Unpacking ccache (4.2-1) ...
Selecting previously unselected package libsnappy1v5:amd64.
Preparing to unpack .../libsnappy1v5_1.1.8-1_amd64.deb ...
Unpacking libsnappy1v5:amd64 (1.1.8-1) ...
Selecting previously unselected package libsnappy-dev:amd64.
Preparing to unpack .../libsnappy-dev_1.1.8-1_amd64.deb ...
Unpacking libsnappy-dev:amd64 (1.1.8-1) ...
Setting up ccache (4.2-1) ...
Updating symlinks in /usr/lib/ccache ...
Setting up libsnappy1v5:amd64 (1.1.8-1) ...
Setting up libsnappy-dev:amd64 (1.1.8-1) ...
Processing triggers for libc-bin (2.31-13+deb11u4) ...
Removing intermediate container ce044b4aa693
---> 47635c702cb5
Step 5/32 : COPY target/base_image_requirements.txt
/tmp/base_image_requirements.txt
---> 931e1ce41483
Step 6/32 : RUN pip install --no-deps -r /tmp/base_image_requirements.txt
&& python -c "import nltk; nltk.download('stopwords')" && rm
/root/nltk_data/corpora/stopwords.zip && python -c "from
google.protobuf.internal import api_implementation; assert
api_implementation._default_implementation_type == 'cpp'; print ('Verified fast
protobuf used.')" && rm -rf /root/.cache/pip && rm -rf
/tmp/base_image_requirements.txt
---> Running in d1c09f861ecb
[91mERROR: Could not find a version that satisfies the requirement
absl-py==1.3.0 (from versions: 0.1.0, 0.1.1, 0.1.2, 0.1.3, 0.1.4, 0.1.5, 0.1.6,
0.1.7, 0.1.8, 0.1.9, 0.1.10, 0.1.11, 0.1.12, 0.1.13, 0.2.0, 0.2.1, 0.2.2,
0.3.0, 0.4.0, 0.4.1, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.8.0, 0.8.1, 0.9.0,
0.10.0, 0.11.0, 0.12.0, 0.13.0, 0.14.0, 0.14.1, 0.15.0, 1.0.0)
[0m[91mERROR: No matching distribution found for absl-py==1.3.0
[0mThe command '/bin/sh -c pip install --no-deps -r
/tmp/base_image_requirements.txt && python -c "import nltk;
nltk.download('stopwords')" && rm /root/nltk_data/corpora/stopwords.zip &&
python -c "from google.protobuf.internal import api_implementation; assert
api_implementation._default_implementation_type == 'cpp'; print ('Verified fast
protobuf used.')" && rm -rf /root/.cache/pip && rm -rf
/tmp/base_image_requirements.txt' returned a non-zero code: 1
> Task :sdks:python:container:py310:docker FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py310:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 1m 36s
242 actionable tasks: 26 executed, 216 up-to-date
Publishing build scan...
https://gradle.com/s/z5sjjr3slkpsg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
