See
<https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/5468/display/redirect?page=changes>
Changes:
[noreply] Open Allow and test pyarrow 8.x and 9.x (#22997)
[noreply] (BQ Python) Pass project field from options or parameter when writing
[noreply] Update python-machine-learning.md (#23209)
[noreply] Pin the version of cloudpickle to 2.1.x (#23120)
[noreply] Add streaming test for Write API sink (#21903)
[noreply] [Go SDK] Proto changes for timer param (#23216)
[noreply] Bump github.com/testcontainers/testcontainers-go in /sdks (#23201)
[noreply] Update to objsize to 0.5.2 which is under BSD-3 license (fixes #23096)
[noreply] Exclude insignificant whitespace from cloud object (#23217)
------------------------------------------
[...truncated 62.00 KB...]
Using cached grpcio_tools-1.37.0-cp37-cp37m-manylinux2014_x86_64.whl (2.5 MB)
Collecting mypy-protobuf==1.18
Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB)
Collecting distlib==0.3.1
Using cached distlib-0.3.1-py2.py3-none-any.whl (335 kB)
Collecting
virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
Using cached virtualenv-20.16.5-py3-none-any.whl (8.8 MB)
Collecting packaging>=14
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting toml>=0.9.4
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting filelock>=3.0.0
Using cached filelock-3.8.0-py3-none-any.whl (10 kB)
Collecting importlib-metadata<3,>=0.12
Using cached importlib_metadata-2.1.3-py2.py3-none-any.whl (10 kB)
Collecting py>=1.4.17
Using cached py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting six>=1.14.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting pluggy>=0.12.0
Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
> Task :sdks:java:extensions:sql:expansion-service:shadowJar FROM-CACHE
> Task :sdks:python:setupVirtualenv
Collecting packaging>=14
Using cached packaging-21.3-py3-none-any.whl (40 kB)
Collecting toml>=0.9.4
Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)
Collecting
virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
Using cached virtualenv-20.16.5-py3-none-any.whl (8.8 MB)
Collecting filelock>=3.0.0
Using cached filelock-3.8.0-py3-none-any.whl (10 kB)
Collecting pluggy>=0.12.0
Using cached pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting six>=1.14.0
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting py>=1.4.17
Using cached py-1.11.0-py2.py3-none-any.whl (98 kB)
Collecting protobuf<4.0dev,>=3.5.0.post1
Using cached protobuf-3.20.2-py2.py3-none-any.whl (162 kB)
Collecting grpcio>=1.37.0
Using cached
grpcio-1.48.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB)
Collecting pyparsing!=3.0.5,>=2.0.2
Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting
virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
Using cached virtualenv-20.16.4-py3-none-any.whl (8.8 MB)
Using cached virtualenv-20.16.3-py2.py3-none-any.whl (8.8 MB)
Using cached virtualenv-20.16.2-py2.py3-none-any.whl (8.8 MB)
Collecting platformdirs<3,>=2
Using cached platformdirs-2.5.2-py3-none-any.whl (14 kB)
Installing collected packages: distlib, wheel, toml, six, setuptools,
pyparsing, py, protobuf, pluggy, platformdirs, filelock, virtualenv, packaging,
mypy-protobuf, grpcio, tox, grpcio-tools
Attempting uninstall: setuptools
Found existing installation: setuptools 44.0.0
Uninstalling setuptools-44.0.0:
Successfully uninstalled setuptools-44.0.0
> Task :release:go-licenses:py:docker
---> Running in 8fb77f9bd077
Removing intermediate container 8fb77f9bd077
---> 7162baf99e18
Step 8/8 : ENTRYPOINT /opt/apache/beam/get-licenses.sh
---> Running in ad971ca1fad9
Removing intermediate container ad971ca1fad9
---> 9de26dc8504a
Successfully built 9de26dc8504a
Successfully tagged golicenses-python:latest
> Task :sdks:python:test-suites:portable:py37:setupVirtualenv
Collecting grpcio>=1.37.0
Using cached
grpcio-1.48.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.6 MB)
Collecting protobuf<4.0dev,>=3.5.0.post1
Using cached protobuf-3.20.2-py2.py3-none-any.whl (162 kB)
Collecting zipp>=0.5
Using cached zipp-3.8.1-py3-none-any.whl (5.6 kB)
Collecting pyparsing!=3.0.5,>=2.0.2
Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB)
Collecting
virtualenv!=20.0.0,!=20.0.1,!=20.0.2,!=20.0.3,!=20.0.4,!=20.0.5,!=20.0.6,!=20.0.7,>=16.0.0
Using cached virtualenv-20.16.4-py3-none-any.whl (8.8 MB)
Using cached virtualenv-20.16.3-py2.py3-none-any.whl (8.8 MB)
Using cached virtualenv-20.16.2-py2.py3-none-any.whl (8.8 MB)
Collecting platformdirs<3,>=2
Using cached platformdirs-2.5.2-py3-none-any.whl (14 kB)
Installing collected packages: distlib, zipp, wheel, toml, six, setuptools,
pyparsing, py, protobuf, platformdirs, filelock, packaging, mypy-protobuf,
importlib-metadata, grpcio, virtualenv, pluggy, grpcio-tools, tox
Attempting uninstall: setuptools
Found existing installation: setuptools 47.1.0
Uninstalling setuptools-47.1.0:
Successfully uninstalled setuptools-47.1.0
> Task :sdks:python:setupVirtualenv
Successfully installed distlib-0.3.1 filelock-3.8.0 grpcio-1.48.1
grpcio-tools-1.37.0 mypy-protobuf-1.18 packaging-21.3 platformdirs-2.5.2
pluggy-1.0.0 protobuf-3.20.2 py-1.11.0 pyparsing-3.0.9 setuptools-65.3.0
six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.16.2 wheel-0.37.1
> Task :release:go-licenses:py:dockerRun
+ rm -rf '/output/*'
+ go get github.com/apache/beam/sdks/v2/python/container
go: downloading github.com/apache/beam/sdks/v2 v2.41.0
go: downloading github.com/apache/beam v2.41.0+incompatible
go: downloading github.com/nightlyone/lockfile v1.0.0
go: downloading github.com/golang/protobuf v1.5.2
go: downloading google.golang.org/grpc v1.48.0
go: downloading google.golang.org/protobuf v1.28.0
go: downloading google.golang.org/genproto v0.0.0-20220714211235-042d03aeabc9
go: downloading golang.org/x/sys v0.0.0-20220624220833-87e55d714810
go: downloading golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e
go: downloading golang.org/x/text v0.3.7
> Task :sdks:python:test-suites:portable:py37:setupVirtualenv
Successfully installed distlib-0.3.1 filelock-3.8.0 grpcio-1.48.1
grpcio-tools-1.37.0 importlib-metadata-2.1.3 mypy-protobuf-1.18 packaging-21.3
platformdirs-2.5.2 pluggy-1.0.0 protobuf-3.20.2 py-1.11.0 pyparsing-3.0.9
setuptools-65.3.0 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.16.2
wheel-0.37.1 zipp-3.8.1
> Task :sdks:python:sdist
org/apache/beam/model/interactive/v1/beam_interactive_api.proto:36:1: warning:
Import google/protobuf/timestamp.proto is unused.
Writing mypy to org/apache/beam/model/pipeline/v1/external_transforms_pb2.pyi
Writing mypy to org/apache/beam/model/pipeline/v1/beam_runner_api_pb2.pyi
Writing mypy to org/apache/beam/model/pipeline/v1/standard_window_fns_pb2.pyi
Writing mypy to org/apache/beam/model/pipeline/v1/metrics_pb2.pyi
Writing mypy to org/apache/beam/model/pipeline/v1/endpoints_pb2.pyi
Writing mypy to org/apache/beam/model/pipeline/v1/schema_pb2.pyi
Writing mypy to
org/apache/beam/model/job_management/v1/beam_artifact_api_pb2.pyi
Writing mypy to
org/apache/beam/model/job_management/v1/beam_expansion_api_pb2.pyi
Writing mypy to org/apache/beam/model/job_management/v1/beam_job_api_pb2.pyi
Writing mypy to org/apache/beam/model/fn_execution/v1/beam_provision_api_pb2.pyi
Writing mypy to org/apache/beam/model/fn_execution/v1/beam_fn_api_pb2.pyi
Writing mypy to
org/apache/beam/model/interactive/v1/beam_interactive_api_pb2.pyi
setup.py:142: UserWarning: Could not import gen_protos, skipping proto
generation.
warnings.warn("Could not import gen_protos, skipping proto generation.")
<https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/setuptools/dist.py>:530:
UserWarning: Normalizing '2.43.0.dev' to '2.43.0.dev0'
warnings.warn(tmpl.format(**locals()))
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
warning: no files found matching 'LICENSE.python'
warning: sdist: standard file not found: should have one of README, README.rst,
README.txt, README.md
> Task :sdks:python:container:py37:copyDockerfileDependencies
> Task :release:go-licenses:py:dockerRun
go: added github.com/apache/beam/sdks/v2 v2.41.0
go: added github.com/golang/protobuf v1.5.2
go: added github.com/nightlyone/lockfile v1.0.0
go: added golang.org/x/net v0.0.0-20220624214902-1bab6f366d9e
go: added golang.org/x/sys v0.0.0-20220624220833-87e55d714810
go: added golang.org/x/text v0.3.7
go: added google.golang.org/genproto v0.0.0-20220714211235-042d03aeabc9
go: added google.golang.org/grpc v1.48.0
go: added google.golang.org/protobuf v1.28.0
+ go-licenses save github.com/apache/beam/sdks/v2/python/container
--save_path=/output/licenses
W0913 22:22:54.050800 24 library.go:94] "golang.org/x/sys/unix" contains
non-Go code that can't be inspected for further dependencies:
/go/pkg/mod/golang.org/x/[email protected]/unix/asm_linux_amd64.s
> Task :sdks:python:test-suites:portable:py37:installGcpTest FAILED
Processing
<https://ci-beam.apache.org/job/beam_PreCommit_Portable_Python_Cron/ws/src/sdks/python/build/apache-beam.tar.gz>
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'error'
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [10 lines of output]
Traceback (most recent call last):
File "<string>", line 36, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/tmp/pip-req-build-qvi34ijo/setup.py", line 169, in <module>
generate_protos_first()
File "/tmp/pip-req-build-qvi34ijo/setup.py", line 139, in
generate_protos_first
gen_protos.generate_proto_files()
File "/tmp/pip-req-build-qvi34ijo/gen_protos.py", line 476, in
generate_proto_files
raise RuntimeError(error_msg)
RuntimeError: Not in apache git tree, unable to find proto definitions.
[end of output]
note: This error originates from a subprocess, and is likely not a problem
with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
> Task :release:go-licenses:py:dockerRun
+ go-licenses csv github.com/apache/beam/sdks/v2/python/container
+ tee /output/licenses/list.csv
> Task :runners:flink:1.15:job-server:shadowJar FROM-CACHE
> Task :release:go-licenses:py:dockerRun
W0913 22:23:00.976556 175 library.go:94] "golang.org/x/sys/unix" contains
non-Go code that can't be inspected for further dependencies:
/go/pkg/mod/golang.org/x/[email protected]/unix/asm_linux_amd64.s
github.com/apache/beam/sdks/v2/go/pkg/beam,https://github.com/apache/beam/blob/sdks/v2.41.0/sdks/go/README.md,Apache-2.0
github.com/apache/beam/sdks/v2/python/container,https://github.com/apache/beam/blob/sdks/v2.41.0/sdks/LICENSE,Apache-2.0
github.com/golang/protobuf,https://github.com/golang/protobuf/blob/v1.5.2/LICENSE,BSD-3-Clause
github.com/nightlyone/lockfile,https://github.com/nightlyone/lockfile/blob/v1.0.0/LICENSE,MIT
golang.org/x/net,https://cs.opensource.google/go/x/net/+/1bab6f36:LICENSE,BSD-3-Clause
golang.org/x/sys,https://cs.opensource.google/go/x/sys/+/87e55d71:LICENSE,BSD-3-Clause
golang.org/x/text,https://cs.opensource.google/go/x/text/+/v0.3.7:LICENSE,BSD-3-Clause
google.golang.org/genproto/googleapis/rpc/status,https://github.com/googleapis/go-genproto/blob/042d03aeabc9/LICENSE,Apache-2.0
google.golang.org/grpc,https://github.com/grpc/grpc-go/blob/v1.48.0/LICENSE,Apache-2.0
google.golang.org/protobuf,https://github.com/protocolbuffers/protobuf-go/blob/v1.28.0/LICENSE,BSD-3-Clause
+ chmod -R a+w /output/licenses
> Task :release:go-licenses:py:createLicenses
> Task :sdks:python:container:py37:copyGolangLicenses
> Task :sdks:python:container:py37:dockerPrepare
> Task :sdks:python:container:py37:docker
The command '/bin/sh -c pip install --no-deps -v
/opt/apache/beam/tars/apache-beam.tar.gz[gcp]' returned a non-zero code: 1
> Task :sdks:python:container:py37:docker FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:installGcpTest'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task ':sdks:python:container:py37:docker'.
> Process 'command 'docker'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 13m 14s
143 actionable tasks: 91 executed, 40 from cache, 12 up-to-date
Publishing build scan...
https://gradle.com/s/4rzfam2dqj3yi
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
