See
<https://builds.apache.org/job/beam_Dependency_Check/256/display/redirect?page=changes>
Changes:
[git] Remove optionality and add sensible defaults to PubsubIO builders.
[boyuanz] Update verify_release_build script to run python tests with dev
version.
[filiperegadas] Add BigQuery useAvroLogicalTypes option
[filiperegadas] fixup! Add BigQuery useAvroLogicalTypes option
[jvilcek] [BEAM-9360] Fix equivalence check for FieldType
[lcwik] [BEAM-9397] Pass all but output receiver parameters to start
[kamil.wasilewski] Add integration test for AnnotateImage transform
[chadrik] [BEAM-7746] Runtime change to timestamp/duration equality
[aromanenko.dev] [BEAM-8925] Tika version update to 1.23
[12602502+Ardagan] [BEAM-8327] Override Gradle cache for community metrics
prober
[ehudm] Reduce warnings in pytest runs.
[heejong] [BEAM-9415] fix postcommit xvr tests
[github] Merge pull request #10968 from [BEAM-9381] Adding display data to
[github] [BEAM-8335] Add PCollection to DataFrame logic for InteractiveRunner.
[robertwb] Remove excessive logging.
[github] [BEAM-2939] Java UnboundedSource SDF wrapper (#10897)
[iemejia] [website] Update link to environment_type (SDK harness configuration)
[iemejia] Fix typo on python code
[kamil.wasilewski] Fix: skip test if GCP dependencies are not installed
[fernandodiaz] [BEAM-9424] Allow grouping by LogicalType
[github] Revert "[BEAM-8335] Add PCollection to DataFrame logic for
[echauchot] Add metrics export to documentation on the website.
[github] [BEAM-8382] Add rate limit policy to KinesisIO.Read (#9765)
[lcwik] [BEAM-9288] Bump version number vendored gRPC build.
[chadrik] [BEAM-9274] Support running yapf in a git pre-commit hook
[rohde.samuel] [BEAM-8335] Add PCollection to Dataframe logic for
InteractiveRunner.
[github] [BEAM-8575] Modified trigger test to work for different runners.
[github] [BEAM-9413] fix beam_PostCommit_Py_ValCon (#11023)
[rohde.samuel] ReverseTestStream Implementation
[github] Update lostluck's info on the Go SDK roadmap
[suztomo] Google-cloud-bigquery 1.108.0
[github] [BEAM-9432] Move expansion service into its own project. (#11035)
[ehudm] [BEAM-3713] Remove nosetests from tox.ini
[github] Merge pull request #11025: [BEAM-6428] Improve select performance with
[github] Switch contact email to apache.org.
[github] [BEAM-6374] Emit PCollection metrics from GoSDK (#10942)
[amaliujia] [BEAM-9288] Not bundle conscrypt in gRPC vendor in META-INF/
[github] Update container image tags used by Dataflow runner for Beam master
[github] [BEAM-8328] Disable community metrics integration test in 'test' task
[iemejia] [BEAM-9450] Update www.apache.org/dist/ links to downloads.apache.org
[iemejia] [BEAM-9450] Convert links available via https to use https
[github] Add integration test for AnnotateVideoWithContext transform (#10986)
[lcwik] [BEAM-9452] Update classgraph to latest version to resolve windows
[hktang] [BEAM-9453] Changed new string creation to use StandardCharsets.UTF_8
[chuck.yang] Use Avro format for file loads to BigQuery
[jkai] [Hotfix] fix rabbitmp spotless check
[github] Revert "[BEAM-6374] Emit PCollection metrics from GoSDK (#10942)"
[github] Merge pull request #11032 from [BEAM-8335] Display rather than logging
[github] Fix a bug in performance test for reading data from BigQuery (#11062)
[lcwik] [BEAM-2939, BEAM-9458] Add deduplication transform for SplittableDoFns
[lcwik] [BEAM-9464] Fix WithKeys to respect parameterized types
[ankurgoenka] [BEAM-9465] Fire repeatedly in reshuffle
------------------------------------------
[...truncated 34.03 KB...]
- org.powermock:powermock-api-mockito2 [2.0.2 -> 2.0.5]
http://www.powermock.org
- org.powermock:powermock-module-junit4 [2.0.2 -> 2.0.5]
http://www.powermock.org
- org.scala-lang:scala-library [2.11.8 -> 2.13.1]
https://www.scala-lang.org/
- org.scala-lang:scala-library [2.12.7 -> 2.13.1]
https://www.scala-lang.org/
- org.slf4j:jcl-over-slf4j [1.7.25 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.slf4j:slf4j-api [1.7.21 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.slf4j:slf4j-api [1.7.25 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.slf4j:slf4j-jdk14 [1.7.25 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.slf4j:slf4j-simple [1.7.25 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.sonarqube:org.sonarqube.gradle.plugin [2.7 -> 2.8]
- org.springframework:spring-expression [4.3.18.RELEASE -> 5.2.4.RELEASE]
https://github.com/spring-projects/spring-framework
- org.testcontainers:clickhouse [1.10.5 -> 1.13.0]
https://testcontainers.org
- org.testcontainers:localstack [1.11.2 -> 1.13.0]
https://testcontainers.org
- org.testcontainers:testcontainers [1.11.3 -> 1.13.0]
https://testcontainers.org
- org.xerial.snappy:snappy-java [1.1.4 -> 1.1.7.3]
https://github.com/xerial/snappy-java
- redis.clients:jedis [3.0.1 -> 3.2.0]
https://github.com/xetorthio/jedis
- ru.yandex.clickhouse:clickhouse-jdbc [0.1.47 -> 0.2.4]
https://github.com/ClickHouse/clickhouse-jdbc
- software.amazon.awssdk:apache-client [2.10.61 -> 2.10.81]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:auth [2.10.61 -> 2.10.81]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:cloudwatch [2.10.61 -> 2.10.81]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:dynamodb [2.10.61 -> 2.10.81]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:sdk-core [2.10.61 -> 2.10.81]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:sns [2.10.61 -> 2.10.81]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:sqs [2.10.61 -> 2.10.81]
https://aws.amazon.com/sdkforjava
- sqlline:sqlline [1.4.0 -> 1.9.0]
https://github.com/julianhyde/sqlline
Gradle updates:
- Gradle: [5.2.1 -> 6.2.2]
Generated report file build/dependencyUpdates/report.txt
> Task :sdks:python:dependencyUpdates
# Virtualenv for the rest of the script to run setup
rm -rf sdks/
virtualenv sdks/python
Using base prefix '/usr'
New python executable in
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin/python3>
Also creating executable in
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin/python>
Installing setuptools, pip, wheel...
done.
. sdks/python/bin/activate
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
unset -f pydoc >/dev/null 2>&1
# reset old environment variables
# ! [ -z ${VAR+_} ] returns true if VAR is declared at all
if ! [ -z "${_OLD_VIRTUAL_PATH+_}" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if ! [ -z "${_OLD_VIRTUAL_PYTHONHOME+_}" ] ; then
PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
if ! [ -z "${_OLD_VIRTUAL_PS1+_}" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV="<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python";>
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="${PS1-}"
if [ "x" != x ] ; then
PS1="${PS1-}"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) ${PS1-}"
fi
export PS1
fi
basename "$VIRTUAL_ENV"
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc || true
pydoc () {
python -m pydoc "$@"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
pip install -e .[gcp,test,docs]
Obtaining
file://<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python>
ERROR: Command errored out with exit status 1:
command:
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin/python3>
-c 'import sys, setuptools, tokenize; sys.argv[0] =
'"'"'<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/setup.py'"'"';>
__file__='"'"'<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/setup.py'"'"';f=getattr(tokenize,>
'"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
cwd:
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/>
Complete output (67 lines):
running egg_info
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/lib/python3.5/site-packages/setuptools/dist.py>:453:
UserWarning: Normalizing '2.21.0.dev' to '2.21.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py>:241:
UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/.eggs/grpcio-wheels>
Collecting grpcio-tools<=1.14.2,>=1.3.5
Using cached grpcio_tools-1.14.2-cp35-cp35m-manylinux1_x86_64.whl (22.2
MB)
Processing
/home/jenkins/.cache/pip/wheels/bf/c9/a3/c538d90ef17cf7823fa51fc701a7a7a910a80f6a405bf15b1a/future-0.16.0-cp35-none-any.whl
Collecting mypy-protobuf==1.18
Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB)
Collecting grpcio>=1.14.2
Using cached grpcio-1.27.2-cp35-cp35m-manylinux2010_x86_64.whl (2.7 MB)
Collecting protobuf>=3.5.0.post1
Using cached protobuf-3.11.3-cp35-cp35m-manylinux1_x86_64.whl (1.3 MB)
Collecting six>=1.5.2
Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting setuptools
Using cached setuptools-46.0.0-py3-none-any.whl (582 kB)
Installing collected packages: six, grpcio, setuptools, protobuf,
grpcio-tools, future, mypy-protobuf
Successfully installed future-0.16.0 grpcio-1.27.2 grpcio-tools-1.14.2
mypy-protobuf-1.18 protobuf-3.11.3 setuptools-46.0.0 six-1.14.0
WARNING:root:Installing grpcio-tools took 7.40 seconds.
INFO:gen_protos:Regenerating Python proto definitions (no output files).
Process Process-1:
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 292, in generate_proto_files
from grpc_tools import protoc
ImportError: No module named 'grpc_tools'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in
_bootstrap
self.run()
File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 378, in _install_grpcio_tools_and_generate_proto_files
generate_proto_files(force=force)
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 315, in generate_proto_files
protoc_gen_mypy = _find_protoc_gen_mypy()
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 233, in _find_protoc_gen_mypy
(fname, ', '.join(search_paths)))
RuntimeError: Could not find protoc-gen-mypy in
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin,>
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin,>
/home/jenkins/tools/java/latest1.8/bin, /usr/local/sbin, /usr/local/bin,
/usr/sbin, /usr/bin, /sbin, /bin, /usr/games, /usr/local/games
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 292, in generate_proto_files
from grpc_tools import protoc
ImportError: No module named 'grpc_tools'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/setup.py";,>
line 317, in <module>
'mypy': generate_protos_first(mypy),
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/lib/python3.5/site-packages/setuptools/__init__.py";,>
line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/setup.py";,>
line 241, in run
gen_protos.generate_proto_files()
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 310, in generate_proto_files
raise ValueError("Proto generation failed (see log for details).")
ValueError: Proto generation failed (see log for details).
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check
the logs for full command output.
> Task :sdks:python:dependencyUpdates FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/build.gradle'>
line: 94
* What went wrong:
Execution failed for task ':sdks:python:dependencyUpdates'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 0s
2 actionable tasks: 2 executed
Publishing build scan...
https://gradle.com/s/l35rzxco67xs2
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
