See
<https://builds.apache.org/job/beam_Dependency_Check/257/display/redirect?page=changes>
Changes:
[robertwb] Supporting infrastructure for dataframes on beam.
[robertwb] Basic deferred data frame implementation.
[robertwb] yapf, py2
[github] typings and docs for expressions.py
[kcweaver] [BEAM-9448] Fix log message for job server cache.
[kcweaver] Downgrade cache log level from warn->info.
[suztomo] grpc 1.27.2 and gax 1.54.0
[suztomo] bigquerystorage 0.125.0-beta
[apilloud] [BEAM-9463] Bump ZetaSQL to 2020.03.1
[lcwik] [BEAM-2939, BEAM-9458] Use deduplication transform for UnboundedSources
[echauchot] Fix wrong generated code comment.
[github] [BEAM-9396] Fix Docker image name in CoGBK test for Python on Flink
[lcwik] [BEAM-9288] Update to use vendored gRPC without shaded conscrypt
[github] [BEAM-9319] Clean up start topic in TestPubsubSignal (#11072)
[lcwik] [BEAM-2939] Follow-up on comment in pr/11065
[lcwik] [BEAM-9473] Dont copy over META-INF index/checksum/signing files during
[apilloud] [BEAM-9411] Enable BigQuery DIRECT_READ by default in SQL
[hannahjiang] update CHANGE.md for 2.20
[lcwik] [BEAM-9475] Fix typos and shore up expectations on type
[rohde.samuel] BEAM[8335] TestStreamService integration with DirectRunner
[github] [BEAM-7926] Update Data Visualization (#11020)
[ankurgoenka] [BEAM-9402] Remove options overwrite
[chadrik] Add pre-commit hook for pylint
[github] Additional new Python Katas (#11078)
[github] [BEAM-9478] Update samza runner page to reflect post 1.0 changes
[suztomo] grpc-google-cloud-pubsub-v1 1.85.1
[pabloem] Updating BigQuery client APIs
[github] [BEAM-9481] Exclude signature files from expansion service test
[github] Install typing package only for Python < 3.5.3 (#10821)
[heejong] [BEAM-9056] Staging artifacts from environment
[sunjincheng121] [BEAM-9295] Add Flink 1.10 build target and Make FlinkRunner
compatible
[ankurgoenka] [BEAM-9485] Raise error when transform urn is not implemented
[12602502+Ardagan] [BEAM-9431] Remove ReadFromPubSub/Read-out0-ElementCount
from the
[github] Update Python roadmap for 2.7 eol
[mxm] [BEAM-9474] Improve robustness of BundleFactory and ProcessEnvironment
[github] [BEAM-7815] update MemoryReporter comments about using guppy3 (#11073)
[rohde.samuel] [BEAM-8335] Modify the StreamingCache to subclass the
CacheManager
[sunjincheng121] [BEAM-9298] Drop support for Flink 1.7
[github] Fixing apache_beam.io.gcp.bigquery_test:PubSubBigQueryIT. at head
[mxm] [BEAM-9490] Guard referencing for environment expiration via a lock
[github] Verify schema early in ToJson and JsonToRow (#11105)
[lcwik] [BEAM-9481] fix indentation
[github] Merge pull request #11103 from [BEAM-9494] Reifying outputs from BQ
file
[github] [BEAM-8335] Implemented Capture Size limitation (#11050)
[github] [BEAM-9294] Move RowJsonException out of RowJsonSerializer (#11102)
[github] Merge pull request #11046: [BEAM-9442] Properly handle nullable fields
[ankurgoenka] [BEAM-9287] disable validates runner test which uses teststreams
for
[sunjincheng121] [BEAM-9299-PR]Upgrade Flink Runner 1.8x to 1.8.3 and 1.9x to
1.9.2
[lcwik] [BEAM-2939] Implement interfaces and concrete watermark estimators
[ankurgoenka] [BEAM-9499] Sickbay test_multi_triggered_gbk_side_input for
streaming
[robertwb] Minor cleanup, lint.
[thw] [BEAM-9490] Use the lock that belongs to the cache when bundle load
[github] Update Dataflow py container version (#11120)
[github] [BEAM-7923] Streaming support and pipeline pruning when instrumenting a
[github] Update default value in Java snippet
[rohde.samuel] [BEAM-8335] Final PR to merge the InteractiveBeam feature branch
[github] [BEAM-9477] RowCoder should be hashable and picklable (#11088)
[github] [BEAM-9346] Improve the efficiency of TFRecordIO (#11122)
------------------------------------------
[...truncated 33.42 KB...]
- org.powermock:powermock-api-mockito2 [2.0.2 -> 2.0.5]
http://www.powermock.org
- org.powermock:powermock-module-junit4 [2.0.2 -> 2.0.5]
http://www.powermock.org
- org.scala-lang:scala-library [2.11.8 -> 2.13.1]
https://www.scala-lang.org/
- org.scala-lang:scala-library [2.12.7 -> 2.13.1]
https://www.scala-lang.org/
- org.slf4j:jcl-over-slf4j [1.7.25 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.slf4j:slf4j-api [1.7.21 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.slf4j:slf4j-api [1.7.25 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.slf4j:slf4j-jdk14 [1.7.25 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.slf4j:slf4j-simple [1.7.25 -> 2.0.0-alpha1]
http://www.slf4j.org
- org.sonarqube:org.sonarqube.gradle.plugin [2.7 -> 2.8]
- org.springframework:spring-expression [4.3.18.RELEASE -> 5.2.4.RELEASE]
https://github.com/spring-projects/spring-framework
- org.testcontainers:clickhouse [1.10.5 -> 1.13.0]
https://testcontainers.org
- org.testcontainers:localstack [1.11.2 -> 1.13.0]
https://testcontainers.org
- org.testcontainers:testcontainers [1.11.3 -> 1.13.0]
https://testcontainers.org
- org.xerial.snappy:snappy-java [1.1.4 -> 1.1.7.3]
https://github.com/xerial/snappy-java
- redis.clients:jedis [3.0.1 -> 3.2.0]
https://github.com/xetorthio/jedis
- ru.yandex.clickhouse:clickhouse-jdbc [0.1.47 -> 0.2.4]
https://github.com/ClickHouse/clickhouse-jdbc
- software.amazon.awssdk:apache-client [2.10.61 -> 2.10.86]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:auth [2.10.61 -> 2.10.86]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:cloudwatch [2.10.61 -> 2.10.86]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:dynamodb [2.10.61 -> 2.10.86]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:sdk-core [2.10.61 -> 2.10.86]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:sns [2.10.61 -> 2.10.86]
https://aws.amazon.com/sdkforjava
- software.amazon.awssdk:sqs [2.10.61 -> 2.10.86]
https://aws.amazon.com/sdkforjava
- sqlline:sqlline [1.4.0 -> 1.9.0]
https://github.com/julianhyde/sqlline
Gradle updates:
- Gradle: [5.2.1 -> 6.2.2 -> 6.3-rc-1]
Generated report file build/dependencyUpdates/report.txt
> Task :sdks:python:dependencyUpdates
# Virtualenv for the rest of the script to run setup
rm -rf sdks/
virtualenv sdks/python
Using base prefix '/usr'
New python executable in
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin/python3>
Also creating executable in
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin/python>
Installing setuptools, pip, wheel...
done.
. sdks/python/bin/activate
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
unset -f pydoc >/dev/null 2>&1
# reset old environment variables
# ! [ -z ${VAR+_} ] returns true if VAR is declared at all
if ! [ -z "${_OLD_VIRTUAL_PATH+_}" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if ! [ -z "${_OLD_VIRTUAL_PYTHONHOME+_}" ] ; then
PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
if ! [ -z "${_OLD_VIRTUAL_PS1+_}" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV="<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python";>
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="${PS1-}"
if [ "x" != x ] ; then
PS1="${PS1-}"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) ${PS1-}"
fi
export PS1
fi
basename "$VIRTUAL_ENV"
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc || true
pydoc () {
python -m pydoc "$@"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
pip install -e .[gcp,test,docs]
Obtaining
file://<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python>
ERROR: Command errored out with exit status 1:
command:
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin/python3>
-c 'import sys, setuptools, tokenize; sys.argv[0] =
'"'"'<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/setup.py'"'"';>
__file__='"'"'<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/setup.py'"'"';f=getattr(tokenize,>
'"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"',
'"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
cwd:
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/>
Complete output (67 lines):
running egg_info
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/lib/python3.5/site-packages/setuptools/dist.py>:453:
UserWarning: Normalizing '2.21.0.dev' to '2.21.0.dev0'
normalized_version,
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py>:241:
UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
WARNING:root:Installing grpcio-tools into
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/.eggs/grpcio-wheels>
Collecting grpcio-tools<=1.14.2,>=1.3.5
Using cached grpcio_tools-1.14.2-cp35-cp35m-manylinux1_x86_64.whl (22.2
MB)
Processing
/home/jenkins/.cache/pip/wheels/bf/c9/a3/c538d90ef17cf7823fa51fc701a7a7a910a80f6a405bf15b1a/future-0.16.0-cp35-none-any.whl
Collecting mypy-protobuf==1.18
Using cached mypy_protobuf-1.18-py3-none-any.whl (7.3 kB)
Collecting protobuf>=3.5.0.post1
Using cached protobuf-3.11.3-cp35-cp35m-manylinux1_x86_64.whl (1.3 MB)
Collecting grpcio>=1.14.2
Using cached grpcio-1.27.2-cp35-cp35m-manylinux2010_x86_64.whl (2.7 MB)
Collecting six>=1.9
Using cached six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting setuptools
Using cached setuptools-46.0.0-py3-none-any.whl (582 kB)
Installing collected packages: six, setuptools, protobuf, grpcio,
grpcio-tools, future, mypy-protobuf
Successfully installed future-0.16.0 grpcio-1.27.2 grpcio-tools-1.14.2
mypy-protobuf-1.18 protobuf-3.11.3 setuptools-46.0.0 six-1.14.0
WARNING:root:Installing grpcio-tools took 9.41 seconds.
INFO:gen_protos:Regenerating Python proto definitions (no output files).
Process Process-1:
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 292, in generate_proto_files
from grpc_tools import protoc
ImportError: No module named 'grpc_tools'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.5/multiprocessing/process.py", line 249, in
_bootstrap
self.run()
File "/usr/lib/python3.5/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 378, in _install_grpcio_tools_and_generate_proto_files
generate_proto_files(force=force)
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 315, in generate_proto_files
protoc_gen_mypy = _find_protoc_gen_mypy()
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 233, in _find_protoc_gen_mypy
(fname, ', '.join(search_paths)))
RuntimeError: Could not find protoc-gen-mypy in
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin,>
<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/bin,>
/home/jenkins/tools/java/latest1.8/bin, /usr/local/sbin, /usr/local/bin,
/usr/sbin, /usr/bin, /sbin, /bin, /usr/games, /usr/local/games
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 292, in generate_proto_files
from grpc_tools import protoc
ImportError: No module named 'grpc_tools'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/setup.py";,>
line 315, in <module>
'mypy': generate_protos_first(mypy),
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/sdks/python/lib/python3.5/site-packages/setuptools/__init__.py";,>
line 144, in setup
return distutils.core.setup(**attrs)
File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
dist.run_commands()
File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/setup.py";,>
line 239, in run
gen_protos.generate_proto_files()
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/gen_protos.py";,>
line 310, in generate_proto_files
raise ValueError("Proto generation failed (see log for details).")
ValueError: Proto generation failed (see log for details).
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check
the logs for full command output.
> Task :sdks:python:dependencyUpdates FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_Dependency_Check/ws/src/sdks/python/build.gradle'>
line: 94
* What went wrong:
Execution failed for task ':sdks:python:dependencyUpdates'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 3m 58s
2 actionable tasks: 2 executed
Publishing build scan...
https://gradle.com/s/fbb5rsfwq7pqa
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
