See
<https://builds.apache.org/job/beam_Dependency_Check/190/display/redirect?page=changes>
Changes:
[sekikn] [BEAM-6483] Add support for SADD operation to RedisIO.Write
[daniel.mescheder] wrap join in PTransform to allow multiple joins in same
pipeline
[daniel.mescheder] fix stylecheck issues
[daniel.mescheder] actually fix stylecheck issues
[github] Update default test in run_integration_test.sh
[heejong] [BEAM-6443] decrease the number of thread for BigQuery streaming
[katarzyna.kucharczyk] [BEAM-5995] Add fanout and reiteration in GBK load test
in Python
[melissapa] [BEAM-6827] Update Python dependencies page for 2.10.0, 2.11.0
[michal.walenia] [BEAM-6703] Added a Jenkins job to test Dataflow portability
API with
[michal.walenia] [BEAM-6703] Changed job type to phrase triggered only
[michal.walenia] [BEAM-6703] Added README entry with cron and phrase triggered
jobs
[iemejia] [BEAM-5115] Make ValueProvider API consistent between XmlIO and
[echauchot] [BEAM-6052] Fix the bug of checkForErrors, add update response parse
[echauchot] [BEAM-6052] Refactor testPartialUpdateWithErrors and related
[echauchot] [BEAM-6279] instead of stopping extra nodes of the test cluster to
have
[echauchot] [BEAM-6052] Fix setIndexMapping
[echauchot] Remove Cassandra PasswordDecrypter fix (tagged for 2.12) from high
level
[michal.walenia] [BEAM-6703] Changed job type to include cron trigger
[mxm] [BEAM-6730] Expose Java SDK's GenerateSequence to Python
[mxm] [BEAM-6730] Support Read translation in streaming portable Flink Runner
[mxm] [BEAM-6730] Support Read translation in batch portable Flink Runner
[mxm] [BEAM-6730] Add portable Read translation test
[katarzyna.kucharczyk] [BEAM-5995] Add fanout to Combine load test in Python
[echauchot] [BEAM-6052] Fix testWritePartialUpdateWithErrors insert only one
invalid
[mxm] [BEAM-6730] Use generic payload to configure external transforms
[mxm] [BEAM-6730] Add Java GenerateSequence expansion test
[kyle.winkelman] [BEAM-6851] Support recursive globs in HadoopFileSystem.
[katarzyna.kucharczyk] [BEAM-6848] changed datatset key name in load test
builder. Added
[mxm] [BEAM-6730] Add Python GenerateSequence integration test
[mxm] [BEAM-6730] Add test for GenerateSequence's external builder
[github] Fixing minor type
[iemejia] [BEAM-6854] Update amazon-web-services aws-sdk dependency to version
[juta.staes] [BEAM-6619] [BEAM-6593] fix autocomplete it test
[lukasz.gajowy] [BEAM-6848] Fix incorrectly passed args to job
[lukasz.gajowy] [BEAM-6703] Added a Jenkins job definition for Java 11 enabled
Dataflow
[katarzyna.kucharczyk] [BEAM-6175] Extract reader and publisher from
MetricsMonitor in Python
[katarzyna.kucharczyk] [BEAM-6175] Extracted common class for all load tests
[kcweaver] [BEAM-6865] DefaultJobBundleFactory: create using default
[kmj] Set the user-agent in the BQ storage client.
[kcweaver] [BEAM-6865] share non-Flink-specific pipeline helper utils
[kcweaver] [BEAM-6864] Pass JobInfo to PortablePipelineRunner.run
[jeff] [BEAM-6770] Correct zstd-jni dependency scope to tests only
[michal.walenia] [BEAM-6627] Refactor metric gathering mechanism to be more
flexible
[michal.walenia] [BEAM-6627] Added metrics to ParquetIOIT
[michal.walenia] [BEAM-6627] Added metrics to AvroIOIT
[michal.walenia] [BEAM-6627] Added metrics to TFRecordIOIT, refactored
namespace names
[michal.walenia] [BEAM-6627] Added metrics to XmlIOIT
[michal.walenia] [BEAM-6627] Added metric reporting to all file based test jobs
[jeff] [BEAM-5638] Exception handling for Java MapElements and FlatMapElements
[iemejia] Revert "Review: Add some comments & tests"
[iemejia] Revert "[BEAM-6292] PasswordDecrypter: Delay decryption / Avoid
[ajhalaria] Create a custom hash paritioner that deals with arrays during
combines
[markliu] [BEAM-6527] Use Gradle to parallel Python tox tests
[markliu] Increase beam_Release_Gradle_NightlySnapshot timeout to 200mins due to
[tweise] [flink] debug logging for state access
[github] [BEAM-6866]: SamzaRunner: support timers in ParDo (#8092)
[boyuanz] Reset Datasource counter to 0 in FinishBundle
[boyuanz] Reset count to 0 after all operations have finished bundle
[huangry] Clean up fixed checkstyle suppressions.
[daniel.o.programmer] [BEAM-6838] Go SDK: Improving error msgs in pipeline
serialization.
[robertwb] [BEAM-6778] Enable Bundle Finalization in Python SDK harness over
FnApi
[migryz] Fix pyhamcrest import missing
[migryz] Remove deprecated Java PVR job from Jenkins"
[github] Correct spark version used by the spark runner in the website
[mxm] [website] Update Flink Runner page on portability
[daniel.o.programmer] [BEAM-6880] Remove deprecated Reference Runner PostCommit
from Jenkins.
[robert] [Go SDK] Use the known varint64 coder for int64s.
[iemejia] [BEAM-6554] Replace latest mentions of FindBugs in the build files
[srfrnk] fix: BEAM-6890 - use vendored Guava class
------------------------------------------
[...truncated 43.84 KB...]
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV="<https://builds.apache.org/job/beam_Dependency_Check/ws/src/dependency/check";>
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
fi
export PS1
fi
basename "$VIRTUAL_ENV"
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc
pydoc () {
python -m pydoc "$@"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
pip install --upgrade google-cloud-bigquery
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020.
Please upgrade your Python as Python 2.7 won't be maintained after that date. A
future version of pip will drop support for Python 2.7.
Collecting google-cloud-bigquery
Using cached
https://files.pythonhosted.org/packages/5b/4e/bc6a8c48d25670d10518fcb117f1d85b2f5452929f59ea302ee9756a576b/google_cloud_bigquery-1.10.0-py2.py3-none-any.whl
Collecting google-resumable-media>=0.3.1 (from google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/e2/5d/4bc5c28c252a62efe69ed1a1561da92bd5af8eca0cdcdf8e60354fae9b29/google_resumable_media-0.3.2-py2.py3-none-any.whl
Collecting google-api-core<2.0.0dev,>=1.6.0 (from google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/7d/73/e4877e921fe59307ec6b1b0b0c2ad9fde2d1c6bab8dd06ec913891a20dc6/google_api_core-1.8.2-py2.py3-none-any.whl
Collecting google-cloud-core<0.30dev,>=0.29.0 (from google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/0c/f2/3c225e7a69cb27d283b68bff867722bd066bc1858611180197f711815ea5/google_cloud_core-0.29.1-py2.py3-none-any.whl
Collecting six (from google-resumable-media>=0.3.1->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting googleapis-common-protos!=1.5.4,<2.0dev,>=1.5.3 (from
google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Collecting protobuf>=3.4.0 (from
google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/65/31/00ba247814289a000f56d89cb0257448efdaa1b79aee62bfb974d0da2e8b/protobuf-3.7.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting google-auth<2.0dev,>=0.4.0 (from
google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/c5/9b/ed0516cc1f7609fb0217e3057ff4f0f9f3e3ce79a369c6af4a6c5ca25664/google_auth-1.6.3-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: setuptools>=34.0.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery) (40.8.0)
Collecting pytz (from google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl
Collecting futures>=3.2.0; python_version < "3.2" (from
google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Collecting requests<3.0.0dev,>=2.18.0 (from
google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/7d/e3/20f3d364d6c8e5d2353c72a67778eb189176f08e873c9900e10c0287b84b/requests-2.21.0-py2.py3-none-any.whl
Collecting rsa>=3.1.4 (from
google-auth<2.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/02/e5/38518af393f7c214357079ce67a317307936896e961e35450b70fad2a9cf/rsa-4.0-py2.py3-none-any.whl
Collecting cachetools>=2.0.0 (from
google-auth<2.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/39/2b/d87fc2369242bd743883232c463f28205902b8579cb68dcf5b11eee1652f/cachetools-3.1.0-py2.py3-none-any.whl
Collecting pyasn1-modules>=0.2.1 (from
google-auth<2.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/da/98/8ddd9fa4d84065926832bcf2255a2b69f1d03330aa4d1c49cc7317ac888e/pyasn1_modules-0.2.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from
requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/60/75/f692a584e85b7eaba0e03827b3d51f45f571c2e793dd731e598828d380aa/certifi-2019.3.9-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from
requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting idna<2.9,>=2.5 (from
requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl
Collecting urllib3<1.25,>=1.21.1 (from
requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from
rsa>=3.1.4->google-auth<2.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.6.0->google-cloud-bigquery)
Using cached
https://files.pythonhosted.org/packages/7b/7c/c9386b82a25115cccf1903441bba3cbadcfae7b678a20167347fa8ded34c/pyasn1-0.4.5-py2.py3-none-any.whl
Installing collected packages: six, google-resumable-media, protobuf,
googleapis-common-protos, pyasn1, rsa, cachetools, pyasn1-modules, google-auth,
pytz, futures, certifi, chardet, idna, urllib3, requests, google-api-core,
google-cloud-core, google-cloud-bigquery
Successfully installed cachetools-3.1.0 certifi-2019.3.9 chardet-3.0.4
futures-3.2.0 google-api-core-1.8.2 google-auth-1.6.3
google-cloud-bigquery-1.10.0 google-cloud-core-0.29.1
google-resumable-media-0.3.2 googleapis-common-protos-1.5.9 idna-2.8
protobuf-3.7.0 pyasn1-0.4.5 pyasn1-modules-0.2.4 pytz-2018.9 requests-2.21.0
rsa-4.0 six-1.12.0 urllib3-1.24.1
pip install --upgrade google-cloud-bigtable
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020.
Please upgrade your Python as Python 2.7 won't be maintained after that date. A
future version of pip will drop support for Python 2.7.
Collecting google-cloud-bigtable
Using cached
https://files.pythonhosted.org/packages/15/81/df8ec275ea39885744ec058f2bbcf393669a423eac18cc6a7707dc4b77d9/google_cloud_bigtable-0.32.1-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade:
google-api-core[grpc]<2.0.0dev,>=1.6.0 in
./dependency/check/lib/python2.7/site-packages (from google-cloud-bigtable)
(1.8.2)
Collecting grpc-google-iam-v1<0.12dev,>=0.11.4 (from google-cloud-bigtable)
Requirement already satisfied, skipping upgrade:
google-cloud-core<0.30dev,>=0.29.0 in
./dependency/check/lib/python2.7/site-packages (from google-cloud-bigtable)
(0.29.1)
Requirement already satisfied, skipping upgrade:
googleapis-common-protos!=1.5.4,<2.0dev,>=1.5.3 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable) (1.5.9)
Requirement already satisfied, skipping upgrade: protobuf>=3.4.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable) (3.7.0)
Requirement already satisfied, skipping upgrade: google-auth<2.0dev,>=0.4.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable) (1.6.3)
Requirement already satisfied, skipping upgrade: setuptools>=34.0.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable) (40.8.0)
Requirement already satisfied, skipping upgrade: six>=1.10.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable) (1.12.0)
Requirement already satisfied, skipping upgrade: pytz in
./dependency/check/lib/python2.7/site-packages (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable) (2018.9)
Requirement already satisfied, skipping upgrade: futures>=3.2.0; python_version
< "3.2" in ./dependency/check/lib/python2.7/site-packages (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable) (3.2.0)
Requirement already satisfied, skipping upgrade: requests<3.0.0dev,>=2.18.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable) (2.21.0)
Collecting grpcio>=1.8.2; extra == "grpc" (from
google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
Using cached
https://files.pythonhosted.org/packages/b8/be/3bb6d8241b5ed1f8437169df53e7dd6ca986174e022585de15087a848c99/grpcio-1.19.0-cp27-cp27mu-manylinux1_x86_64.whl
Requirement already satisfied, skipping upgrade: rsa>=3.1.4 in
./dependency/check/lib/python2.7/site-packages (from
google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
(4.0)
Requirement already satisfied, skipping upgrade: cachetools>=2.0.0 in
./dependency/check/lib/python2.7/site-packages (from
google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
(3.1.0)
Requirement already satisfied, skipping upgrade: pyasn1-modules>=0.2.1 in
./dependency/check/lib/python2.7/site-packages (from
google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
(0.2.4)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in
./dependency/check/lib/python2.7/site-packages (from
requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
(2019.3.9)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in
./dependency/check/lib/python2.7/site-packages (from
requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
(3.0.4)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in
./dependency/check/lib/python2.7/site-packages (from
requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
(2.8)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in
./dependency/check/lib/python2.7/site-packages (from
requests<3.0.0dev,>=2.18.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
(1.24.1)
Collecting enum34>=1.0.4 (from grpcio>=1.8.2; extra ==
"grpc"->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
Using cached
https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl
Requirement already satisfied, skipping upgrade: pyasn1>=0.1.3 in
./dependency/check/lib/python2.7/site-packages (from
rsa>=3.1.4->google-auth<2.0dev,>=0.4.0->google-api-core[grpc]<2.0.0dev,>=1.6.0->google-cloud-bigtable)
(0.4.5)
Installing collected packages: enum34, grpcio, grpc-google-iam-v1,
google-cloud-bigtable
Successfully installed enum34-1.1.6 google-cloud-bigtable-0.32.1
grpc-google-iam-v1-0.11.4 grpcio-1.19.0
pip install --upgrade google-cloud-core
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020.
Please upgrade your Python as Python 2.7 won't be maintained after that date. A
future version of pip will drop support for Python 2.7.
Requirement already up-to-date: google-cloud-core in
./dependency/check/lib/python2.7/site-packages (0.29.1)
Requirement already satisfied, skipping upgrade:
google-api-core<2.0.0dev,>=1.0.0 in
./dependency/check/lib/python2.7/site-packages (from google-cloud-core) (1.8.2)
Requirement already satisfied, skipping upgrade:
googleapis-common-protos!=1.5.4,<2.0dev,>=1.5.3 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.0.0->google-cloud-core) (1.5.9)
Requirement already satisfied, skipping upgrade: protobuf>=3.4.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.0.0->google-cloud-core) (3.7.0)
Requirement already satisfied, skipping upgrade: google-auth<2.0dev,>=0.4.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.0.0->google-cloud-core) (1.6.3)
Requirement already satisfied, skipping upgrade: setuptools>=34.0.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.0.0->google-cloud-core) (40.8.0)
Requirement already satisfied, skipping upgrade: six>=1.10.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.0.0->google-cloud-core) (1.12.0)
Requirement already satisfied, skipping upgrade: pytz in
./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.0.0->google-cloud-core) (2018.9)
Requirement already satisfied, skipping upgrade: futures>=3.2.0; python_version
< "3.2" in ./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.0.0->google-cloud-core) (3.2.0)
Requirement already satisfied, skipping upgrade: requests<3.0.0dev,>=2.18.0 in
./dependency/check/lib/python2.7/site-packages (from
google-api-core<2.0.0dev,>=1.0.0->google-cloud-core) (2.21.0)
Requirement already satisfied, skipping upgrade: rsa>=3.1.4 in
./dependency/check/lib/python2.7/site-packages (from
google-auth<2.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-core)
(4.0)
Requirement already satisfied, skipping upgrade: cachetools>=2.0.0 in
./dependency/check/lib/python2.7/site-packages (from
google-auth<2.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-core)
(3.1.0)
Requirement already satisfied, skipping upgrade: pyasn1-modules>=0.2.1 in
./dependency/check/lib/python2.7/site-packages (from
google-auth<2.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-core)
(0.2.4)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in
./dependency/check/lib/python2.7/site-packages (from
requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-core)
(2019.3.9)
Requirement already satisfied, skipping upgrade: chardet<3.1.0,>=3.0.2 in
./dependency/check/lib/python2.7/site-packages (from
requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-core)
(3.0.4)
Requirement already satisfied, skipping upgrade: idna<2.9,>=2.5 in
./dependency/check/lib/python2.7/site-packages (from
requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-core)
(2.8)
Requirement already satisfied, skipping upgrade: urllib3<1.25,>=1.21.1 in
./dependency/check/lib/python2.7/site-packages (from
requests<3.0.0dev,>=2.18.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-core)
(1.24.1)
Requirement already satisfied, skipping upgrade: pyasn1>=0.1.3 in
./dependency/check/lib/python2.7/site-packages (from
rsa>=3.1.4->google-auth<2.0dev,>=0.4.0->google-api-core<2.0.0dev,>=1.0.0->google-cloud-core)
(0.4.5)
rm -f build/dependencyUpdates/beam-dependency-check-report.txt
# Insall packages and run the unit tests of the report generator and the jira
manager
pip install mock jira pyyaml
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020.
Please upgrade your Python as Python 2.7 won't be maintained after that date. A
future version of pip will drop support for Python 2.7.
Collecting mock
Using cached
https://files.pythonhosted.org/packages/e6/35/f187bdf23be87092bd0f1200d43d23076cee4d0dec109f195173fd3ebc79/mock-2.0.0-py2.py3-none-any.whl
Collecting jira
Using cached
https://files.pythonhosted.org/packages/cb/c2/4203b6cad72d115b0ad62ae7028567b538c8933745f665d0eee5082baf67/jira-2.0.0-py2.py3-none-any.whl
Collecting pyyaml
Collecting pbr>=0.11 (from mock)
Using cached
https://files.pythonhosted.org/packages/14/09/12fe9a14237a6b7e0ba3a8d6fcf254bf4b10ec56a0185f73d651145e9222/pbr-5.1.3-py2.py3-none-any.whl
Collecting funcsigs>=1; python_version < "3.3" (from mock)
Using cached
https://files.pythonhosted.org/packages/69/cb/f5be453359271714c01b9bd06126eaf2e368f1fddfff30818754b5ac2328/funcsigs-1.0.2-py2.py3-none-any.whl
Requirement already satisfied: six>=1.9 in
./dependency/check/lib/python2.7/site-packages (from mock) (1.12.0)
Requirement already satisfied: requests>=2.10.0 in
./dependency/check/lib/python2.7/site-packages (from jira) (2.21.0)
Collecting oauthlib[signedtoken]>=1.0.0 (from jira)
Using cached
https://files.pythonhosted.org/packages/16/95/699466b05b72b94a41f662dc9edf87fda4289e3602ecd42d27fcaddf7b56/oauthlib-3.0.1-py2.py3-none-any.whl
Collecting defusedxml (from jira)
Using cached
https://files.pythonhosted.org/packages/87/1c/17f3e3935a913dfe2a5ca85fa5ccbef366bfd82eb318b1f75dadbf0affca/defusedxml-0.5.0-py2.py3-none-any.whl
Collecting requests-toolbelt (from jira)
Using cached
https://files.pythonhosted.org/packages/60/ef/7681134338fc097acef8d9b2f8abe0458e4d87559c689a8c306d0957ece5/requests_toolbelt-0.9.1-py2.py3-none-any.whl
Collecting requests-oauthlib>=0.6.1 (from jira)
Using cached
https://files.pythonhosted.org/packages/c2/e2/9fd03d55ffb70fe51f587f20bcf407a6927eb121de86928b34d162f0b1ac/requests_oauthlib-1.2.0-py2.py3-none-any.whl
Requirement already satisfied: setuptools>=20.10.1 in
./dependency/check/lib/python2.7/site-packages (from jira) (40.8.0)
Requirement already satisfied: certifi>=2017.4.17 in
./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira)
(2019.3.9)
Requirement already satisfied: chardet<3.1.0,>=3.0.2 in
./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira)
(3.0.4)
Requirement already satisfied: idna<2.9,>=2.5 in
./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira)
(2.8)
Requirement already satisfied: urllib3<1.25,>=1.21.1 in
./dependency/check/lib/python2.7/site-packages (from requests>=2.10.0->jira)
(1.24.1)
Collecting cryptography; extra == "signedtoken" (from
oauthlib[signedtoken]>=1.0.0->jira)
Using cached
https://files.pythonhosted.org/packages/c3/c1/cf8665c955c9393e9ff0872ba6cd3dc6f46ef915e94afcf6e0410508ca69/cryptography-2.6.1-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pyjwt>=1.0.0; extra == "signedtoken" (from
oauthlib[signedtoken]>=1.0.0->jira)
Using cached
https://files.pythonhosted.org/packages/87/8b/6a9f14b5f781697e51259d81657e6048fd31a113229cf346880bb7545565/PyJWT-1.7.1-py2.py3-none-any.whl
Requirement already satisfied: enum34; python_version < "3" in
./dependency/check/lib/python2.7/site-packages (from cryptography; extra ==
"signedtoken"->oauthlib[signedtoken]>=1.0.0->jira) (1.1.6)
Collecting cffi!=1.11.3,>=1.8 (from cryptography; extra ==
"signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached
https://files.pythonhosted.org/packages/9d/6f/aea9f5559fb593da07ff34e67513bd62483b45715b4a5f5fae6a0a5792ea/cffi-1.12.2-cp27-cp27mu-manylinux1_x86_64.whl
Collecting asn1crypto>=0.21.0 (from cryptography; extra ==
"signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached
https://files.pythonhosted.org/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting ipaddress; python_version < "3" (from cryptography; extra ==
"signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Using cached
https://files.pythonhosted.org/packages/fc/d0/7fc3a811e011d4b388be48a0e381db8d990042df54aa4ef4599a31d39853/ipaddress-1.0.22-py2.py3-none-any.whl
Collecting pycparser (from cffi!=1.11.3,>=1.8->cryptography; extra ==
"signedtoken"->oauthlib[signedtoken]>=1.0.0->jira)
Installing collected packages: pbr, funcsigs, mock, pycparser, cffi,
asn1crypto, ipaddress, cryptography, pyjwt, oauthlib, defusedxml,
requests-toolbelt, requests-oauthlib, jira, pyyaml
Successfully installed asn1crypto-0.24.0 cffi-1.12.2 cryptography-2.6.1
defusedxml-0.5.0 funcsigs-1.0.2 ipaddress-1.0.22 jira-2.0.0 mock-2.0.0
oauthlib-3.0.1 pbr-5.1.3 pycparser-2.19 pyjwt-1.7.1 pyyaml-5.1
requests-oauthlib-1.2.0 requests-toolbelt-0.9.1
cd $WORKSPACE/src/.test-infra/jenkins
python -m dependency_check.dependency_check_report_generator_test
jira_utils/jira_manager.py:43: YAMLLoadWarning: calling yaml.load() without
Loader=... is deprecated, as the default Loader is unsafe. Please read
https://msg.pyyaml.org/load for full details.
EINFO:root:
Start processing: - group1:artifact1 [0.rc1.0 -> 0.rc2.0]
.INFO:root:Clean Up Starts
.INFO:root:
Start processing: - group1:artifact1 (1.0.0, 2.0.0)
Traceback (most recent call last):
File "dependency_check/dependency_check_report_generator.py", line 107, in
prioritize_dependencies
dep_name, curr_ver, latest_ver = extract_single_dep(dep)
File "dependency_check/dependency_check_report_generator.py", line 76, in
extract_single_dep
raise InvalidFormatError("Failed to extract the dependency information:
{}".format(dep))
InvalidFormatError: Failed to extract the dependency information: -
group1:artifact1 (1.0.0, 2.0.0)
INFO:root:
Start processing: - group2:artifact2 [1.0.0 -> 2.0.0]
.INFO:root:
Start processing: - group1:artifact1 [1.0.0 -> 3.0.0]
INFO:root:
Start processing: - group2:artifact2 [1.0.0 -> 1.3.0]
INFO:root:
Start processing: - group3:artifact3 [1.0.0 -> 1.1.0]
INFO:root:
Start processing: - group4:artifact4 [1.0.0 -> 1.1.0]
.
======================================================================
ERROR: test_dep_with_nondigit_major_versions
(__main__.DependencyCheckReportGeneratorTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/dependency/check/local/lib/python2.7/site-packages/mock/mock.py";,>
line 1305, in patched
return func(*args, **keywargs)
File
"<https://builds.apache.org/job/beam_Dependency_Check/ws/src/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py";,>
line 118, in test_dep_with_nondigit_major_versions
report = prioritize_dependencies(deps, _SDK_TYPE)
File "dependency_check/dependency_check_report_generator.py", line 102, in
prioritize_dependencies
ReportGeneratorConfig.get_owners_file(sdk_type))
File "jira_utils/jira_manager.py", line 44, in __init__
TypeError: 'NoneType' object has no attribute '__getitem__'
----------------------------------------------------------------------
Ran 5 tests in 0.058s
FAILED (errors=1)
Test : test_dep_with_nondigit_major_versions
Test : test_dep_with_nondigit_minor_versions
Test : test_empty_dep_input
Test : test_invalid_dep_input
Test : test_normal_dep_input
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
