See
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/72/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-9263] Bump up python sdk dataflow environment major versions
[jozsi] Update Jet version to 4.0
[kyoungha] [BEAM-9325] Added Proper Write Method in UnownedOutputStream
[kyoungha] fix format warning
[kcweaver] [BEAM-9638] Strengthen worker region & zone options tests.
[samuelw] [BEAM-9399] Change the redirection of System.err to be a custom
[kyoungha] [BEAM-9325] reflect comment : inline testing methods
[boyuanz] [BEAM-9454] Add Deduplication PTransform
[robertwb] [BEAM-9577] Rename the Artifact{Staging,Retrieval}Service.
[robertwb] [BEAM-9577] Define the new Artifact{Staging,Retrieval}Service.
[robertwb] [BEAM-9577] Regenerate protos.
[jozsi] Update Jet Runner web page with info about 4.0
[jozsi] Add Beam-Jet compatibility table
[robertwb] [BEAM-9577] Implement the new Artifact{Staging,Retrieval}Services in
[samuelw] Fix missing test import
[kyoungha] [BEAM-9325] reflect comment : Fix JAXBCoder + change test
[chamikara] Refactors X-Lang test pipelines.
[pabloem] [BEAM-9608] Increase reliance on Context Managers for FnApiRunner
------------------------------------------
[...truncated 120.81 KB...]
Using cached terminado-0.8.3-py2.py3-none-any.whl (33 kB)
Collecting ipython-genutils
Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)
Processing
/home/jenkins/.cache/pip/wheels/8e/28/49/fad4e7f0b9a1227708cbbee4487ac8558a7334849cb81c813d/absl_py-0.9.0-cp27-none-any.whl
Collecting backports.weakref>=1.0rc1
Using cached backports.weakref-1.0.post1-py2.py3-none-any.whl (5.2 kB)
Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0
Using cached tensorflow_estimator-1.14.0-py2.py3-none-any.whl (488 kB)
Requirement already satisfied: grpcio>=1.8.6 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow==1.14.0->-r requirements.txt (line 19)) (1.27.2)
Collecting tensorboard<1.15.0,>=1.14.0
Using cached tensorboard-1.14.0-py2-none-any.whl (3.1 MB)
Processing
/home/jenkins/.cache/pip/wheels/dd/f2/5d/2689b5547f32c4e258c3b7ccbe7f1d0f2afbb84fb01e830792/Keras_Applications-1.0.8-cp27-none-any.whl
Processing
/home/jenkins/.cache/pip/wheels/5b/d8/8e/81a83cb5321b940a954996f5b57fddc8976e712b3ac3a1a54b/wrapt-1.12.1-cp27-cp27mu-linux_x86_64.whl
Requirement already satisfied: protobuf>=3.6.1 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow==1.14.0->-r requirements.txt (line 19)) (3.11.3)
Collecting google-pasta>=0.1.6
Using cached google_pasta-0.2.0-py2-none-any.whl (57 kB)
Collecting keras-preprocessing>=1.0.5
Using cached Keras_Preprocessing-1.1.0-py2.py3-none-any.whl (41 kB)
Requirement already satisfied: mock>=2.0.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow==1.14.0->-r requirements.txt (line 19)) (2.0.0)
Requirement already satisfied: numpy<2.0,>=1.14.5 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow==1.14.0->-r requirements.txt (line 19)) (1.16.6)
Requirement already satisfied: six>=1.10.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow==1.14.0->-r requirements.txt (line 19)) (1.14.0)
Requirement already satisfied: wheel in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow==1.14.0->-r requirements.txt (line 19)) (0.34.2)
Collecting astor>=0.6.0
Using cached astor-0.8.1-py2.py3-none-any.whl (27 kB)
Requirement already satisfied: enum34>=1.1.6 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow==1.14.0->-r requirements.txt (line 19)) (1.1.10)
Processing
/home/jenkins/.cache/pip/wheels/7c/06/54/bc84598ba1daf8f970247f550b175aaaee85f68b4b0c5ab2c6/termcolor-1.1.0-cp27-none-any.whl
Collecting gast>=0.2.0
Using cached gast-0.3.3-py2.py3-none-any.whl (9.7 kB)
Collecting scikit-learn<1,>=0.18
Using cached scikit_learn-0.20.4-cp27-cp27mu-manylinux1_x86_64.whl (5.5 MB)
Requirement already satisfied: apache-beam[gcp]<3,>=2.11 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20))
(2.21.0.dev0)
Collecting joblib<1,>=0.12
Using cached joblib-0.14.1-py2.py3-none-any.whl (294 kB)
Collecting IPython>=5.0
Using cached ipython-5.9.0-py2-none-any.whl (760 kB)
Requirement already satisfied: pandas<1,>=0.18 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20))
(0.24.2)
Requirement already satisfied: googleapis-common-protos in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorflow-metadata<0.14,>=0.13.0->-r requirements.txt (line 21))
(1.51.0)
Collecting scipy==1.1.0
Using cached scipy-1.1.0-cp27-cp27mu-manylinux1_x86_64.whl (30.8 MB)
Processing
/home/jenkins/.cache/pip/wheels/6a/a5/14/25541ebcdeaf97a37b6d05c7ff15f5bd20f5e91b99d313e5b4/pydot-1.2.4-cp27-none-any.whl
Collecting qtpy
Using cached QtPy-1.9.0-py2.py3-none-any.whl (54 kB)
Collecting pygments
Using cached Pygments-2.5.2-py2.py3-none-any.whl (896 kB)
Collecting widgetsnbextension~=3.5.0
Using cached widgetsnbextension-3.5.1-py2.py3-none-any.whl (2.2 MB)
Collecting prompt-toolkit<2.0.0,>=1.0.0
Using cached prompt_toolkit-1.0.18-py2-none-any.whl (245 kB)
Collecting mistune<2,>=0.8.1
Using cached mistune-0.8.4-py2.py3-none-any.whl (16 kB)
Collecting defusedxml
Using cached defusedxml-0.6.0-py2.py3-none-any.whl (23 kB)
Collecting entrypoints>=0.2.2
Using cached entrypoints-0.3-py2.py3-none-any.whl (11 kB)
Collecting bleach
Downloading bleach-3.1.4-py2.py3-none-any.whl (151 kB)
Processing
/home/jenkins/.cache/pip/wheels/39/01/56/f1b08a6275acc59e846fa4c1e1b65dbc1919f20157d9e66c20/pandocfilters-1.4.2-cp27-none-any.whl
Collecting testpath
Using cached testpath-0.4.4-py2.py3-none-any.whl (163 kB)
Collecting jsonschema!=2.5.0,>=2.4
Using cached jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting decorator
Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting MarkupSafe>=0.23
Using cached MarkupSafe-1.1.1-cp27-cp27mu-manylinux1_x86_64.whl (24 kB)
Requirement already satisfied: python-dateutil>=2.1 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from jupyter-client>=5.2.0->notebook<5.8,>=5.7.8->-r requirements.txt (line
18)) (2.8.1)
Collecting backports-abc>=0.4
Using cached backports_abc-0.5-py2.py3-none-any.whl (5.2 kB)
Requirement already satisfied: singledispatch in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tornado<7,>=4.1->notebook<5.8,>=5.7.8->-r requirements.txt (line 18))
(3.4.0.3)
Requirement already satisfied: futures in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tornado<7,>=4.1->notebook<5.8,>=5.7.8->-r requirements.txt (line 18))
(3.3.0)
Collecting ptyprocess; os_name != "nt"
Using cached ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB)
Collecting markdown>=2.6.8
Using cached Markdown-3.1.1-py2.py3-none-any.whl (87 kB)
Requirement already satisfied: setuptools>=41.0.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from tensorboard<1.15.0,>=1.14.0->tensorflow==1.14.0->-r requirements.txt
(line 19)) (44.1.0)
Collecting werkzeug>=0.11.15
Downloading Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB)
Collecting h5py
Using cached h5py-2.10.0-cp27-cp27mu-manylinux1_x86_64.whl (2.8 MB)
Requirement already satisfied: pbr>=0.11 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from mock>=2.0.0->tensorflow==1.14.0->-r requirements.txt (line 19)) (5.4.4)
Requirement already satisfied: funcsigs>=1; python_version < "3.3" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from mock>=2.0.0->tensorflow==1.14.0->-r requirements.txt (line 19)) (1.0.2)
Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (2.5.8)
Requirement already satisfied: avro<1.10.0,>=1.8.1; python_version < "3.0" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.9.2)
Requirement already satisfied: httplib2<=0.12.0,>=0.8 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.12.0)
Requirement already satisfied: crcmod<2.0,>=1.7 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.7)
Requirement already satisfied: oauth2client<4,>=2.0.1 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (3.0.0)
Requirement already satisfied: typing-extensions<3.8.0,>=3.7.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (3.7.4.1)
Requirement already satisfied: typing<3.8.0,>=3.7.0; python_version < "3.5.3"
in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (3.7.4.1)
Requirement already satisfied: fastavro<0.22,>=0.21.4 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.21.24)
Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (3.10.1)
Requirement already satisfied: future<1.0.0,>=0.16.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.16.0)
Requirement already satisfied: pytz>=2018.3 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (2019.3)
Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.3.1.1)
Requirement already satisfied: pyarrow<0.17.0,>=0.15.1; python_version >= "3.0"
or platform_system != "Windows" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.16.0)
Requirement already satisfied: pyvcf<0.7.0,>=0.6.8; python_version < "3.0" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.6.8)
Requirement already satisfied: cachetools<4,>=3.1.0; extra == "gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (3.1.1)
Requirement already satisfied: google-cloud-dlp<=0.13.0,>=0.12.0; extra ==
"gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.13.0)
Requirement already satisfied: google-cloud-core<2,>=0.28.1; extra == "gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.3.0)
Requirement already satisfied: grpcio-gcp<1,>=0.2.2; extra == "gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.2.2)
Requirement already satisfied: google-cloud-pubsub<1.1.0,>=0.39.0; extra ==
"gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.0.2)
Requirement already satisfied: google-cloud-videointelligence<1.14.0,>=1.8.0;
extra == "gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.13.0)
Requirement already satisfied:
proto-google-cloud-datastore-v1<=0.90.4,>=0.90.0; python_version < "3.0" and
extra == "gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.90.4)
Requirement already satisfied: googledatastore<7.1,>=7.0.1; python_version <
"3.0" and extra == "gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (7.0.2)
Requirement already satisfied: google-cloud-spanner<1.14.0,>=1.13.0; extra ==
"gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.13.0)
Requirement already satisfied: google-cloud-bigtable<1.1.0,>=0.31.1; extra ==
"gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.0.0)
Requirement already satisfied: google-cloud-vision<0.43.0,>=0.38.0; extra ==
"gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.42.0)
Requirement already satisfied: google-cloud-language<2,>=1.3.0; extra == "gcp"
in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.3.0)
Requirement already satisfied: google-cloud-bigquery<=1.24.0,>=1.6.0; extra ==
"gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.24.0)
Requirement already satisfied: google-apitools<0.5.29,>=0.5.28; extra == "gcp"
in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.5.28)
Requirement already satisfied: google-cloud-datastore<1.8.0,>=1.7.1; extra ==
"gcp" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.7.4)
Collecting backports.shutil-get-terminal-size; python_version == "2.7"
Using cached backports.shutil_get_terminal_size-1.0.0-py2.py3-none-any.whl
(6.5 kB)
Requirement already satisfied: pathlib2; python_version == "2.7" or
python_version == "3.3" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from IPython>=5.0->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (2.3.5)
Collecting pexpect; sys_platform != "win32"
Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)
Processing
/home/jenkins/.cache/pip/wheels/a9/28/53/f24776b4c5bcbe91aaf1f1e247bd6fadd17191aa12fac63902/simplegeneric-0.8.1-cp27-none-any.whl
Collecting pickleshare
Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)
Requirement already satisfied: pyparsing>=2.1.4 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from pydot<1.3,>=1.2.0->tensorflow-transform<0.14,>=0.13.0->-r
requirements.txt (line 24)) (2.4.6)
Requirement already satisfied: wcwidth in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from prompt-toolkit<2.0.0,>=1.0.0->jupyter-console->jupyter<2,>=1.0->-r
requirements.txt (line 17)) (0.1.9)
Requirement already satisfied: configparser>=3.5; python_version == "2.7" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from entrypoints>=0.2.2->nbconvert->jupyter<2,>=1.0->-r requirements.txt
(line 17)) (4.0.2)
Collecting webencodings
Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Processing
/home/jenkins/.cache/pip/wheels/b5/18/32/77a1030457155606ba5e3ec3a8a57132b1a04b1c4f765177b2/functools32-3.2.3.post2-cp27-none-any.whl
Requirement already satisfied: attrs>=17.4.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from jsonschema!=2.5.0,>=2.4->nbformat->notebook<5.8,>=5.7.8->-r
requirements.txt (line 18)) (19.3.0)
Collecting pyrsistent>=0.14.0
Downloading pyrsistent-0.16.0.tar.gz (108 kB)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from jsonschema!=2.5.0,>=2.4->nbformat->notebook<5.8,>=5.7.8->-r
requirements.txt (line 18)) (1.6.0)
Requirement already satisfied: docopt in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.6.2)
Requirement already satisfied: requests>=2.7.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (2.23.0)
Requirement already satisfied: rsa>=3.1.4 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
oauth2client<4,>=2.0.1->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (4.0)
Requirement already satisfied: pyasn1>=0.1.7 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
oauth2client<4,>=2.0.1->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.4.8)
Requirement already satisfied: pyasn1-modules>=0.0.5 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
oauth2client<4,>=2.0.1->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.2.8)
Requirement already satisfied: google-api-core[grpc]<2.0.0dev,>=1.14.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from google-cloud-dlp<=0.13.0,>=0.12.0; extra ==
"gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.16.0)
Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from google-cloud-pubsub<1.1.0,>=0.39.0; extra ==
"gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.12.3)
Requirement already satisfied: google-auth<2.0dev,>=1.9.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from google-cloud-bigquery<=1.24.0,>=1.6.0; extra ==
"gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.12.0)
Requirement already satisfied: google-resumable-media<0.6dev,>=0.5.0 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from google-cloud-bigquery<=1.24.0,>=1.6.0; extra ==
"gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.5.0)
Requirement already satisfied: fasteners>=0.14 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from google-apitools<0.5.29,>=0.5.28; extra ==
"gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (0.15)
Requirement already satisfied: scandir; python_version < "3.5" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from pathlib2; python_version == "2.7" or python_version ==
"3.3"->IPython>=5.0->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.10.0)
Requirement already satisfied: contextlib2; python_version < "3" in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from importlib-metadata; python_version <
"3.8"->jsonschema!=2.5.0,>=2.4->nbformat->notebook<5.8,>=5.7.8->-r
requirements.txt (line 18)) (0.6.0.post1)
Requirement already satisfied: zipp>=0.5 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from importlib-metadata; python_version <
"3.8"->jsonschema!=2.5.0,>=2.4->nbformat->notebook<5.8,>=5.7.8->-r
requirements.txt (line 18)) (1.2.0)
Requirement already satisfied: certifi>=2017.4.17 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (2019.11.28)
Requirement already satisfied: idna<3,>=2.5 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (2.9)
Requirement already satisfied: chardet<4,>=3.0.2 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from
requests>=2.7.0->hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.25.8)
Requirement already satisfied: monotonic>=0.1 in
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages>
(from fasteners>=0.14->google-apitools<0.5.29,>=0.5.28; extra ==
"gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r
requirements.txt (line 20)) (1.5)
Building wheels for collected packages: pyrsistent
Building wheel for pyrsistent (setup.py): started
Building wheel for pyrsistent (setup.py): finished with status 'done'
Created wheel for pyrsistent:
filename=pyrsistent-0.16.0-cp27-cp27mu-linux_x86_64.whl size=93321
sha256=01ea995076176de10a146c3fcae00f5ae2a2cb155b1704ff599416a1913b5f30
Stored in directory:
/home/jenkins/.cache/pip/wheels/1f/d0/6d/4eb101c505cdc5e2be006684c0b93f3095a8ac5e0cf9abb35c
Successfully built pyrsistent
Installing collected packages: decorator, ipython-genutils, traitlets,
backports.shutil-get-terminal-size, ptyprocess, pexpect, prompt-toolkit,
simplegeneric, pickleshare, pygments, IPython, backports-abc, tornado, pyzmq,
jupyter-core, jupyter-client, ipykernel, qtpy, qtconsole, mistune, defusedxml,
entrypoints, MarkupSafe, jinja2, webencodings, bleach, pandocfilters,
functools32, pyrsistent, jsonschema, nbformat, testpath, nbconvert,
prometheus-client, ipaddress, Send2Trash, terminado, notebook,
widgetsnbextension, ipywidgets, jupyter-console, jupyter, absl-py,
backports.weakref, tensorflow-estimator, markdown, werkzeug, tensorboard, h5py,
keras-applications, wrapt, google-pasta, keras-preprocessing, astor, termcolor,
gast, tensorflow, scipy, scikit-learn, tensorflow-metadata, joblib, pydot,
tensorflow-transform, tensorflow-data-validation, tensorflow-model-analysis,
tensorflow-serving-api
Attempting uninstall: pydot
Found existing installation: pydot 1.4.1
Uninstalling pydot-1.4.1:
Successfully uninstalled pydot-1.4.1
Successfully installed IPython-5.9.0 MarkupSafe-1.1.1 Send2Trash-1.5.0
absl-py-0.9.0 astor-0.8.1 backports-abc-0.5
backports.shutil-get-terminal-size-1.0.0 backports.weakref-1.0.post1
bleach-3.1.4 decorator-4.4.2 defusedxml-0.6.0 entrypoints-0.3
functools32-3.2.3.post2 gast-0.3.3 google-pasta-0.2.0 h5py-2.10.0
ipaddress-1.0.23 ipykernel-4.10.1 ipython-genutils-0.2.0 ipywidgets-7.5.1
jinja2-2.11.1 joblib-0.14.1 jsonschema-3.2.0 jupyter-1.0.0 jupyter-client-5.3.4
jupyter-console-5.2.0 jupyter-core-4.6.3 keras-applications-1.0.8
keras-preprocessing-1.1.0 markdown-3.1.1 mistune-0.8.4 nbconvert-5.6.1
nbformat-4.4.0 notebook-5.7.8 pandocfilters-1.4.2 pexpect-4.8.0
pickleshare-0.7.5 prometheus-client-0.7.1 prompt-toolkit-1.0.18
ptyprocess-0.6.0 pydot-1.2.4 pygments-2.5.2 pyrsistent-0.16.0 pyzmq-19.0.0
qtconsole-4.7.2 qtpy-1.9.0 scikit-learn-0.20.4 scipy-1.1.0 simplegeneric-0.8.1
tensorboard-1.14.0 tensorflow-1.14.0 tensorflow-data-validation-0.13.1
tensorflow-estimator-1.14.0 tensorflow-metadata-0.13.0
tensorflow-model-analysis-0.13.2 tensorflow-serving-api-1.13.1
tensorflow-transform-0.13.0 termcolor-1.1.0 terminado-0.8.3 testpath-0.4.4
tornado-5.1.1 traitlets-4.3.3 webencodings-0.5.1 werkzeug-1.0.1
widgetsnbextension-3.5.1 wrapt-1.12.1
> Task :sdks:python:test-suites:portable:py2:chicagoTaxiExample
Using GCP project: apache-beam-testing
Job output path:
gs://temp-storage-for-perf-tests/chicago-taxi/chicago-taxi-tfdv-20200401-143636/chicago_taxi_output
TFDV output path:
gs://temp-storage-for-perf-tests/chicago-taxi/chicago-taxi-tfdv-20200401-143636/chicago_taxi_output/tfdv_output
Starting distributed TFDV stats computation and schema generation...
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/__init__.py>:82:
UserWarning: You are using Apache Beam with Python 2. New releases of Apache
Beam will soon support Python 3 only.
'You are using Apache Beam with Python 2. '
tfdv_analyze_and_validate.py:117: FutureWarning: _ReadFromBigQuery is
experimental.
| 'ConvertToTFDVInput' >> beam.Map(
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py>:1618:
BeamDeprecationWarning: options is deprecated since First stable release.
References to <pipeline>.options will not be supported
temp_location = pcoll.pipeline.options.view_as(
WARNING:tensorflow:From
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/tensorflow_transform/analyzers.py>:948:
The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
WARNING:tensorflow:From
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/tensorflow_transform/analyzers.py>:948:
The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.
2020-04-01 14:36:42.131269: I
tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports
instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2020-04-01 14:36:42.141900: I
tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:
2300000000 Hz
2020-04-01 14:36:42.143448: I tensorflow/compiler/xla/service/service.cc:168]
XLA service 0x8859ea0 executing computations on platform Host. Devices:
2020-04-01 14:36:42.143481: I tensorflow/compiler/xla/service/service.cc:175]
StreamExecutor device (0): <undefined>, <undefined>
2020-04-01 14:36:42.157356: W
tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning):
Not using XLA:CPU for cluster because envvar
TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either
set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm
that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper
command-line flag, not via TF_XLA_FLAGS) or set the envvar
XLA_FLAGS=--xla_hlo_profile.
WARNING:tensorflow:From
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/tensorflow_transform/analyzers.py>:994:
The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder
instead.
WARNING:tensorflow:From
<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/tensorflow_transform/analyzers.py>:994:
The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder
instead.
WARNING:root:Make sure that locally built Python SDK docker image has Python
2.7 interpreter.
warning: sdist: standard file not found: should have one of README, README.rst,
README.txt, README.md
warning: check: missing required meta-data: url
warning: check: missing meta-data: either (author and author_email) or
(maintainer and maintainer_email) must be supplied
Traceback (most recent call last):
File "tfdv_analyze_and_validate.py", line 227, in <module>
main()
File "tfdv_analyze_and_validate.py", line 212, in main
project=known_args.metric_reporting_project)
File "tfdv_analyze_and_validate.py", line 131, in compute_stats
result = pipeline.run()
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/pipeline.py";,>
line 498, in run
self._options).run(False)
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/pipeline.py";,>
line 511, in run
return self.runner.run_pipeline(self, self._options)
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/runners/portability/portable_runner.py";,>
line 406, in run_pipeline
job_service_handle.submit(proto_pipeline)
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/runners/portability/portable_runner.py";,>
line 107, in submit
prepare_response.staging_session_token)
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/runners/portability/portable_runner.py";,>
line 204, in stage
stager.stage_job_resources(resources, staging_location='')
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/runners/portability/stager.py";,>
line 305, in stage_job_resources
file_path, FileSystems.join(staging_location, staged_path))
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/runners/portability/portable_stager.py";,>
line 98, in stage_artifact
self._artifact_staging_stub.PutArtifact(artifact_request_generator())
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 1011, in __call__
return _end_unary_response_blocking(state, call, False, None)
File
"<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/grpc/_channel.py";,>
line 729, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNIMPLEMENTED
details = "Method not found:
org.apache.beam.model.job_management.v1.LegacyArtifactStagingService/PutArtifact"
debug_error_string =
"{"created":"@1585751807.391901784","description":"Error received from peer
ipv4:127.0.0.1:8098","file":"src/core/lib/surface/call.cc","file_line":1056,"grpc_message":"Method
not found:
org.apache.beam.model.job_management.v1.LegacyArtifactStagingService/PutArtifact","grpc_status":12}"
>
> Task :sdks:python:test-suites:portable:py2:chicagoTaxiExample FAILED
FAILURE: Build failed with an exception.
* Where:
Build file
'<https://builds.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/sdks/python/test-suites/portable/py2/build.gradle'>
line: 197
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py2:chicagoTaxiExample'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug
option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 6.0.
Use '--warning-mode all' to show the individual deprecation warnings.
See
https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 2m 0s
6 actionable tasks: 5 executed, 1 up-to-date
Publishing build scan...
https://gradle.com/s/a2gdlpsl54g4y
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
