See <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/228/display/redirect?page=changes>
Changes: [Omkar_Deshpande] [BEAM-10829] pass kafka header to producer.send [Kyle Weaver] [BEAM-10849] Test that default Dataflow region option is set. [noreply] [Beam-4379] Make ParquetIO read splittable (#12223) [noreply] Identify pair-with-none DoFn in CombineGlobally with a named URN. [Kyle Weaver] [BEAM-10849] Refactor to fix DataflowRunnerTest mocks. [noreply] Merge pull request #12761 from [BEAM-10853] Fixing copy name issue: [noreply] Merge pull request #12618 from [BEAM-10669] Add support for Dataflow [noreply] [BEAM-10615] Eliminate nullability errors from [noreply] [BEAM-10720] Add implementation for elementwise str methods (#12749) [noreply] Badge fixes in README and PR Template (#12769) [noreply] [BEAM-5757] Add ElasticsearchIO: delete document support (#12670) [noreply] More logging for missing next work index. (#12718) ------------------------------------------ [...truncated 70.25 KB...] Collecting ptyprocess; os_name != "nt" Using cached ptyprocess-0.6.0-py2.py3-none-any.whl (39 kB) Requirement already satisfied: pbr>=0.11 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from mock>=2.0.0->tensorflow==1.14.0->-r requirements.txt (line 19)) (5.5.0) Requirement already satisfied: funcsigs>=1; python_version < "3.3" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from mock>=2.0.0->tensorflow==1.14.0->-r requirements.txt (line 19)) (1.0.2) Collecting h5py Using cached h5py-2.10.0-cp27-cp27mu-manylinux1_x86_64.whl (2.8 MB) Requirement already satisfied: setuptools in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from protobuf>=3.6.1->tensorflow==1.14.0->-r requirements.txt (line 19)) (44.1.1) Collecting markdown>=2.6.8 Using cached Markdown-3.1.1-py2.py3-none-any.whl (87 kB) Collecting werkzeug>=0.11.15 Using cached Werkzeug-1.0.1-py2.py3-none-any.whl (298 kB) Requirement already satisfied: avro<1.10.0,>=1.8.1; python_version < "3.0" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.9.2) Requirement already satisfied: crcmod<2.0,>=1.7 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.7) Requirement already satisfied: oauth2client<4,>=2.0.1 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (3.0.0) Requirement already satisfied: typing-extensions<3.8.0,>=3.7.0 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (3.7.4.3) Requirement already satisfied: future<1.0.0,>=0.18.2 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.18.2) Requirement already satisfied: pymongo<4.0.0,>=3.8.0 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (3.11.0) Requirement already satisfied: pyarrow<0.18.0,>=0.15.1; python_version >= "3.0" or platform_system != "Windows" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.16.0) Requirement already satisfied: requests<3.0.0,>=2.24.0 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (2.24.0) Requirement already satisfied: pyvcf<0.7.0,>=0.6.8; python_version < "3.0" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.6.8) Requirement already satisfied: typing<3.8.0,>=3.7.0; python_full_version < "3.5.3" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (3.7.4.3) Requirement already satisfied: pytz>=2018.3 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (2020.1) Requirement already satisfied: httplib2<0.18.0,>=0.8 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.17.4) Requirement already satisfied: hdfs<3.0.0,>=2.1.0 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (2.5.8) Requirement already satisfied: dill<0.3.2,>=0.3.1.1 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.3.1.1) Requirement already satisfied: fastavro<0.24,>=0.21.4 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.23.6) Requirement already satisfied: google-cloud-spanner<2,>=1.13.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.18.0) Requirement already satisfied: google-apitools<0.5.32,>=0.5.31; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.5.31) Requirement already satisfied: google-cloud-core<2,>=0.28.1; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.4.1) Requirement already satisfied: google-cloud-bigtable<2,>=0.31.1; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.4.0) Requirement already satisfied: grpcio-gcp<1,>=0.2.2; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.2.2) Requirement already satisfied: google-cloud-vision<2,>=0.38.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.0.0) Requirement already satisfied: google-cloud-language<2,>=1.3.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.3.0) Requirement already satisfied: google-cloud-videointelligence<2,>=1.8.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.15.0) Requirement already satisfied: google-cloud-dlp<2,>=0.12.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.0.0) Requirement already satisfied: google-cloud-pubsub<2,>=0.39.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.7.0) Requirement already satisfied: google-cloud-bigquery<2,>=1.6.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.27.2) Requirement already satisfied: google-auth<2,>=1.18.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.21.1) Requirement already satisfied: cachetools<4,>=3.1.0; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (3.1.1) Requirement already satisfied: google-cloud-datastore<2,>=1.7.1; extra == "gcp" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.15.0) Collecting backports.shutil-get-terminal-size; python_version == "2.7" Using cached backports.shutil_get_terminal_size-1.0.0-py2.py3-none-any.whl (6.5 kB) Requirement already satisfied: pathlib2; python_version == "2.7" or python_version == "3.3" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from IPython>=5.0->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (2.3.5) Collecting pexpect; sys_platform != "win32" Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB) Processing /home/jenkins/.cache/pip/wheels/1b/83/23/9d866729c0090bfc62d03a5bd43080fb07a5f4fad10a09399a/simplegeneric-0.8.1-py2-none-any.whl Collecting pickleshare Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB) Requirement already satisfied: pyparsing>=2.1.4 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from pydot<1.3,>=1.2.0->tensorflow-transform<0.14,>=0.13.0->-r requirements.txt (line 24)) (2.4.7) Requirement already satisfied: wcwidth in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from prompt-toolkit<2.0.0,>=1.0.0->jupyter-console->jupyter<2,>=1.0->-r requirements.txt (line 17)) (0.2.5) Requirement already satisfied: configparser>=3.5; python_version == "2.7" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from entrypoints>=0.2.2->nbconvert->jupyter<2,>=1.0->-r requirements.txt (line 17)) (4.0.2) Collecting webencodings Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB) Requirement already satisfied: packaging in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from bleach->nbconvert->jupyter<2,>=1.0->-r requirements.txt (line 17)) (20.4) Processing /home/jenkins/.cache/pip/wheels/c2/ea/a3/25af52265fad6418a74df0b8d9ca8b89e0b3735dbd4d0d3794/functools32-3.2.3.post2-py2-none-any.whl Processing /home/jenkins/.cache/pip/wheels/1f/d0/6d/4eb101c505cdc5e2be006684c0b93f3095a8ac5e0cf9abb35c/pyrsistent-0.16.0-cp27-cp27mu-linux_x86_64.whl Requirement already satisfied: attrs>=17.4.0 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from jsonschema!=2.5.0,>=2.4->nbformat->notebook<5.8,>=5.7.8->-r requirements.txt (line 18)) (20.1.0) Requirement already satisfied: importlib-metadata; python_version < "3.8" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from jsonschema!=2.5.0,>=2.4->nbformat->notebook<5.8,>=5.7.8->-r requirements.txt (line 18)) (1.7.0) Requirement already satisfied: rsa>=3.1.4 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (4.0) Requirement already satisfied: pyasn1>=0.1.7 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.4.8) Requirement already satisfied: pyasn1-modules>=0.0.5 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from oauth2client<4,>=2.0.1->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.2.8) Requirement already satisfied: certifi>=2017.4.17 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (2020.6.20) Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.25.10) Requirement already satisfied: idna<3,>=2.5 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (2.10) Requirement already satisfied: chardet<4,>=3.0.2 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from requests<3.0.0,>=2.24.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (3.0.4) Requirement already satisfied: docopt in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from hdfs<3.0.0,>=2.1.0->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.6.2) Requirement already satisfied: google-api-core[grpc,grpcgcp]<2.0.0dev,>=1.14.0 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from google-cloud-spanner<2,>=1.13.0; extra == "gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.22.2) Requirement already satisfied: grpc-google-iam-v1<0.13dev,>=0.12.3 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from google-cloud-spanner<2,>=1.13.0; extra == "gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.12.3) Requirement already satisfied: fasteners>=0.14 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from google-apitools<0.5.32,>=0.5.31; extra == "gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (0.15) Requirement already satisfied: google-resumable-media<2.0dev,>=0.5.0 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from google-cloud-bigquery<2,>=1.6.0; extra == "gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.0.0) Requirement already satisfied: scandir; python_version < "3.5" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from pathlib2; python_version == "2.7" or python_version == "3.3"->IPython>=5.0->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.10.0) Requirement already satisfied: backports.functools-lru-cache>=1.2.1; python_version < "3.2" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from wcwidth->prompt-toolkit<2.0.0,>=1.0.0->jupyter-console->jupyter<2,>=1.0->-r requirements.txt (line 17)) (1.6.1) Requirement already satisfied: contextlib2; python_version < "3" in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from importlib-metadata; python_version < "3.8"->jsonschema!=2.5.0,>=2.4->nbformat->notebook<5.8,>=5.7.8->-r requirements.txt (line 18)) (0.6.0.post1) Requirement already satisfied: zipp>=0.5 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from importlib-metadata; python_version < "3.8"->jsonschema!=2.5.0,>=2.4->nbformat->notebook<5.8,>=5.7.8->-r requirements.txt (line 18)) (1.2.0) Requirement already satisfied: monotonic>=0.1 in <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/lib/python2.7/site-packages> (from fasteners>=0.14->google-apitools<0.5.32,>=0.5.31; extra == "gcp"->apache-beam[gcp]<3,>=2.11->tensorflow-data-validation<0.14,>=0.13.1->-r requirements.txt (line 20)) (1.5) Installing collected packages: decorator, ipython-genutils, traitlets, jupyter-core, backports-abc, tornado, pyzmq, jupyter-client, backports.shutil-get-terminal-size, prompt-toolkit, ptyprocess, pexpect, pygments, simplegeneric, pickleshare, IPython, ipykernel, qtpy, qtconsole, functools32, pyrsistent, jsonschema, nbformat, Send2Trash, MarkupSafe, jinja2, mistune, defusedxml, entrypoints, webencodings, bleach, pandocfilters, testpath, nbconvert, terminado, prometheus-client, notebook, widgetsnbextension, ipywidgets, jupyter-console, jupyter, termcolor, absl-py, backports.weakref, tensorflow-estimator, h5py, keras-applications, wrapt, google-pasta, keras-preprocessing, gast, astor, markdown, werkzeug, tensorboard, tensorflow, scipy, scikit-learn, tensorflow-metadata, joblib, pydot, tensorflow-transform, tensorflow-data-validation, tensorflow-model-analysis, tensorflow-serving-api Attempting uninstall: pydot Found existing installation: pydot 1.4.1 Uninstalling pydot-1.4.1: Successfully uninstalled pydot-1.4.1 Successfully installed IPython-5.10.0 MarkupSafe-1.1.1 Send2Trash-1.5.0 absl-py-0.10.0 astor-0.8.1 backports-abc-0.5 backports.shutil-get-terminal-size-1.0.0 backports.weakref-1.0.post1 bleach-3.1.5 decorator-4.4.2 defusedxml-0.6.0 entrypoints-0.3 functools32-3.2.3.post2 gast-0.4.0 google-pasta-0.2.0 h5py-2.10.0 ipykernel-4.10.1 ipython-genutils-0.2.0 ipywidgets-7.5.1 jinja2-2.11.2 joblib-0.14.1 jsonschema-3.2.0 jupyter-1.0.0 jupyter-client-5.3.5 jupyter-console-5.2.0 jupyter-core-4.6.3 keras-applications-1.0.8 keras-preprocessing-1.1.2 markdown-3.1.1 mistune-0.8.4 nbconvert-5.6.1 nbformat-4.4.0 notebook-5.7.10 pandocfilters-1.4.2 pexpect-4.8.0 pickleshare-0.7.5 prometheus-client-0.8.0 prompt-toolkit-1.0.18 ptyprocess-0.6.0 pydot-1.2.4 pygments-2.5.2 pyrsistent-0.16.0 pyzmq-19.0.2 qtconsole-4.7.7 qtpy-1.9.0 scikit-learn-0.20.4 scipy-1.1.0 simplegeneric-0.8.1 tensorboard-1.14.0 tensorflow-1.14.0 tensorflow-data-validation-0.13.1 tensorflow-estimator-1.14.0 tensorflow-metadata-0.13.0 tensorflow-model-analysis-0.13.2 tensorflow-serving-api-1.13.1 tensorflow-transform-0.13.0 termcolor-1.1.0 terminado-0.8.3 testpath-0.4.4 tornado-5.1.1 traitlets-4.3.3 webencodings-0.5.1 werkzeug-1.0.1 widgetsnbextension-3.5.1 wrapt-1.12.1 > Task :sdks:python:test-suites:portable:py2:chicagoTaxiExample Using GCP project: apache-beam-testing Job output path: gs://temp-storage-for-perf-tests/chicago-taxi/chicago-taxi-tfdv-20200904-142623/chicago_taxi_output TFDV output path: gs://temp-storage-for-perf-tests/chicago-taxi/chicago-taxi-tfdv-20200904-142623/chicago_taxi_output/tfdv_output Starting distributed TFDV stats computation and schema generation... <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/__init__.py>:82: UserWarning: You are using the final Apache Beam release with Python 2 support. New releases of Apache Beam will require Python 3.6 or a newer version. 'You are using the final Apache Beam release with Python 2 support. ' <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/azure/storage/blob/_shared/encryption.py>:15: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release. from cryptography.hazmat.backends import default_backend <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/io/gcp/bigquery.py>:1875: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported temp_location = pcoll.pipeline.options.view_as( WARNING:tensorflow:From <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/tensorflow_transform/analyzers.py>:948: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. WARNING:tensorflow:From <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/tensorflow_transform/analyzers.py>:948: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead. 2020-09-04 14:26:30.138480: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2020-09-04 14:26:30.152174: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz 2020-09-04 14:26:30.154040: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x8623d80 executing computations on platform Host. Devices: 2020-09-04 14:26:30.154073: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined> 2020-09-04 14:26:30.166811: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1412] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set. If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU. To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile. WARNING:tensorflow:From <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/tensorflow_transform/analyzers.py>:994: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead. WARNING:tensorflow:From <https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/tensorflow_transform/analyzers.py>:994: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead. warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md warning: check: missing required meta-data: url warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied WARNING:root:Make sure that locally built Python SDK docker image has Python 2.7 interpreter. ERROR:root:java.lang.RuntimeException: Error received from SDK harness for instruction 8: Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 258, in _execute response = task() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 315, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 485, in do_instruction getattr(request, request_type), request.instruction_id) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 514, in process_bundle instruction_id, request.process_bundle_descriptor_id) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 387, in get self.data_channel_factory) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 853, in __init__ self.ops = self.create_execution_tree(self.process_bundle_descriptor) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 908, in create_execution_tree descriptor.transforms, key=topological_height, reverse=True) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in get_operation pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in <dictcomp> pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in get_operation pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in <dictcomp> pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in get_operation pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in <dictcomp> pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in get_operation pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in <dictcomp> pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 894, in get_operation transform_id, transform_consumers) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1181, in create_operation return creator(self, transform_id, transform_proto, payload, consumers) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1519, in create_par_do assert parameter.do_fn.urn == python_urns.PICKLED_DOFN_INFO AssertionError Traceback (most recent call last): File "tfdv_analyze_and_validate.py", line 228, in <module> main() File "tfdv_analyze_and_validate.py", line 213, in main project=known_args.metric_reporting_project) File "tfdv_analyze_and_validate.py", line 133, in compute_stats result.wait_until_finish() File "<https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/build/gradleenv/1866363813/local/lib/python2.7/site-packages/apache_beam/runners/portability/portable_runner.py";,> line 551, in wait_until_finish raise self._runtime_exception RuntimeError: Pipeline chicago-taxi-tfdv-20200904-142623_90d6da75-e9ba-4904-92cb-229247232af6 failed in state FAILED: java.lang.RuntimeException: Error received from SDK harness for instruction 8: Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 258, in _execute response = task() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 315, in <lambda> lambda: self.create_worker().do_instruction(request), request) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 485, in do_instruction getattr(request, request_type), request.instruction_id) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 514, in process_bundle instruction_id, request.process_bundle_descriptor_id) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/sdk_worker.py", line 387, in get self.data_channel_factory) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 853, in __init__ self.ops = self.create_execution_tree(self.process_bundle_descriptor) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 908, in create_execution_tree descriptor.transforms, key=topological_height, reverse=True) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in get_operation pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in <dictcomp> pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in get_operation pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in <dictcomp> pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in get_operation pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in <dictcomp> pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in get_operation pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 891, in <dictcomp> pcoll_id in descriptor.transforms[transform_id].outputs.items() File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 797, in wrapper result = cache[args] = func(*args) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 894, in get_operation transform_id, transform_consumers) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1181, in create_operation return creator(self, transform_id, transform_proto, payload, consumers) File "/usr/local/lib/python2.7/site-packages/apache_beam/runners/worker/bundle_processor.py", line 1519, in create_par_do assert parameter.do_fn.urn == python_urns.PICKLED_DOFN_INFO AssertionError > Task :sdks:python:test-suites:portable:py2:chicagoTaxiExample FAILED FAILURE: Build failed with an exception. * Where: Build file '<https://ci-beam.apache.org/job/beam_PostCommit_Python_Chicago_Taxi_Flink/ws/src/sdks/python/test-suites/portable/py2/build.gradle'> line: 125 * What went wrong: Execution failed for task ':sdks:python:test-suites:portable:py2:chicagoTaxiExample'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 6.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/5.2.1/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 6m 37s 6 actionable tasks: 6 executed Publishing build scan... https://gradle.com/s/7jrm2jkns2k32 Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
