See <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/4695/display/redirect?page=changes>
Changes: [suztomo] [BEAM-11205] Added libraries-bom version to CHANGES.md [zhoufek] [BEAM-11811] Disallow num_workers > max_num_workers for Go Dataflow [noreply] [BEAM-11811] Disallow num_workers > max_num_workers for Python [noreply] Minor: Remove pandas 0.x information in dataframe schema notes (#14965) [noreply] [BEAM-12074] Define docstrings for all DataFrame operations (#14876) ------------------------------------------ [...truncated 40.15 KB...] Collecting grpcio>=1.37.0 Using cached grpcio-1.38.0-cp38-cp38-manylinux2014_x86_64.whl (4.2 MB) Collecting protobuf<4.0dev,>=3.5.0.post1 Using cached protobuf-3.17.3-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.whl (1.0 MB) Requirement already satisfied: setuptools in <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/build/gradleenv/2022703442/lib/python3.8/site-packages> (from grpcio-tools==1.37.0->-r <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/build-requirements.txt> (line 17)) (57.0.0) Collecting pyparsing>=2.0.2 Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Collecting appdirs<2,>=1.4.3 Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB) Installing collected packages: six, pyparsing, filelock, distlib, appdirs, virtualenv, toml, py, protobuf, pluggy, packaging, grpcio, tox, mypy-protobuf, grpcio-tools, future Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12 future-0.18.2 grpcio-1.38.0 grpcio-tools-1.37.0 mypy-protobuf-1.18 packaging-20.9 pluggy-0.13.1 protobuf-3.17.3 py-1.10.0 pyparsing-2.4.7 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.4.7 > Task :sdks:python:test-suites:portable:py36:setupVirtualenv Collecting grpcio>=1.37.0 Using cached grpcio-1.38.0-cp36-cp36m-manylinux2014_x86_64.whl (4.2 MB) Collecting zipp>=0.5 Using cached zipp-3.4.1-py3-none-any.whl (5.2 kB) Collecting pyparsing>=2.0.2 Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB) Collecting importlib-resources>=1.0 Using cached importlib_resources-5.1.4-py3-none-any.whl (26 kB) Collecting appdirs<2,>=1.4.3 Using cached appdirs-1.4.4-py2.py3-none-any.whl (9.6 kB) Installing collected packages: zipp, six, pyparsing, importlib-resources, importlib-metadata, filelock, distlib, appdirs, virtualenv, toml, py, protobuf, pluggy, packaging, grpcio, tox, mypy-protobuf, grpcio-tools, future Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12 future-0.18.2 grpcio-1.38.0 grpcio-tools-1.37.0 importlib-metadata-2.1.1 importlib-resources-5.1.4 mypy-protobuf-1.18 packaging-20.9 pluggy-0.13.1 protobuf-3.17.3 py-1.10.0 pyparsing-2.4.7 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.4.7 zipp-3.4.1 > Task :sdks:python:test-suites:portable:py37:setupVirtualenv Successfully installed appdirs-1.4.4 distlib-0.3.1 filelock-3.0.12 future-0.18.2 grpcio-1.38.0 grpcio-tools-1.37.0 importlib-metadata-2.1.1 mypy-protobuf-1.18 packaging-20.9 pluggy-0.13.1 protobuf-3.17.3 py-1.10.0 pyparsing-2.4.7 six-1.16.0 toml-0.10.2 tox-3.20.1 virtualenv-20.4.7 zipp-3.4.1 > Task :sdks:python:sdist <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/build/gradleenv/1922375555/lib/python3.8/site-packages/setuptools/dist.py>:484: UserWarning: Normalizing '2.32.0.dev' to '2.32.0.dev0' warnings.warn(tmpl.format(**locals())) INFO:gen_protos:Regenerating Python proto definitions (no output files). INFO:gen_protos:Found protoc_gen_mypy at <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/build/gradleenv/1922375555/bin/protoc-gen-mypy> beam_interactive_api.proto:36:1: warning: Import google/protobuf/timestamp.proto is unused. Writing mypy to external_transforms_pb2.pyi Writing mypy to beam_runner_api_pb2.pyi Writing mypy to standard_window_fns_pb2.pyi Writing mypy to metrics_pb2.pyi Writing mypy to endpoints_pb2.pyi Writing mypy to schema_pb2.pyi Writing mypy to beam_artifact_api_pb2.pyi Writing mypy to beam_expansion_api_pb2.pyi Writing mypy to beam_job_api_pb2.pyi Writing mypy to beam_provision_api_pb2.pyi Writing mypy to beam_fn_api_pb2.pyi Writing mypy to beam_interactive_api_pb2.pyi RefactoringTool: Skipping optional fixer: idioms RefactoringTool: Skipping optional fixer: ws_comma RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_artifact_api_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_artifact_api_pb2_grpc.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_expansion_api_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_expansion_api_pb2_grpc.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_fn_api_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_fn_api_pb2_grpc.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_interactive_api_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_job_api_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_job_api_pb2_grpc.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_provision_api_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_provision_api_pb2_grpc.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_runner_api_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_runner_api_pb2_grpc.py> RefactoringTool: No changes to <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/endpoints_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/external_transforms_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/metrics_pb2.py> RefactoringTool: No changes to <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/schema_pb2.py> RefactoringTool: Refactored <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/standard_window_fns_pb2.py> RefactoringTool: Files that were modified: RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_artifact_api_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_artifact_api_pb2_grpc.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_expansion_api_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_expansion_api_pb2_grpc.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_fn_api_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_fn_api_pb2_grpc.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_interactive_api_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_job_api_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_job_api_pb2_grpc.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_provision_api_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_provision_api_pb2_grpc.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_runner_api_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_runner_api_pb2_grpc.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/endpoints_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/external_transforms_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/metrics_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/schema_pb2.py> RefactoringTool: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/standard_window_fns_pb2.py> INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/metrics_pb2_urns.py> INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_artifact_api_pb2_urns.py> INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/standard_window_fns_pb2_urns.py> INFO:gen_protos:Writing urn stubs: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/apache_beam/portability/api/beam_runner_api_pb2_urns.py> warning: no files found matching 'README.md' warning: no files found matching 'NOTICE' warning: no files found matching 'LICENSE' warning: sdist: standard file not found: should have one of README, README.rst, README.txt, README.md > Task :sdks:python:test-suites:portable:py36:sparkCompatibilityMatrixLOOPBACK [1mspark-runner-test create: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test> [0m[1mspark-runner-test installdeps: -rbuild-requirements.txt [0m[1mspark-runner-test inst: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/target/.tox-spark-runner-test/.tmp/package/1/apache-beam.tar.gz> [0m[1mspark-runner-test installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/target/.tox-spark-runner-test/.tmp/package/1/apache-beam.tar.gz,apipkg==1.5,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,chardet==4.0.0,crcmod==1.7,dataclasses==0.8,deprecation==2.1.0,dill==0.3.1.1,distlib==0.3.1,docker==5.0.0,docopt==0.6.2,execnet==1.8.1,fastavro==1.4.1,freezegun==1.1.0,future==0.18.2,greenlet==1.1.0,grpcio==1.38.0,grpcio-tools==1.37.0,hdfs==2.6.0,httplib2==0.19.1,idna==2.10,importlib-metadata==4.5.0,mock==2.0.0,more-itertools==8.8.0,mypy-protobuf==1.18,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.19.5,oauth2client==4.1.3,packaging==20.9,pandas==1.1.5,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.8.6,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.11.4,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.1,pytz==2021.1,PyYAML==5.4.1,requests==2.25.1,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.17,tenacity==5.1.5,testcontainers==3.4.1,typing-extensions==3.7.4.3,urllib3==1.26.5,wcwidth==0.2.5,websocket-client==1.0.1,wrapt==1.12.1,zipp==3.4.1> [0m[1mspark-runner-test run-test-pre: PYTHONHASHSEED='3976866721' [0m[1mspark-runner-test run-test-pre: commands[0] | python --version [0mPython 3.6.8 [1mspark-runner-test run-test-pre: commands[1] | pip --version [0mpip 21.1.2 from <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.6/site-packages/pip> (python 3.6) [1mspark-runner-test run-test-pre: commands[2] | pip check [0mNo broken requirements found. [1mspark-runner-test run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/scripts/run_tox_cleanup.sh> [0m[1mspark-runner-test run-test: commands[0] | <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/apache_beam/runners/portability/spark_runner_test.py> '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.32.0-SNAPSHOT.jar> --environment_type=LOOPBACK' [0m============================= test session starts ============================== platform linux -- Python 3.6.8, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 cachedir: target/.tox-spark-runner-test/spark-runner-test/.pytest_cache rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 50 items apache_beam/runners/portability/spark_runner_test.py .s...ss.......ss... [ 38%] ...s............s.ssss..sssss.. [100%] - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/pytest_spark-runner-test.xml> - =================== 34 passed, 16 skipped in 479.70 seconds ==================== [1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py36/build/srcs/sdks/python/scripts/run_tox_cleanup.sh> [0m___________________________________ summary ____________________________________ [32m spark-runner-test: commands succeeded [0m[32m congratulations :) [0m > Task :sdks:python:test-suites:portable:py36:sparkValidatesRunner > Task :sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK [1mspark-runner-test create: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test> [0m[1mspark-runner-test installdeps: -rbuild-requirements.txt [0m[1mspark-runner-test inst: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/target/.tox-spark-runner-test/.tmp/package/1/apache-beam.tar.gz> [0m[1mspark-runner-test installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/target/.tox-spark-runner-test/.tmp/package/1/apache-beam.tar.gz,apipkg==1.5,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,chardet==4.0.0,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,distlib==0.3.1,docker==5.0.0,docopt==0.6.2,execnet==1.8.1,fastavro==1.4.1,freezegun==1.1.0,future==0.18.2,greenlet==1.1.0,grpcio==1.38.0,grpcio-tools==1.37.0,hdfs==2.6.0,httplib2==0.19.1,idna==2.10,importlib-metadata==4.5.0,mock==2.0.0,more-itertools==8.8.0,mypy-protobuf==1.18,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,packaging==20.9,pandas==1.2.4,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.8.6,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.11.4,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.1,pytz==2021.1,PyYAML==5.4.1,requests==2.25.1,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.17,tenacity==5.1.5,testcontainers==3.4.1,typing-extensions==3.7.4.3,urllib3==1.26.5,wcwidth==0.2.5,websocket-client==1.0.1,wrapt==1.12.1,zipp==3.4.1> [0m[1mspark-runner-test run-test-pre: PYTHONHASHSEED='4046906717' [0m[1mspark-runner-test run-test-pre: commands[0] | python --version [0mPython 3.7.3 [1mspark-runner-test run-test-pre: commands[1] | pip --version [0mpip 21.1.2 from <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.7/site-packages/pip> (python 3.7) [1mspark-runner-test run-test-pre: commands[2] | pip check [0mNo broken requirements found. [1mspark-runner-test run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/scripts/run_tox_cleanup.sh> [0m[1mspark-runner-test run-test: commands[0] | <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/apache_beam/runners/portability/spark_runner_test.py> '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.32.0-SNAPSHOT.jar> --environment_type=LOOPBACK' [0m============================= test session starts ============================== platform linux -- Python 3.7.3, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 cachedir: target/.tox-spark-runner-test/spark-runner-test/.pytest_cache rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 50 items apache_beam/runners/portability/spark_runner_test.py Fs...ss.......ss... [ 38%] ...s............s.ssss..sssss.. [100%] =================================== FAILURES =================================== _______________________ SparkRunnerTest.test_assert_that _______________________ RuntimeError: Pipeline timed out waiting for job service subprocess. During handling of the above exception, another exception occurred: self = <apache_beam.runners.portability.spark_runner_test.SparkRunnerTest testMethod=test_assert_that> def test_assert_that(self): # TODO: figure out a way for fn_api_runner to parse and raise the # underlying exception. with self.assertRaisesRegex(Exception, 'Failed assert'): with self.create_pipeline() as p: > assert_that(p | beam.Create(['a', 'b']), equal_to(['a'])) E AssertionError: "Failed assert" does not match "Pipeline timed out waiting for job service subprocess." apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:108: AssertionError - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/pytest_spark-runner-test.xml> - ============== 1 failed, 33 passed, 16 skipped in 644.64 seconds =============== [31mERROR: InvocationError for command <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test apache_beam/runners/portability/spark_runner_test.py '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.32.0-SNAPSHOT.jar> --environment_type=LOOPBACK' (exited with code 1) [0m[1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/scripts/run_tox_cleanup.sh> [0m___________________________________ summary ____________________________________ [31mERROR: spark-runner-test: commands failed [0m > Task :sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK > FAILED > Task :sdks:python:test-suites:portable:py38:sparkCompatibilityMatrixLOOPBACK [1mspark-runner-test create: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test> [0m[1mspark-runner-test installdeps: -rbuild-requirements.txt [0m[1mspark-runner-test inst: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/.tmp/package/1/apache-beam.tar.gz> [0m[1mspark-runner-test installed: apache-beam @ file://<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/.tmp/package/1/apache-beam.tar.gz,apipkg==1.5,atomicwrites==1.4.0,attrs==21.2.0,avro-python3==1.9.2.1,certifi==2021.5.30,chardet==4.0.0,crcmod==1.7,deprecation==2.1.0,dill==0.3.1.1,distlib==0.3.1,docker==5.0.0,docopt==0.6.2,execnet==1.8.1,fastavro==1.4.1,freezegun==1.1.0,future==0.18.2,greenlet==1.1.0,grpcio==1.38.0,grpcio-tools==1.37.0,hdfs==2.6.0,httplib2==0.19.1,idna==2.10,mock==2.0.0,more-itertools==8.8.0,mypy-protobuf==1.18,nose==1.3.7,nose-xunitmp==0.4.1,numpy==1.20.3,oauth2client==4.1.3,packaging==20.9,pandas==1.2.4,parameterized==0.7.5,pbr==5.6.0,pluggy==0.13.1,protobuf==3.17.3,psycopg2-binary==2.8.6,py==1.10.0,pyarrow==4.0.1,pyasn1==0.4.8,pyasn1-modules==0.2.8,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.11.4,pyparsing==2.4.7,pytest==4.6.11,pytest-forked==1.3.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.1,pytz==2021.1,PyYAML==5.4.1,requests==2.25.1,requests-mock==1.9.3,rsa==4.7.2,six==1.16.0,SQLAlchemy==1.4.17,tenacity==5.1.5,testcontainers==3.4.1,typing-extensions==3.7.4.3,urllib3==1.26.5,wcwidth==0.2.5,websocket-client==1.0.1,wrapt==1.12.1> [0m[1mspark-runner-test run-test-pre: PYTHONHASHSEED='2899265009' [0m[1mspark-runner-test run-test-pre: commands[0] | python --version [0mPython 3.8.5 [1mspark-runner-test run-test-pre: commands[1] | pip --version [0mpip 21.1.2 from <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/pip> (python 3.8) [1mspark-runner-test run-test-pre: commands[2] | pip check [0mNo broken requirements found. [1mspark-runner-test run-test-pre: commands[3] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh> [0m[1mspark-runner-test run-test: commands[0] | <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh> spark-runner-test <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/apache_beam/runners/portability/spark_runner_test.py> '--spark_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/runners/spark/2/job-server/build/libs/beam-runners-spark-job-server-2.32.0-SNAPSHOT.jar> --environment_type=LOOPBACK' [0m============================= test session starts ============================== platform linux -- Python 3.8.5, pytest-4.6.11, py-1.10.0, pluggy-0.13.1 cachedir: target/.tox-spark-runner-test/spark-runner-test/.pytest_cache rootdir: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python,> inifile: pytest.ini plugins: xdist-1.34.0, timeout-1.4.2, forked-1.3.0, requests-mock-1.9.3 timeout: 600.0s timeout method: signal timeout func_only: False collected 50 items apache_beam/runners/portability/spark_runner_test.py .s...ss.......ss... [ 38%] ...s............s.ssss..sssss.. [100%] =============================== warnings summary =============================== target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42 <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-spark-runner-test/spark-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42: DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use "async def" instead def call(self, fn, *args, **kwargs): -- Docs: https://docs.pytest.org/en/latest/warnings.html - generated xml file: <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_spark-runner-test.xml> - ============= 34 passed, 16 skipped, 1 warnings in 576.96 seconds ============== [1mspark-runner-test run-test-post: commands[0] | bash <https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Spark/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh> [0m___________________________________ summary ____________________________________ [32m spark-runner-test: commands succeeded [0m[32m congratulations :) [0m > Task :sdks:python:test-suites:portable:py38:sparkValidatesRunner FAILURE: Build failed with an exception. * What went wrong: Execution failed for task ':sdks:python:test-suites:portable:py37:sparkCompatibilityMatrixLOOPBACK'. > Process 'command 'sh'' finished with non-zero exit value 1 * Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights. * Get more help at https://help.gradle.org Deprecated Gradle features were used in this build, making it incompatible with Gradle 7.0. Use '--warning-mode all' to show the individual deprecation warnings. See https://docs.gradle.org/6.8.3/userguide/command_line_interface.html#sec:command_line_warnings BUILD FAILED in 39m 21s 67 actionable tasks: 44 executed, 23 from cache Publishing build scan... https://gradle.com/s/22runhplrvxzk Build step 'Invoke Gradle script' changed build result to FAILURE Build step 'Invoke Gradle script' marked build as failure --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
