See
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/2877/display/redirect?page=changes>
Changes:
[bvolpato] Do not use .get() on ValueProvider during pipeline creation
------------------------------------------
[...truncated 94.46 KB...]
......s....s...s..s............s..ss.....ssss..F.....s...s...ss......... [ 78%]
...s....s...s..s............s..ss.....ss.s.. [100%]
=================================== FAILURES ===================================
__________________ FlinkRunnerTestStreaming.test_assert_that ___________________
RuntimeError: Pipeline timed out waiting for job service subprocess.
During handling of the above exception, another exception occurred:
self =
<apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestStreaming
testMethod=test_assert_that>
def test_assert_that(self):
# TODO: figure out a way for fn_api_runner to parse and raise the
# underlying exception.
with self.assertRaisesRegex(Exception, 'Failed assert'):
with self.create_pipeline() as p:
> assert_that(p | beam.Create(['a', 'b']), equal_to(['a']))
E AssertionError: "Failed assert" does not match "Pipeline timed out
waiting for job service subprocess."
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:113:
AssertionError
----------------------------- Captured stderr call -----------------------------
Sep 22, 2022 10:56:11 AM
software.amazon.awssdk.regions.internal.util.EC2MetadataUtils getItems
WARNING: Unable to retrieve the requested metadata.
Sep 22, 2022 10:56:11 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createArtifactStagingService
INFO: ArtifactStagingService started on localhost:45237
Sep 22, 2022 10:56:11 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createExpansionService
INFO: Java ExpansionService started on localhost:45907
Sep 22, 2022 10:56:11 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createJobServer
INFO: JobService started on localhost:40375
Sep 22, 2022 10:56:11 AM org.apache.beam.runners.jobsubmission.JobServerDriver
run
INFO: Job server now running, terminate with Ctrl+C
=============================== warnings summary ===============================
target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_pardo_unfusable_side_inputs
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/_pytest/threadexception.py>:73:
PytestUnhandledThreadExceptionWarning: Exception in thread
read_grpc_client_inputs
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/apache_beam/runners/worker/data_plane.py";,>
line 671, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/apache_beam/runners/worker/data_plane.py";,>
line 654, in _read_inputs
for elements in elements_iterator:
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/grpc/_channel.py";,>
line 426, in __next__
return self._next()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/grpc/_channel.py";,>
line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "UNKNOWN:Error received from peer
ipv6:%5B::1%5D:38567 {grpc_message:"Multiplexer hanging up", grpc_status:1,
created_time:"2022-09-22T10:55:22.637828932+00:00"}"
>
warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_pardo_side_inputs
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/_pytest/threadexception.py>:73:
PytestUnhandledThreadExceptionWarning: Exception in thread
read_grpc_client_inputs
Traceback (most recent call last):
File "/usr/lib/python3.8/threading.py", line 932, in _bootstrap_inner
self.run()
File "/usr/lib/python3.8/threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/apache_beam/runners/worker/data_plane.py";,>
line 671, in <lambda>
target=lambda: self._read_inputs(elements_iterator),
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/apache_beam/runners/worker/data_plane.py";,>
line 654, in _read_inputs
for elements in elements_iterator:
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/grpc/_channel.py";,>
line 426, in __next__
return self._next()
File
"<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/grpc/_channel.py";,>
line 826, in _next
raise self
grpc._channel._MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that
terminated with:
status = StatusCode.CANCELLED
details = "Multiplexer hanging up"
debug_error_string = "UNKNOWN:Error received from peer
ipv6:%5B::1%5D:45973 {grpc_message:"Multiplexer hanging up", grpc_status:1,
created_time:"2022-09-22T10:59:27.495117482+00:00"}"
>
warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_flink-runner-test.xml>
-
=========================== short test summary info ============================
FAILED
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_assert_that
====== 1 failed, 160 passed, 46 skipped, 4 warnings in 836.39s (0:13:56) =======
[31mERROR: InvocationError for command
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test apache_beam/runners/portability/flink_runner_test.py
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.43.0-SNAPSHOT.jar>
--environment_type=LOOPBACK' (exited with code 1)
[0m[1mflink-runner-test run-test-post: commands[0] | bash
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary
____________________________________
[31mERROR: flink-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py38:flinkCompatibilityMatrixLOOPBACK
> FAILED
> Task :sdks:python:test-suites:portable:py39:flinkCompatibilityMatrixLOOPBACK
[1mflink-runner-test create:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test>
[0m[1mflink-runner-test installdeps: -rbuild-requirements.txt
[0m[1mflink-runner-test inst:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/.tmp/package/1/apache-beam.tar.gz>
[0m[1mflink-runner-test installed: apache-beam @
file://<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/.tmp/package/1/apache-beam.tar.gz,attrs==22.1.0,certifi==2022.9.14,cffi==1.15.1,charset-normalizer==2.1.1,cloudpickle==2.2.0,crcmod==1.7,cryptography==38.0.1,deprecation==2.1.0,dill==0.3.1.1,distlib==0.3.1,docker==6.0.0,docopt==0.6.2,execnet==1.9.0,fastavro==1.6.1,freezegun==1.2.2,greenlet==1.1.3,grpcio==1.49.1,grpcio-tools==1.37.0,hdfs==2.7.0,httplib2==0.20.4,idna==3.4,iniconfig==1.1.1,joblib==1.2.0,mock==2.0.0,mypy-protobuf==1.18,numpy==1.22.4,objsize==0.5.2,orjson==3.8.0,packaging==21.3,pandas==1.5.0,parameterized==0.8.1,pbr==5.10.0,pluggy==1.0.0,proto-plus==1.22.1,protobuf==3.20.2,psycopg2-binary==2.9.3,py==1.11.0,pyarrow==9.0.0,pycparser==2.21,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.3,PyMySQL==1.0.2,pyparsing==3.0.9,pytest==7.1.3,pytest-forked==1.4.0,pytest-timeout==2.1.0,pytest-xdist==2.5.0,python-dateutil==2.8.2,pytz==2022.2.1,PyYAML==6.0,regex==2022.9.13,requests==2.28.1,requests-mock==1.10.0,scikit-learn==1.1.2,scipy==1.9.1,six==1.16.0,SQLAlchemy==1.4.41,tenacity==5.1.5,testcontainers==3.7.0,threadpoolctl==3.1.0,tomli==2.0.1,typing_extensions==4.3.0,urllib3==1.26.12,websocket-client==1.4.1,wrapt==1.14.1,zstandard==0.18.0>
[0m[1mflink-runner-test run-test-pre: PYTHONHASHSEED='3438343301'
[0m[1mflink-runner-test run-test-pre: commands[0] | python --version
[0mPython 3.9.10
[1mflink-runner-test run-test-pre: commands[1] | pip --version
[0mpip 22.2.2 from
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/pip>
(python 3.9)
[1mflink-runner-test run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mflink-runner-test run-test-pre: commands[3] | bash
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mflink-runner-test run-test: commands[0] |
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/apache_beam/runners/portability/flink_runner_test.py>
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.43.0-SNAPSHOT.jar>
--environment_type=LOOPBACK'
[0m============================= test session starts
==============================
platform linux -- Python 3.9.10, pytest-7.1.3, pluggy-1.0.0
cachedir: target/.tox-flink-runner-test/flink-runner-test/.pytest_cache
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python,>
configfile: pytest.ini
plugins: xdist-2.5.0, timeout-2.1.0, forked-1.4.0, requests-mock-1.10.0
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 207 items
apache_beam/runners/portability/flink_runner_test.py ......s...s...ss... [ 9%]
.........s....s...s..s............s..ss.....ss.s..F.....s...s...ss...sss [ 43%]
......s....s...s..s............s..ss.....ssss..F.....s...s...ss......... [ 78%]
...s....s...s..s............s..ss.....ss.s.. [100%]
=================================== FAILURES ===================================
__________________ FlinkRunnerTestOptimized.test_assert_that ___________________
RuntimeError: Pipeline timed out waiting for job service subprocess.
During handling of the above exception, another exception occurred:
self =
<apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestOptimized
testMethod=test_assert_that>
def test_assert_that(self):
# TODO: figure out a way for fn_api_runner to parse and raise the
# underlying exception.
with self.assertRaisesRegex(Exception, 'Failed assert'):
with self.create_pipeline() as p:
> assert_that(p | beam.Create(['a', 'b']), equal_to(['a']))
E AssertionError: "Failed assert" does not match "Pipeline timed out
waiting for job service subprocess."
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:113:
AssertionError
----------------------------- Captured stderr call -----------------------------
Sep 22, 2022 11:07:06 AM
software.amazon.awssdk.regions.internal.util.EC2MetadataUtils getItems
WARNING: Unable to retrieve the requested metadata.
Sep 22, 2022 11:07:06 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createArtifactStagingService
INFO: ArtifactStagingService started on localhost:35267
Sep 22, 2022 11:07:06 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createExpansionService
INFO: Java ExpansionService started on localhost:49877
Sep 22, 2022 11:07:06 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createJobServer
INFO: JobService started on localhost:37149
Sep 22, 2022 11:07:06 AM org.apache.beam.runners.jobsubmission.JobServerDriver
run
INFO: Job server now running, terminate with Ctrl+C
__________________ FlinkRunnerTestStreaming.test_assert_that ___________________
RuntimeError: Pipeline timed out waiting for job service subprocess.
During handling of the above exception, another exception occurred:
self =
<apache_beam.runners.portability.flink_runner_test.FlinkRunnerTestStreaming
testMethod=test_assert_that>
def test_assert_that(self):
# TODO: figure out a way for fn_api_runner to parse and raise the
# underlying exception.
with self.assertRaisesRegex(Exception, 'Failed assert'):
with self.create_pipeline() as p:
> assert_that(p | beam.Create(['a', 'b']), equal_to(['a']))
E AssertionError: "Failed assert" does not match "Pipeline timed out
waiting for job service subprocess."
apache_beam/runners/portability/fn_api_runner/fn_runner_test.py:113:
AssertionError
----------------------------- Captured stderr call -----------------------------
Sep 22, 2022 11:11:26 AM
software.amazon.awssdk.regions.internal.util.EC2MetadataUtils getItems
WARNING: Unable to retrieve the requested metadata.
Sep 22, 2022 11:11:26 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createArtifactStagingService
INFO: ArtifactStagingService started on localhost:36073
Sep 22, 2022 11:11:26 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createExpansionService
INFO: Java ExpansionService started on localhost:49889
Sep 22, 2022 11:11:26 AM org.apache.beam.runners.jobsubmission.JobServerDriver
createJobServer
INFO: JobService started on localhost:47811
Sep 22, 2022 11:11:26 AM org.apache.beam.runners.jobsubmission.JobServerDriver
run
INFO: Job server now running, terminate with Ctrl+C
=============================== warnings summary ===============================
target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/hdfs/config.py:15
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/hdfs/config.py>:15:
DeprecationWarning: the imp module is deprecated in favour of importlib; see
the module's documentation for alternative uses
from imp import load_source
target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/pytest_flink-runner-test.xml>
-
=========================== short test summary info ============================
FAILED
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_assert_that
FAILED
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_assert_that
====== 2 failed, 159 passed, 46 skipped, 2 warnings in 863.36s (0:14:23) =======
[31mERROR: InvocationError for command
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test apache_beam/runners/portability/flink_runner_test.py
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.43.0-SNAPSHOT.jar>
--environment_type=LOOPBACK' (exited with code 1)
[0m[1mflink-runner-test run-test-post: commands[0] | bash
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary
____________________________________
[31mERROR: flink-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py39:flinkCompatibilityMatrixLOOPBACK
> FAILED
FAILURE: Build completed with 2 failures.
1: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py38:flinkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
2: Task failed with an exception.
-----------
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py39:flinkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
==============================================================================
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 50m 40s
151 actionable tasks: 104 executed, 43 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ytqsyyrwah7xq
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
