See
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/2479/display/redirect?page=changes>
Changes:
[Jan Lukavský] [BEAM-14265] Add watermark hold for all timers
[noreply] Bump Python beam-master container (#21820)
[noreply] Split PytorchModelHandler into PytorchModelHandlerTensor and
------------------------------------------
[...truncated 218.27 KB...]
INFO: Stopping SessionDispatcherLeaderProcess.
Jun 14, 2022 4:34:06 PM org.apache.flink.runtime.taskexecutor.KvStateService
shutdown
INFO: Shutting down the kvState service and its components.
Jun 14, 2022 4:34:06 PM
org.apache.flink.runtime.taskexecutor.DefaultJobLeaderService stop
INFO: Stop job leader service.
Jun 14, 2022 4:34:06 PM
org.apache.flink.runtime.resourcemanager.ResourceManagerServiceImpl closeAsync
INFO: Stopping resource manager service.
Jun 14, 2022 4:34:06 PM org.apache.flink.runtime.filecache.FileCache shutdown
INFO: removed file cache directory
/tmp/flink-dist-cache-4d28ea79-7c71-4be4-bc64-71c49aa6e971
Jun 14, 2022 4:34:06 PM
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager
close
INFO: Closing the slot manager.
Jun 14, 2022 4:34:06 PM
org.apache.flink.runtime.resourcemanager.slotmanager.DeclarativeSlotManager
suspend
INFO: Suspending the slot manager.
------------------------------ Captured log call -------------------------------
WARNING apache_beam.options.pipeline_options:pipeline_options.py:350
Discarding invalid overrides: {'max_bundle_time_millis': 95130590130}
--------------------------- Captured stderr teardown ---------------------------
Jun 14, 2022 4:34:07 PM org.apache.flink.runtime.metrics.MetricRegistryImpl
unregister
WARNING: Error while unregistering metric: RequestedMemoryUsage.
java.lang.ArithmeticException: / by zero
at
org.apache.flink.runtime.io.network.buffer.NetworkBufferPool.getRequestedSegmentsUsage(NetworkBufferPool.java:385)
at
org.apache.flink.runtime.io.network.metrics.NettyShuffleMetricFactory$RequestedMemoryUsageMetric.getValue(NettyShuffleMetricFactory.java:247)
at
org.apache.flink.runtime.io.network.metrics.NettyShuffleMetricFactory$RequestedMemoryUsageMetric.getValue(NettyShuffleMetricFactory.java:238)
at
org.apache.beam.runners.flink.metrics.Metrics.toString(Metrics.java:33)
at
org.apache.beam.runners.flink.metrics.FileReporter.notifyOfRemovedMetric(FileReporter.java:73)
at
org.apache.flink.runtime.metrics.MetricRegistryImpl.unregister(MetricRegistryImpl.java:436)
at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.close(AbstractMetricGroup.java:333)
at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.close(AbstractMetricGroup.java:327)
at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.close(AbstractMetricGroup.java:327)
at
org.apache.flink.runtime.metrics.groups.AbstractMetricGroup.close(AbstractMetricGroup.java:327)
at
org.apache.flink.runtime.metrics.groups.ComponentMetricGroup.close(ComponentMetricGroup.java:62)
at
org.apache.flink.runtime.taskexecutor.TaskExecutor.stopTaskExecutorServices(TaskExecutor.java:537)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$runAfterwardsAsync$18(FutureUtils.java:696)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
at
org.apache.flink.util.concurrent.DirectExecutorService.execute(DirectExecutorService.java:217)
at
java.util.concurrent.CompletableFuture$UniCompletion.claim(CompletableFuture.java:543)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:765)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
at
org.apache.flink.util.concurrent.FutureUtils.doForward(FutureUtils.java:1389)
at
org.apache.flink.util.concurrent.FutureUtils.lambda$forwardTo$24(FutureUtils.java:1372)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
at
java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
at
java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:575)
at
java.util.concurrent.CompletableFuture$UniRun.tryFire(CompletableFuture.java:704)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.lambda$handleRunAsync$4(AkkaRpcActor.java:443)
at
org.apache.flink.runtime.concurrent.akka.ClassLoadingUtils.runWithContextClassLoader(ClassLoadingUtils.java:68)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRunAsync(AkkaRpcActor.java:443)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:213)
at
org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:163)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:24)
at akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:20)
at scala.PartialFunction.applyOrElse(PartialFunction.scala:123)
at scala.PartialFunction.applyOrElse$(PartialFunction.scala:122)
at akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:20)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:172)
at akka.actor.Actor.aroundReceive(Actor.scala:537)
at akka.actor.Actor.aroundReceive$(Actor.scala:535)
at akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:220)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:580)
at akka.actor.ActorCell.invoke(ActorCell.scala:548)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:270)
at akka.dispatch.Mailbox.run(Mailbox.scala:231)
at akka.dispatch.Mailbox.exec(Mailbox.scala:243)
at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
at
java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
at
java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
Jun 14, 2022 4:34:07 PM
org.apache.flink.runtime.jobmaster.slotpool.DefaultDeclarativeSlotPool
releaseSlots
INFO: Releasing slot [872193a4492bf2a791b67a63ba4e5734].
Jun 14, 2022 4:34:07 PM org.apache.flink.runtime.jobmaster.JobMaster
dissolveResourceManagerConnection
INFO: Close ResourceManager connection 4faf39b77a8d5b03778a5c587def275e:
ResourceManager leader changed to new address null
Jun 14, 2022 4:34:07 PM org.apache.flink.runtime.taskexecutor.TaskExecutor
handleOnStopException
INFO: Stopped TaskExecutor akka://flink/user/rpc/taskmanager_76.
=============================== warnings summary ===============================
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTest::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_element_to_batch_pardo
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/apache_beam/runners/worker/operations.py>:358:
InefficientExecutionWarning: Input to operation <DoOperation
ParDo(ArrayMultiplyDoFn) output_tags=['None'],
receivers=[GeneralPurposeConsumerSet[ParDo(ArrayMultiplyDoFn).out0,
coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]> must be
rebatched from type NumpyArray[dtype('int64')] to NumpyArray[dtype('O')].
This is very inefficient, consider re-structuring your pipeline or adding a
DoFn to directly convert between these types.
InefficientExecutionWarning)
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTest::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_element_to_batch_pardo
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/apache_beam/runners/common.py>:1691:
FutureWarning: arrays to stack must be passed as a "sequence" type such as
list or tuple. Support for non-sequence iterables such as generators is
deprecated as of NumPy 1.16 and will raise an error in the future.
self.main_receivers.receive_batch(windowed_batch)
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/pytest_flink-runner-test.xml>
-
======= 1 failed, 157 passed, 46 skipped, 6 warnings in 1112.40 seconds ========
[31mERROR: InvocationError for command
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test apache_beam/runners/portability/flink_runner_test.py
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.40.0-SNAPSHOT.jar>
--environment_type=LOOPBACK' (exited with code 1)
[0m[1mflink-runner-test run-test-post: commands[0] | bash
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py37/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary
____________________________________
[31mERROR: flink-runner-test: commands failed
[0m
> Task :sdks:python:test-suites:portable:py37:flinkCompatibilityMatrixLOOPBACK
> FAILED
> Task :sdks:python:test-suites:portable:py38:flinkCompatibilityMatrixLOOPBACK
[1mflink-runner-test create:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test>
[0m[1mflink-runner-test installdeps: -rbuild-requirements.txt
[0m[1mflink-runner-test inst:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/.tmp/package/1/apache-beam.tar.gz>
[0m[1mflink-runner-test installed: apache-beam @
file://<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/.tmp/package/1/apache-beam.tar.gz,atomicwrites==1.4.0,attrs==21.4.0,certifi==2022.5.18.1,cffi==1.15.0,charset-normalizer==2.0.12,cloudpickle==2.1.0,crcmod==1.7,cryptography==37.0.2,deprecation==2.1.0,dill==0.3.5.1,distlib==0.3.1,docker==5.0.3,docopt==0.6.2,execnet==1.9.0,fastavro==1.5.1,freezegun==1.2.1,greenlet==1.1.2,grpcio==1.46.3,grpcio-tools==1.37.0,hdfs==2.7.0,httplib2==0.20.4,idna==3.3,joblib==1.1.0,mock==2.0.0,more-itertools==8.13.0,mypy-protobuf==1.18,numpy==1.22.4,orjson==3.7.2,packaging==21.3,pandas==1.4.2,parameterized==0.7.5,pbr==5.9.0,pluggy==0.13.1,proto-plus==1.20.6,protobuf==3.20.1,psycopg2-binary==2.9.3,py==1.11.0,pyarrow==7.0.0,pycparser==2.21,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.3,PyMySQL==1.0.2,pyparsing==3.0.9,pytest==4.6.11,pytest-forked==1.4.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2022.1,PyYAML==6.0,requests==2.28.0,requests-mock==1.9.3,scikit-learn==1.1.1,scipy==1.8.1,six==1.16.0,SQLAlchemy==1.4.37,tenacity==5.1.5,testcontainers==3.5.4,threadpoolctl==3.1.0,typing_extensions==4.2.0,urllib3==1.26.9,wcwidth==0.2.5,websocket-client==1.3.2,wrapt==1.14.1>
[0m[1mflink-runner-test run-test-pre: PYTHONHASHSEED='2596210734'
[0m[1mflink-runner-test run-test-pre: commands[0] | python --version
[0mPython 3.8.10
[1mflink-runner-test run-test-pre: commands[1] | pip --version
[0mpip 22.1.2 from
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/pip>
(python 3.8)
[1mflink-runner-test run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mflink-runner-test run-test-pre: commands[3] | bash
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mflink-runner-test run-test: commands[0] |
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/apache_beam/runners/portability/flink_runner_test.py>
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.40.0-SNAPSHOT.jar>
--environment_type=LOOPBACK'
[0m============================= test session starts
==============================
platform linux -- Python 3.8.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
cachedir: target/.tox-flink-runner-test/flink-runner-test/.pytest_cache
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 204 items
apache_beam/runners/portability/flink_runner_test.py .....s...s...ss.... [ 9%]
........s....s...s..s............s..ss.....ss.s.......s...s...ss...sss.. [ 44%]
....s....s...s..s............s..ss.....ssss.......s...s...ss............ [ 79%]
s....s...s..s............s..ss.....ss.s.. [100%]
=============================== warnings summary ===============================
target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.8/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTest::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_element_to_batch_pardo
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/apache_beam/runners/worker/operations.py>:352:
InefficientExecutionWarning: Input to operation <DoOperation
ParDo(ArrayMultiplyDoFn) output_tags=['None'],
receivers=[GeneralPurposeConsumerSet[ParDo(ArrayMultiplyDoFn).out0,
coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]> must be
rebatched from type NumpyArray[dtype('int64')] to NumpyArray[dtype('O')].
This is very inefficient, consider re-structuring your pipeline or adding a
DoFn to directly convert between these types.
warnings.warn(
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTest::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_element_to_batch_pardo
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/apache_beam/runners/common.py>:1691:
FutureWarning: arrays to stack must be passed as a "sequence" type such as
list or tuple. Support for non-sequence iterables such as generators is
deprecated as of NumPy 1.16 and will raise an error in the future.
self.main_receivers.receive_batch(windowed_batch)
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/pytest_flink-runner-test.xml>
-
============= 158 passed, 46 skipped, 7 warnings in 812.68 seconds =============
[1mflink-runner-test run-test-post: commands[0] | bash
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py38/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary
____________________________________
[32m flink-runner-test: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:portable:py38:flinkValidatesRunner
> Task :sdks:python:test-suites:portable:py39:flinkCompatibilityMatrixLOOPBACK
[1mflink-runner-test create:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test>
[0m[1mflink-runner-test installdeps: -rbuild-requirements.txt
[0m[1mflink-runner-test inst:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/.tmp/package/1/apache-beam.tar.gz>
[0m[1mflink-runner-test installed: apache-beam @
file://<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/.tmp/package/1/apache-beam.tar.gz,atomicwrites==1.4.0,attrs==21.4.0,certifi==2022.5.18.1,cffi==1.15.0,charset-normalizer==2.0.12,cloudpickle==2.1.0,crcmod==1.7,cryptography==37.0.2,deprecation==2.1.0,dill==0.3.5.1,distlib==0.3.1,docker==5.0.3,docopt==0.6.2,execnet==1.9.0,fastavro==1.5.1,freezegun==1.2.1,greenlet==1.1.2,grpcio==1.46.3,grpcio-tools==1.37.0,hdfs==2.7.0,httplib2==0.20.4,idna==3.3,joblib==1.1.0,mock==2.0.0,more-itertools==8.13.0,mypy-protobuf==1.18,numpy==1.22.4,orjson==3.7.2,packaging==21.3,pandas==1.4.2,parameterized==0.7.5,pbr==5.9.0,pluggy==0.13.1,proto-plus==1.20.6,protobuf==3.20.1,psycopg2-binary==2.9.3,py==1.11.0,pyarrow==7.0.0,pycparser==2.21,pydot==1.4.2,PyHamcrest==1.10.1,pymongo==3.12.3,PyMySQL==1.0.2,pyparsing==3.0.9,pytest==4.6.11,pytest-forked==1.4.0,pytest-timeout==1.4.2,pytest-xdist==1.34.0,python-dateutil==2.8.2,pytz==2022.1,PyYAML==6.0,requests==2.28.0,requests-mock==1.9.3,scikit-learn==1.1.1,scipy==1.8.1,six==1.16.0,SQLAlchemy==1.4.37,tenacity==5.1.5,testcontainers==3.5.4,threadpoolctl==3.1.0,typing_extensions==4.2.0,urllib3==1.26.9,wcwidth==0.2.5,websocket-client==1.3.2,wrapt==1.14.1>
[0m[1mflink-runner-test run-test-pre: PYTHONHASHSEED='2951528908'
[0m[1mflink-runner-test run-test-pre: commands[0] | python --version
[0mPython 3.9.10
[1mflink-runner-test run-test-pre: commands[1] | pip --version
[0mpip 22.1.2 from
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/pip>
(python 3.9)
[1mflink-runner-test run-test-pre: commands[2] | pip check
[0mNo broken requirements found.
[1mflink-runner-test run-test-pre: commands[3] | bash
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m[1mflink-runner-test run-test: commands[0] |
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/pytest_validates_runner.sh>
flink-runner-test
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/apache_beam/runners/portability/flink_runner_test.py>
'--flink_job_server_jar=<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/runners/flink/1.15/job-server/build/libs/beam-runners-flink-1.15-job-server-2.40.0-SNAPSHOT.jar>
--environment_type=LOOPBACK'
[0m============================= test session starts
==============================
platform linux -- Python 3.9.10, pytest-4.6.11, py-1.11.0, pluggy-0.13.1
cachedir: target/.tox-flink-runner-test/flink-runner-test/.pytest_cache
rootdir:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python,>
inifile: pytest.ini
plugins: xdist-1.34.0, timeout-1.4.2, forked-1.4.0, requests-mock-1.9.3
timeout: 600.0s
timeout method: signal
timeout func_only: False
collected 204 items
apache_beam/runners/portability/flink_runner_test.py .....s...s...ss.... [ 9%]
........s....s...s..s............s..ss.....ss.s.......s...s...ss...sss.. [ 44%]
....s....s...s..s............s..ss.....ssss.......s...s...ss............ [ 79%]
s....s...s..s............s..ss.....ss.s.. [100%]
=============================== warnings summary ===============================
target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/tenacity/_asyncio.py:42
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/target/.tox-flink-runner-test/flink-runner-test/lib/python3.9/site-packages/tenacity/_asyncio.py>:42:
DeprecationWarning: "@coroutine" decorator is deprecated since Python 3.8, use
"async def" instead
def call(self, fn, *args, **kwargs):
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTest::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_element_to_batch_pardo
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/apache_beam/runners/worker/operations.py>:352:
InefficientExecutionWarning: Input to operation <DoOperation
ParDo(ArrayMultiplyDoFn) output_tags=['None'],
receivers=[GeneralPurposeConsumerSet[ParDo(ArrayMultiplyDoFn).out0,
coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]> must be
rebatched from type NumpyArray[dtype('int64')] to NumpyArray[dtype('O')].
This is very inefficient, consider re-structuring your pipeline or adding a
DoFn to directly convert between these types.
warnings.warn(
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTest::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestOptimized::test_element_to_batch_pardo
apache_beam/runners/portability/flink_runner_test.py::FlinkRunnerTestStreaming::test_element_to_batch_pardo
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/apache_beam/runners/common.py>:1691:
FutureWarning: arrays to stack must be passed as a "sequence" type such as
list or tuple. Support for non-sequence iterables such as generators is
deprecated as of NumPy 1.16 and will raise an error in the future.
self.main_receivers.receive_batch(windowed_batch)
-- Docs: https://docs.pytest.org/en/latest/warnings.html
- generated xml file:
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/pytest_flink-runner-test.xml>
-
============= 158 passed, 46 skipped, 7 warnings in 772.43 seconds =============
[1mflink-runner-test run-test-post: commands[0] | bash
<https://ci-beam.apache.org/job/beam_PostCommit_Python_VR_Flink/ws/src/sdks/python/test-suites/portable/py39/build/srcs/sdks/python/scripts/run_tox_cleanup.sh>
[0m___________________________________ summary
____________________________________
[32m flink-runner-test: commands succeeded
[0m[32m congratulations :)
[0m
> Task :sdks:python:test-suites:portable:py39:flinkValidatesRunner
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task
':sdks:python:test-suites:portable:py37:flinkCompatibilityMatrixLOOPBACK'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
Deprecated Gradle features were used in this build, making it incompatible with
Gradle 8.0.
You can use '--warning-mode all' to show the individual deprecation warnings
and determine if they come from your own scripts or plugins.
See
https://docs.gradle.org/7.4/userguide/command_line_interface.html#sec:command_line_warnings
Execution optimizations have been disabled for 1 invalid unit(s) of work during
this build to ensure correctness.
Please consult deprecation warnings for more details.
BUILD FAILED in 50m 22s
151 actionable tasks: 103 executed, 44 from cache, 4 up-to-date
Publishing build scan...
https://gradle.com/s/ykxtj4bt4v4xg
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
