[
https://issues.apache.org/jira/browse/FLINK-17724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Metzger reassigned FLINK-17724:
--------------------------------------
Assignee: Wei Zhong
> PyFlink end-to-end test fails with Cannot run program
> "venv.zip/.conda/bin/python": error=2, No such file or directory
> ----------------------------------------------------------------------------------------------------------------------
>
> Key: FLINK-17724
> URL: https://issues.apache.org/jira/browse/FLINK-17724
> Project: Flink
> Issue Type: Bug
> Components: API / Python, Tests
> Reporter: Robert Metzger
> Assignee: Wei Zhong
> Priority: Major
> Labels: pull-request-available, test-stability
> Fix For: 1.11.0
>
>
> CI:
> https://dev.azure.com/rmetzger/Flink/_build/results?buildId=8001&view=logs&j=1f3ed471-1849-5d3c-a34c-19792af4ad16&t=2f5b54d0-1d28-5b01-d344-aa50ffe0cdf8
> {code}
> 2020-05-15T09:21:20.5198725Z Verifying transaction: ...working... done
> 2020-05-15T09:21:20.5918557Z Executing transaction: ...working... done
> 2020-05-15T09:22:18.8776831Z DeprecationWarning: 'source deactivate' is
> deprecated. Use 'conda deactivate'.
> 2020-05-15T09:22:19.0555591Z Starting cluster.
> 2020-05-15T09:22:21.9516388Z Starting standalonesession daemon on host
> fv-az678.
> 2020-05-15T09:22:23.5833254Z Starting taskexecutor daemon on host fv-az678.
> 2020-05-15T09:22:23.6192099Z Waiting for Dispatcher REST endpoint to come
> up...
> 2020-05-15T09:22:24.6699447Z Waiting for Dispatcher REST endpoint to come
> up...
> 2020-05-15T09:22:26.0376695Z Waiting for Dispatcher REST endpoint to come
> up...
> 2020-05-15T09:22:27.1345574Z Waiting for Dispatcher REST endpoint to come
> up...
> 2020-05-15T09:22:28.1809673Z Dispatcher REST endpoint is up.
> 2020-05-15T09:22:28.1842051Z Test submitting python job:\n
> 2020-05-15T09:22:29.6520483Z Results directory: /tmp/result
> 2020-05-15T09:23:23.2171819Z Traceback (most recent call last):
> 2020-05-15T09:23:23.2174222Z File
> "/home/vsts/work/1/s/flink-end-to-end-tests/flink-python-test/python/python_job.py",
> line 82, in <module>
> 2020-05-15T09:23:23.2174857Z word_count()
> 2020-05-15T09:23:23.2175685Z File
> "/home/vsts/work/1/s/flink-end-to-end-tests/flink-python-test/python/python_job.py",
> line 76, in word_count
> 2020-05-15T09:23:23.2176310Z t_env.execute("word_count")
> 2020-05-15T09:23:23.2177228Z File
> "/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/opt/python/pyflink.zip/pyflink/table/table_environment.py",
> line 1049, in execute
> 2020-05-15T09:23:23.2179484Z File
> "/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/opt/python/py4j-0.10.8.1-src.zip/py4j/java_gateway.py",
> line 1286, in __call__
> 2020-05-15T09:23:23.2181045Z File
> "/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/opt/python/pyflink.zip/pyflink/util/exceptions.py",
> line 147, in deco
> 2020-05-15T09:23:23.2182205Z File
> "/home/vsts/work/1/s/flink-dist/target/flink-1.11-SNAPSHOT-bin/flink-1.11-SNAPSHOT/opt/python/py4j-0.10.8.1-src.zip/py4j/protocol.py",
> line 328, in get_return_value
> 2020-05-15T09:23:23.2182889Z py4j.protocol.Py4JJavaError: An error occurred
> while calling o2.execute.
> 2020-05-15T09:23:23.2183634Z : java.lang.RuntimeException:
> java.util.concurrent.ExecutionException:
> org.apache.flink.client.program.ProgramInvocationException: Job failed
> (JobID: b0e54b0a1c99b57e04ec32d7879437c4)
> 2020-05-15T09:23:23.2184387Z at
> org.apache.flink.util.ExceptionUtils.rethrow(ExceptionUtils.java:290)
> 2020-05-15T09:23:23.2184960Z at
> org.apache.flink.table.api.internal.BatchTableEnvImpl.execute(BatchTableEnvImpl.scala:325)
> 2020-05-15T09:23:23.2185508Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2020-05-15T09:23:23.2186320Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2020-05-15T09:23:23.2186930Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2020-05-15T09:23:23.2187472Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2020-05-15T09:23:23.2188027Z at
> org.apache.flink.api.python.shaded.py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
> 2020-05-15T09:23:23.2188672Z at
> org.apache.flink.api.python.shaded.py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
> 2020-05-15T09:23:23.2189287Z at
> org.apache.flink.api.python.shaded.py4j.Gateway.invoke(Gateway.java:282)
> 2020-05-15T09:23:23.2189911Z at
> org.apache.flink.api.python.shaded.py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
> 2020-05-15T09:23:23.2190546Z at
> org.apache.flink.api.python.shaded.py4j.commands.CallCommand.execute(CallCommand.java:79)
> 2020-05-15T09:23:23.2191178Z at
> org.apache.flink.api.python.shaded.py4j.GatewayConnection.run(GatewayConnection.java:238)
> 2020-05-15T09:23:23.2191702Z at java.lang.Thread.run(Thread.java:748)
> 2020-05-15T09:23:23.2192357Z Caused by:
> java.util.concurrent.ExecutionException:
> org.apache.flink.client.program.ProgramInvocationException: Job failed
> (JobID: b0e54b0a1c99b57e04ec32d7879437c4)
> 2020-05-15T09:23:23.2193084Z at
> java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
> 2020-05-15T09:23:23.2193645Z at
> java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
> 2020-05-15T09:23:23.2194213Z at
> org.apache.flink.table.api.internal.BatchTableEnvImpl.execute(BatchTableEnvImpl.scala:319)
> 2020-05-15T09:23:23.2194827Z ... 11 more
> 2020-05-15T09:23:23.2196472Z Caused by:
> org.apache.flink.client.program.ProgramInvocationException: Job failed
> (JobID: b0e54b0a1c99b57e04ec32d7879437c4)
> 2020-05-15T09:23:23.2198222Z at
> org.apache.flink.client.deployment.ClusterClientJobClientAdapter.lambda$null$6(ClusterClientJobClientAdapter.java:116)
> 2020-05-15T09:23:23.2198778Z at
> java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:616)
> 2020-05-15T09:23:23.2199403Z at
> java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:591)
> 2020-05-15T09:23:23.2199858Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2020-05-15T09:23:23.2200313Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2020-05-15T09:23:23.2200826Z at
> org.apache.flink.client.program.rest.RestClusterClient.lambda$pollResourceAsync$22(RestClusterClient.java:602)
> 2020-05-15T09:23:23.2201336Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2020-05-15T09:23:23.2201822Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2020-05-15T09:23:23.2202290Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2020-05-15T09:23:23.2224141Z at
> java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:1975)
> 2020-05-15T09:23:23.2224660Z at
> org.apache.flink.runtime.concurrent.FutureUtils.lambda$retryOperationWithDelay$8(FutureUtils.java:307)
> 2020-05-15T09:23:23.2225150Z at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:774)
> 2020-05-15T09:23:23.2225628Z at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:750)
> 2020-05-15T09:23:23.2226080Z at
> java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488)
> 2020-05-15T09:23:23.2226519Z at
> java.util.concurrent.CompletableFuture.postFire(CompletableFuture.java:575)
> 2020-05-15T09:23:23.2226971Z at
> java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:943)
> 2020-05-15T09:23:23.2227416Z at
> java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:456)
> 2020-05-15T09:23:23.2227864Z at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> 2020-05-15T09:23:23.2228293Z at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> 2020-05-15T09:23:23.2228601Z ... 1 more
> 2020-05-15T09:23:23.2228905Z Caused by:
> org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
> 2020-05-15T09:23:23.2229343Z at
> org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:147)
> 2020-05-15T09:23:23.2229861Z at
> org.apache.flink.client.deployment.ClusterClientJobClientAdapter.lambda$null$6(ClusterClientJobClientAdapter.java:114)
> 2020-05-15T09:23:23.2230242Z ... 19 more
> 2020-05-15T09:23:23.2230569Z Caused by:
> org.apache.flink.runtime.JobException: Recovery is suppressed by
> NoRestartBackoffTimeStrategy
> 2020-05-15T09:23:23.2231404Z at
> org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.handleFailure(ExecutionFailureHandler.java:112)
> 2020-05-15T09:23:23.2232059Z at
> org.apache.flink.runtime.executiongraph.failover.flip1.ExecutionFailureHandler.getFailureHandlingResult(ExecutionFailureHandler.java:78)
> 2020-05-15T09:23:23.2232647Z at
> org.apache.flink.runtime.scheduler.DefaultScheduler.handleTaskFailure(DefaultScheduler.java:189)
> 2020-05-15T09:23:23.2233188Z at
> org.apache.flink.runtime.scheduler.DefaultScheduler.maybeHandleTaskFailure(DefaultScheduler.java:183)
> 2020-05-15T09:23:23.2233734Z at
> org.apache.flink.runtime.scheduler.DefaultScheduler.updateTaskExecutionStateInternal(DefaultScheduler.java:177)
> 2020-05-15T09:23:23.2234253Z at
> org.apache.flink.runtime.scheduler.SchedulerBase.updateTaskExecutionState(SchedulerBase.java:505)
> 2020-05-15T09:23:23.2234748Z at
> org.apache.flink.runtime.jobmaster.JobMaster.updateTaskExecutionState(JobMaster.java:386)
> 2020-05-15T09:23:23.2235150Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2020-05-15T09:23:23.2235515Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2020-05-15T09:23:23.2235967Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2020-05-15T09:23:23.2236466Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2020-05-15T09:23:23.2236860Z at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcInvocation(AkkaRpcActor.java:284)
> 2020-05-15T09:23:23.2237478Z at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleRpcMessage(AkkaRpcActor.java:199)
> 2020-05-15T09:23:23.2238115Z at
> org.apache.flink.runtime.rpc.akka.FencedAkkaRpcActor.handleRpcMessage(FencedAkkaRpcActor.java:74)
> 2020-05-15T09:23:23.2238594Z at
> org.apache.flink.runtime.rpc.akka.AkkaRpcActor.handleMessage(AkkaRpcActor.java:152)
> 2020-05-15T09:23:23.2238985Z at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:26)
> 2020-05-15T09:23:23.2239352Z at
> akka.japi.pf.UnitCaseStatement.apply(CaseStatements.scala:21)
> 2020-05-15T09:23:23.2239726Z at
> scala.PartialFunction$class.applyOrElse(PartialFunction.scala:123)
> 2020-05-15T09:23:23.2240095Z at
> akka.japi.pf.UnitCaseStatement.applyOrElse(CaseStatements.scala:21)
> 2020-05-15T09:23:23.2240569Z at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:170)
> 2020-05-15T09:23:23.2240938Z at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> 2020-05-15T09:23:23.2241326Z at
> scala.PartialFunction$OrElse.applyOrElse(PartialFunction.scala:171)
> 2020-05-15T09:23:23.2241685Z at
> akka.actor.Actor$class.aroundReceive(Actor.scala:517)
> 2020-05-15T09:23:23.2242024Z at
> akka.actor.AbstractActor.aroundReceive(AbstractActor.scala:225)
> 2020-05-15T09:23:23.2242385Z at
> akka.actor.ActorCell.receiveMessage(ActorCell.scala:592)
> 2020-05-15T09:23:23.2242702Z at
> akka.actor.ActorCell.invoke(ActorCell.scala:561)
> 2020-05-15T09:23:23.2243033Z at
> akka.dispatch.Mailbox.processMailbox(Mailbox.scala:258)
> 2020-05-15T09:23:23.2243357Z at akka.dispatch.Mailbox.run(Mailbox.scala:225)
> 2020-05-15T09:23:23.2243644Z at akka.dispatch.Mailbox.exec(Mailbox.scala:235)
> 2020-05-15T09:23:23.2243987Z at
> akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
> 2020-05-15T09:23:23.2244373Z at
> akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
> 2020-05-15T09:23:23.2244784Z at
> akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
> 2020-05-15T09:23:23.2245181Z at
> akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
> 2020-05-15T09:23:23.2246306Z Caused by: java.lang.Exception: The user defined
> 'open(Configuration)' method in class
> org.apache.flink.table.runtime.functions.python.PythonScalarFunctionFlatMap
> caused an exception: Failed to create stage bundle factory!
> 2020-05-15T09:23:23.2246989Z at
> org.apache.flink.runtime.operators.BatchTask.openUserCode(BatchTask.java:1352)
> 2020-05-15T09:23:23.2247464Z at
> org.apache.flink.runtime.operators.chaining.ChainedFlatMapDriver.openTask(ChainedFlatMapDriver.java:47)
> 2020-05-15T09:23:23.2247930Z at
> org.apache.flink.runtime.operators.BatchTask.openChainedTasks(BatchTask.java:1392)
> 2020-05-15T09:23:23.2248370Z at
> org.apache.flink.runtime.operators.DataSourceTask.invoke(DataSourceTask.java:157)
> 2020-05-15T09:23:23.2248765Z at
> org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:713)
> 2020-05-15T09:23:23.2249133Z at
> org.apache.flink.runtime.taskmanager.Task.run(Task.java:539)
> 2020-05-15T09:23:23.2249456Z at java.lang.Thread.run(Thread.java:748)
> 2020-05-15T09:23:23.2249774Z Caused by: java.lang.RuntimeException: Failed to
> create stage bundle factory!
> 2020-05-15T09:23:23.2250246Z at
> org.apache.flink.python.AbstractPythonFunctionRunner.createStageBundleFactory(AbstractPythonFunctionRunner.java:197)
> 2020-05-15T09:23:23.2250755Z at
> org.apache.flink.python.AbstractPythonFunctionRunner.open(AbstractPythonFunctionRunner.java:164)
> 2020-05-15T09:23:23.2251342Z at
> org.apache.flink.table.runtime.runners.python.scalar.AbstractGeneralPythonScalarFunctionRunner.open(AbstractGeneralPythonScalarFunctionRunner.java:65)
> 2020-05-15T09:23:23.2252007Z at
> org.apache.flink.table.runtime.functions.python.AbstractPythonStatelessFunctionFlatMap.open(AbstractPythonStatelessFunctionFlatMap.java:211)
> 2020-05-15T09:23:23.2252624Z at
> org.apache.flink.table.runtime.functions.python.AbstractPythonScalarFunctionFlatMap.open(AbstractPythonScalarFunctionFlatMap.java:69)
> 2020-05-15T09:23:23.2253173Z at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> 2020-05-15T09:23:23.2253718Z at
> org.apache.flink.runtime.operators.BatchTask.openUserCode(BatchTask.java:1350)
> 2020-05-15T09:23:23.2253997Z ... 6 more
> 2020-05-15T09:23:23.2254328Z Caused by: java.io.IOException: Cannot run
> program "venv.zip/.conda/bin/python": error=2, No such file or directory
> 2020-05-15T09:23:23.2254737Z at
> java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
> 2020-05-15T09:23:23.2255178Z at
> org.apache.flink.python.util.PythonEnvironmentManagerUtils.execute(PythonEnvironmentManagerUtils.java:176)
> 2020-05-15T09:23:23.2255723Z at
> org.apache.flink.python.util.PythonEnvironmentManagerUtils.getSitePackagesPath(PythonEnvironmentManagerUtils.java:156)
> 2020-05-15T09:23:23.2256434Z at
> org.apache.flink.python.util.PythonEnvironmentManagerUtils.pipInstallRequirements(PythonEnvironmentManagerUtils.java:98)
> 2020-05-15T09:23:23.2257046Z at
> org.apache.flink.python.env.ProcessPythonEnvironmentManager.createEnvironment(ProcessPythonEnvironmentManager.java:172)
> 2020-05-15T09:23:23.2257609Z at
> org.apache.flink.python.AbstractPythonFunctionRunner.createPythonExecutionEnvironment(AbstractPythonFunctionRunner.java:249)
> 2020-05-15T09:23:23.2258213Z at
> org.apache.flink.table.runtime.runners.python.AbstractPythonStatelessFunctionRunner.createExecutableStage(AbstractPythonStatelessFunctionRunner.java:158)
> 2020-05-15T09:23:23.2258821Z at
> org.apache.flink.python.AbstractPythonFunctionRunner.createStageBundleFactory(AbstractPythonFunctionRunner.java:195)
> 2020-05-15T09:23:23.2259172Z ... 12 more
> 2020-05-15T09:23:23.2259405Z Caused by: java.io.IOException: error=2, No such
> file or directory
> 2020-05-15T09:23:23.2259710Z at java.lang.UNIXProcess.forkAndExec(Native
> Method)
> 2020-05-15T09:23:23.2259991Z at
> java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
> 2020-05-15T09:23:23.2260313Z at
> java.lang.ProcessImpl.start(ProcessImpl.java:134)
> 2020-05-15T09:23:23.2260623Z at
> java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
> 2020-05-15T09:23:23.2260875Z ... 19 more
> 2020-05-15T09:23:23.2260980Z
> 2020-05-15T09:23:23.2406372Z
> org.apache.flink.client.program.ProgramAbortException
> 2020-05-15T09:23:23.2406771Z at
> org.apache.flink.client.python.PythonDriver.main(PythonDriver.java:95)
> 2020-05-15T09:23:23.2407331Z at
> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2020-05-15T09:23:23.2407892Z at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2020-05-15T09:23:23.2408342Z at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2020-05-15T09:23:23.2408748Z at
> java.lang.reflect.Method.invoke(Method.java:498)
> 2020-05-15T09:23:23.2409138Z at
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:288)
> 2020-05-15T09:23:23.2409654Z at
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:198)
> 2020-05-15T09:23:23.2410157Z at
> org.apache.flink.client.ClientUtils.executeProgram(ClientUtils.java:148)
> 2020-05-15T09:23:23.2410664Z at
> org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:707)
> 2020-05-15T09:23:23.2411172Z at
> org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:245)
> 2020-05-15T09:23:23.2411577Z at
> org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:924)
> 2020-05-15T09:23:23.2412014Z at
> org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1000)
> 2020-05-15T09:23:23.2412499Z at
> org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:30)
> 2020-05-15T09:23:23.2413178Z at
> org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1000)
> 2020-05-15T09:23:23.8192063Z Stopping taskexecutor daemon (pid: 89153) on
> host fv-az678.
> 2020-05-15T09:23:23.9113567Z Stopping standalonesession daemon (pid: 88857)
> on host fv-az678.
> 2020-05-15T09:23:24.0749389Z [FAIL] Test script contains errors.
> 2020-05-15T09:23:24.0757935Z Checking of logs skipped.
> 2020-05-15T09:23:24.0758481Z
> 2020-05-15T09:23:24.0759921Z [FAIL] 'PyFlink end-to-end test' failed after 3
> minutes and 24 seconds! Test exited with exit code 1
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
