Hi Schmid, Just as you have seen that the jar package of flink-python is located in the directory $FLINK_HOME/opt. It seems that this jar doesn't exist in your environment. Could you double check that?
Regards, Dian On Thu, Oct 28, 2021 at 3:35 PM Schmid Christian <[email protected]> wrote: > Hi, > > > > When I execute a PyFlink-Job locally in a mini cluster, then everything > works fine: > > > > (env) > > user@bla /cygdrive/c/flink1_13_3/examples/python/table/batch > > $ python word_count.py > > Results directory: C:\cygwin64\tmp/result > > > > But when I try to execute the PyFlink-Job in a remote cluster, the job > execution fails: > > > > user@bla /cygdrive/c/flink1_13_3/examples/python/table/batch > > $ flink run -pyarch env.zip -pyexec env.zip/env/Scripts/python -py > word_count.py > > > > Although I set the environment variable (FLINK_HOME = C:\flink1_13_3), > Flink is trying to fetch the flink-python_2.11-1.13.3.jar from the wrong > directory (\cygdrive\c\flink1_13_3\opt). > > > > The stacktrace in the logfiles looks as follow: > > > > java.lang.RuntimeException: Exception encountered during finding the > flink-python jar. This should not happen. > > at > org.apache.flink.client.program.PackagedProgramUtils.getPythonJar(PackagedProgramUtils.java:222) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.ProgramOptionsUtils.getPythonClassLoader(ProgramOptionsUtils.java:98) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.ProgramOptionsUtils.createPythonProgramOptions(ProgramOptionsUtils.java:76) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.ProgramOptions.create(ProgramOptions.java:186) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:236) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28) > [flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132) > [flink-dist_2.11-1.13.3.jar:1.13.3] > > Caused by: java.nio.file.NoSuchFileException: \cygdrive\c\flink1_13_3\opt > > at > sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79) > ~[?:1.8.0_302] > > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97) > ~[?:1.8.0_302] > > at > sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102) > ~[?:1.8.0_302] > > at > sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53) > ~[?:1.8.0_302] > > at > sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38) > ~[?:1.8.0_302] > > at > sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193) > ~[?:1.8.0_302] > > at java.nio.file.Files.readAttributes(Files.java:1737) > ~[?:1.8.0_302] > > at > java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219) > ~[?:1.8.0_302] > > at > java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276) ~[?:1.8.0_302] > > at > java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322) ~[?:1.8.0_302] > > at java.nio.file.Files.walkFileTree(Files.java:2662) > ~[?:1.8.0_302] > > at java.nio.file.Files.walkFileTree(Files.java:2742) > ~[?:1.8.0_302] > > at > org.apache.flink.client.program.PackagedProgramUtils.getPythonJar(PackagedProgramUtils.java:208) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > ... 8 more > > 2021-10-28 09:20:35,663 ERROR > org.apache.flink.client.cli.CliFrontend [] - Invalid > command line arguments. > > org.apache.flink.client.cli.CliArgsException: Python command line option > detected but the flink-python module seems to be missing or not working as > expected. > > at > org.apache.flink.client.cli.ProgramOptionsUtils.createPythonProgramOptions(ProgramOptionsUtils.java:88) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.ProgramOptions.create(ProgramOptions.java:186) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:236) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.CliFrontend.parseAndRun(CliFrontend.java:1054) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.CliFrontend.lambda$main$10(CliFrontend.java:1132) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.runtime.security.contexts.NoOpSecurityContext.runSecured(NoOpSecurityContext.java:28) > [flink-dist_2.11-1.13.3.jar:1.13.3] > > at > org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1132) > [flink-dist_2.11-1.13.3.jar:1.13.3] > > Caused by: java.lang.ClassNotFoundException: > org.apache.flink.client.cli.PythonProgramOptions > > at > java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_302] > > at java.lang.ClassLoader.loadClass(ClassLoader.java:419) > ~[?:1.8.0_302] > > at > sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) ~[?:1.8.0_302] > > at java.lang.ClassLoader.loadClass(ClassLoader.java:352) > ~[?:1.8.0_302] > > at java.lang.Class.forName0(Native Method) ~[?:1.8.0_302] > > at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_302] > > at > org.apache.flink.client.cli.ProgramOptionsUtils.createPythonProgramOptions(ProgramOptionsUtils.java:78) > ~[flink-dist_2.11-1.13.3.jar:1.13.3] > > ... 6 more > > > > How can I fix this? > > > > I'm using Flink Version 1.13.3 (Windows – Cygwin) > > > > Any help would be highly appreciated. > > > > *Diese E-Mail ist ausschliesslich für den Adressaten bestimmt. Sollten Sie > diese E-Mail irrtümlich erhalten haben oder wünschen Sie künftig keine > Kontakte mehr per E-Mail, bitten wir Sie, die Bank Cler hierüber sofort zu > orientieren. Die irrtümlich erhaltene E-Mail ist mit allen Anhängen > unwiderruflich zu löschen, allfällige Ausdrucke sind zu vernichten und auf > die Verwendung des Inhalts ist zu verzichten. Der Versand unverschlüsselter > E-Mail birgt erhebliche Risiken in sich (mangelnde Vertraulichkeit, > Manipulation von Inhalt/Absender, Fehlleitung, Viren etc.). Bank Cler lehnt > jede Haftung für Schäden hieraus ab. Bank Cler akzeptiert grundsätzlich > keine per E-Mail übermittelten Aufträge, Widerrufe von Aufträgen oder > sonstige Weisungen etc., ohne verpflichtet zu sein, diese ausdrücklich > zurück zu weisen. Kündigungen von Verträgen per E-Mail sind nicht > rechtswirksam. * >
