[jira] [Commented] (FLINK-1840) Job execution fails on Windows (native and Cygwin)
[ https://issues.apache.org/jira/browse/FLINK-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484908#comment-14484908 ] ASF GitHub Bot commented on FLINK-1840: --- Github user asfgit closed the pull request at: https://github.com/apache/flink/pull/578 > Job execution fails on Windows (native and Cygwin) > -- > > Key: FLINK-1840 > URL: https://issues.apache.org/jira/browse/FLINK-1840 > Project: Flink > Issue Type: Bug > Components: JobManager >Affects Versions: 0.9 > Environment: Windows 8.1 (Cygwin and native) >Reporter: Fabian Hueske >Assignee: Fabian Hueske >Priority: Critical > Fix For: 0.9 > > > Job execution fails on Windows using > - {{./bin/flink.bat}} (native) > - {{./bin/flink.sh}} (Cygwin) > - Web submission client > The following exception is thrown: > {code} > org.apache.flink.client.program.ProgramInvocationException: Could not upload > the program's JAR files tothe JobManager. > at org.apache.flink.client.program.Client.run(Client.java:341) > at org.apache.flink.client.program.Client.run(Client.java:313) > at org.apache.flink.client.program.Client.run(Client.java:306) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:63) > at > org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:82) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedPro >gram.java:353) > at org.apache.flink.client.program.Client.run(Client.java:273) > at > org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:584) > at org.apache.flink.client.CliFrontend.run(CliFrontend.java:290) > at > org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:886) > at org.apache.flink.client.CliFrontend.main(CliFrontend.java:928) > Caused by: java.io.IOException: PUT operation failed: Server side error: > Cannot move staging file C:\cyg > win64\tmp\blobStore-17f49f61-bcb7-45ba-909e-56beca159fe4\incoming\temp- > to BLOB file C:\cygwin64 > \tmp\blobStore-17f49f61-bcb7-45ba-909e-56beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:465) > at org.apache.flink.runtime.blob.BlobClient.put(BlobClient.java:327) > at > org.apache.flink.runtime.jobgraph.JobGraph.uploadRequiredJarFiles(JobGraph.java:457) > at > org.apache.flink.runtime.client.JobClient$.uploadJarFiles(JobClient.scala:288) > at > org.apache.flink.runtime.client.JobClient.uploadJarFiles(JobClient.scala) > at org.apache.flink.client.program.Client.run(Client.java:338) > ... 15 more > Caused by: java.io.IOException: Server side error: Cannot move staging file > C:\cygwin64\tmp\blobStore-17 > f49f61-bcb7-45ba-909e-56beca159fe4\incoming\temp- to BLOB file > C:\cygwin64\tmp\blobStore-17f49f6 > 1-bcb7-45ba-909e-56beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobClient.receivePutResponseAndCompare(BlobClient.java:491) > at > org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:461) > ... 20 more > Caused by: java.io.IOException: Cannot move staging file > C:\cygwin64\tmp\blobStore-17f49f61-bcb7-45ba-90 > 9e-56beca159fe4\incoming\temp- to BLOB file > C:\cygwin64\tmp\blobStore-17f49f61-bcb7-45ba-909e-56 > beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobServerConnection.put(BlobServerConnection.java:325) > at > org.apache.flink.runtime.blob.BlobServerConnection.run(BlobServerConnection.java:109) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1840) Job execution fails on Windows (native and Cygwin)
[ https://issues.apache.org/jira/browse/FLINK-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484172#comment-14484172 ] ASF GitHub Bot commented on FLINK-1840: --- Github user hsaputra commented on the pull request: https://github.com/apache/flink/pull/578#issuecomment-90745068 +1 LGTM > Job execution fails on Windows (native and Cygwin) > -- > > Key: FLINK-1840 > URL: https://issues.apache.org/jira/browse/FLINK-1840 > Project: Flink > Issue Type: Bug > Components: JobManager >Affects Versions: 0.9 > Environment: Windows 8.1 (Cygwin and native) >Reporter: Fabian Hueske >Assignee: Fabian Hueske >Priority: Critical > Fix For: 0.9 > > > Job execution fails on Windows using > - {{./bin/flink.bat}} (native) > - {{./bin/flink.sh}} (Cygwin) > - Web submission client > The following exception is thrown: > {code} > org.apache.flink.client.program.ProgramInvocationException: Could not upload > the program's JAR files tothe JobManager. > at org.apache.flink.client.program.Client.run(Client.java:341) > at org.apache.flink.client.program.Client.run(Client.java:313) > at org.apache.flink.client.program.Client.run(Client.java:306) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:63) > at > org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:82) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedPro >gram.java:353) > at org.apache.flink.client.program.Client.run(Client.java:273) > at > org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:584) > at org.apache.flink.client.CliFrontend.run(CliFrontend.java:290) > at > org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:886) > at org.apache.flink.client.CliFrontend.main(CliFrontend.java:928) > Caused by: java.io.IOException: PUT operation failed: Server side error: > Cannot move staging file C:\cyg > win64\tmp\blobStore-17f49f61-bcb7-45ba-909e-56beca159fe4\incoming\temp- > to BLOB file C:\cygwin64 > \tmp\blobStore-17f49f61-bcb7-45ba-909e-56beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:465) > at org.apache.flink.runtime.blob.BlobClient.put(BlobClient.java:327) > at > org.apache.flink.runtime.jobgraph.JobGraph.uploadRequiredJarFiles(JobGraph.java:457) > at > org.apache.flink.runtime.client.JobClient$.uploadJarFiles(JobClient.scala:288) > at > org.apache.flink.runtime.client.JobClient.uploadJarFiles(JobClient.scala) > at org.apache.flink.client.program.Client.run(Client.java:338) > ... 15 more > Caused by: java.io.IOException: Server side error: Cannot move staging file > C:\cygwin64\tmp\blobStore-17 > f49f61-bcb7-45ba-909e-56beca159fe4\incoming\temp- to BLOB file > C:\cygwin64\tmp\blobStore-17f49f6 > 1-bcb7-45ba-909e-56beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobClient.receivePutResponseAndCompare(BlobClient.java:491) > at > org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:461) > ... 20 more > Caused by: java.io.IOException: Cannot move staging file > C:\cygwin64\tmp\blobStore-17f49f61-bcb7-45ba-90 > 9e-56beca159fe4\incoming\temp- to BLOB file > C:\cygwin64\tmp\blobStore-17f49f61-bcb7-45ba-909e-56 > beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobServerConnection.put(BlobServerConnection.java:325) > at > org.apache.flink.runtime.blob.BlobServerConnection.run(BlobServerConnection.java:109) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (FLINK-1840) Job execution fails on Windows (native and Cygwin)
[ https://issues.apache.org/jira/browse/FLINK-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484129#comment-14484129 ] ASF GitHub Bot commented on FLINK-1840: --- GitHub user fhueske opened a pull request: https://github.com/apache/flink/pull/578 [FLINK-1840] Fix BLOB manager on Windows - closes open `FileOutputStream` before the field is moved - replaces `java.io.File.renameTo()` by `com.google.common.io.Files.move()` to avoid platform-dependent behavior You can merge this pull request into a Git repository by running: $ git pull https://github.com/fhueske/flink blobWindows Alternatively you can review and apply these changes as the patch at: https://github.com/apache/flink/pull/578.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #578 commit b83ab465ba727751bc13db26989c40a94ffe378b Author: Fabian Hueske Date: 2015-04-07T21:04:59Z [FLINK-1840] Fix BLOB manager on Windows > Job execution fails on Windows (native and Cygwin) > -- > > Key: FLINK-1840 > URL: https://issues.apache.org/jira/browse/FLINK-1840 > Project: Flink > Issue Type: Bug > Components: JobManager >Affects Versions: 0.9 > Environment: Windows 8.1 (Cygwin and native) >Reporter: Fabian Hueske >Assignee: Fabian Hueske >Priority: Critical > Fix For: 0.9 > > > Job execution fails on Windows using > - {{./bin/flink.bat}} (native) > - {{./bin/flink.sh}} (Cygwin) > - Web submission client > The following exception is thrown: > {code} > org.apache.flink.client.program.ProgramInvocationException: Could not upload > the program's JAR files tothe JobManager. > at org.apache.flink.client.program.Client.run(Client.java:341) > at org.apache.flink.client.program.Client.run(Client.java:313) > at org.apache.flink.client.program.Client.run(Client.java:306) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:63) > at > org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:82) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedPro >gram.java:353) > at org.apache.flink.client.program.Client.run(Client.java:273) > at > org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:584) > at org.apache.flink.client.CliFrontend.run(CliFrontend.java:290) > at > org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:886) > at org.apache.flink.client.CliFrontend.main(CliFrontend.java:928) > Caused by: java.io.IOException: PUT operation failed: Server side error: > Cannot move staging file C:\cyg > win64\tmp\blobStore-17f49f61-bcb7-45ba-909e-56beca159fe4\incoming\temp- > to BLOB file C:\cygwin64 > \tmp\blobStore-17f49f61-bcb7-45ba-909e-56beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:465) > at org.apache.flink.runtime.blob.BlobClient.put(BlobClient.java:327) > at > org.apache.flink.runtime.jobgraph.JobGraph.uploadRequiredJarFiles(JobGraph.java:457) > at > org.apache.flink.runtime.client.JobClient$.uploadJarFiles(JobClient.scala:288) > at > org.apache.flink.runtime.client.JobClient.uploadJarFiles(JobClient.scala) > at org.apache.flink.client.program.Client.run(Client.java:338) > ... 15 more > Caused by: java.io.IOException: Server side error: Cannot move staging file > C:\cygwin64\tmp\blobStore-17 > f49f61-bcb7-45ba-909e-56beca159fe4\incoming\temp- to BLOB file > C:\cygwin64\tmp\blobStore-17f49f6 > 1-bcb7-45ba-909e-56beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobClient.receivePutResponseAndCompare(BlobClient.java:491) > at > org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:461) > ... 20 more > Caused by: java.io.IOException: Cannot move staging file > C:\cygwin64\tmp\blobStore-17f49f61-bcb7-45ba-90 > 9e-56beca159fe4\incoming\temp-
[jira] [Commented] (FLINK-1840) Job execution fails on Windows (native and Cygwin)
[ https://issues.apache.org/jira/browse/FLINK-1840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14484001#comment-14484001 ] Fabian Hueske commented on FLINK-1840: -- The exception is cause by the {{java.io.File.renameTo()}} method call. The JavaDocs say that the behavior is ??inherently platform-depended??. Therefore, I tried to substitute the call by an alternative but neither {{com.google.common.io.Files.move()}} or {{java.nio.file.Files.move()}} worked. It seems like the source file is still used/accessed by a process and can therefore not be deleted. > Job execution fails on Windows (native and Cygwin) > -- > > Key: FLINK-1840 > URL: https://issues.apache.org/jira/browse/FLINK-1840 > Project: Flink > Issue Type: Bug > Components: JobManager >Affects Versions: 0.9 > Environment: Windows 8.1 (Cygwin and native) >Reporter: Fabian Hueske >Priority: Critical > Fix For: 0.9 > > > Job execution fails on Windows using > - {{./bin/flink.bat}} (native) > - {{./bin/flink.sh}} (Cygwin) > - Web submission client > The following exception is thrown: > {code} > org.apache.flink.client.program.ProgramInvocationException: Could not upload > the program's JAR files tothe JobManager. > at org.apache.flink.client.program.Client.run(Client.java:341) > at org.apache.flink.client.program.Client.run(Client.java:313) > at org.apache.flink.client.program.Client.run(Client.java:306) > at > org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:63) > at > org.apache.flink.examples.java.wordcount.WordCount.main(WordCount.java:82) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at > org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437) > at > org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedPro >gram.java:353) > at org.apache.flink.client.program.Client.run(Client.java:273) > at > org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:584) > at org.apache.flink.client.CliFrontend.run(CliFrontend.java:290) > at > org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:886) > at org.apache.flink.client.CliFrontend.main(CliFrontend.java:928) > Caused by: java.io.IOException: PUT operation failed: Server side error: > Cannot move staging file C:\cyg > win64\tmp\blobStore-17f49f61-bcb7-45ba-909e-56beca159fe4\incoming\temp- > to BLOB file C:\cygwin64 > \tmp\blobStore-17f49f61-bcb7-45ba-909e-56beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:465) > at org.apache.flink.runtime.blob.BlobClient.put(BlobClient.java:327) > at > org.apache.flink.runtime.jobgraph.JobGraph.uploadRequiredJarFiles(JobGraph.java:457) > at > org.apache.flink.runtime.client.JobClient$.uploadJarFiles(JobClient.scala:288) > at > org.apache.flink.runtime.client.JobClient.uploadJarFiles(JobClient.scala) > at org.apache.flink.client.program.Client.run(Client.java:338) > ... 15 more > Caused by: java.io.IOException: Server side error: Cannot move staging file > C:\cygwin64\tmp\blobStore-17 > f49f61-bcb7-45ba-909e-56beca159fe4\incoming\temp- to BLOB file > C:\cygwin64\tmp\blobStore-17f49f6 > 1-bcb7-45ba-909e-56beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobClient.receivePutResponseAndCompare(BlobClient.java:491) > at > org.apache.flink.runtime.blob.BlobClient.putInputStream(BlobClient.java:461) > ... 20 more > Caused by: java.io.IOException: Cannot move staging file > C:\cygwin64\tmp\blobStore-17f49f61-bcb7-45ba-90 > 9e-56beca159fe4\incoming\temp- to BLOB file > C:\cygwin64\tmp\blobStore-17f49f61-bcb7-45ba-909e-56 > beca159fe4\cache\blob_9d5dc35b7c40b2ddda2df12d7ee8702f5683b131 > at > org.apache.flink.runtime.blob.BlobServerConnection.put(BlobServerConnection.java:325) > at > org.apache.flink.runtime.blob.BlobServerConnection.run(BlobServerConnection.java:109) > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)