[jira] [Updated] (FLINK-9769) FileUploads may be shared across requests
[
https://issues.apache.org/jira/browse/FLINK-9769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated FLINK-9769:
--
Labels: pull-request-available (was: )
> FileUploads may be shared across requests
> -
>
> Key: FLINK-9769
> URL: https://issues.apache.org/jira/browse/FLINK-9769
> Project: Flink
> Issue Type: Bug
> Components: Job-Submission, REST, Webfrontend
>Affects Versions: 1.5.1, 1.6.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.5.1, 1.6.0
>
>
> Files uploaded by the {{FileUploadHandler}} are passed on to subsequent
> handlers by storing them in a channel attribute.
> The files are retrieved from said attribute by the {{AbstractHandler}}.
> Apparently, since the attribute isn't set to null when retrieving the
> contained value, it can happen that other handlers still see the value, if
> the channel is shared across several requests. (This behavior is surprising
> as i thought that each requests has it's own channel.)
> However, the retrieved files will no longer exist for any handler but the
> original recipient, because he ensures that the files are cleaned up after
> processing.
> Note that this issue existed for a quite a while, it just didn't surface as
> only a single handler ever accessed these attributes.
> {code}
> 2018-07-05 21:55:09,297 ERROR
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Request
> processing failed.
> java.nio.file.NoSuchFileException:
> C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
> at
> sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
> at java.nio.file.Files.readAttributes(Files.java:1737)
> at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
> at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
> at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
> at java.nio.file.Files.walkFileTree(Files.java:2662)
> at java.nio.file.Files.walkFileTree(Files.java:2742)
> at
> org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
> at
> org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
> at
> org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
> at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> at
> java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
> at java.lang.Thread.run(Thread.java:745)
> 2018-07-05 21:55:09,485 ERROR
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler - Request
> processing failed.
> java.nio.file.NoSuchFileException:
> C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
> at
> sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
> at
[jira] [Updated] (FLINK-9769) FileUploads may be shared across requests
[
https://issues.apache.org/jira/browse/FLINK-9769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-9769:
Description:
Files uploaded by the {{FileUploadHandler}} are passed on to subsequent
handlers by storing them in a channel attribute.
The files are retrieved from said attribute by the {{AbstractHandler}}.
Apparently, since the attribute isn't set to null when retrieving the contained
value, it can happen that other handlers still see the value, if the channel is
shared across several requests. (This behavior is surprising as i thought that
each requests has it's own channel.)
However, the retrieved files will no longer exist for any handler but the
original recipient, because he ensures that the files are cleaned up after
processing.
Note that this issue existed for a quite a while, it just didn't surface as
only a single handler ever accessed these attributes.
{code}
2018-07-05 21:55:09,297 ERROR
org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Request
processing failed.
java.nio.file.NoSuchFileException:
C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at
sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
at
sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
at
sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at java.nio.file.Files.walkFileTree(Files.java:2742)
at
org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
at
org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
at
org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at
org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
2018-07-05 21:55:09,485 ERROR
org.apache.flink.runtime.webmonitor.handlers.JarListHandler - Request
processing failed.
java.nio.file.NoSuchFileException:
C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at
sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
at
sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
at
sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at java.nio.file.Files.walkFileTree(Files.java:2742)
at
org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
at
org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
at
[jira] [Updated] (FLINK-9769) FileUploads may be shared across requests
[
https://issues.apache.org/jira/browse/FLINK-9769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-9769:
Description:
Files uploaded by the {{FileUploadHandler}} are passed on to subsequent
handlers by storing them in a channel attribute.
The files are retrieved from said attribute by the {{AbstractHandler}}.
Apparently, since the attribute isn't set to null when retrieving the contained
value, it can happen that other handlers still see the value, if the channel is
shared across several requests. (This behavior is surprising as i thought that
each requests has it's own channel.)
However, the retrieved files will no longer exist for any handler but the
original recipient, because he ensures that the files are cleaned up after
processing.
{code}
2018-07-05 21:55:09,297 ERROR
org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Request
processing failed.
java.nio.file.NoSuchFileException:
C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at
sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
at
sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
at
sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at java.nio.file.Files.walkFileTree(Files.java:2742)
at
org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
at
org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
at
org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
at
java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
at
java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
at
org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
at
org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
at java.lang.Thread.run(Thread.java:745)
2018-07-05 21:55:09,485 ERROR
org.apache.flink.runtime.webmonitor.handlers.JarListHandler - Request
processing failed.
java.nio.file.NoSuchFileException:
C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
at
sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
at
sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
at
sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
at
sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
at
sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
at java.nio.file.Files.readAttributes(Files.java:1737)
at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
at java.nio.file.Files.walkFileTree(Files.java:2662)
at java.nio.file.Files.walkFileTree(Files.java:2742)
at
org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
at
org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
at
org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
at
java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
[jira] [Updated] (FLINK-9769) FileUploads may be shared across requests
[
https://issues.apache.org/jira/browse/FLINK-9769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chesnay Schepler updated FLINK-9769:
Summary: FileUploads may be shared across requests (was: Job submission
via WebUI broken)
> FileUploads may be shared across requests
> -
>
> Key: FLINK-9769
> URL: https://issues.apache.org/jira/browse/FLINK-9769
> Project: Flink
> Issue Type: Bug
> Components: Job-Submission, REST, Webfrontend
>Affects Versions: 1.5.1, 1.6.0
>Reporter: Chesnay Schepler
>Assignee: Chesnay Schepler
>Priority: Blocker
> Fix For: 1.5.1, 1.6.0
>
>
> The rework of the {{FileUploadHandler}} apparently broke the Web UI job
> submission.
> It would be great if someone could check whether this also occurs on 1.6.
> {code}
> 2018-07-05 21:55:09,297 ERROR
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Request
> processing failed.
> java.nio.file.NoSuchFileException:
> C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
> at
> sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
> at java.nio.file.Files.readAttributes(Files.java:1737)
> at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
> at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
> at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
> at java.nio.file.Files.walkFileTree(Files.java:2662)
> at java.nio.file.Files.walkFileTree(Files.java:2742)
> at
> org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
> at
> org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
> at
> org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
> at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> at
> java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
> at java.lang.Thread.run(Thread.java:745)
> 2018-07-05 21:55:09,485 ERROR
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler - Request
> processing failed.
> java.nio.file.NoSuchFileException:
> C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
> at
> sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
> at java.nio.file.Files.readAttributes(Files.java:1737)
> at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
> at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
> at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
> at java.nio.file.Files.walkFileTree(Files.java:2662)
> at java.nio.file.Files.walkFileTree(Files.java:2742)
> at
> org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
> at
> org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
> at
>
