[
https://issues.apache.org/jira/browse/FLINK-9769?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16534185#comment-16534185
]
ASF GitHub Bot commented on FLINK-9769:
---------------------------------------
GitHub user zentol opened a pull request:
https://github.com/apache/flink/pull/6270
[FLINK-9769][rest] Clear FileUpload attribute after access
## What is the purpose of the change
Prevents a resource leakage by clearing the `UPLOADED_FILES` attribute
after accessing it.
Previously it could happen that a handler might see the result of a file
upload operation if it happened to be sharing the same channel. In this case
said files were already cleaned up, leading to `FileNotFoundExceptions`.
## Verifying this change
Manually verified. Submit a job via the Web UI, no exception should be
logged.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zentol/flink 9769
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/6270.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6270
----
commit 56af2054970dc3282b28dfc8cac4b0a4142bf1ab
Author: zentol <chesnay@...>
Date: 2018-07-05T21:28:47Z
[FLINK-9769][rest] Clear FileUpload attribute after access
----
> FileUploads may be shared across requests
> -----------------------------------------
>
> Key: FLINK-9769
> URL: https://issues.apache.org/jira/browse/FLINK-9769
> Project: Flink
> Issue Type: Bug
> Components: Job-Submission, REST, Webfrontend
> Affects Versions: 1.5.1, 1.6.0
> Reporter: Chesnay Schepler
> Assignee: Chesnay Schepler
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 1.5.1, 1.6.0
>
>
> Files uploaded by the {{FileUploadHandler}} are passed on to subsequent
> handlers by storing them in a channel attribute.
> The files are retrieved from said attribute by the {{AbstractHandler}}.
> Apparently, since the attribute isn't set to null when retrieving the
> contained value, it can happen that other handlers still see the value, if
> the channel is shared across several requests. (This behavior is surprising
> as i thought that each requests has it's own channel.)
> However, the retrieved files will no longer exist for any handler but the
> original recipient, because he ensures that the files are cleaned up after
> processing.
> Note that this issue existed for a quite a while, it just didn't surface as
> only a single handler ever accessed these attributes.
> {code}
> 2018-07-05 21:55:09,297 ERROR
> org.apache.flink.runtime.rest.handler.job.JobsOverviewHandler - Request
> processing failed.
> java.nio.file.NoSuchFileException:
> C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
> at
> sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
> at java.nio.file.Files.readAttributes(Files.java:1737)
> at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
> at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
> at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
> at java.nio.file.Files.walkFileTree(Files.java:2662)
> at java.nio.file.Files.walkFileTree(Files.java:2742)
> at
> org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
> at
> org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
> at
> org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
> at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> at
> java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
> at java.lang.Thread.run(Thread.java:745)
> 2018-07-05 21:55:09,485 ERROR
> org.apache.flink.runtime.webmonitor.handlers.JarListHandler - Request
> processing failed.
> java.nio.file.NoSuchFileException:
> C:\Users\Zento\AppData\Local\Temp\flink-web-2c7cae9f-e2d0-4a0e-8696-ef6894238a2e\flink-web-upload\b002df81-2d6f-4727-ae6e-aaa20be22b3b
> at
> sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:79)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
> at
> sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
> at
> sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
> at
> sun.nio.fs.WindowsFileSystemProvider.readAttributes(WindowsFileSystemProvider.java:193)
> at java.nio.file.Files.readAttributes(Files.java:1737)
> at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:219)
> at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
> at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
> at java.nio.file.Files.walkFileTree(Files.java:2662)
> at java.nio.file.Files.walkFileTree(Files.java:2742)
> at
> org.apache.flink.runtime.rest.handler.FileUploads.getUploadedFiles(FileUploads.java:68)
> at
> org.apache.flink.runtime.rest.AbstractHandler.respondAsLeader(AbstractHandler.java:107)
> at
> org.apache.flink.runtime.rest.handler.RedirectHandler.lambda$null$0(RedirectHandler.java:139)
> at
> java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760)
> at
> java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736)
> at
> java.util.concurrent.CompletableFuture$Completion.run(CompletableFuture.java:442)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:357)
> at
> org.apache.flink.shaded.netty4.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:357)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
> at
> org.apache.flink.shaded.netty4.io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:137)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
