[
https://issues.apache.org/jira/browse/HADOOP-18521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Loughran updated HADOOP-18521:
------------------------------------
Description:
AbfsInputStream.close() can trigger the return of buffers used for active
prefetch GET requests into the ReadBufferManager free buffer pool.
A subsequent prefetch by a different stream in the same process may acquire
this same buffer. This can lead to risk of corruption of its own prefetched
data, data which may then be returned to that other thread.
On releases without the fix for this (3.3.2+), the bug can be avoided by
disabling all prefetching
{code}
fs.azure.readaheadqueue.depth = 0
{code}
was:
AbfsInputStream.close() can trigger the return of buffers used for active
prefetch GET requests into the ReadBufferManager free buffer pool.
A subsequent prefetch by a different stream in the same process may acquire
this same buffer. This can lead to risk of corruption of its own prefetched
data, data which may then be returned to that other thread.
On releases without the fix for this (3.3.2 to 3.3.4), the bug can be avoided
by disabling all prefetching
{code}
fs.azure.readaheadqueue.depth = 0
{code}
> ABFS ReadBufferManager buffer sharing across concurrent HTTP requests
> ---------------------------------------------------------------------
>
> Key: HADOOP-18521
> URL: https://issues.apache.org/jira/browse/HADOOP-18521
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs/azure
> Affects Versions: 3.3.2, 3.3.3, 3.3.4
> Reporter: Steve Loughran
> Assignee: Steve Loughran
> Priority: Critical
> Labels: pull-request-available
>
> AbfsInputStream.close() can trigger the return of buffers used for active
> prefetch GET requests into the ReadBufferManager free buffer pool.
> A subsequent prefetch by a different stream in the same process may acquire
> this same buffer. This can lead to risk of corruption of its own prefetched
> data, data which may then be returned to that other thread.
> On releases without the fix for this (3.3.2+), the bug can be avoided by
> disabling all prefetching
> {code}
> fs.azure.readaheadqueue.depth = 0
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
