[
https://issues.apache.org/jira/browse/HADOOP-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17646332#comment-17646332
]
ASF GitHub Bot commented on HADOOP-18546:
-----------------------------------------
steveloughran commented on PR #5205:
URL: https://github.com/apache/hadoop/pull/5205#issuecomment-1347337424
@snvijaya @mukund-thakur @mehakmeet can I get a review of this -i want this
in so there is a programmatic check for the presence of the fix. I'm adding a
"safeprefetch" command to cloudstore which will identify when an abfs release
has the bug (everything with etag_aware), has the fix (the new probe) and if
vulnerable review the options, printing out the correct settings in xml and
spark conf. we need this probe for it to see when things are good
https://github.com/steveloughran/cloudstore/blob/trunk/src/main/java/org/apache/hadoop/fs/store/abfs/SafePrefetch.java
> disable purging list of in progress reads in abfs stream closed
> ---------------------------------------------------------------
>
> Key: HADOOP-18546
> URL: https://issues.apache.org/jira/browse/HADOOP-18546
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: fs/azure
> Affects Versions: 3.3.4
> Reporter: Steve Loughran
> Assignee: Pranav Saxena
> Priority: Major
> Labels: pull-request-available
> Fix For: 3.4.0, 3.3.5
>
>
> turn off the prune of in progress reads in
> ReadBufferManager::purgeBuffersForStream
> this will ensure active prefetches for a closed stream complete. they wiill
> then get to the completed list and hang around until evicted by timeout, but
> at least prefetching will be safe.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
