[
https://issues.apache.org/jira/browse/HDFS-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16491363#comment-16491363
]
Konstantin Shvachko commented on HDFS-12978:
--------------------------------------------
[~csun] I agree re-initializing input streams on each iteration with
{{selectInputStreams()}} may be expensive. As I [mentioned
earlier|https://issues.apache.org/jira/browse/HDFS-12978?focusedCommentId=16488293&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16488293]
optimal solution will require a substantial refactoring of the edits loading
logic, as it is tide now closely to reading closed segments. Which we can
target for the next jira, but let's first see if it is actually a problem.
[~vagarychen] unfortunately this is the redundant lock I was talking earlier
about. Another write lock is taken in {{EditLogTailer.doTailEdits()}} further
up the call stack. So adding the loop as you proposed wont release the write
lock.
> Fine-grained locking while consuming journal stream.
> ----------------------------------------------------
>
> Key: HDFS-12978
> URL: https://issues.apache.org/jira/browse/HDFS-12978
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Reporter: Konstantin Shvachko
> Assignee: Konstantin Shvachko
> Priority: Major
> Attachments: HDFS-12978.001.patch, HDFS-12978.002.patch,
> HDFS-12978.003.patch
>
>
> In current implementation SBN consumes the entire segment of transactions
> under a single namesystem lock, which does not allow reads over a long period
> of time until the segment is processed. We should break the lock into fine
> grained chunks. In extreme case each transaction should release the lock once
> it is applied.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
