[
https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Thomas updated HDFS-6634:
-------------------------------
Attachment: HDFS-6634.6.patch
Responded to most of Colin's comments and made some changes to improve the
impact of inotify load on NameNode throughput (introduced finer-grained locking
in FSEditLog and added logic to prefer reading from local journals if they are
available).
[~cmccabe], regarding HDFS-6777, considering I only have a week left in my
internship, I figured it would be easier to just do everything together and try
to push through one JIRA :)
Regarding the filtering out of in-progress segments if there are finalized
segments, it's not possible for the client to easily do this because everything
is packaged into a RedundantEditLogInputStream by the time it reaches the
client. The behavior I've added is good in general because we know (by the
invariants maintained by the JournalManagers) that the finalized segment
contains committed transactions, whereas the in-progress segment can contain
uncommitted transactions written by a previous active NameNode, so we
definitely don't want to read from it. Previously no one was really reading
in-progress segments, so this is fairly uncharted territory.
> inotify in HDFS
> ---------------
>
> Key: HDFS-6634
> URL: https://issues.apache.org/jira/browse/HDFS-6634
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs-client, namenode, qjm
> Reporter: James Thomas
> Assignee: James Thomas
> Attachments: HDFS-6634.2.patch, HDFS-6634.3.patch, HDFS-6634.4.patch,
> HDFS-6634.5.patch, HDFS-6634.6.patch, HDFS-6634.patch, inotify-design.2.pdf,
> inotify-design.3.pdf, inotify-design.4.pdf, inotify-design.pdf,
> inotify-intro.2.pdf, inotify-intro.pdf
>
>
> Design a mechanism for applications like search engines to access the HDFS
> edit stream.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
