[
https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Thomas updated HDFS-6634:
-------------------------------
Attachment: HDFS-6634.7.patch
Made the change Colin suggested and also removed the logic to choose the
in-progress segment with the highest {{lastWrittenEpoch}} since it was shown to
be unnecessary (will update HDFS-6777 with a more complete discussion of this).
Also, instead of eliminating calls to {{validateLog}}, I refactored
{{IPCLoggerChannel}} to use a separate executor for the {{getEditLogManifest}}
calls -- previously there was only a single executor, and it was
single-threaded, so the {{getEditLogManifest}} calls (made lengthy by the
expensive work in {{validateLog}}) were blocking writes of edits to the
JournalNodes, reducing NameNode edit throughput. So now the inotify logic
should have little impact on the critical path for NameNode edit logging.
Eliminating the calls to {{validateLog}} would still be valuable, but I think
some thinking is required there, so I'll file a follow-up JIRA.
> inotify in HDFS
> ---------------
>
> Key: HDFS-6634
> URL: https://issues.apache.org/jira/browse/HDFS-6634
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs-client, namenode, qjm
> Reporter: James Thomas
> Assignee: James Thomas
> Attachments: HDFS-6634.2.patch, HDFS-6634.3.patch, HDFS-6634.4.patch,
> HDFS-6634.5.patch, HDFS-6634.6.patch, HDFS-6634.7.patch, HDFS-6634.patch,
> inotify-design.2.pdf, inotify-design.3.pdf, inotify-design.4.pdf,
> inotify-design.pdf, inotify-intro.2.pdf, inotify-intro.pdf
>
>
> Design a mechanism for applications like search engines to access the HDFS
> edit stream.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
