[
https://issues.apache.org/jira/browse/HADOOP-5188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luca Telloli updated HADOOP-5188:
---------------------------------
Attachment: HADOOP-5188.patch
This patch adds the following features:
- supports for URI for property dfs.name.edits.dir (HADOOP-5832)
- abstraction of logging systems
The abstraction might be still incomplete, but I think it's in a good shape.
To achieve the first point (URI in dfs.name.edits.dir), the patch introduces a
new abstraction called LoggingDevice.
The trunk implementation currently assumes that every attribute inside the
property dfs.name.edits.dir is a StorageDirectory, which is false if other
types of logging are specified. LoggingDevice should help with it;
additionally, a Logging device would incorporate all needed parameters for its
instantiation by reading them from the initialization URI.
Ideally, a new logging device should:
- extend class LoggingDevice
- implement interfaces EditLogInput/OutputStream
Additionally, in the current implementation there is one data structure that
keeps the set of edit streams (FSEditLog.editStreams) and one structure that
keeps the list of StorageDirectories. The two structures are maintained in sync
whenever a edit stream fails (the corresponding storage directory is removed)
or a storage directory fails (the corresponding edit stream is removed).
My hope is that, by using logging devices, we could keep only one data
structure.
Finally:
- the patch passes all NameNode unit tests
- I'm starting the submit patch process to get more feedback from users
> Modifications to enable multiple types of logging
> --------------------------------------------------
>
> Key: HADOOP-5188
> URL: https://issues.apache.org/jira/browse/HADOOP-5188
> Project: Hadoop Core
> Issue Type: Improvement
> Components: dfs
> Affects Versions: 0.20.0
> Reporter: Luca Telloli
> Fix For: 0.21.0
>
> Attachments: HADOOP-5188.patch, HADOOP-5188.patch, HADOOP-5188.patch,
> HADOOP-5188.patch, HADOOP-5188.pdf
>
>
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
