[
https://issues.apache.org/jira/browse/HDFS-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507032#comment-13507032
]
Todd Lipcon commented on HDFS-4045:
-----------------------------------
Took a look through the patch. I generally think this approach is better than
the first. Here are some more specific comments:
- I agree with Colin above that EditLogServletOutputStream doesn't seems like
it should inherit from EditLogFileOutputStream. There's no need to
double-buffer in this case, so it really reuses very little of its superclass.
Instead, I think you could just operate directly on the ServletOutputStream
itself, pushing the raw bytes directly to the output. This means you also
wouldn't need the new {{writeRaw(FSEditLogOp)}} API -- that API doesn't make a
lot of sense to me either, since it seems kind of like a contradiction in terms.
- Instead of the trick in {{decodeOp()}} where you mark/reset/re-read to get
the raw bytes, did you consider adding a {{readRawOp()}} API? I agree
conceptually with Colin that the EditLogInputStream is a stream of operations,
but if the API were still "give me the raw bytes corresponding to the next op",
that doesn't seem to break the abstraction as much as the previously proposed
API which had {{getInputStream}} in it.
With the above, serveEdits could be implemented something like:
{code}
os = response.getOutputStream();
byte[] op;
while ((op = es.readOpRaw()) != null) {
os.write(op, 0, op.length);
}
{code}
(assuming that the "readOpRaw" includes both the txid and the checksum.
Even if the second suggestion above doesn't work out, I think avoiding the
EditLogServletOutputStream would make the code a little cleaner as well.
Colin, any thoughts on this?
> SecondaryNameNode cannot read from QuorumJournal URI
> ----------------------------------------------------
>
> Key: HDFS-4045
> URL: https://issues.apache.org/jira/browse/HDFS-4045
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: namenode
> Affects Versions: 3.0.0
> Reporter: Vinithra Varadharajan
> Assignee: Andy Isaacson
> Attachments: hdfs-4045-2.txt, hdfs4045-3.txt, hdfs4045-4.txt,
> hdfs-4045.txt
>
>
> If HDFS is set up in basic mode (non-HA) with QuorumJournal, and the
> dfs.namenode.edits.dir is set to only the QuorumJournal URI and no local dir,
> the SecondaryNameNode is unable to do a checkpoint.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
