[
https://issues.apache.org/jira/browse/HDFS-6634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14093481#comment-14093481
]
Andrew Wang commented on HDFS-6634:
-----------------------------------
Thanks for revving James, few more comments. I haven't dug into the QJM stuff
in detail yet.
Misc/meta:
* We have some TODOs sprinkled around. I like to turn these into JIRAs, and
then reference the JIRA in the comment. JIRA is a much better tracking tool
than TODOs :)
* Comment in HdfsAdmin has "TODO complete this"?
* hdfs-default.xml, would be good if you could provide guidance about how to
size this appropriately. Basically, why and how was 1000 chosen?
* Maybe we leave the InterfaceStability at Unstable for now just in case, we
might want to change things as future work flows in.
* CNPServerSideTranslatorPB, unrelated whitespace changes
* EditsDoubleBuffer, does that visibility need to be changed to public?
* NNRpcServer, unnecessary RejectedExecutionException import?
* The PB definition for getCurrentTxid is a uint64, but I see that we return a
-1 if the edit log is not open. Seems like it should be an int64. Same issue
for lastWriterEpoch in QJournalProtocol.
* QuorumJournalManager, unused import to RemoteEditLogManifest
DFSInotifyEventInputStream
* You could add a "_MS" suffix to the two constants, self-documents that
they're in milliseconds. I think one of these also isn't being used.
* Some javadoc lines are longer than 80 chars
* Can we add a {{@link}} to the best-documented version of {{next}} from the
other versions of {{next}}?
* Using LinkedBlockingQueue as a reference, it'd be nice to provide blocking
calls as well for convenience. You could use the same terminology of poll and
take from LinkedBlockingQueue if you like.
* The {{namenode}} RPC proxy has its own timeout and retry policy built in, so
the user could very well end up waiting for minutes regardless of what they
pass in. I think if we used Futures and cancellation (or some other method of
interrupting), it would be more timely. You could also create a new RPC proxy
with a more fitting timeout/retry policy.
* Also I don't think doing additional backoff is necessary, since the RPC proxy
already has timeout/retry policy. I think since our timeouts are quite
conservative (60s?), backoff isn't as important.
Tests:
* some lines longer than 80 chars
* Nit: "Uncomitted" should have two m's
* Wish I had told you about this before, but there's DFSTestUtil#runOperations
which runs through a bunch of different operations. Is this reusable in
testBasic? It'd be nice, since TestOfflineEditsViewer checks that all opcodes
are generated by runOperations, so we'll only need to update this function when
we add more opcodes in the future. Or we could do the same assertion about all
the opcodes being present and tested.
* Can we add conservative timeouts on the new tests, e.g. 120000 ?
> inotify in HDFS
> ---------------
>
> Key: HDFS-6634
> URL: https://issues.apache.org/jira/browse/HDFS-6634
> Project: Hadoop HDFS
> Issue Type: New Feature
> Components: hdfs-client, namenode, qjm
> Reporter: James Thomas
> Assignee: James Thomas
> Attachments: HDFS-6634.2.patch, HDFS-6634.3.patch, HDFS-6634.patch,
> inotify-design.2.pdf, inotify-design.pdf, inotify-intro.2.pdf,
> inotify-intro.pdf
>
>
> Design a mechanism for applications like search engines to access the HDFS
> edit stream.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
