[
https://issues.apache.org/jira/browse/HDFS-1111?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12894223#action_12894223
]
Rodrigo Schmidt commented on HDFS-1111:
---------------------------------------
One other thought I have about paging is that it might introduce unnecessary
complexity. The number of corrupted files is usually low. If it's too high, it
might be better to run a full fsck. But if there is a strong case for paging,
I'm fine with it.
ClientProtocol vs. jsp: As I mentioned before, I'm opposed to the fsck strategy
because it increases the load on the namenode. I've seen a complete cluster
with thousands of nodes almost go down because there were parallel executions
of fsck running internally to the namenode and they couldn't be stopped.
Besides that, using HTTP to get data from the namenode is just another way to
implement an RPC. The advantage of JSP is that it allows for longer or more
dynamic outputs, which is not the case here. I'm fine moving this specific
topic to another JIRA or discussion list.
> getCorruptFiles() should give some hint that the list is not complete
> ---------------------------------------------------------------------
>
> Key: HDFS-1111
> URL: https://issues.apache.org/jira/browse/HDFS-1111
> Project: Hadoop HDFS
> Issue Type: New Feature
> Reporter: Rodrigo Schmidt
> Assignee: Rodrigo Schmidt
> Attachments: HADFS-1111.0.patch
>
>
> If the list of corruptfiles returned by the namenode doesn't say anything if
> the number of corrupted files is larger than the call output limit (which
> means the list is not complete). There should be a way to hint incompleteness
> to clients.
> A simple hack would be to add an extra entry to the array returned with the
> value null. Clients could interpret this as a sign that there are other
> corrupt files in the system.
> We should also do some rephrasing of the fsck output to make it more
> confident when the list is not complete and less confident when the list is
> known to be incomplete.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
