[
https://issues.apache.org/jira/browse/HDFS-13381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16459263#comment-16459263
]
Chris Douglas commented on HDFS-13381:
--------------------------------------
bq. We need one unified implementation that uses minimal external calls that
can be abstracted in the context to either make rpc or fsn calls
Do you have a particular refactoring in mind? In this case, the separate
implementations batch their requests differently, because the internal
implementation uses {{FSTreeTraverser}}. Looking at the patch, one could make
the two implementations more similar, but not significantly more modular or
easier to maintain. IIRC, each context implements "queue up the next batch of
work".
Your point on maintenance is well taken, but {{FileCollector}} looks like a
vestigial abstraction. It's not actually pluggable; the internal/external
contexts put their scan logic into a class, to match the EC pattern. We are
only talking about two implementations of an explicitly internal (i.e.,
{{Unstable}}) API...
Is the problem with the {{FSTreeTraverser}} pattern, generally? You've stated
the goal clearly, but if the traversal is generalized then the internal context
will do no batching under the lock. Did anyone- including the EC folks- publish
any benchmarks/testing that justify this abstraction? If the impl benefits from
it, then as a specialization it has merit, but it should be significant. How
should we test that?
> [SPS]: Use DFSUtilClient#makePathFromFileId() to prepare satisfier file path
> ----------------------------------------------------------------------------
>
> Key: HDFS-13381
> URL: https://issues.apache.org/jira/browse/HDFS-13381
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Reporter: Rakesh R
> Assignee: Rakesh R
> Priority: Major
> Attachments: HDFS-13381-HDFS-10285-00.patch,
> HDFS-13381-HDFS-10285-01.patch
>
>
> This Jira task will address the following comments:
> # Use DFSUtilClient::makePathFromFileId, instead of generics(one for string
> path and another for inodeId) like today.
> # Only the context impl differs for external/internal sps. Here, it can
> simply move FileCollector and BlockMoveTaskHandler to Context interface.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
