[
https://issues.apache.org/jira/browse/HDFS-13826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580595#comment-16580595
]
Wei-Chiu Chuang commented on HDFS-13826:
----------------------------------------
Hey [~tlipcon] sounds like a good proposal.
What would be the scope of this change? I imagine you just want to return a
fake LocatedBlocks in FSNamesystem#getBlockLocations()? Or do you intend to
have a larger change? (Just FYI LinkedIn's Dynamometer tool can generate fake
blocks on simulated DataNodes)
> Add a hidden configuration for NameNode to generate fake block locations
> ------------------------------------------------------------------------
>
> Key: HDFS-13826
> URL: https://issues.apache.org/jira/browse/HDFS-13826
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namenode
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Priority: Minor
>
> In doing testing and benchmarking of the NameNode and dependent systems, it's
> often useful to be able to use an fsimage provided by some production system
> in a controlled environment without actually having access to any of the
> data. For example, while doing some recent work on Apache Impala I was trying
> to optimize the transmission and storage of block locations and tokens and
> measure the results based on metadata from a production user. In order to
> achieve this, it would be useful for the NN to expose a developer-only
> (undocumented) configuration to generate fake block locations and return them
> to callers. The "fake" locations should be randomly distributed across a
> fixed set of fake datanodes.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
