[ https://issues.apache.org/jira/browse/HADOOP-1989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12540526 ]
Doug Cutting commented on HADOOP-1989: -------------------------------------- > For example startDataNode refers to the simulated impl if it is starting the > simulated fsdataset [ ... ] This could be handled other ways, that do not require test code in trunk, no? For example, you could replace 'new SimulatedFoo' with (FooInterface)ReflectionUtils.newInstance("org.apache.hadoop.dfs.SimulatedFoo"), so that there's no compile-time dependency. This could be inside a factory method, so that it's only done once per simulated class. > Add support for simulated Data Nodes - helpful for testing and performance > benchmarking of the Name Node without having a large cluster > ---------------------------------------------------------------------------------------------------------------------------------------- > > Key: HADOOP-1989 > URL: https://issues.apache.org/jira/browse/HADOOP-1989 > Project: Hadoop > Issue Type: Improvement > Components: dfs > Reporter: Sanjay Radia > Priority: Minor > Attachments: SimulatedStoragePatchSubmit.txt, > SimulatedStoragePatchSubmit5.txt > > > Proposal is to add an implementation for a Simulated Data Node. > This will > - allow one to test certain parts of the system (especially the Name Node, > protocols) much more easily and efficiently. > - allow one to run performance benchmarks on the Name node without having a > large cluster. > - Inject faults for testing (e.g. one can add random faults based > probability parameters). > The idea is that the Simulated Data Node will > - discard any data written to blocks (but remember the blocks and their > sizes) > - generate fixed data on the fly when blocks are read (e.g. block is fixed > set of bytes or repeated sequence of strings). > The Simulated Data Node can also be used for fault injection. > The data node can be parameterized with probabilities that allow one to > control: > - Delays on reads and writes, creates, etc > - IO Exceptions > - Loss of blocks > - Failures -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.