[
https://issues.apache.org/jira/browse/HDFS-587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12779589#action_12779589
]
Erik Steffl commented on HDFS-587:
----------------------------------
Patches for 0.22 added:
https://issues.apache.org/jira/secure/attachment/12425362/jira.HDFS-587.mapreduce.branch-0.22.patch
https://issues.apache.org/jira/secure/attachment/12425361/jira.HDFS-587.hdfs.branch-0.22.patch
Most of the Tests were moved to mapreduce, except of TestLoadGenerator.java
which remains in hdfs, that's why there are two patches for 0.22.
Patches were done against:
URL: http://svn.apache.org/repos/asf/hadoop/mapreduce/trunk
Repository Root: http://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: 880959
Node Kind: directory
Schedule: normal
Last Changed Author: omalley
Last Changed Rev: 836063
Last Changed Date: 2009-11-13 15:58:21 -0800 (Fri, 13 Nov 2009)
URL: http://svn.apache.org/repos/asf/hadoop/hdfs/trunk
Repository Root: http://svn.apache.org/repos/asf
Repository UUID: 13f79535-47bb-0310-9956-ffa450edef68
Revision: 881058
Node Kind: directory
Schedule: normal
Last Changed Author: cos
Last Changed Rev: 881017
Last Changed Date: 2009-11-16 14:38:46 -0800 (Mon, 16 Nov 2009)
Running tests for 0.22:
./hadoop/bin/hadoop jar ./hadoop/hadoop-mapred-test-0.22.0-dev.jar TestDFSIO
-Dmapred.job.queue.name=anotherQueue -write
./hadoop/bin/hadoop jar ./hadoop/hadoop-mapred-test-0.22.0-dev.jar DFSCIOTest
-Dmapred.job.queue.name=anotherQueue -write
./hadoop/bin/hadoop jar ./hadoop/hadoop-mapred-test-0.22.0-dev.jar
DistributedFSCheck -Dmapred.job.queue.name=anotherQueue -write
mkdir /tmp/loadGenerator
./hadoop/bin/hadoop org.apache.hadoop.fs.loadGenerator.StructureGenerator
-maxDepth 3 -minWidth 1 -maxWidth 3 -numOfFiles 100 -avgFileSize 2 -outDir
/tmp/loadGenerator -seed 1000
./hadoop/bin/hadoop org.apache.hadoop.fs.loadGenerator.DataGenerator -inDir
/tmp/loadGenerator -root /loadGenerator
./hadoop/bin/hadoop org.apache.hadoop.fs.loadGenerator.LoadGenerator
-readProbability 0.5 -writeProbability 0.5 -root /loadGenerator
-maxDelayBetweenOps 0 -numOfThreads 10 -elapsedTime 15 -startTime 0 -seed 40
./hadoop/bin/hadoop jar ./hadoop/hadoop-mapred-test-0.22.0-dev.jar nnbench
-Dmapred.job.queue.name=anotherQueue -operation create_write
./hadoop/bin/hadoop jar ./hadoop/hadoop-mapred-test-0.22.0-dev.jar
NNBenchWithoutMR -Dmapred.job.queue.name=anotherQueue -operation createWrite
-bytesPerBlock 512 -numFiles 2 -blocksPerFile 2 -baseDir /NNBenchWithoutMR
Note: DFSCIOTest error: ot complains that
$HADOOP_HOME/hadoop/libhdfs/libhdfs.so.1 does not exist, it's a bit weird, the
test needs libhdfs.so.1, hdfs_read and hdfs_write in $HADOOP_HOME/libhdfs so
you might want to touch those files (it copies them to /tmp/DFSCIOTest and uses
them as data files, it does not load libhdfs.so.1 as a library).
You need to add queue (name anotherQueue) to
$HADOOP_HOME/conf/mapred-queues.xml (OR $HADOOP_CONF_DIR/mapred-queues.xml if
you have a separate config dir)
> Test programs support only default queue.
> -----------------------------------------
>
> Key: HDFS-587
> URL: https://issues.apache.org/jira/browse/HDFS-587
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Reporter: Sreekanth Ramakrishnan
> Assignee: Erik Steffl
> Attachments: jira.HDFS-587.branch-0.20-internal.1.patch,
> jira.HDFS-587.branch-0.20-internal.patch, jira.HDFS-587.branch-0.21.patch,
> jira.HDFS-587.hdfs.branch-0.22.patch,
> jira.HDFS-587.mapreduce.branch-0.22.patch
>
>
> Following test programs always run on "default" queue even when other queues
> are passed as job parameter.
> DFSCIOTest
> DistributedFSCheck
> TestDFSIO
> Filebench
> Loadgen
> Nnbench
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
