Re: How to configure RandomWriter to generate less amount of data

[Heshan Lin]

Thanks a bunch, Amar! It works fine.

/Heshan


On Jul 1, 2008, at 1:44 AM, Amar Kamat wrote:

Heshan Lin wrote:

Hi,

I'm trying to configure RandomWriter to generate less data than does the default configuration.

bin/hadoop jar hadoop-*-examples.jar randomwriter - Dtest.randomwrite.bytes_per_map=<value> - Dtest.randomwrite.total_bytes=<value> - Dtest.randomwriter.maps_per_host=<value> <input-filename> The number of maps that will be spawned in this case will be total_bytes/bytes_per_map. Other parameters are test.randomwrite.min_key (size in bytes), test.randomwrite.max_key (size in bytes), test.randomwrite.min_value (size in bytes) and test.randomwrite.max_value (size in bytes).

Amar

I created a job configuration file job.xml and added in variables given at http://wiki.apache.org/hadoop/RandomWriter. Tried a couple of ways of running the program below, but configurations in job.xml were not taken by RandomWriter.

1) bin/hadoop jar hadoop-*-examples.jar randomwriter rand job.xml

2) bin/hadoop jar hadoop-*-examples.jar randomwriter rand --conf job.xml 3) bin/hadoop jar --conf job.xml hadoop-*-examples.jar randomwriter rand

Passing property values via the -D option didn't seem to work either. Can anybody advise on how to use the job configuration file properly?

Thanks,
Heshan