[
https://issues.apache.org/jira/browse/HBASE-4916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13174365#comment-13174365
]
Phabricator commented on HBASE-4916:
------------------------------------
jdcryans has commented on the revision "HBASE-4916 [jira] LoadTest MR Job".
I tried using it on my 0.92 cluster, had to backport HexStringSplit but no
biggie, but I encountered a few usability issues.
- Why is -zk mandatory? It seems that by default it should use the one
provided in HBaseConfiguration.
- There is no help, not even for basic usage like when I tried to use it the
first time without arguments all I got was this and it doesn't tell me what the
switch is:
jdcryans@sv4r11s38:~/hbase$ ./bin/hbase
org.apache.hadoop.hbase.mapreduce.LoadTest
ZooKeeper quorum must be specified
- The workloads need their own documentation too, currently unless you dig
in the code you won't know how to properly tune them.
- VersionWorkloadGenerator and MixedWorkloadGenerator don't work with their
default values, they both hit this in their reducers and the reason is that
delayNS ends up at 0:
11/12/21 19:45:43 INFO mapred.JobClient: Task Id :
attempt_201112142134_0063_r_000008_2, Status : FAILED
java.lang.IllegalArgumentException
at
java.util.concurrent.ScheduledThreadPoolExecutor.scheduleAtFixedRate(ScheduledThreadPoolExecutor.java:420)
at
org.apache.hadoop.hbase.loadtest.Workload$Executor.start(Workload.java:229)
at
org.apache.hadoop.hbase.mapreduce.LoadTest$Reduce.reduce(LoadTest.java:158)
at
org.apache.hadoop.hbase.mapreduce.LoadTest$Reduce.reduce(LoadTest.java:124)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:176)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:572)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:414)
at org.apache.hadoop.mapred.Child$4.run(Child.java:270)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1154)
at org.apache.hadoop.mapred.Child.main(Child.java:264)
- Finally, and I'm not sure if you can do something about it, some tasks
fail in this way on my cluster:
11/12/21 19:45:31 INFO mapred.JobClient: Task Id :
attempt_201112142134_0063_r_000008_1, Status : FAILED
java.lang.Throwable: Child Error
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:242)
Caused by: java.io.IOException: Task process exit with nonzero status of 1.
at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:229)
And in the stderr:
Error: Exception thrown by the agent : java.rmi.server.ExportException: Port
already in use: 8991; nested exception is:
java.net.BindException: Address already in use
It would be nice if it was bubbled up, but I understand if you don't have
control over it.
REVISION DETAIL
https://reviews.facebook.net/D741
> LoadTest MR Job
> ---------------
>
> Key: HBASE-4916
> URL: https://issues.apache.org/jira/browse/HBASE-4916
> Project: HBase
> Issue Type: Sub-task
> Components: client, regionserver
> Reporter: Nicolas Spiegelberg
> Assignee: Christopher Gist
> Fix For: 0.94.0
>
> Attachments: HBASE-4916.D741.1.patch
>
>
> Add a script to start a streaming map-reduce job where each map tasks runs an
> instance of the load tester for a partition of the key-space. Ensure that the
> load tester takes a parameter indicating the start key for write operations.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
