[
https://issues.apache.org/jira/browse/HBASE-8611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13686195#comment-13686195
]
Nick Dimiduk commented on HBASE-8611:
-------------------------------------
Sorry Valdim, you are correct. I've reviewed your patch more closely and I'm
now +1 for commit...
Except, {{TestTableMapReduceUtil#shoudBeValidMapReduceEvaluation}} fails when I
run locally on Hadoop2. Would you mind taking a look?
Here's the commands I ran and the surefire log output:
{noformat}
$ mvn -Dhadoop.profile=2.0 clean package -DskipTests
$ mvn -Dhadoop.profile=2.0 test
-Dtest=org.apache.hadoop.hbase.mapred.TestDriver,org.apache.hadoop.hbase.mapred.TestGroupingTableMap,org.apache.hadoop.hbase.mapred.TestIdentityTableMap,org.apache.hadoop.hbase.mapred.TestRowCounter,org.apache.hadoop.hbase.mapred.TestSplitTable,org.apache.hadoop.hbase.mapred.TestTableMapReduceUtil
{noformat}
{noformat}
2013-06-17 16:41:03,900 INFO [main] hbase.ResourceChecker(171): after:
mapred.TestTableMapReduceUtil#shoudBeValidMapReduceWithPartitionerEvaluation
Thread=174 (was 174), OpenFileDescriptor=334 (was 336), MaxFileDescriptor=10240
(was 10240), SystemLoadAverage=298 (was 315), ProcessCount=147 (was 147),
AvailableMemoryMB=455 (was 458), ConnectionCount=4 (was 4)
2013-06-17 16:41:03,949 INFO [main] hbase.ResourceChecker(147): before:
mapred.TestTableMapReduceUtil#shoudBeValidMapReduceEvaluation Thread=174,
OpenFileDescriptor=336, MaxFileDescriptor=10240, SystemLoadAverage=298,
ProcessCount=147, AvailableMemoryMB=456, ConnectionCount=4
2013-06-17 16:41:03,950 INFO [main] mapred.TestTableMapReduceUtil(100): before
2013-06-17 16:41:03,950 INFO [main] mapred.TestTableMapReduceUtil(102): before
done
2013-06-17 16:41:03,952 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.zookeeper.ZooKeeper, using jar
/Users/ndimiduk/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar
2013-06-17 16:41:03,953 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class com.google.common.base.Function, using jar
/Users/ndimiduk/.m2/repository/com/google/guava/guava/12.0.1/guava-12.0.1.jar
2013-06-17 16:41:03,954 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class com.google.protobuf.Message, using jar
/Users/ndimiduk/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar
2013-06-17 16:41:03,954 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking
for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:03,954 DEBUG [main] mapreduce.TableMapReduceUtil(692):
org.apache.hadoop.util.JarFinder found.
2013-06-17 16:41:04,041 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.io.ImmutableBytesWritable, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-2053724549545712152.jar
2013-06-17 16:41:04,042 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking
for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:04,042 DEBUG [main] mapreduce.TableMapReduceUtil(692):
org.apache.hadoop.util.JarFinder found.
2013-06-17 16:41:04,262 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.client.Put, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-4313891093512368515.jar
2013-06-17 16:41:04,263 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.io.LongWritable, using jar
/Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-common/2.0.2-alpha/hadoop-common-2.0.2-alpha.jar
2013-06-17 16:41:04,264 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.io.Text, using jar
/Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-common/2.0.2-alpha/hadoop-common-2.0.2-alpha.jar
2013-06-17 16:41:04,265 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.mapred.lib.HashPartitioner, using jar
/Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.0.2-alpha/hadoop-mapreduce-client-core-2.0.2-alpha.jar
2013-06-17 16:41:04,266 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking
for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:04,266 DEBUG [main] mapreduce.TableMapReduceUtil(692):
org.apache.hadoop.util.JarFinder found.
2013-06-17 16:41:05,406 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.mapred.TableInputFormat, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-5943365653915313487.jar
2013-06-17 16:41:05,406 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.mapred.TextOutputFormat, using jar
/Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.0.2-alpha/hadoop-mapreduce-client-core-2.0.2-alpha.jar
2013-06-17 16:41:05,407 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.zookeeper.ZooKeeper, using jar
/Users/ndimiduk/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5/zookeeper-3.4.5.jar
2013-06-17 16:41:05,408 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class com.google.common.base.Function, using jar
/Users/ndimiduk/.m2/repository/com/google/guava/guava/12.0.1/guava-12.0.1.jar
2013-06-17 16:41:05,409 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class com.google.protobuf.Message, using jar
/Users/ndimiduk/.m2/repository/com/google/protobuf/protobuf-java/2.4.1/protobuf-java-2.4.1.jar
2013-06-17 16:41:05,409 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking
for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:05,409 DEBUG [main] mapreduce.TableMapReduceUtil(692):
org.apache.hadoop.util.JarFinder found.
2013-06-17 16:41:05,488 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.io.ImmutableBytesWritable, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-7878386878750133423.jar
2013-06-17 16:41:05,488 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking
for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:05,488 DEBUG [main] mapreduce.TableMapReduceUtil(692):
org.apache.hadoop.util.JarFinder found.
2013-06-17 16:41:05,691 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.client.Put, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-2651672527795445135.jar
2013-06-17 16:41:05,692 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.io.ImmutableBytesWritable, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-7878386878750133423.jar
2013-06-17 16:41:05,692 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.client.Put, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-2651672527795445135.jar
2013-06-17 16:41:05,693 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.mapred.lib.HashPartitioner, using jar
/Users/ndimiduk/.m2/repository/org/apache/hadoop/hadoop-mapreduce-client-core/2.0.2-alpha/hadoop-mapreduce-client-core-2.0.2-alpha.jar
2013-06-17 16:41:05,694 DEBUG [main] mapreduce.TableMapReduceUtil(690): Looking
for org.apache.hadoop.util.JarFinder.
2013-06-17 16:41:05,694 DEBUG [main] mapreduce.TableMapReduceUtil(692):
org.apache.hadoop.util.JarFinder found.
2013-06-17 16:41:06,864 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.mapred.TableInputFormat, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-1149375339751659803.jar
2013-06-17 16:41:06,864 DEBUG [main] mapreduce.TableMapReduceUtil(614): For
class org.apache.hadoop.hbase.mapred.TableOutputFormat, using jar
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-1149375339751659803.jar
2013-06-17 16:41:06,895 WARN [main] mapreduce.JobSubmitter(138): Hadoop
command-line option parsing not performed. Implement the Tool interface and
execute your application with ToolRunner to remedy this.
2013-06-17 16:41:06,950 WARN [main] mapreduce.JobSubmitter(247): No job jar
file set. User classes may not be found. See Job or Job#setJar(String).
2013-06-17 16:41:06,953 ERROR [main] security.UserGroupInformation(1370):
PriviledgedActionException as:ndimiduk (auth:SIMPLE)
cause:java.io.FileNotFoundException: File does not exist:
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-7878386878750133423.jar
2013-06-17 16:41:06,953 ERROR [main] security.UserGroupInformation(1370):
PriviledgedActionException as:ndimiduk (auth:SIMPLE)
cause:java.io.FileNotFoundException: File does not exist:
/Users/ndimiduk/repos/hbase/hbase-server/target/test-dir/hadoop-7878386878750133423.jar
{noformat}
Interestingly, {{hadoop-7878386878750133423.jar}} does exist on my filesystem
after the test execution terminates.
> Improve test coverage in pkg org.apache.hadoop.hbase.mapred
> -----------------------------------------------------------
>
> Key: HBASE-8611
> URL: https://issues.apache.org/jira/browse/HBASE-8611
> Project: HBase
> Issue Type: Test
> Reporter: Vadim Bondarev
> Assignee: Vadim Bondarev
> Attachments: HBASE-8611-0.94-a.patch, HBASE-8611-0.94-c.patch,
> HBASE-8611-trunk-a.patch, HBASE-8611-trunk-b.patch, HBASE-8611-trunk-c.patch
>
>
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
