[ 
https://issues.apache.org/jira/browse/ACCUMULO-4467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15511021#comment-15511021
 ] 

Hadoop QA commented on ACCUMULO-4467:
-------------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s {color} 
| {color:red} ACCUMULO-4467 does not apply to 1.7. Rebase required? Wrong 
Branch? See http://accumulo.apache.org/git.html#contributors for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12829670/ACCUMULO-4467-1.7.v1.patch
 |
| JIRA Issue | ACCUMULO-4467 |
| Console output | 
https://builds.apache.org/job/PreCommit-ACCUMULO-Build/43/console |
| Powered by | Apache Yetus 0.3.0   http://yetus.apache.org |


This message was automatically generated.



> Random Walk broken because of unmet dependency on commons-math
> --------------------------------------------------------------
>
>                 Key: ACCUMULO-4467
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4467
>             Project: Accumulo
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 1.6.6, 1.7.2
>            Reporter: Dima Spivak
>            Assignee: Sean Busbey
>             Fix For: 1.7.3, 1.8.1, 2.0.0
>
>         Attachments: ACCUMULO-4467-1.6.v1.patch, ACCUMULO-4467-1.7.v1.patch
>
>
> When trying to run the Random Walk with {{LongEach.xml}} module, I hit a 
> failure once we reach the {{Shard.xml}} step:
> {code}
> 16 19:52:05,146 [randomwalk.Framework] ERROR: Error during random walk
> java.lang.Exception: Error running node Shard.xml
>       at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:346)
>       at org.apache.accumulo.test.randomwalk.Framework.run(Framework.java:59)
>       at 
> org.apache.accumulo.test.randomwalk.Framework.main(Framework.java:119)
>       at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>       at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>       at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>       at java.lang.reflect.Method.invoke(Method.java:606)
>       at org.apache.accumulo.start.Main$2.run(Main.java:157)
>       at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.Exception: Error running node shard.BulkInsert
>       at org.apache.accumulo.test.randomwalk.Module.visit(Module.java:346)
>       at org.apache.accumulo.test.randomwalk.Module$1.call(Module.java:283)
>       at org.apache.accumulo.test.randomwalk.Module$1.call(Module.java:278)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>       at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>       at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>       at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>       at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>       at 
> org.apache.accumulo.fate.util.LoggingRunnable.run(LoggingRunnable.java:35)
>       ... 1 more
> Caused by: java.lang.Exception: Failed to run map/red verify
>       at 
> org.apache.accumulo.test.randomwalk.shard.BulkInsert.sort(BulkInsert.java:186)
>       at 
> org.apache.accumulo.test.randomwalk.shard.BulkInsert.visit(BulkInsert.java:132)
>       ... 9 more
> {code}
> Digging into YARN to see why the MR job became unhappy, I see the following:
> {code}
> Error: java.lang.ClassNotFoundException: 
> org.apache.commons.math.stat.descriptive.SummaryStatistics at 
> java.net.URLClassLoader$1.run(URLClassLoader.java:366) at 
> java.net.URLClassLoader$1.run(URLClassLoader.java:355) at 
> java.security.AccessController.doPrivileged(Native Method) at 
> java.net.URLClassLoader.findClass(URLClassLoader.java:354) at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:425) at 
> sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308) at 
> java.lang.ClassLoader.loadClass(ClassLoader.java:358) at 
> org.apache.accumulo.core.file.rfile.RFile$Writer.<init>(RFile.java:310) at 
> org.apache.accumulo.core.file.rfile.RFileOperations.openWriter(RFileOperations.java:127)
>  at 
> org.apache.accumulo.core.file.rfile.RFileOperations.openWriter(RFileOperations.java:106)
>  at 
> org.apache.accumulo.core.file.DispatchingFileFactory.openWriter(DispatchingFileFactory.java:78)
>  at 
> org.apache.accumulo.core.client.mapreduce.AccumuloFileOutputFormat$1.write(AccumuloFileOutputFormat.java:172)
>  at 
> org.apache.accumulo.core.client.mapreduce.AccumuloFileOutputFormat$1.write(AccumuloFileOutputFormat.java:152)
>  at 
> org.apache.hadoop.mapred.ReduceTask$NewTrackingRecordWriter.write(ReduceTask.java:558)
>  at 
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
>  at 
> org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.write(WrappedReducer.java:105)
>  at org.apache.hadoop.mapreduce.Reducer.reduce(Reducer.java:150) at 
> org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171) at 
> org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) at 
> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) at 
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163) at 
> java.security.AccessController.doPrivileged(Native Method) at 
> javax.security.auth.Subject.doAs(Subject.java:415) at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
>  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) 
> {code}
> It looks like [this 
> commit|https://github.com/apache/accumulo/commit/63a8a5d7eec7b0ab6dce107d382143562e21b527]
>  introduced a dependency on the {{commons-math}} JAR at runtime (in the 
> RFiles Writer class), but tests weren't updated to ensure that the same 
> dependency would be put onto the classpath of MR jobs submitted by Random 
> Walk.
> Props to [~busbey] for helping to figure out the root cause here. On a 
> separate note, we may want to start running this test before releases, as it 
> appears this regression also snuck into 1.8.0 and at least one 1.6 release 
> (though, since I don't have any easy way to test this against non-1.7.2 
> cluster, I'm limiting the affects versions to what I've confirmed myself). 
> Ping [~kturner], who might know the simplest way to fix this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to