Christopher Tubbs commented on ACCUMULO-4611:

In hindsight, I should have broken this into subtasks.

The first half is done... that is to remove commons-configuration (1) from the 
public API in the 2.0 branch (with deprecation in 1.9.0). The second half was 
to migrate the 2.0 branch to commons-configuration2. Since the subject of this 
issue was to migrate, I was leaving this open until that part is completed, but 
I realize now it probably makes more sense to create a sub-task or follow-on 
issue and close this one.

I started the second half work, but ran into an obstacle migrating the Metrics 
configuration file reloading watcher code (lack of clear docs for file 
reloading, and unclear intent in our original code), then got side-tracked by 
other work stuff (and, TBH, family stuff too).

Whether or not it is important for Hadoop 3 compatibility depends on whether 
Hadoop is shipping with shaded jars or not, and whether or not [~elserj]'s 
experiment with Maven profiles in https://github.com/apache/accumulo/pull/332 
is successful at supporting Hadoop 3, and I haven't had any time to follow up 
with any of that.

If it turns out that the migration to commons-configuration2 is essential for 
Hadoop 3 compatibility, then I can re-prioritize to get this done, so it won't 
hold up any future 2.0 releases... but right now, I want to focus on getting 
1.7.4 and 1.9.0 out the door and getting the ITs fixed in the master branch so 
CI is useful again. In the meantime, as I said, I'll create a follow-on issue 
for the second half of the migration to commons-configuration2.

> Commons Configuration either needs bumped or needs to be provided for Hadoop 3
> ------------------------------------------------------------------------------
>                 Key: ACCUMULO-4611
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-4611
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.7.2, 1.8.1
>         Environment: CentOS 7
> Accumulo 1.7.x, 1.8.x
> Hadoop 3.0.0-alpha 2
> Zookeeper 3.4.9
>            Reporter: Michael Hogue
>            Assignee: Christopher Tubbs
>            Priority: Critical
>              Labels: pull-request-available
>             Fix For: 1.9.0, 2.0.0
>          Time Spent: 2h 20m
>  Remaining Estimate: 0h
> I was investigating running either Accumulo 1.7.x or 1.8.x on Hadoop 
> 3.0.0-alpha 2 and ran into a couple of issues. Since Accumulo assumes some 
> dependencies will be provided by Hadoop (per ACCUMULO-1244), if those 
> dependencies change then Accumulo might also need to change.  
> HADOOP-13660 bumped {{commons-configuration}} from 1.6 -> 2.1 for Hadoop 
> 3.0.0 and so NoClassDefFoundErrors are thrown per the below stack trace on 
> {{accumulo init}}:
> {noformat}
> [mike@localhost cloud]$ accumulo init
> 2017-03-22 15:38:34,106 [start.Main] ERROR: Uncaught exception
> java.util.ServiceConfigurationError: 
> org.apache.accumulo.start.spi.KeywordExecutable: Provider 
> org.apache.accumulo.proxy.Proxy could not be instantiated
>     at java.util.ServiceLoader.fail(ServiceLoader.java:232)
>     at java.util.ServiceLoader.access$100(ServiceLoader.java:185)
>     at 
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:384)
>     at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
>     at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
>     at org.apache.accumulo.start.Main.checkDuplicates(Main.java:223)
>     at org.apache.accumulo.start.Main.getExecutables(Main.java:215)
>     at org.apache.accumulo.start.Main.main(Main.java:78)
> Caused by: java.lang.NoClassDefFoundError: 
> org/apache/commons/configuration/Configuration
>     at java.lang.Class.getDeclaredConstructors0(Native Method)
>     at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
>     at java.lang.Class.getConstructor0(Class.java:3075)
>     at java.lang.Class.newInstance(Class.java:412)
>     at 
> java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
>     ... 5 more
> Caused by: java.lang.ClassNotFoundException: 
> org.apache.commons.configuration.Configuration
>     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>     at 
> org.apache.accumulo.start.classloader.AccumuloClassLoader$2.loadClass(AccumuloClassLoader.java:284)
>     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>     ... 10 more
> {noformat}
> I worked around this by dropping the {{commons-configuration}} 1.6 jar on 
> Accumulo's classpath, but this should either be provided by Accumulo now or 
> Accumulo should be bumped to {{commons-configuration2}} as well. Note that 
> the latter change would cause problems for Hadoop 2 installations.
> There was actually one more thing that needed added to the accumulo-site.xml 
> in order to get Accumulo to run in Hadoop 3:
> {noformat}
> $HADOOP_PREFIX/share/hadoop/client/[^.].*.jar
> {noformat}
> The reason for this is because {{Hdfs.java}} moved from the hadoop-hdfs jar 
> to the hadoop-client-api jar.

This message was sent by Atlassian JIRA

Reply via email to