[
https://issues.apache.org/jira/browse/AVRO-1567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14110873#comment-14110873
]
Tony Reix commented on AVRO-1567:
---------------------------------
There is another issue when using IBM JVM, preventing 3 remaining tests to pass.
In some places, the pom.xml file ony handles hadoop version 1, like in
lang/java/trevni/avro/pom.xml :
./lang/java/trevni/avro/pom.xml: <groupId>org.apache.hadoop</groupId>
./lang/java/trevni/avro/pom.xml: <artifactId>hadoop-core</artifactId>
./lang/java/trevni/avro/pom.xml: <version>${hadoop1.version}</version>
That means that using "-Phadoop2 -Dhadoop.version=2" with mvn when testing Java
language does not imply to use Hadoop2 in all places.
So, in some places, though I want to compile/test with Hadoop2, it makes use of
Hadoop1. Since Hadoop1 is not installed in my environment, Maven downloads it
from the Web and put in my Maven .m2 repository some old version of Hadoop
(version 0.20.205.0 for me, since this version appears in lang/java/pom.xml for
version 1.7.4 of Avro : <hadoop1.version>0.20.205.0</hadoop1.version>).
When using Oracle or OpenJDK JVMs, there is no issue, since they have
"com.sun...." classes. But, when using IBM JVM, that creates an issue.
So, it would be useful to use the IBM JVM in order to locate the remaining
places (pom.xml files) where version 2 of Hadoop is not handled, implying
download/use of old Hadoop code.
I have a simple work-around for fixing this issue when using IBM JVM:
<hadoop1.version>2.4.1</hadoop1.version> in lang/java/pom.xml .
However, it cannot be used as a patch ! ;)
So, it would be useful to fix the remaining places in Avro where version 2 of
Hadoop is not handled.
> Avro java tools tests fail with IBM JVM
> ---------------------------------------
>
> Key: AVRO-1567
> URL: https://issues.apache.org/jira/browse/AVRO-1567
> Project: Avro
> Issue Type: Bug
> Components: java
> Affects Versions: 1.7.4, 1.7.7
> Environment: RHEL 6.5 on x86_64
> IBM JVM 7.1.1.1
> HADOOP 2.4.1
> Reporter: Tony Reix
> Priority: Blocker
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> When using IBM JVM, compared to Oracle JVM, 25 of the Avro Tools tests fail.
> This is due to Avro using Hadoop which uses class:
> org/apache/hadoop/security/UserGroupInformation.java
> which makes use of:
> com.sun.security.auth.module.UnixLoginModule
> which does not exist in IBM JVM.
> Instead there is the class:
> com.ibm.security.auth.module.LinuxLoginModule
> that can be used in UserGroupInformation.java if the JVM is IBM.
> With a IBM-JVM patched version of Hadoop that takes care of the kind of JVM,
> these 25 Avro Java Tools tests still fail because the pom.xml file of:
> lang/java/tools/
> says unconditionnaly (starting line 146 in Avro 1.7.7) :
> <dependency>
> <groupId>org.apache.hadoop</groupId>
> <artifactId>hadoop-core</artifactId>
> Using:
> mvn -Phadoop2 -Dhadoop.version=2 test
> is of no help.
> In fact, hadoop-core exists only for old Hadoop versions (here version
> 0.20.205.0 is used by Avro), and not for Hadoop 2.4.1 .
> Replacing hadoop-core by hadoop-client in lang/java/tools/pom.xml file does
> fix the issue, as a work-around.
> However, a more rigorous solution is required, like it is done in
> lang/java/mapred/pom.xm , where hadoop-core is associated with hadoop1 and
> hadoop-client is associated with hadoop2 .
> I'm not an expert of Maven/pom.xml, and the pom.xml file of tools contains
> <exclusions> tags I have no idea. So, I'm not sure I can provide a correct
> patch.
> I guess that a Maven/pom.xml expert should be able to fix this in some
> minutes, plus testing.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
