Jarek Jarcec Cecho created SQOOP-1685:
-----------------------------------------
Summary: HCatalog integration is not working on JDK8
Key: SQOOP-1685
URL: https://issues.apache.org/jira/browse/SQOOP-1685
Project: Sqoop
Issue Type: Bug
Affects Versions: 1.4.5
Reporter: Jarek Jarcec Cecho
Assignee: Jarek Jarcec Cecho
Fix For: 1.4.6
I was looking into JDK8 compatibility in Sqoop 1 and I found out that HCatalog
tests are broken, failing on exceptions such as this one:
{code}
14/11/05 16:29:06 ERROR tool.ImportTool: Encountered IOException running import
job: java.io.IOException: Projected column id not in list of columns from
database
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.initDBColumnInfo(SqoopHCatUtilities.java:483)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:314)
at
org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:753)
{code}
Looking deeper into our HCatalog code, we do have class
{{[LCKeyMap|https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java#L169]}}
that is suppose to store all it's keys in lower case form and provide case
independent lookup. Indeed we're overriding methods {{put}} and {{get}} and
lowercasing the key parameter to achieve desired behavior.
Now to the actual problem. We are in the code calling method
{{[putAll|https://github.com/apache/sqoop/blob/trunk/src/java/org/apache/sqoop/mapreduce/hcat/SqoopHCatUtilities.java#L483]}}
to bulk insert values from another Map. This worked in JDK7 as the
[underlaying
implementation|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/7-b147/java/util/HashMap.java#HashMap.putAll%28java.util.Map%29]
of {{putAll}} method is calling {{put}} method that we're overriding. It seems
that this [has changed in
JDK8|http://grepcode.com/file/repository.grepcode.com/java/root/jdk/openjdk/8-b132/java/util/HashMap.java#HashMap.putAll%28java.util.Map%29]
where the {{putAll}} method is no longer calling {{put}} and hence when we
call {{putAll}} all the keys are stored with their original case and subsequent
lookup fails
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
