[
https://issues.apache.org/jira/browse/SQOOP-1533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shigeki Hirose updated SQOOP-1533:
----------------------------------
Summary: Java 8 : sqoop import --create-hcatalog-table for mysql causes
NullPointerException when column name contains upper case char (was: sqoop
import --create-hcatalog-table for mysql causes NullPointerException when
column name contains upper case char)
> Java 8 : sqoop import --create-hcatalog-table for mysql causes
> NullPointerException when column name contains upper case char
> -----------------------------------------------------------------------------------------------------------------------------
>
> Key: SQOOP-1533
> URL: https://issues.apache.org/jira/browse/SQOOP-1533
> Project: Sqoop
> Issue Type: Bug
> Environment: ubuntu, Sqoop 1.4.4-cdh5.1.2, java 8
> Reporter: Shigeki Hirose
> Priority: Critical
>
> Sqoop command example:
> {noformat}
> sqoop import --connect "jdbc:mysql://127.0.0.1:3306/xxx" --username xxx
> --password xxx --table SampleTable --hcatalog-database xxx
> --create-hcatalog-table --hcatalog-storage-stanza 'stored as orc'
> --hcatalog-table sampletable
> {noformat}
> Call stack:
> {noformat}
> sqoop.Sqoop: Got exception running Sqoop: java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.createHCatTable(SqoopHCatUtilities.java:491)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureHCat(SqoopHCatUtilities.java:293)
> at
> org.apache.sqoop.mapreduce.hcat.SqoopHCatUtilities.configureImportOutputFormat(SqoopHCatUtilities.java:657)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.configureOutputFormat(ImportJobBase.java:98)
> at
> org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:240)
> at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:614)
> at
> org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
> at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:413)
> at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:506)
> at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:222)
> at org.apache.sqoop.Sqoop.runTool(Sqoop.java:231)
> at org.apache.sqoop.Sqoop.main(Sqoop.java:240)
> {noformat}
> This seems to be because source MySQL table contains a column name that has
> an upper case letter.
> Sample table below. The column name, 'A', causes the exception.
> {code:sql}
> DROP TABLE `SampleTable`;
> CREATE TABLE `SampleTable` (
> `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
> `A` bigint(20) unsigned NOT NULL DEFAULT '0',
> PRIMARY KEY (`id`)
> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
> {code}
> Once column name is changed to 'a', it works properly:
> {code:sql}
> DROP TABLE `SampleTable`;
> CREATE TABLE `SampleTable` (
> `id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
> `a` bigint(20) unsigned NOT NULL DEFAULT '0',
> PRIMARY KEY (`id`)
> ) ENGINE=InnoDB DEFAULT CHARSET=utf8;
> {code}
> This is probably because LCKeyMap (and HCKeyMap) class in
> SqoopHCatUtilities.java line 160 - 177 are expecting HashMap to call *put*
> method for each element when *putAll* is invoked, which seem to have been the
> case in Java 6/7, but it doesn't behave that way in Java 8.
> {code:java}
> /**
> * A Map using String as key type that ignores case of its key and stores
> the
> * key in lower case.
> */
> private static class LCKeyMap<V> extends HashMap<String, V> {
> private static final long serialVersionUID = -6751510232323094216L;
> @Override
> public V put(String key, V value) {
> return super.put(key.toLowerCase(), value);
> }
> @Override
> public V get(Object key) {
> return super.get(((String) key).toLowerCase());
> }
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
