[
https://issues.apache.org/jira/browse/SQOOP-1037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13662600#comment-13662600
]
wenzhi wang commented on SQOOP-1037:
------------------------------------
Jecec,
Thanks, I opened #SQOOP-1054 to request Sqoop to support binary type splitter.
Although there are integer columns in the table, I cannot define key on them.
Another question: I also tried import table by "--split-by
currentMapScreeningProfile" and without "-m 1". This time, It costs 219s to
import same amount of data. Currently, field "currentMapScreeningProfile" is
NULL for all records:
mysql> SELECT MIN(`currentMapScreeningProfile`),
MAX(`currentMapScreeningProfile`) FROM `IMSI`;
+-----------------------------------+-----------------------------------+
| MIN(`currentMapScreeningProfile`) | MAX(`currentMapScreeningProfile`) |
+-----------------------------------+-----------------------------------+
| NULL | NULL |
+-----------------------------------+-----------------------------------+
1 row in set (19.64 sec)
Why the import duation shortens amost a half? Does Sqoop import table with more
than 1 mapper even when splliter's values are same?
Thanks,
Wenzhi
> Cannot import mysql table with into Hadoop
> ------------------------------------------
>
> Key: SQOOP-1037
> URL: https://issues.apache.org/jira/browse/SQOOP-1037
> Project: Sqoop
> Issue Type: Bug
> Components: tools
> Affects Versions: 1.4.3
> Environment: bl1253{wenzhiw}/home/wenzhiw>cat /etc/redhat-release
> Scientific Linux SL release 5.3 (Boron)
> bl1253{wenzhiw}/home/wenzhiw>uname -a
> Linux bl1253.cn.alcatel-lucent.com 2.6.18-128.7.1.el5PAE #1 SMP Mon Aug 24
> 08:41:30 EDT 2009 i686 athlon i386 GNU/Linux
> Reporter: wenzhi wang
> Priority: Critical
> Attachments: sqoop_import_error.log
>
> Original Estimate: 504h
> Remaining Estimate: 504h
>
> Hi,
> I run Sqoop 1.4.3 on Apache Hadoop 1.0.4 and want to import table from mysql,
> but met below errors. I also searched jira site for this issue, but didn't
> find any useful info, please help to check. thanks!
> {code}
> bl1253{wenzhiw}/local/wenzhiw/sqoop2/sqoop-1.4.3.bin__hadoop-0.23/bin>sqoop
> import --table IMSI --connect jdbc:mysql://135.252.158.250/hlr --username
> root -P
> Warning: /usr/lib/hbase does not exist! HBase imports will fail.
> Please set $HBASE_HOME to the root of your HBase installation.
> Warning: $HADOOP_HOME is deprecated.
> Enter password:
> 13/05/08 10:20:53 INFO manager.MySQLManager: Preparing to use a MySQL
> streaming resultset.
> 13/05/08 10:20:53 INFO tool.CodeGenTool: Beginning code generation
> 13/05/08 10:20:54 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM `IMSI` AS t LIMIT 1
> 13/05/08 10:20:55 INFO manager.SqlManager: Executing SQL statement: SELECT
> t.* FROM `IMSI` AS t LIMIT 1
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type
> org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type
> org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type
> org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type
> org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type
> org.apache.hadoop.io.BytesWritable
> 13/05/08 10:20:55 ERROR orm.ClassWriter: No parser available for Java type
> org.apache.hadoop.io.BytesWritable
> ...
> {code}
> Table definition:
> {code}
> mysql> desc IMSI;
> +-------------------------------+-------------+------+-----+---------+-------+
> | Field | Type | Null | Key | Default | Extra |
> +-------------------------------+-------------+------+-----+---------+-------+
> | mSubIdentificationNumberId | binary(6) | NO | PRI | NULL | |
> | mobileNetworkCodeId | binary(3) | NO | PRI | NULL | |
> | mobileCountryCodeId | binary(3) | NO | PRI | NULL | |
> | mainMSISDN | binary(12) | YES | | NULL | |
> | alertMSISDN | binary(12) | YES | | NULL | |
> | associatedMSISDN | binary(88) | YES | | NULL | |
> | gprsNotReachableReason | tinyint(4) | YES | | NULL | |
> | sgsnLastGlobalUpdateTime | int(11) | YES | | NULL | |
> | vlrIsdn | binary(10) | YES | | NULL | |
> | mscIsdn | binary(10) | YES | | NULL | |
> | supOfLocRegCount | tinyint(4) | YES | | NULL | |
> | locRegSubsPeriod | smallint(6) | YES | | NULL | |
> | inducedCbocBsg | binary(7) | YES | | NULL | |
> | inducedCboiBsg | binary(7) | YES | | NULL | |
> | inducedCboehBsg | binary(7) | YES | | NULL | |
> | vlrLastGlobalUpdateTime | int(11) | YES | | NULL | |
> | currentMapScreeningProfile | int(11) | YES | | NULL | |
> | smscIsdn | binary(70) | YES | | NULL | |
> | gsmNotReachableReason | tinyint(4) | YES | | NULL | |
> | timeStampWaitingSms | binary(28) | YES | | NULL | |
> | checkSSindicator | tinyint(4) | YES | | NULL | |
> | supOfAutReqCount | tinyint(4) | YES | | NULL | |
> | autReqSubsPeriod | smallint(6) | YES | | NULL | |
> | underTestImsi | bit(1) | YES | | NULL | |
> | cardStatus | bit(1) | YES | | NULL | |
> |
> +-------------------------------+-------------+------+-----+---------+-------+
> {code}
> Thanks,
> Wenzhi
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
