[
https://issues.apache.org/jira/browse/SQOOP-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Crotty updated SQOOP-2628:
---------------------------------
Description:
Sqoop doesn't honor UTF-8 chars when import --direct on a MySQL table.
Here is the key comma delimited output from attached example script w/o and w/
--direct:
{code}
1,Τη γλώσσα,"/fox/\jumps
1,���� ������������,"/fox/\jumps
{code}
I looked over sqoop --verbose output and hadoop logs but can't find anything
suspicious.
As an aside run the example script w/ --mysql-delimiters to get this comma
delimited output:
{code}
1,Τη γλώσσα,"/fox/\\jumps
1,'���� ������������','\"/fox/\\jumps'
{code}
Attached are scripts to create the MySQL utest example table and bash script I
used to demonstrate the --direct problem.
Environment
{code}
$ sqoop version
Warning: /home/hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../hcatalog
does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../accumulo
does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/10/20 17:28:21 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
Sqoop 1.4.6
git commit id c0c5a81723759fa575844a0a1eae8f510fa32c25
Compiled by root on Mon Apr 27 14:38:36 CST 2015
$ hadoop version
Hadoop 2.6.0-amzn-1
Subversion [email protected]:/pkg/Aws157BigTop -r
edd5a97db145470a8723dde24f38c83724e0959c
Compiled by ec2-user on 2015-09-25T14:59Z
Compiled with protoc 2.5.0
>From source with checksum 7beeae31f3c4554b23d92f1e63dc85
This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-amzn-1.jar
{code}
was:
Sqoop doesn't honor UTF-8 chars when import --direct on a MySQL table.
Here is the key comma delimited output from attached example script w/o and w/
--direct:
{code}
1,Τη γλώσσα,"/fox/\jumps
1,���� ������������,"/fox/\jumps
{code}
I looked over sqoop --verbose output and hadoop logs but can't find anything
suspicious.
Attached are scripts to create the MySQL utest example table and bash script I
used to demonstrate the --direct problem.
Environment
{code}
$ sqoop version
Warning: /home/hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../hcatalog
does not exist! HCatalog jobs will fail.
Please set $HCAT_HOME to the root of your HCatalog installation.
Warning: /home/hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../accumulo
does not exist! Accumulo imports will fail.
Please set $ACCUMULO_HOME to the root of your Accumulo installation.
15/10/20 17:28:21 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
Sqoop 1.4.6
git commit id c0c5a81723759fa575844a0a1eae8f510fa32c25
Compiled by root on Mon Apr 27 14:38:36 CST 2015
$ hadoop version
Hadoop 2.6.0-amzn-1
Subversion [email protected]:/pkg/Aws157BigTop -r
edd5a97db145470a8723dde24f38c83724e0959c
Compiled by ec2-user on 2015-09-25T14:59Z
Compiled with protoc 2.5.0
>From source with checksum 7beeae31f3c4554b23d92f1e63dc85
This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-amzn-1.jar
{code}
> Import MySQL table --direct UTF-8 data corrupted
> ------------------------------------------------
>
> Key: SQOOP-2628
> URL: https://issues.apache.org/jira/browse/SQOOP-2628
> Project: Sqoop
> Issue Type: Bug
> Components: sqoop2-jdbc-connector
> Affects Versions: 1.4.6
> Environment: sqoop 1.4.6 hadoop 2.6.0-amzn-1
> Reporter: Joseph Crotty
> Attachments: create_utest_table.sql, sqoop_import.sh, sqoop_utest.log
>
>
> Sqoop doesn't honor UTF-8 chars when import --direct on a MySQL table.
> Here is the key comma delimited output from attached example script w/o and
> w/ --direct:
> {code}
> 1,Τη γλώσσα,"/fox/\jumps
> 1,���� ������������,"/fox/\jumps
> {code}
> I looked over sqoop --verbose output and hadoop logs but can't find anything
> suspicious.
> As an aside run the example script w/ --mysql-delimiters to get this comma
> delimited output:
> {code}
> 1,Τη γλώσσα,"/fox/\\jumps
> 1,'���� ������������','\"/fox/\\jumps'
> {code}
> Attached are scripts to create the MySQL utest example table and bash script
> I used to demonstrate the --direct problem.
> Environment
> {code}
> $ sqoop version
> Warning: /home/hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../hcatalog
> does not exist! HCatalog jobs will fail.
> Please set $HCAT_HOME to the root of your HCatalog installation.
> Warning: /home/hadoop/sqoop-1.4.6.bin__hadoop-2.0.4-alpha/bin/../../accumulo
> does not exist! Accumulo imports will fail.
> Please set $ACCUMULO_HOME to the root of your Accumulo installation.
> 15/10/20 17:28:21 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6
> Sqoop 1.4.6
> git commit id c0c5a81723759fa575844a0a1eae8f510fa32c25
> Compiled by root on Mon Apr 27 14:38:36 CST 2015
> $ hadoop version
> Hadoop 2.6.0-amzn-1
> Subversion [email protected]:/pkg/Aws157BigTop -r
> edd5a97db145470a8723dde24f38c83724e0959c
> Compiled by ec2-user on 2015-09-25T14:59Z
> Compiled with protoc 2.5.0
> From source with checksum 7beeae31f3c4554b23d92f1e63dc85
> This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-amzn-1.jar
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
