liviu created SQOOP-3227:
----------------------------
Summary: Sqoop Avro import with decimal mapping issue
Key: SQOOP-3227
URL: https://issues.apache.org/jira/browse/SQOOP-3227
Project: Sqoop
Issue Type: Bug
Reporter: liviu
Priority: Blocker
Hi,
We are using Sqoop version 1.4.6-cdh5.8.5 for importing numeric data types from
Oracle to Avro Decimal Logical type (Bytes as Decimal with precision and scale)
The import works ok except for cases in which the values stored in Oracle have
the scale lower than the one defined in table ddl.
Ex. the column is defined as NUMERIC(20,2) and the value stored is "3.2" (only
one digit after decimal point); in this case we receive the error message:
*_Error: org.apache.avro.file.DataFileWriter$AppendWriteException:
org.apache.avro.AvroTypeException: Cannot encode decimal with scale 1 as scale
2_*
This error was discussed in [https://issues.apache.org/jira/browse/AVRO-1864]
with the resolution that this sqoop behavior is correct (it cannot add by
default an extra info for transforming "3.2" to "3.20")
We tried below methods for conversion to AVRO Decimal(20,2) Logical datatype:
1). use *--map-column-java "COLNAME=java.math.BigDecimal"* in sqoop import
command
error received:
_*Error: org.apache.avro.file.DataFileWriter$AppendWriteException:
org.apache.avro.AvroTypeException: Cannot encode decimal with scale 1 as scale
2*_
2). use _*--map-column-java "COL1=Decimal(20%2C2)"*_ in sqoop command
error received:
*_ERROR tool.ImportTool: Import failed: No ResultSet method for Java type
Decimal(20,2)_*
3). made the column as varchar2(100) in Oracle database, stored the value as
"3.20" and use _*--map-column-java "COLNAME=java.math.BigDecimal"* _in sqoop
command
error received:
*_ERROR tool.ImportTool: Import failed: Cannot convert to AVRO type
java.math.BigDecimal_*
There is any way in which we can instruct sqoop to map the column from Oracle
table to AVRO Decimal logical type (precision=20, scale=2) and import the "3.2"
value from Oracle database as "3.20" type Decimal(20,2) in AVRO file?
Thanks,
Liviu
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
