[
https://issues.apache.org/jira/browse/SPARK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Andrushenko updated SPARK-20427:
------------------------------------------
Description:
In Oracle exists data type NUMBER. When defining a filed in a table of type
NUMBER the field has two components, precision and scale.
For example, NUMBER(p,s) has precision p and scale s.
Precision can range from 1 to 38.
Scale can range from -84 to 127.
When reading such a filed Spark can create numbers with precision exceeding 38.
In our case it has created fields with precision 44,
calculated as sum of the precision (in our case 34 digits) and the scale (10):
"...java.lang.IllegalArgumentException: requirement failed: Decimal precision
44 exceeds max precision 38...".
The result was, that a data frame was read from a table on one schema but could
not be inserted in the identical table on other schema.
was:
In Oracle exists data type NUMBER. When defining a filed in a table of type
NUMBER the field has two components, precision and scale.
For example, NUMBER(p,s) has precision p and scale s.
Precision can range from 1 to 38.
Scale can range from -84 to 127.
When reading such a filed Spark can create numbers with precision exceeding 38.
In our case it has created fields with precision 44:
"...java.lang.IllegalArgumentException: requirement failed: Decimal precision
44 exceeds max precision 38...".
The result was, that a data frame was read from a table on one schema but could
not be inserted in the identical table on other schema.
> Issue with number conversions
> -----------------------------
>
> Key: SPARK-20427
> URL: https://issues.apache.org/jira/browse/SPARK-20427
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.1.0
> Reporter: Alexander Andrushenko
>
> In Oracle exists data type NUMBER. When defining a filed in a table of type
> NUMBER the field has two components, precision and scale.
> For example, NUMBER(p,s) has precision p and scale s.
> Precision can range from 1 to 38.
> Scale can range from -84 to 127.
> When reading such a filed Spark can create numbers with precision exceeding
> 38. In our case it has created fields with precision 44,
> calculated as sum of the precision (in our case 34 digits) and the scale (10):
> "...java.lang.IllegalArgumentException: requirement failed: Decimal precision
> 44 exceeds max precision 38...".
> The result was, that a data frame was read from a table on one schema but
> could not be inserted in the identical table on other schema.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
