[
https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15757924#comment-15757924
]
Dongjoon Hyun edited comment on SPARK-18877 at 12/18/16 1:11 AM:
-----------------------------------------------------------------
Hi, [~Navya Krishnappa].
As you see in the stack trace, that is a different exception from Apache
Parquet code.
{code}
Caused by: java.lang.IllegalArgumentException: Invalid DECIMAL scale: -9
at org.apache.parquet.Preconditions.checkArgument(Preconditions.java:55)
{code}
And, this is the Parquet code
[here|https://github.com/Parquet/parquet-mr/blob/master/parquet-column/src/main/java/parquet/schema/Types.java#L405-L417].
It apparently does not fully support BigDecimal.
{code}
protected DecimalMetadata decimalMetadata() {
DecimalMetadata meta = null;
if (OriginalType.DECIMAL == originalType) {
Preconditions.checkArgument(precision > 0,
"Invalid DECIMAL precision: " + precision);
Preconditions.checkArgument(scale >= 0,
"Invalid DECIMAL scale: " + scale);
Preconditions.checkArgument(scale <= precision,
"Invalid DECIMAL scale: cannot be greater than precision");
meta = new DecimalMetadata(precision, scale);
}
return meta;
}
}
{code}
I cannot make a PR for Parquet.
I hope you register another issue for that in Apache Parquet JIRA.
was (Author: dongjoon):
Hi, [~Navya Krishnappa].
As you see in the stack trace, that is a different exception from Apache
Parquet code.
{code}
Caused by: java.lang.IllegalArgumentException: Invalid DECIMAL scale: -9
at org.apache.parquet.Preconditions.checkArgument(Preconditions.java:55)
{code}
You can register another issue for that, but maybe in Apache Parquet JIRA.
> Unable to read given csv data. Excepion: java.lang.IllegalArgumentException:
> requirement failed: Decimal precision 28 exceeds max precision 20
> ----------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: SPARK-18877
> URL: https://issues.apache.org/jira/browse/SPARK-18877
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 2.0.2
> Reporter: Navya Krishnappa
>
> When reading below mentioned csv data, even though the maximum decimal
> precision is 38, following exception is thrown
> java.lang.IllegalArgumentException: requirement failed: Decimal precision 28
> exceeds max precision 20
> Decimal
> 2323366225312000000000000000
> 24335739714000000
> 23233662253000
> 232336622530000
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
