[
https://issues.apache.org/jira/browse/SPARK-55932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-55932:
-----------------------------------
Assignee: Stevo Mitric
> XML variant parser hangs on BigDecimal.setScale(0) for extreme negative scale
> -----------------------------------------------------------------------------
>
> Key: SPARK-55932
> URL: https://issues.apache.org/jira/browse/SPARK-55932
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 4.2.0
> Reporter: Stevo Mitric
> Assignee: Stevo Mitric
> Priority: Major
> Labels: pull-request-available
>
> When parsing XML or CSV data to Variant type, a decimal string with an
> extreme positive exponent (e.g., "1e9999999") causes the executor to hang
> indefinitely. Java's BigDecimal parses such strings into a compact
> representation (unscaledValue=1, scale=-9999999).
> The variant parser then calls d.setScale(0) to normalize the scale before
> storing in the variant binary format. This forces the JDK to compute
> BigInteger.pow(10, 9999999), materializing a number with ~10 million digits
> via Toom-Cook-3/Karatsuba multiplication - an operation that takes hours and
> effectively hangs the task. The value would ultimately be rejected anyway
> since its precision exceeds MAX_DECIMAL16_PRECISION (38), but the expensive
> computation runs before that check.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
