Stevo Mitric created SPARK-55932:
------------------------------------
Summary: XML variant parser hangs on BigDecimal.setScale(0) for
extreme negative scale
Key: SPARK-55932
URL: https://issues.apache.org/jira/browse/SPARK-55932
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 4.2.0
Reporter: Stevo Mitric
When parsing XML or CSV data to Variant type, a decimal string with an extreme
positive exponent (e.g., "1e9999999") causes the executor to hang indefinitely.
Java's BigDecimal parses such strings into a compact representation
(unscaledValue=1, scale=-9999999).
The variant parser then calls d.setScale(0) to normalize the scale before
storing in the variant binary format. This forces the JDK to compute
BigInteger.pow(10, 9999999), materializing a number with ~10 million digits via
Toom-Cook-3/Karatsuba multiplication - an operation that takes hours and
effectively hangs the task. The value would ultimately be rejected anyway since
its precision exceeds MAX_DECIMAL16_PRECISION (38), but the expensive
computation runs before that check.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
