Hi Shrirang, Both your points are well-taken and folded into the SPIP document.
1) Kernel: added pros/cons & trade-offs for JNI vs Java impl (expanded Appendix E, #6). 2) Coercion: agreed and tightened the Note 3 for type precedence (under Appendix C). Thank you for reviewing the doc! Further feedback is still welcome. Best, Uroš On 2026/08/18 18:53:05 Shrirang Mhalgi wrote: > Hi Uroš. > Thank you so much for the thorough SPIP. The IEEE 754 alignment, the > config-gated incubation pattern, and the explicit type table are really > helpful - excited to see this move forward. > > I had two questions: > 1. Has a pure - Java arithmetic implementation been evaluated as an > alternative to JNI / libbid? Spark's existing native dependencies (snappy, > zstd, lz4) are compression codecs that ship as multi-platform fat JARs. > libbid would be the first native dependency on the arithmetic hot path with > no fallback - every DECFLOAT operation would be a JNI call. There is a > potential risk of introducing platform-specific build complexity, > crash-mode failures (SIGSEGV vs exceptions), and distribution overhead for > downstream packagers. Curious if the tradeoff was considered. > 2. On type precedence (Note 3): is the DECIMAL(35..38) -> DECFLOAT(34) > coercion implicit in a UNION or JOIN? Users could silently lose precision. > Should ANSI mode reject this implicit widening (require explicit CAST)? > Silent precision loss seems inconsistent with ANSI mode's fail-fast > behavior for other numeric overflows (CAST_OVERFLOW). > > Thanks and regards, > Shrirang > > On Tue, Aug 18, 2026 at 9:01 AM Uroš Bojanić <[email protected]> wrote: > > > Hi all, > > > > I would like to start a discussion on the SPIP to add a new Spark SQL data > > type: DECFLOAT (IEEE 754 decimal64 / decimal128), for base-10 > > floating-point decimals with per-value exponents. > > > > JIRA ID: https://issues.apache.org/jira/browse/SPARK-58820 > > > > Brief summary: DECFLOAT is a new IEEE 754 decimal floating-point type > > (decimal64 / decimal128, i.e. DECFLOAT(16) / DECFLOAT(34)) that closes the > > gap between DECIMAL, which caps at precision 38 with a fixed per-column > > scale, and DOUBLE, whose binary rounding makes fractions like 0.1 inexact: > > each value keeps its own exponent, arithmetic is decimal, and it supports > > signed zero, Inf, and NaN. It is additive, round-trips through a proposed > > Parquet logical type for cross-engine interop, and would ship config-gated > > during incubation like TIME, leaving existing DECIMAL / DOUBLE behavior > > unchanged. > > > > Additional information is available in the SPIP document: > > > > https://docs.google.com/document/d/1qnLXm0ldHSwPSJs_Q5SzGyyTKMMEqeX5rm8Fh4rvV3E > > > > Please provide your feedback on the approach, scope, and the proposal > > described in the document. > > > > Thank you! > > > > Best, > > Uroš > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: [email protected] > > > > > --------------------------------------------------------------------- To unsubscribe e-mail: [email protected]
