Hi Uroš. Thank you so much for the thorough SPIP. The IEEE 754 alignment, the config-gated incubation pattern, and the explicit type table are really helpful - excited to see this move forward.
I had two questions: 1. Has a pure - Java arithmetic implementation been evaluated as an alternative to JNI / libbid? Spark's existing native dependencies (snappy, zstd, lz4) are compression codecs that ship as multi-platform fat JARs. libbid would be the first native dependency on the arithmetic hot path with no fallback - every DECFLOAT operation would be a JNI call. There is a potential risk of introducing platform-specific build complexity, crash-mode failures (SIGSEGV vs exceptions), and distribution overhead for downstream packagers. Curious if the tradeoff was considered. 2. On type precedence (Note 3): is the DECIMAL(35..38) -> DECFLOAT(34) coercion implicit in a UNION or JOIN? Users could silently lose precision. Should ANSI mode reject this implicit widening (require explicit CAST)? Silent precision loss seems inconsistent with ANSI mode's fail-fast behavior for other numeric overflows (CAST_OVERFLOW). Thanks and regards, Shrirang On Tue, Aug 18, 2026 at 9:01 AM Uroš Bojanić <[email protected]> wrote: > Hi all, > > I would like to start a discussion on the SPIP to add a new Spark SQL data > type: DECFLOAT (IEEE 754 decimal64 / decimal128), for base-10 > floating-point decimals with per-value exponents. > > JIRA ID: https://issues.apache.org/jira/browse/SPARK-58820 > > Brief summary: DECFLOAT is a new IEEE 754 decimal floating-point type > (decimal64 / decimal128, i.e. DECFLOAT(16) / DECFLOAT(34)) that closes the > gap between DECIMAL, which caps at precision 38 with a fixed per-column > scale, and DOUBLE, whose binary rounding makes fractions like 0.1 inexact: > each value keeps its own exponent, arithmetic is decimal, and it supports > signed zero, Inf, and NaN. It is additive, round-trips through a proposed > Parquet logical type for cross-engine interop, and would ship config-gated > during incubation like TIME, leaving existing DECIMAL / DOUBLE behavior > unchanged. > > Additional information is available in the SPIP document: > > https://docs.google.com/document/d/1qnLXm0ldHSwPSJs_Q5SzGyyTKMMEqeX5rm8Fh4rvV3E > > Please provide your feedback on the approach, scope, and the proposal > described in the document. > > Thank you! > > Best, > Uroš > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
