This is an automated email from the ASF dual-hosted git repository.
mbutrovich pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/datafusion-comet.git
The following commit(s) were added to refs/heads/main by this push:
new 42528dbcd5 docs: drop compatibility notes for bugs that are now fixed
(#5154)
42528dbcd5 is described below
commit 42528dbcd5ea95416429ed995f2917ec037db4ba
Author: Andy Grove <[email protected]>
AuthorDate: Mon Aug 3 08:05:23 2026 -0600
docs: drop compatibility notes for bugs that are now fixed (#5154)
Follow-up to #5085. Three of the divergences it documented have since been
fixed, but none of the fixes removed the note describing them:
- #5065, fixed by #5084 -- `CometIntegralDivide` still claims `Long.MinValue
div -1` silently returns the wrapped value under ANSI.
- #5066, fixed by #5080 -- `CometMakeDecimal` still says the native
implementation ignores `nullOnOverflow`.
- #5070, fixed by #5082 -- `CometRound` still says `LongType` input with
`scale <= -19` returns `0` instead of overflowing.
Remove all three `getCompatibleNotes` overrides. The per-category
compatibility pages (`math.md`, `misc.md`, ...) are generated from these
strings at docs-build time and are not committed to the tree, so stale text
publishes on the next docs build rather than showing up in a diff. #5127 set
the precedent by removing its `null IN ()` bullet as part of the fix.
The string-cast trim note this PR originally rescoped no longer needs
touching: #5150 fixed the boolean, integral, float, double and decimal
targets and rescoped the `compatibility/index.md` bullet to the remaining
timestamp and `to_time` targets in the same commit.
The other notes from #5085 are unchanged and still accurate: #5073, #4680,
#4947, #5071, #5072, #4967, #5068, #5022, #4993, and the datetime-rebase
note
in `scans.md` -- `spark.comet.exceptionOnDatetimeRebase` is still only its
`CometConf.scala` definition with no reader, so the dead-code note holds.
After this change the only issue referenced by any `getCompatibleNotes`
override is #5073, which is still open.
Co-authored-by: Matt Butrovich <[email protected]>
---
spark/src/main/scala/org/apache/comet/serde/arithmetic.scala | 12 ------------
.../scala/org/apache/comet/serde/decimalExpressions.scala | 8 --------
2 files changed, 20 deletions(-)
diff --git a/spark/src/main/scala/org/apache/comet/serde/arithmetic.scala
b/spark/src/main/scala/org/apache/comet/serde/arithmetic.scala
index 614fa4a5c0..8023231b35 100644
--- a/spark/src/main/scala/org/apache/comet/serde/arithmetic.scala
+++ b/spark/src/main/scala/org/apache/comet/serde/arithmetic.scala
@@ -292,12 +292,6 @@ object CometDivide extends CometExpressionSerde[Divide]
with MathBase {
object CometIntegralDivide extends CometExpressionSerde[IntegralDivide] with
MathBase {
- override def getCompatibleNotes(): Seq[String] = Seq(
- "On `LongType` input, `Long.MinValue div -1` does not overflow-check:
Comet's decimal-backed" +
- " path returns the wrapped value `Long.MinValue` under legacy mode and
silently returns" +
- " the same wrapped value under ANSI, where Spark raises
`ARITHMETIC_OVERFLOW`" +
- " ([#5065](https://github.com/apache/datafusion-comet/issues/5065)).")
-
override def getSupportLevel(expr: IntegralDivide): SupportLevel =
mathDataTypeSupportLevel(expr.left.dataType)
@@ -385,12 +379,6 @@ object CometRemainder extends
CometExpressionSerde[Remainder] with MathBase {
object CometRound extends CometExpressionSerde[Round] {
- override def getCompatibleNotes(): Seq[String] = Seq(
- "On `LongType` input with a negative `scale` of `-19` or lower,
`10^(-scale)` overflows the" +
- " native integer type and Comet returns `0` instead of Spark's
overflowed value. Under" +
- " ANSI mode Spark raises `ARITHMETIC_OVERFLOW` at the same input; Comet
still returns `0`" +
- " ([#5070](https://github.com/apache/datafusion-comet/issues/5070)).")
-
override def getSupportLevel(expr: Round): SupportLevel =
expr.child.dataType match {
case t: DecimalType if t.scale < 0 => // Spark disallows negative scale
SPARK-30252
Unsupported(Some("Decimal type has negative scale"))
diff --git
a/spark/src/main/scala/org/apache/comet/serde/decimalExpressions.scala
b/spark/src/main/scala/org/apache/comet/serde/decimalExpressions.scala
index 3af32e1d11..f41e0fafaa 100644
--- a/spark/src/main/scala/org/apache/comet/serde/decimalExpressions.scala
+++ b/spark/src/main/scala/org/apache/comet/serde/decimalExpressions.scala
@@ -41,14 +41,6 @@ object CometMakeDecimal extends
CometExpressionSerde[MakeDecimal] {
override def getUnsupportedReasons(): Seq[String] = Seq("Only `LongType`
input is supported")
- override def getCompatibleNotes(): Seq[String] = Seq(
- "The native implementation silently returns `NULL` on overflow regardless
of the" +
- " `nullOnOverflow` flag, so under ANSI mode Comet returns `NULL` where
Spark raises" +
- " `NUMERIC_VALUE_OUT_OF_RANGE`. `MakeDecimal` is inserted by the
`DecimalAggregates`" +
- " optimizer rule, so this affects `sum`/`avg` over low-precision
decimals when the" +
- " unscaled long overflows the target precision" +
- " ([#5066](https://github.com/apache/datafusion-comet/issues/5066)).")
-
override def getSupportLevel(expr: MakeDecimal): SupportLevel = {
expr.child.dataType match {
case LongType => Compatible()
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
