andygrove opened a new issue, #285:
URL: https://github.com/apache/datafusion-spark/issues/285

   Part of #271 and apache/datafusion-comet#7300.
   
   ### What the function does
   
   `xpath_int(xml, path)` evaluates an XPath 1.0 expression, converts the 
result with XPath `number()` into a
   Java `Double`, and narrows it with `Double.intValue()`, giving an `INT`. 
Both arguments are strings
   (collation-aware from 4.0), and `path` must be foldable. The result is 
always nullable. Available since Spark
   2.0.0.
   
   The `number()` conversion is in the `xpath_double` issue. Parsing, DTD and 
entity handling, namespaces, NULL
   handling and errors are in the `xpath` issue. This issue covers only the 
double-to-int step, which is Java's
   narrowing conversion `(int) d`:
   
   - `NaN` becomes `0`;
   - values are truncated toward zero;
   - out-of-range values saturate at `Integer.MAX_VALUE` / `Integer.MIN_VALUE`, 
and so do `+/-Infinity`.
   
   This matches Rust's `d as i32`. Nothing raises an error, and ANSI mode has 
no effect.
   
   ### Examples
   
   ```sql
   SELECT xpath_int('<a><b>1</b><b>2</b></a>', 'sum(a/b)');
   -- 3
   SELECT xpath_int('<a>this is not a number</a>', 'a');
   -- 0
   SELECT xpath_int('<a>this 2 is not a number</a>', 'a');
   -- 0
   SELECT xpath_int('<a><b>2000000000</b><c>40000000000</c></a>', 'a/b * a/c');
   -- 2147483647
   SELECT xpath_int('<a>try a boolean</a>', 'a = 10');
   -- 0
   SELECT xpath_int('<a><b class="odd">1</b><b class="even">2</b><b 
class="odd">4</b><c>8</c></a>', 'a/b');
   -- 1
   SELECT xpath_int('<a><b class="odd">1</b><b class="even">2</b><b 
class="odd">4</b><c>8</c></a>', 'sum(a/*)');
   -- 15
   SELECT xpath_int('<a><b class="odd">100000</b><b class="even">2</b><b 
class="odd">4</b><c>8</c></a>', 'sum(a/b[@class="odd"])');
   -- 100004
   ```
   
   These come from the `ExpressionDescription`, the Hive `udf_xpath_int` golden 
files run by
   `HiveCompatibilitySuite`, and `XPathExpressionSuite`.
   
   ### Edge cases to test
   
   - Saturation at both ends: `8.0E19` gives `2147483647` (above). By the same 
rule, `-1 * a/b * a/c` gives
     `-2147483648`, `1 div 0` gives `2147483647`, and `-1 div 0` gives 
`-2147483648`.
   - Truncation: `<a>2.7</a>` gives `2` and `<a>-2.7</a>` gives `-2`, by code 
reading. Note that XPath `round()`
     and `floor()` give different results, so test `xpath_int(x, 'round(a)')` 
as well.
   - `NaN` gives `0` (non-numeric text, `0 div 0`, and the empty node-set, 
which is `NaN` by XPath 1.0 rules).
     The usage string says "zero if no match is found, or a match is found but 
the value is non-numeric".
   - Values just at and beyond `2^31`: `2147483647`, `2147483648` and 
`-2147483649`.
   - `NULL` or `''` arguments give NULL. A non-literal path gives 
`DATATYPE_MISMATCH.NON_FOLDABLE_INPUT`.
     Malformed XML and an invalid path raise runtime errors (see the `xpath` 
issue).
   
   ### Differences across Spark versions
   
   None found beyond the shared ones in the `xpath` issue.
   
   ### Implementation notes
   
   - Spark: `XPathInt` in 
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala`.
     It calls `XPathIntEvaluator` (`ret.intValue()`) in 
`.../xml/XmlExpressionEvalUtils.scala`.
   - DataFusion core has no equivalent.
   - Reuse the shared `number()` evaluator from the `xpath` and `xpath_double` 
issues in
     `crates/spark/src/function/xml/`, and narrow with `as i32`. Rust's 
saturating float-to-int cast matches
     Java's here.
   - Tests: add `crates/sqllogictest/test_files/xml/xpath_int.slt` and document 
the function with `#[user_doc]`.
   
   ---
   _This issue was generated by an LLM (Claude) from the Apache Spark source 
code and tests. Examples come from Spark's `ExpressionDescription` examples, 
test suites and `.sql.out` golden files, but please verify them against Spark 
before relying on them._


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to