andygrove opened a new issue, #282:
URL: https://github.com/apache/datafusion-spark/issues/282

   Part of #271 and apache/datafusion-comet#7300.
   
   ### What the function does
   
   `xpath_boolean(xml, path)` evaluates an XPath 1.0 expression and converts 
the result to `BOOLEAN` using the
   XPath `boolean()` rules (`XPathConstants.BOOLEAN`). Both arguments are 
strings (collation-aware from 4.0), and
   `path` must be foldable. The result is always nullable. Available since 
Spark 2.0.0. The expression is a
   `Predicate`, so it can be used directly in `WHERE`.
   
   Parsing, DTD and entity handling, namespaces, NULL and empty-string 
handling, invalid-XML and invalid-path
   errors, and lazy path compilation are shared with the rest of the family. 
They are specified in the `xpath`
   issue, and the implementation should reuse that evaluator. This issue covers 
only the boolean conversion.
   
   ### Examples
   
   ```sql
   SELECT xpath_boolean('<a><b>1</b></a>','a/b');
   -- true
   SELECT xpath_boolean('<a><b>b</b></a>', 'a/c');
   -- false
   SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "b"');
   -- true
   SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "c"');
   -- false
   SELECT xpath_boolean('<a><b>10</b></a>', 'a/b < 10');
   -- false
   SELECT xpath_boolean('<a><b>10</b></a>', 'a/b = 10');
   -- true
   ```
   
   These come from the `ExpressionDescription`, `XPathExpressionSuite` and the 
Hive `udf_xpath_boolean` golden
   files run by `HiveCompatibilitySuite`. In `UDFXPathUtilSuite`, 
`a/b[1]/text()` is true and `a/b[4]` (no match)
   is false.
   
   ### Edge cases to test
   
   - Conversion rules from XPath 1.0 `boolean()`:
     - a node-set is true if it is non-empty, whatever the text. `<b>false</b>` 
with `a/b/text()` gives true;
     - a number is false for `0`, `-0` and `NaN`, and true otherwise;
     - a string is true if it is non-empty, so `string(a/c)` on a missing node 
gives false;
     - a boolean result is used as is.
   - Comparisons between a node-set and a value are existential: `a/b = 2` is 
true if any `b` equals 2. Also test
     number versus string comparison (`a/b = "10"` versus `a/b = 10` with 
`<b>10.0</b>`).
   - `NULL` or `''` for either argument gives NULL. A non-literal `path` fails 
with
     `DATATYPE_MISMATCH.NON_FOLDABLE_INPUT`.
   - Malformed XML and an invalid path raise the runtime errors described in 
the `xpath` issue. There is no
     `false` fallback.
   - Use in a filter (`WHERE xpath_boolean(xml, 'a/b')`) with NULL rows. NULL 
rows are filtered out.
   
   ### Differences across Spark versions
   
   None found beyond the shared ones listed in the `xpath` issue (4.0 moved to 
`RuntimeReplaceable` and added
   collation support).
   
   ### Implementation notes
   
   - Spark: `XPathBoolean` in 
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala`.
     It calls `XPathBooleanEvaluator`, which calls `UDFXPathUtil.evalBoolean`.
   - DataFusion core has no equivalent.
   - Put it in `crates/spark/src/function/xml/` on top of the shared evaluator 
from the `xpath` issue.
     The existing `crates/sqllogictest/test_files/xml/xpath_boolean.slt` is a 
commented-out stub that should be
     enabled and extended. Document the function with `#[user_doc]`.
   
   ---
   _This issue was generated by an LLM (Claude) from the Apache Spark source 
code and tests. Examples come from Spark's `ExpressionDescription` examples, 
test suites and `.sql.out` golden files, but please verify them against Spark 
before relying on them._


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to