andygrove opened a new issue, #282:
URL: https://github.com/apache/datafusion-spark/issues/282
Part of #271 and apache/datafusion-comet#7300.
### What the function does
`xpath_boolean(xml, path)` evaluates an XPath 1.0 expression and converts
the result to `BOOLEAN` using the
XPath `boolean()` rules (`XPathConstants.BOOLEAN`). Both arguments are
strings (collation-aware from 4.0), and
`path` must be foldable. The result is always nullable. Available since
Spark 2.0.0. The expression is a
`Predicate`, so it can be used directly in `WHERE`.
Parsing, DTD and entity handling, namespaces, NULL and empty-string
handling, invalid-XML and invalid-path
errors, and lazy path compilation are shared with the rest of the family.
They are specified in the `xpath`
issue, and the implementation should reuse that evaluator. This issue covers
only the boolean conversion.
### Examples
```sql
SELECT xpath_boolean('<a><b>1</b></a>','a/b');
-- true
SELECT xpath_boolean('<a><b>b</b></a>', 'a/c');
-- false
SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "b"');
-- true
SELECT xpath_boolean('<a><b>b</b></a>', 'a/b = "c"');
-- false
SELECT xpath_boolean('<a><b>10</b></a>', 'a/b < 10');
-- false
SELECT xpath_boolean('<a><b>10</b></a>', 'a/b = 10');
-- true
```
These come from the `ExpressionDescription`, `XPathExpressionSuite` and the
Hive `udf_xpath_boolean` golden
files run by `HiveCompatibilitySuite`. In `UDFXPathUtilSuite`,
`a/b[1]/text()` is true and `a/b[4]` (no match)
is false.
### Edge cases to test
- Conversion rules from XPath 1.0 `boolean()`:
- a node-set is true if it is non-empty, whatever the text. `<b>false</b>`
with `a/b/text()` gives true;
- a number is false for `0`, `-0` and `NaN`, and true otherwise;
- a string is true if it is non-empty, so `string(a/c)` on a missing node
gives false;
- a boolean result is used as is.
- Comparisons between a node-set and a value are existential: `a/b = 2` is
true if any `b` equals 2. Also test
number versus string comparison (`a/b = "10"` versus `a/b = 10` with
`<b>10.0</b>`).
- `NULL` or `''` for either argument gives NULL. A non-literal `path` fails
with
`DATATYPE_MISMATCH.NON_FOLDABLE_INPUT`.
- Malformed XML and an invalid path raise the runtime errors described in
the `xpath` issue. There is no
`false` fallback.
- Use in a filter (`WHERE xpath_boolean(xml, 'a/b')`) with NULL rows. NULL
rows are filtered out.
### Differences across Spark versions
None found beyond the shared ones listed in the `xpath` issue (4.0 moved to
`RuntimeReplaceable` and added
collation support).
### Implementation notes
- Spark: `XPathBoolean` in
`sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/xml/xpath.scala`.
It calls `XPathBooleanEvaluator`, which calls `UDFXPathUtil.evalBoolean`.
- DataFusion core has no equivalent.
- Put it in `crates/spark/src/function/xml/` on top of the shared evaluator
from the `xpath` issue.
The existing `crates/sqllogictest/test_files/xml/xpath_boolean.slt` is a
commented-out stub that should be
enabled and extended. Document the function with `#[user_doc]`.
---
_This issue was generated by an LLM (Claude) from the Apache Spark source
code and tests. Examples come from Spark's `ExpressionDescription` examples,
test suites and `.sql.out` golden files, but please verify them against Spark
before relying on them._
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]