dongjoon-hyun opened a new pull request, #525:
URL: https://github.com/apache/spark-connect-swift/pull/525
### What changes were proposed in this pull request?
This PR aims to support the following 12 XML functions in a new file,
`XmlFunctions.swift`.
**Conversion functions**
| Function | Overloads added | Since |
| -------- | --------------- | ----- |
| `from_xml` | `schema: String` (DDL), `schema: StructType`, `schema:
Column` | 4.0.0 |
| `to_xml` | | 4.0.0 |
| `schema_of_xml` | `xml: String`, `xml: Column` | 4.0.0 |
**XPath functions**
| Function | Result type | Since |
| -------- | ----------- | ----- |
| `xpath` | `ARRAY<STRING>` | 3.5.0 |
| `xpath_boolean` | `BOOLEAN` | 3.5.0 |
| `xpath_double`, `xpath_number` | `DOUBLE` | 3.5.0 |
| `xpath_float` | `FLOAT` | 3.5.0 |
| `xpath_short`, `xpath_int`, `xpath_long` | `SMALLINT`, `INT`, `BIGINT` |
3.5.0 |
| `xpath_string` | `STRING` | 3.5.0 |
Like the JSON and CSV functions added by SPARK-59216, `from_xml`, `to_xml`
and
`schema_of_xml` take an option map which Apache Spark encodes as a plain
trailing
`map(k1, v1, ...)` argument, so this PR reuses the existing internal
`fn(_ name:options:_ args:)` helper. When there is no option, no extra
argument is
appended at all.
For the `schema` argument, `from_xml` follows the upstream signature
(`sql/api/src/main/scala/org/apache/spark/sql/functions.scala`) which
accepts a
`StructType`, a DDL `String`, or a `Column` such as a `schema_of_xml`
result. Unlike
`from_json`, XML has no `MapType`/`ArrayType` form in the upstream API. The
`StructType`
overload delegates to the DDL string form via `StructType.toDDL`; the server
resolves it
through `ExprUtils.evalSchemaExpr` / `DataType.fromDDL`.
The `path` argument of the XPath functions is a `Column`, matching
`def xpath(xml: Column, path: Column)` upstream. Callers pass a literal with
`lit("a/b/text()")`. This mirrors the existing `get_json_object(_ col:
Column, _ path: String)`
precedent, which takes a `String` only because the upstream
`get_json_object` does.
### Why are the changes needed?
To improve the API coverage of the Swift Spark Connect client. Currently, no
XML function
is supported, so Swift users cannot parse, produce, or query XML strings
inside a `Column`
expression without falling back to `selectExpr`. Note that
`DataFrameReader.xml` and
`DataFrameWriter.xml` are already supported; this PR adds the column-level
counterparts.
### Does this PR introduce _any_ user-facing change?
No, this is a new feature which adds 12 new functions.
```swift
let df = try await spark.sql("SELECT 1 AS id, 'a' AS name")
let parsed = from_xml(to_xml(`struct`(col("id"), col("name"))), "id INT,
name STRING")
try await df.select(parsed.alias("parsed")).selectExpr("parsed.id",
"parsed.name").show()
try await
spark.range(1).select(schema_of_xml("<p><a>1</a><b>x</b></p>")).show()
// STRUCT<a: BIGINT, b: STRING>
try await spark.range(1).select(
xpath(lit("<a><b>b1</b><b>b2</b></a>"), lit("a/b/text()"))
).show()
// [b1, b2]
```
### How was this patch tested?
Pass the CIs with the newly added test suite, `XmlFunctionsTests`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
