dongjoon-hyun opened a new pull request, #457:
URL: https://github.com/apache/spark-connect-swift/pull/457
### What changes were proposed in this pull request?
This PR aims to support predicate, string, and extraction methods in
`Column`.
- Predicates: `isNull`, `isNotNull`, `isin`, `between`, `eqNullSafe`
- String methods: `like`, `rlike`, `ilike`, `contains`, `startsWith`,
`endsWith`, `substr`
- Extraction: `getItem`, `getField`
The function name mappings follow the Scala client's `Column.scala`:
`isnull`/`isnotnull`, `in`, `<=>`, `like`/`rlike`/`ilike`,
`contains`/`startswith`/`endswith`, `substr`. `between` is composed as `(self
>= lower) && (self <= upper)` like the Scala client, and `getItem`/`getField`
use `Expression.UnresolvedExtractValue` like the Scala client's
`apply(extraction)`. Methods taking operands accept both `Column` and
`SparkLiteral` values.
### Why are the changes needed?
To allow Swift users to build filter and projection expressions
idiomatically without falling back to SQL strings.
```swift
df.filter(col("name").isNotNull() && col("age").between(20, 30))
df.filter(col("name").like("Al%"))
df.select(col("arr").getItem(0), col("s").getField("a"))
```
### Does this PR introduce _any_ user-facing change?
No. This is a new API addition.
### How was this patch tested?
Pass the CIs with newly added unit tests (`predicates`, `between`,
`stringMethods`, `extraction`) and integration tests (`filterWithPredicates`,
`filterWithStringMethods`, `selectWithSubstrAndExtraction`) in
`FunctionsTests`, verified against a local Apache Spark 4.2.0 Connect server.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Fable 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
