dongjoon-hyun opened a new pull request, #515:
URL: https://github.com/apache/spark-connect-swift/pull/515
### What changes were proposed in this pull request?
This PR adds the 9 missing predicate functions to the Swift Spark Connect
client, in a new
`PredicateFunctions.swift`:
- `equal_null`, `isnan`, `isnotnull`, `isnull`, `like`, `ilike`, `regexp`,
`regexp_like`, `rlike`
`like` and `ilike` are provided as two overloads each, with and without the
optional `escapeChar`
argument, mirroring the Scala API.
The grouping follows upstream: these are exactly the members of the
`Predicate Functions` section
in PySpark's `functions.rst`, and of the `@group predicate_funcs` annotation
in Scala's
`functions.scala` (whose only other member, `not`, is already covered here
by the `!` prefix
operator on `Column`). The new file also matches the existing per-category
layout of
`MathFunctions.swift`, `StringFunctions.swift` and
`CollectionFunctions.swift`.
Following the existing convention in this repository, all new functions
accept `Column` only;
no `String` overloads are added.
### Why are the changes needed?
These are among the most frequently used functions in the PySpark API and
have no Swift
equivalent today. `Column` already exposes `isNull()`, `isNotNull()`,
`like()`, `rlike()` and
`ilike()` as methods, but PySpark and the Scala API additionally provide
them as free functions,
which is what this PR fills in.
### Does this PR introduce _any_ user-facing change?
No because there are new public APIs. For example:
```swift
let df = try await spark.sql("SELECT * FROM VALUES ('Alice', 20), ('Bob',
NULL) T(name, age)")
try await df.filter(isnotnull(col("age"))).show()
try await df.filter(like(col("name"), lit("Al%"))).show()
```
### How was this patch tested?
Pass the CIs with the newly added `PredicateFunctionsTests`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
