Myracle commented on PR #27577:
URL: https://github.com/apache/flink/pull/27577#issuecomment-6572505638
> @Myracle Thanks for applying the feedbacks. Could please do some research
if other vendors have a similar method? How does the function behaves there?
Thanks for the suggestion. I compared REGEXP_SPLIT with similar functions in
other engines:
- Trino/Presto `regexp_split(string, pattern)`: Java-style regex, keeps
leading/consecutive/trailing empty strings, e.g. regexp_split('1a 2b 14m',
'\s*[a-z]+\s*') -> [1, 2, 14, ''].
- Spark `split(str, regex[, limit])` (default limit -1) and Hive `split(str,
regex)`: Java regex, keep all empty strings; an empty regex splits into single
characters in Spark.
- DuckDB `regexp_split_to_array(str, regex[, options])` and ClickHouse
`splitByRegexp(regexp, s[, max_substrings])`: same results, e.g. ',a,,b,' ->
['', 'a', '', 'b', '']; an empty regex splits into characters, and an empty
input returns [''].
- PostgreSQL `regexp_split_to_array(str, pattern[, flags])` is the outlier:
it ignores zero-length matches at the start/end of the string or immediately
after a previous match.
Flink's REGEXP_SPLIT follows the majority
(Trino/Spark/Hive/DuckDB/ClickHouse) and the existing Flink SPLIT function.
Based on this I:
- made REGEXP_SPLIT('', '') return [''] (previously []), aligned with
DuckDB/ClickHouse,
- added tests for leading/consecutive/trailing matches and empty input,
- documented the empty-string, empty-regex and no-match semantics.
Invalid literal regexes are rejected at planning time; invalid non-literal
ones return NULL, consistent with the other Flink REGEXP_* functions. Optional
`limit`/`flags` arguments, as offered by some vendors, could be a follow-up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
