panbingkun commented on code in PR #46880:
URL: https://github.com/apache/spark/pull/46880#discussion_r1746592433
##########
python/pyspark/sql/functions/builtin.py:
##########
@@ -11241,13 +11241,18 @@ def sentences(
) -> Column:
"""
Splits a string into arrays of sentences, where each sentence is an array
of words.
- The 'language' and 'country' arguments are optional, and if omitted, the
default locale is used.
+ The `language` and `country` arguments are optional,
+ if they are both omitted, the default locale (the language is 'en' and the
country is 'US')
+ is used, if the `country` is omitted, the default `country` ('') is used.
Review Comment:
> It's surprising that we don't return null for this case
Yes, the original expression `Sentences` did not inherit `NullIntolerant`.
If we inherit it now, it may make this `breaking change`.
https://github.com/apache/spark/blob/9269a0bfed56429e999269dfdfd89aefcb1b7261/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala#L3347-L3351
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
