koertkuipers opened a new pull request, #41903: URL: https://github.com/apache/spark/pull/41903
### What changes were proposed in this pull request? To correct a bug introduced in SPARK-37829 where a Scala Option can become null instead of None in typed operations (e.g. Dataset, Aggregator). ### Why are the changes needed? When we use Option we expect to get back Some(...) or None, but never null. ### Does this PR introduce _any_ user-facing change? Yes it restores the behavior where no nulls show up for Option[_] ### How was this patch tested? Added a test for Aggregator where the output without this PR has a null where it shouldn't.l -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
