cdegroc opened a new pull request #35139: URL: https://github.com/apache/spark/pull/35139
### What changes were proposed in this pull request? Add a unit test demonstrating the regression on `DataFrame.joinWith`. Update `ExpressionEncoder` to make the test pass (equivalent to reverting the [commit](https://github.com/apache/spark/commit/cd92f25be5a221e0d4618925f7bc9dfd3bb8cb59) that introduced the issue). ### Why are the changes needed? Doing an outer-join using joinWith on DataFrames used to return missing values as null in Spark 2.4.8, but returns them as Rows with null values in Spark 3.0.0+. The regression has been introduced in [this commit](https://github.com/apache/spark/commit/cd92f25be5a221e0d4618925f7bc9dfd3bb8cb59). ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? A unit test was added. Ran unit tests for the `sql-core` and `sql-catalyst` submodules with `./build/mvn clean package -pl sql/core,cql/catalyst` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
