HeartSaVioR opened a new pull request, #38115:
URL: https://github.com/apache/spark/pull/38115
### What changes were proposed in this pull request?
This PR fixes a bug which occurs NPE when the input schema of
applyInPandasWithState has "non-nullable" column(s).
This PR also leaves a code comment explaining the fix. Quoting:
```
// See processTimedOutState: we create a row which contains the actual
values for grouping key,
// but all nulls for value side by intention. This technically changes the
schema of input to
// be "nullable", hence the schema information and the internal projection
of row should take
// this into consideration. Strictly saying, it's not applied to the part
of grouping key, but
// it doesn't hurt much to we apply the same for grouping key as well.
```
### Why are the changes needed?
There's a bug which we didn't take the non-null columns into account. This
PR fixes the bug.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
New UT. The new test case failed with NPE without the fix, and succeeded
with the fix.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
