qingfureal commented on PR #58875: URL: https://github.com/apache/spark/pull/58875#issuecomment-6048810261
Thanks @cloud-fan. Pushed 6b6aba9 addressing the three nits. On the producer boundary: agreed that this PR is lookup hardening, not the complete fix. The description used to point at SPARK-26174 for the read-side fix, but that ticket was closed as Incomplete in 2021, so nothing actually tracks it. I filed [SPARK-60070](https://issues.apache.org/jira/browse/SPARK-60070) for rejecting null keys in the Parquet/ORC/Avro/Hive readers and linked it here. I've updated the Scope section to match. I'd prefer to keep the reader change separate: it touches four readers and needs a decision on config and default behavior, while this PR stands on its own as a correctness fix for the lookup paths. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
