sadhen commented on pull request #32332: URL: https://github.com/apache/spark/pull/32332#issuecomment-829105798
> BTW, can we avoid doing the verification twice (#32332 (comment)) too? Yes. The original implementation mixed schema verification with `prepare` to avoid performance loss. Let me make it clear in pseudo code to describe what I have done: ``` switch schema: + case DataType --> require prepare -> optional schema verification + case None/a list/tuple of names: --> require schema inferences and conversion + case StructType --> optional schema verification ``` ``` switch schema: + case DataType --> require prepare -> optional schema verification + case None/a list/tuple of names: --> require schema inferences and optional verification and conversion + case StructType --> optional schema verification ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
