naveenp2708 commented on PR #57644: URL: https://github.com/apache/spark/pull/57644#issuecomment-5320964107
@szehon-ho thanks for the detailed review, this was really helpful. I went with aligning the read path like you suggested instead of hiding the column. There's one shared helper now that both materializeTable and VirtualTableInput.load use to build the target schema, so a downstream SELECT * in the same graph plans against the columns the target actually gets created with. On the incremental question, I added a test that creates the target and then does a second run through evolveTable and checks the schema doesn't change between runs. The existing schema-evolution suites already cover the evolve path more broadly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
