manbuyun opened a new pull request #30410: URL: https://github.com/apache/spark/pull/30410
### What changes were proposed in this pull request? Insert overwrite same table can overwrite a path that is also being read under dynamic partition overwrite ### Why are the changes needed? Currently, Insert overwrite cannot overwrite a path that is also being read under dynamic partition overwrite. In the class DataSourceAnalysis, DDLUtils.verifyNotReadPath is called to determine whether it can be overwrite, but in the InsertIntoHadoopFsRelationCommand.dynamicPartitionOverwrite method, we need to consider the scenario where staticPartitions.size equals partitionColumns.length, which should also be allowed to overwrite path this is also being read. Consider the following statement: CREATE TABLE insertTable(i int, part1 int, part2 int) USING PARQUET PARTITIONED BY (part1, part2); INSERT OVERWRITE TABLE insertTable PARTITION(part1=1, part2=1) SELECT i + 1 FROM insertTable; ### Does this PR introduce _any_ user-facing change? no ### How was this patch tested? Add test in InsertSuite, start line:307 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
