coalchan opened a new pull request #35549: URL: https://github.com/apache/spark/pull/35549
### What changes were proposed in this pull request? Add a spark conf in order to just fetch partitions' name instead of fetching partitions' details. This can reduce requests on hive metastore. ### Why are the changes needed? 1. method `listPartitions` is order to get locations of partitions and compute custom partition locations(variable `customPartitionLocations`), but in most cases we do not have custom partition locations. 2. method `listPartitionNames` just fetchs partitions' name, it can reduce requests on hive metastore db. ### Does this PR introduce _any_ user-facing change? Yes, we should config "spark.sql.hasCustomPartitionLocations = false" ### How was this patch tested? 1. recompile InsertIntoHadoopFsRelationCommand.scala 2. update spark-sql_2.12-3.0.2.jar 3. run insert into cases -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
