Github user jinxing64 commented on a diff in the pull request:
https://github.com/apache/spark/pull/19868#discussion_r180733361
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -176,12 +176,13 @@ class HadoopTableReader(
val matches = fs.globStatus(pathPattern)
matches.foreach(fileStatus => existPathSet +=
fileStatus.getPath.toString)
}
- // convert /demo/data/year/month/day to /demo/data/*/*/*/
+ // convert /demo/data/year/month/day to
/demo/data/year/month/*/
--- End diff --
@cloud-fan
Thanks a lot for review;
Like I mentioned in description, "/demo/data/year/month/day" will not be
converted to "/demo/data/*/*/*/", instead it's converted to
"/demo/data/year/month/*/" as a path pattern. When this path pattern is passed
to `updateExistPathSetByPathPattern`, less matched paths will be returned.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
