dengzhhu653 commented on code in PR #4520:
URL: https://github.com/apache/hive/pull/4520#discussion_r1299352282
##########
ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java:
##########
@@ -229,11 +249,20 @@ public List<Task<?>> getTasks(HiveConf conf, Object
objCtx) {
private void generateActualTasks(HiveConf conf, List<Task<?>> resTsks,
long trgtSize, long avgConditionSize, Task<?> mvTask,
Task<?> mrTask, Task<?> mrAndMvTask, Path dirPath,
- FileSystem inpFs, ConditionalResolverMergeFilesCtx ctx, MapWork work,
int dpLbLevel)
+ FileSystem inpFs, ConditionalResolverMergeFilesCtx ctx, MapWork work,
int dpLbLevel,
+ boolean manifestFilePresent)
throws IOException {
DynamicPartitionCtx dpCtx = ctx.getDPCtx();
- // get list of dynamic partitions
- List<FileStatus> statusList = HiveStatsUtils.getFileStatusRecurse(dirPath,
dpLbLevel, inpFs);
+ List<FileStatus> statusList;
+ if (manifestFilePresent) {
+ // Get the list of files from manifest file.
+ List<FileStatus> fileStatuses = getManifestFilePaths(conf, dirPath);
+ // Setup the work to include all the files present in the manifest.
+ setupWorkWhenUsingManifestFile(work, fileStatuses, dirPath);
+ statusList =
HiveStatsUtils.getFileStatusRecurse(Utilities.toTempPath(dirPath), dpLbLevel,
inpFs);
Review Comment:
Why do we need to `HiveStatsUtils.getFileStatusRecurse` again?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
