ashutoshc commented on a change in pull request #552: Hive 21279
URL: https://github.com/apache/hive/pull/552#discussion_r260600921
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java
##########
@@ -204,6 +219,16 @@ public void compile(final ParseContext pCtx,
fetch.setIsUsingThriftJDBCBinarySerDe(false);
}
+ Collection<Operator<? extends OperatorDesc>> tableScanOps =
+ Lists.<Operator<?>>newArrayList(pCtx.getTopOps().values());
+ Set<FileSinkOperator> fsOps = OperatorUtils.findOperators(tableScanOps,
FileSinkOperator.class);
+ if(fsOps != null && fsOps.size() == 1) {
+ FileSinkOperator op = fsOps.iterator().next();
+ Set<Path> filesToFetch = new HashSet<>();
+ op.getConf().setFilesToFetch(filesToFetch);
Review comment:
I am not sure how this works. filesToFetch field is populated in tasks.
Having this same reference won't help, because plan (and FilesinkDesc) is sent
to cluster from HS2, but any changes in plan won't reflect here after job is
finished on cluster.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
