saihemanth-cloudera commented on a change in pull request #1503:
URL: https://github.com/apache/hive/pull/1503#discussion_r527035484
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/util/ResourceDownloader.java
##########
@@ -71,42 +87,61 @@ public static boolean isFileUri(String value) {
}
}
- public List<URI> resolveAndDownload(String source, boolean convertToUnix)
+ public List<URI> resolveAndDownload(String source, boolean convertToUnix,
boolean useCache)
throws URISyntaxException, IOException {
- return resolveAndDownloadInternal(createURI(source), null, convertToUnix,
true);
+ return resolveAndDownloadInternal(createURI(source), null, convertToUnix,
true, useCache);
}
public List<URI> downloadExternal(URI source, String subDir, boolean
convertToUnix)
throws URISyntaxException, IOException {
- return resolveAndDownloadInternal(source, subDir, convertToUnix, false);
+ return resolveAndDownloadInternal(source, subDir, convertToUnix, false,
false);
+ }
+ public List<URI> downloadExternal(URI source, String subDir, boolean
convertToUnix, boolean useCache)
+ throws URISyntaxException, IOException {
+ return resolveAndDownloadInternal(source, subDir, convertToUnix, false,
useCache);
}
private List<URI> resolveAndDownloadInternal(URI source, String subDir,
- boolean convertToUnix, boolean isLocalAllowed) throws
URISyntaxException, IOException {
+ boolean convertToUnix, boolean isLocalAllowed, boolean useCache) throws
URISyntaxException, IOException {
switch (getURLType(source)) {
case FILE: return isLocalAllowed ? Collections.singletonList(source) :
null;
case IVY: return dependencyResolver.downloadDependencies(source);
case HDFS:
case OTHER:
- return Collections.singletonList(createURI(downloadResource(source,
subDir, convertToUnix)));
+ return Collections.singletonList(createURI(downloadResource(source,
subDir, convertToUnix, useCache)));
Review comment:
Caching for HDFS is more abstract optimization than what we are working
on. It can be tracked in a separate JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For additional commands, e-mail: gitbox-h...@hive.apache.org
