[
https://issues.apache.org/jira/browse/HIVE-24169?focusedWorklogId=514206&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-514206
]
ASF GitHub Bot logged work on HIVE-24169:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 19/Nov/20 16:44
Start Date: 19/Nov/20 16:44
Worklog Time Spent: 10m
Work Description: saihemanth-cloudera commented on a change in pull
request #1503:
URL: https://github.com/apache/hive/pull/1503#discussion_r527035484
##########
File path: ql/src/java/org/apache/hadoop/hive/ql/util/ResourceDownloader.java
##########
@@ -71,42 +87,61 @@ public static boolean isFileUri(String value) {
}
}
- public List<URI> resolveAndDownload(String source, boolean convertToUnix)
+ public List<URI> resolveAndDownload(String source, boolean convertToUnix,
boolean useCache)
throws URISyntaxException, IOException {
- return resolveAndDownloadInternal(createURI(source), null, convertToUnix,
true);
+ return resolveAndDownloadInternal(createURI(source), null, convertToUnix,
true, useCache);
}
public List<URI> downloadExternal(URI source, String subDir, boolean
convertToUnix)
throws URISyntaxException, IOException {
- return resolveAndDownloadInternal(source, subDir, convertToUnix, false);
+ return resolveAndDownloadInternal(source, subDir, convertToUnix, false,
false);
+ }
+ public List<URI> downloadExternal(URI source, String subDir, boolean
convertToUnix, boolean useCache)
+ throws URISyntaxException, IOException {
+ return resolveAndDownloadInternal(source, subDir, convertToUnix, false,
useCache);
}
private List<URI> resolveAndDownloadInternal(URI source, String subDir,
- boolean convertToUnix, boolean isLocalAllowed) throws
URISyntaxException, IOException {
+ boolean convertToUnix, boolean isLocalAllowed, boolean useCache) throws
URISyntaxException, IOException {
switch (getURLType(source)) {
case FILE: return isLocalAllowed ? Collections.singletonList(source) :
null;
case IVY: return dependencyResolver.downloadDependencies(source);
case HDFS:
case OTHER:
- return Collections.singletonList(createURI(downloadResource(source,
subDir, convertToUnix)));
+ return Collections.singletonList(createURI(downloadResource(source,
subDir, convertToUnix, useCache)));
Review comment:
Caching for HDFS is more abstract optimization than what we are working
on. It can be tracked in a separate JIRA.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 514206)
Time Spent: 1h 10m (was: 1h)
> HiveServer2 UDF cache
> ---------------------
>
> Key: HIVE-24169
> URL: https://issues.apache.org/jira/browse/HIVE-24169
> Project: Hive
> Issue Type: Improvement
> Components: HiveServer2
> Affects Versions: 4.0.0
> Reporter: Sam An
> Assignee: Sam An
> Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> UDF is cache per session. This optional feature can help speed up UDF access
> in S3 scenario.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
