[jira] [Commented] (HIVE-9664) Hive "add jar" command should be able to download and add jars from a repository
[
https://issues.apache.org/jira/browse/HIVE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317840#comment-14317840
]
Lefty Leverenz commented on HIVE-9664:
--
Here's where you can find information about Hive's compile syntax and Groovy:
* [Hive Commands |
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Commands]:
{{compile `` AS GROOVY NAMED }}
* [Nov. 2013 Hive Contributors Meetup Presentations -- Using Dynamic
Compilation with Hive |
https://cwiki.apache.org/confluence/download/attachments/27362054/HiveContrib-Nov13-groovy_plus_hive.pptx?version=1&modificationDate=1385171856000&api=v2]
Note: The last slide in the presentation has a typo. Instead of HIVE-520 see
HIVE-5250 and its subtask HIVE-5252 (Add ql syntax for inline java code
creation).
> Hive "add jar" command should be able to download and add jars from a
> repository
>
>
> Key: HIVE-9664
> URL: https://issues.apache.org/jira/browse/HIVE-9664
> Project: Hive
> Issue Type: Improvement
>Reporter: Anant Nag
> Labels: hive
>
> Currently Hive's "add jar" command takes a local path to the dependency jar.
> This clutters the local file-system as users may forget to remove this jar
> later
> It would be nice if Hive supported a Gradle like notation to download the jar
> from a repository.
> Example: add jar org:module:version
>
> It should also be backward compatible and should take jar from the local
> file-system as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9664) Hive "add jar" command should be able to download and add jars from a repository
[ https://issues.apache.org/jira/browse/HIVE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317581#comment-14317581 ] Edward Capriolo commented on HIVE-9664: --- Just so you know. You can use a groovy for writing UDFs currently, and groovy has some @GRAB support for chasing down dependencies. Check hives COMPILE syntax > Hive "add jar" command should be able to download and add jars from a > repository > > > Key: HIVE-9664 > URL: https://issues.apache.org/jira/browse/HIVE-9664 > Project: Hive > Issue Type: Improvement >Reporter: Anant Nag > Labels: hive > > Currently Hive's "add jar" command takes a local path to the dependency jar. > This clutters the local file-system as users may forget to remove this jar > later > It would be nice if Hive supported a Gradle like notation to download the jar > from a repository. > Example: add jar org:module:version > > It should also be backward compatible and should take jar from the local > file-system as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-9664) Hive "add jar" command should be able to download and add jars from a repository
[
https://issues.apache.org/jira/browse/HIVE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317574#comment-14317574
]
Anant Nag commented on HIVE-9664:
-
Using the above syntax will make parsing much simpler with the help of URI.
syntax: *ivy://org:module:version?e=exclusion_list*
URI can easily parse the syntax.
Ex:
{code:java}
URI uri = new URI("gradle://group:module:version?exclude=group:module1");
System.out.println("uri.getScheme()); // prints gradle
System.out.println("uri.getAuthority()); // prints group:module:version
System.out.println("uri.getQuery()); // prints exclude=group:module1
{code}
> Hive "add jar" command should be able to download and add jars from a
> repository
>
>
> Key: HIVE-9664
> URL: https://issues.apache.org/jira/browse/HIVE-9664
> Project: Hive
> Issue Type: Improvement
>Reporter: Anant Nag
> Labels: hive
>
> Currently Hive's "add jar" command takes a local path to the dependency jar.
> This clutters the local file-system as users may forget to remove this jar
> later
> It would be nice if Hive supported a Gradle like notation to download the jar
> from a repository.
> Example: add jar org:module:version
>
> It should also be backward compatible and should take jar from the local
> file-system as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-9664) Hive "add jar" command should be able to download and add jars from a repository
[
https://issues.apache.org/jira/browse/HIVE-9664?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14317573#comment-14317573
]
Anant Nag commented on HIVE-9664:
-
The gradle like notation can also be extended to other commands such as ADD
FILE and ADD ARCHIVE. The LIST and DELETE commands should continue to work
and produce sensible output.
The following syntax for the command is proposed:
{code}add [FILE|JAR|ARCHIVE]
*{code}
{code}add [FILE|JAR|ARCHIVE]
*{code}
{code}add [FILE|JAR|ARCHIVE]
*{code}
{code}add [FILE|JAR|ARCHIVE]
*{code}
{code}add [FILE|JAR|ARCHIVE] *{code}
The motivation for the above syntax is being able to differentiate how a
jar(file) is obtained. Having something like helps
us to identify that the file is being downloaded from the artifact such as
maven repository whereas helps us to identify that the
jar is being added from the local system.
We're assuming that the jar can be added by either of the following methods:
1. A jar can be added from the artifactory( like maven repository). In such a
case, transitive dependencies( if enabled) should also be downloaded and added
to the classpath. If some dependencies have to be excluded then those should be
mentioned in the command itself.
Command:
{code}
add jar
*
{code}
exclude=org1:module1 denotes that these dependencies should be excluded while
satisfying transitive dependencies.
2) A http or https url of the jar can directly be provided. In such a case, the
jar will be downloaded and added to the classpath. This might be useful in
cases where a single jar is required which is not present in the artifactory
but a download link is available.
Command:
{code}
add jar
add jar
{code}
3) The jar can be added from the local filesystem. This is basically what hive
already supports with the add command. The file is already there in the
filesystem and it is just added to the classpath.
Command:
{code}
add jar jarname
add jar file:///tmp/sample.jar
{code}
4) The jar can be added from the hdfs file system.
Command:
{code}
add jar hdfs:/user/abc/dwh-udf.jar;
add jar hdfs:///user/abc/dwh-udf.jar;
{code}
Having syntax like the commands above helps us to clearly distinct the location
of the jar and also the method used to obtain the jar. Please mention your
queries and thoughts in the comments.
> Hive "add jar" command should be able to download and add jars from a
> repository
>
>
> Key: HIVE-9664
> URL: https://issues.apache.org/jira/browse/HIVE-9664
> Project: Hive
> Issue Type: Improvement
>Reporter: Anant Nag
> Labels: hive
>
> Currently Hive's "add jar" command takes a local path to the dependency jar.
> This clutters the local file-system as users may forget to remove this jar
> later
> It would be nice if Hive supported a Gradle like notation to download the jar
> from a repository.
> Example: add jar org:module:version
>
> It should also be backward compatible and should take jar from the local
> file-system as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
