[jira] [Commented] (HIVE-3017) hive-exec jar, contains classes from other modules(hive-serde, hive-shims, hive-common etc) duplicating those classes in two jars
[
https://issues.apache.org/jira/browse/HIVE-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14077969#comment-14077969
]
murali namburi commented on HIVE-3017:
--
I suggest to have hive-ql jar which has related to hive-ql and hive-exec could
have ql and shade jars(jackson,thrift..etc)
In one of our app, we have latest jackson which has conflicts to hive-exec jar
(shade jackson classes) , our app doesn't work if we include hive-exec, also we
cannot remove from dependency as it is shaded jar. we have remove the jackson
class from the jar and manually pack it make them available to the app, so that
app is working as expected
hive-exec jar, contains classes from other modules(hive-serde, hive-shims,
hive-common etc) duplicating those classes in two jars
-
Key: HIVE-3017
URL: https://issues.apache.org/jira/browse/HIVE-3017
Project: Hive
Issue Type: Bug
Reporter: Jakob Homan
HIVE-2646 added the jars from hive-serde to the hive-exec class:
{noformat}
...
0 Wed May 09 20:56:30 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/
1971 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/ListTypeInfo.class
2396 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/MapTypeInfo.class
2788 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/PrimitiveTypeInfo.class
4408 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/StructTypeInfo.class
900 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.class
6576 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.class
1231 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$1.class
1239 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser$Token.class
7145 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser.class
14482 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.class
2594 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/UnionTypeInfo.class
144 Wed May 09 20:56:30 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/package-info.class
...{noformat}
Was this intentional? If so, the serde jar should be deprecated. If not, the
serde classes should be removed since this creates two sources of truth for
them and can cause other problems (see HCATALOG-407).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-3017) hive-exec jar, contains classes from other modules(hive-serde, hive-shims, hive-common etc) duplicating those classes in two jars
[
https://issues.apache.org/jira/browse/HIVE-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499514#comment-13499514
]
Edward Capriolo commented on HIVE-3017:
---
The core issue is that hive needs things to work. Example hive might use
CommonUtil.String.isEmpty() from 2.5. Well some user code might require
CommonUtil.String.isEmpty() from 2.6. The only full proof solution is that hive
will have to shade (or rename every class or) everything it uses so it will not
possibly conflict with anything in the world.
For example, I use hive+cassandra both have different versions of thrift at the
moment, both have different versions of antlr as well. The only answer is
upgrade one projects so the libs match or make a fat jar and rename all the
conflicts.
These options have come up and been discussed before.
hive-exec jar, contains classes from other modules(hive-serde, hive-shims,
hive-common etc) duplicating those classes in two jars
-
Key: HIVE-3017
URL: https://issues.apache.org/jira/browse/HIVE-3017
Project: Hive
Issue Type: Bug
Reporter: Jakob Homan
HIVE-2646 added the jars from hive-serde to the hive-exec class:
{noformat}
...
0 Wed May 09 20:56:30 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/
1971 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/ListTypeInfo.class
2396 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/MapTypeInfo.class
2788 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/PrimitiveTypeInfo.class
4408 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/StructTypeInfo.class
900 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.class
6576 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.class
1231 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$1.class
1239 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser$Token.class
7145 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser.class
14482 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.class
2594 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/UnionTypeInfo.class
144 Wed May 09 20:56:30 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/package-info.class
...{noformat}
Was this intentional? If so, the serde jar should be deprecated. If not, the
serde classes should be removed since this creates two sources of truth for
them and can cause other problems (see HCATALOG-407).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3017) hive-exec jar, contains classes from other modules(hive-serde, hive-shims, hive-common etc) duplicating those classes in two jars
[
https://issues.apache.org/jira/browse/HIVE-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499550#comment-13499550
]
Travis Crawford commented on HIVE-3017:
---
From my perspective I don't see an issue with {{hive-exec.jar}} being a fat
jar, since it really simplifies running Hive queries. With all dependencies
repackaged as a fat jar there's nothing else to add jar or put on your
classpath.
It is challenging to build tools on top of Hive though because of the fat jar
classpath surprises. The area I think could be improved is publishing jars of
each subproject independently, as well as hive-exec. For example, I think users
would get a lot of value from publishing {{hive-ql.jar}} for use by other
project that integrate with Hive.
Hive queries would continue to work as they do today, and people building tools
on top of Hive could also control their classpath. Thoughts?
hive-exec jar, contains classes from other modules(hive-serde, hive-shims,
hive-common etc) duplicating those classes in two jars
-
Key: HIVE-3017
URL: https://issues.apache.org/jira/browse/HIVE-3017
Project: Hive
Issue Type: Bug
Reporter: Jakob Homan
HIVE-2646 added the jars from hive-serde to the hive-exec class:
{noformat}
...
0 Wed May 09 20:56:30 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/
1971 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/ListTypeInfo.class
2396 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/MapTypeInfo.class
2788 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/PrimitiveTypeInfo.class
4408 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/StructTypeInfo.class
900 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.class
6576 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.class
1231 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$1.class
1239 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser$Token.class
7145 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser.class
14482 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.class
2594 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/UnionTypeInfo.class
144 Wed May 09 20:56:30 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/package-info.class
...{noformat}
Was this intentional? If so, the serde jar should be deprecated. If not, the
serde classes should be removed since this creates two sources of truth for
them and can cause other problems (see HCATALOG-407).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3017) hive-exec jar, contains classes from other modules(hive-serde, hive-shims, hive-common etc) duplicating those classes in two jars
[
https://issues.apache.org/jira/browse/HIVE-3017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13499300#comment-13499300
]
Arup Malakar commented on HIVE-3017:
I figured that hive-exec also contains all the classes from
hive-shims-0.10.0-SNAPSHOT.jar, hive-common-0.10.0-SNAPSHOT.jar etc.
I haven't checked what else it duplicates. Having the same class present in two
jars lead to confusion and is more error prone. In case we have both the jars
in classpath and they don't contain the same version of a class, wouldn't know
which class got used.
(Edited the title)
hive-exec jar, contains classes from other modules(hive-serde, hive-shims,
hive-common etc) duplicating those classes in two jars
-
Key: HIVE-3017
URL: https://issues.apache.org/jira/browse/HIVE-3017
Project: Hive
Issue Type: Bug
Reporter: Jakob Homan
HIVE-2646 added the jars from hive-serde to the hive-exec class:
{noformat}
...
0 Wed May 09 20:56:30 PDT 2012 org/apache/hadoop/hive/serde2/typeinfo/
1971 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/ListTypeInfo.class
2396 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/MapTypeInfo.class
2788 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/PrimitiveTypeInfo.class
4408 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/StructTypeInfo.class
900 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfo.class
6576 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoFactory.class
1231 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$1.class
1239 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser$Token.class
7145 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils$TypeInfoParser.class
14482 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/TypeInfoUtils.class
2594 Wed May 09 20:56:28 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/UnionTypeInfo.class
144 Wed May 09 20:56:30 PDT 2012
org/apache/hadoop/hive/serde2/typeinfo/package-info.class
...{noformat}
Was this intentional? If so, the serde jar should be deprecated. If not, the
serde classes should be removed since this creates two sources of truth for
them and can cause other problems (see HCATALOG-407).
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
