[jira] [Updated] (HIVE-6336) hive 12 : local mode and datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
[
https://issues.apache.org/jira/browse/HIVE-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Savage updated HIVE-6336:
---
Priority: Minor (was: Blocker)
hive 12 : local mode and datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
--
Key: HIVE-6336
URL: https://issues.apache.org/jira/browse/HIVE-6336
Project: Hive
Issue Type: Wish
Components: HiveServer2
Affects Versions: 0.12.0
Environment: Hadoop 2.2 local derby Meatastore embedded
Reporter: Nigel Savage
Priority: Minor
Labels: HADOOP
There is an with hive 12 datanucleus incompatability which seems to have
invompatibility with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
The main question:
*IF hive 0.12.0 and datanucleus are compatabile, then what is the version of
datanucleus I should be using with Hive 12 and Hadoop 2.2?*
The error which Im getting (this blocks me from properly running hive queries
invoked from the test phase of a maven project)
*To reproduce*
I have hadoop and hive running as a pseudo cluster local mode and derby as
the metastore
I have the following environment variables
{noformat}
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
{noformat}
I have the RegexSerDe declared in the hive-site.xml
{noformat}
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
{noformat}
If I run with
{noformat}
datanucleus.version3.0.2/datanucleus.version
{noformat}
I get the following 1 exception only
'java.lang.ClassNotFoundException...org.datanucleus.store.types.backed.Ma'
HOWEVER, If I run with
{noformat}
datanucleus.version3.2.0-release/datanucleus.version
{noformat}
I get the following 1 exception exception only
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
EXPLANATION
The RegexSerDe class is picked up at run time but the datanucleus Map class
is not available, I have checked in the datanucleus-core 3.0.2 jar and it is
missing, Upgrading to the first datanucleus above 3.0.2 that includes the
Map class throws the ClassNotFoundException for RegexSerDe.
The earlier *3.0.2* datanucleus, code fails with the missing Map class but
the RegexSerDe class is found, then when I upgrade to the
3.2.0-release the Map class is found but for some unkown reason the code/Hive
no longer finds the RegexSerDe class
I started using the same datanucleus dependencies found in this hive pom
http://maven-repository.com/artifact/org.apache.hive/hive-metastore/0.12.0/pom
below are the dependencies my latest attempts to get a functioning pom
{noformat}
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase-server/artifactId
version0.96.0-hadoop2/version
/dependency
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase-client/artifactId
version0.96.0-hadoop2/version
/dependency
!-- misc --
dependency
groupIdorg.apache.commons/groupId
artifactIdcommons-lang3/artifactId
version3.1/version
/dependency
dependency
groupIdcom.google.guava/groupId
artifactIdguava/artifactId
version${guava.version}/version
/dependency
dependency
groupIdorg.apache.derby/groupId
artifactIdderby/artifactId
version${derby.version}/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-core/artifactId
version${datanucleus.version}/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-rdbms/artifactId
version${datanucleus-rdbms.version}/version
/dependency
dependency
groupIdjavax.jdo/groupId
artifactIdjdo-api/artifactId
version3.0.1/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-api-jdo/artifactId
version${datanucleus.jdo.version}/version
exclusions
exclusion
groupIdjavax.jdo/groupId
artifactIdjdo2-api/artifactId
/exclusion
exclusion
groupIdjunit/groupId
artifactIdjunit/artifactId
/exclusion
exclusion
[jira] [Commented] (HIVE-6336) Issue is hive 12 datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
[
https://issues.apache.org/jira/browse/HIVE-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13888781#comment-13888781
]
Nigel Savage commented on HIVE-6336:
Thanks andy for your response
you are correct re datanucleus-api-jdo exclusions, thanks for the link to the
pom, the reason the exclusions are there is because i used the
org.apache.hive:hive-metastore:0.12.0 pom as a guide, I assumed that this pom
would be correct, that is if the exclusions are there then you need them seeing
we use this class in our code and it is the main database that is jdo in terms
of transactions
http://maven-repository.com/artifact/org.apache.hive/hive-metastore/0.12.0/pom
basically I am getting the exceptions outlined and I am trying to resolve the
issue so I am trying diferent things, basically trying to harmonize the pom so
it uses the same datanucleus as the dependencies like the
org.apache.hive:hive-metastore:0.12.0
I notice you have a datanucleus email so if possible would you be willing to
comment on what versions of datanucleus we should be using ?
for hive 12 and hadoop 2.2.0
datanucleus.version3.2.2/datanucleus.version for
artifactIddatanucleus-core/artifactId
datanucleus-rdbms.version3.2.1/datanucleus-rdbms.version
artifactIddatanucleus-rdbms/artifactId
datanucleus.jdo.version3.2.1/datanucleus.jdo.version
artifactIddatanucleus-api-jdo/artifactId
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
--
Key: HIVE-6336
URL: https://issues.apache.org/jira/browse/HIVE-6336
Project: Hive
Issue Type: Wish
Components: HiveServer2
Affects Versions: 0.12.0
Environment: Hadoop 2.2 local derby Meatastore embedded
Reporter: Nigel Savage
Priority: Blocker
Labels: HADOOP
There is an with hive 12 datanucleus incompatability which seems to have
invompatibility with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
The main question:
*IF hive 0.12.0 and datanucleus are compatabile, then what is the version of
datanucleus I should be using with Hive 12 and Hadoop 2.2?*
The error which Im getting (this blocks me from properly running hive queries
invoked from the test phase of a maven project)
*To reproduce*
I have hadoop and hive running as a pseudo cluster local mode and derby as
the metastore
I have the following environment variables
{noformat}
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
{noformat}
I have the RegexSerDe declared in the hive-site.xml
{noformat}
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
{noformat}
If I run with
{noformat}
datanucleus.version3.0.2/datanucleus.version
{noformat}
I get the following 1 exception only
'java.lang.ClassNotFoundException...org.datanucleus.store.types.backed.Ma'
HOWEVER, If I run with
{noformat}
datanucleus.version3.2.0-release/datanucleus.version
{noformat}
I get the following 1 exception exception only
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
EXPLANATION
The RegexSerDe class is picked up at run time but the datanucleus Map class
is not available, I have checked in the datanucleus-core 3.0.2 jar and it is
missing, Upgrading to the first datanucleus above 3.0.2 that includes the
Map class throws the ClassNotFoundException for RegexSerDe.
The earlier *3.0.2* datanucleus, code fails with the missing Map class but
the RegexSerDe class is found, then when I upgrade to the
3.2.0-release the Map class is found but for some unkown reason the code/Hive
no longer finds the RegexSerDe class
I started using the same datanucleus dependencies found in this hive pom
http://maven-repository.com/artifact/org.apache.hive/hive-metastore/0.12.0/pom
below are the dependencies my latest attempts to get a functioning pom
{noformat}
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase-server/artifactId
version0.96.0-hadoop2/version
/dependency
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase-client/artifactId
version0.96.0-hadoop2/version
/dependency
!-- misc --
dependency
groupIdorg.apache.commons/groupId
artifactIdcommons-lang3/artifactId
version3.1/version
/dependency
dependency
groupIdcom.google.guava/groupId
artifactIdguava/artifactId
version${guava.version}/version
/dependency
[jira] [Updated] (HIVE-6336) Issue is hive 12 datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
[
https://issues.apache.org/jira/browse/HIVE-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Savage updated HIVE-6336:
---
Description:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in the hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following 1 exception only
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following 1 exception exception only
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not available, I have checked in the datanucleus-core 3.0.2 jar and it
is missing
upgrading to the first datanucleus above 3.0.2 that includse the Map class
throws the ClassNotFoundException for RegexSerDe
that is with the earlier 3.0.2 datanucleus the code fails with the missing Map
class but the RegexSerDe class is found, then when I upgrade to the
3.2.0-release the Map class is found but for some unkown reason the code/Hive
no longer finds the RegexSerDe class
if hive 0.12.0 and datanucleus are compatabile what is the version of
datanucleus I should be using with Hive 12 and Hadoop 2.2, thanks for your time
and effort
here are the dependencies from the pom
dependency
groupIdjavax.jdo/groupId
artifactIdjdo-api/artifactId
version3.0.1/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-core/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-rdbms/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-api-jdo/artifactId
version${datanucleus.version}/version
/dependency
!-- hadoop --
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-client/artifactId
version${hadoop.version}/version
/dependency
!-- hive --
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-common/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-serde/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-exec/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-jdbc/artifactId
version${hive.version}/version
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-contrib/artifactId
version${hive.version}/version
/dependency
dependency
groupIdcom.jolbox/groupId
artifactIdbonecp/artifactId
version${bonecp.version}/version
/dependency
was:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in the hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following 1 exception only
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following 1 exception exception only
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe
[jira] [Updated] (HIVE-6336) Issue is hive 12 datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
[
https://issues.apache.org/jira/browse/HIVE-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Savage updated HIVE-6336:
---
Description:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in the hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following 1 exception only
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following 1 exception exception only
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not available, I have checked in the datanucleus-core 3.0.2 jar and it
is missing
upgrading to the first datanucleus above 3.0.2 that includse the Map class
throws the ClassNotFoundException for RegexSerDe
that is with the earlier 3.0.2 datanucleus the code fails with the missing Map
class but the RegexSerDe class is found, then when I upgrade to the
3.2.0-release the Map class is found but for some unkown reason the code/Hive
no longer finds the RegexSerDe class
what is the version of datanucleus I should be using with Hive 12 and Hadoop
2.2, thanks for your time and effort
here are the dependencies from the pom
dependency
groupIdjavax.jdo/groupId
artifactIdjdo-api/artifactId
version3.0.1/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-core/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-rdbms/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-api-jdo/artifactId
version${datanucleus.version}/version
/dependency
!-- hadoop --
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-client/artifactId
version${hadoop.version}/version
/dependency
!-- hive --
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-common/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-serde/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-exec/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-jdbc/artifactId
version${hive.version}/version
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-contrib/artifactId
version${hive.version}/version
/dependency
dependency
groupIdcom.jolbox/groupId
artifactIdbonecp/artifactId
version${bonecp.version}/version
/dependency
was:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in thew hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following exception
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following exception
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not
[jira] [Updated] (HIVE-6336) Issue is hive 12 datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
[
https://issues.apache.org/jira/browse/HIVE-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Savage updated HIVE-6336:
---
Description:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in the hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following 1 exception only
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following 1 exception exception only
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not available, I have checked in the datanucleus-core 3.0.2 jar and it
is missing
upgrading to the first datanucleus above 3.0.2 that includse the Map class
throws the ClassNotFoundException for RegexSerDe
that is with the earlier 3.0.2 datanucleus the code fails with the missing Map
class but the RegexSerDe class is found, then when I upgrade to the
3.2.0-release the Map class is found but for some unkown reason the code/Hive
no longer finds the RegexSerDe class
if hive 0.12.0 and datanucleus are compatabile what is the version of
datanucleus I should be using with Hive 12 and Hadoop 2.2, thanks for your time
and effort
I started using the same datanucleus dependencies found in this hive pom
http://maven-repository.com/artifact/org.apache.hive/hive-metastore/0.12.0/pom
below is my latest attempts to get a functioning pom
here are the dependencies from the pom
dependency
groupIdjavax.jdo/groupId
artifactIdjdo-api/artifactId
version3.0.1/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-core/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-rdbms/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-api-jdo/artifactId
version${datanucleus.version}/version
/dependency
!-- hadoop --
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-client/artifactId
version${hadoop.version}/version
/dependency
!-- hive --
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-common/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-serde/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-exec/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-jdbc/artifactId
version${hive.version}/version
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-contrib/artifactId
version${hive.version}/version
/dependency
dependency
groupIdcom.jolbox/groupId
artifactIdbonecp/artifactId
version${bonecp.version}/version
/dependency
was:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in the hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following 1 exception only
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with
[jira] [Updated] (HIVE-6336) Issue is hive 12 datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
[
https://issues.apache.org/jira/browse/HIVE-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Savage updated HIVE-6336:
---
Description:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in the hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following 1 exception only
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following 1 exception exception only
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not available, I have checked in the datanucleus-core 3.0.2 jar and it
is missing
upgrading to the first datanucleus above 3.0.2 that includse the Map class
throws the ClassNotFoundException for RegexSerDe
that is with the earlier 3.0.2 datanucleus the code fails with the missing Map
class but the RegexSerDe class is found, then when I upgrade to the
3.2.0-release the Map class is found but for some unkown reason the code/Hive
no longer finds the RegexSerDe class
if hive 0.12.0 and datanucleus are compatabile what is the version of
datanucleus I should be using with Hive 12 and Hadoop 2.2, thanks for your time
and effort
I started using the same datanucleus dependencies found in this hive pom
http://maven-repository.com/artifact/org.apache.hive/hive-metastore/0.12.0/pom
below is my latest attempts to get a functioning pom
here are the dependencies from the pom
dependency
groupIdjavax.jdo/groupId
artifactIdjdo-api/artifactId
version3.0.1/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-core/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-rdbms/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-api-jdo/artifactId
version${datanucleus.version}/version
/dependency
!-- hadoop --
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-client/artifactId
version${hadoop.version}/version
/dependency
!-- hive --
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-common/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-serde/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-exec/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-jdbc/artifactId
version${hive.version}/version
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-contrib/artifactId
version${hive.version}/version
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-metastore/artifactId
version${hive.version}/version
/dependency
dependency
groupIdcom.jolbox/groupId
artifactIdbonecp/artifactId
version${bonecp.version}/version
/dependency
was:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in the hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with
[jira] [Updated] (HIVE-6336) Issue is hive 12 datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
[
https://issues.apache.org/jira/browse/HIVE-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Savage updated HIVE-6336:
---
Description:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in the hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following 1 exception only
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following 1 exception exception only
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not available, I have checked in the datanucleus-core 3.0.2 jar and it
is missing
upgrading to the first datanucleus above 3.0.2 that includse the Map class
throws the ClassNotFoundException for RegexSerDe
that is with the earlier 3.0.2 datanucleus the code fails with the missing Map
class but the RegexSerDe class is found, then when I upgrade to the
3.2.0-release the Map class is found but for some unkown reason the code/Hive
no longer finds the RegexSerDe class
if hive 0.12.0 and datanucleus are compatabile what is the version of
datanucleus I should be using with Hive 12 and Hadoop 2.2, thanks for your time
and effort
I started using the same datanucleus dependencies found in this hive pom
http://maven-repository.com/artifact/org.apache.hive/hive-metastore/0.12.0/pom
below are the dependencies my latest attempts to get a functioning pom
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase-server/artifactId
version0.96.0-hadoop2/version
/dependency
dependency
groupIdorg.apache.hbase/groupId
artifactIdhbase-client/artifactId
version0.96.0-hadoop2/version
/dependency
!-- misc --
dependency
groupIdorg.apache.commons/groupId
artifactIdcommons-lang3/artifactId
version3.1/version
/dependency
dependency
groupIdcom.google.guava/groupId
artifactIdguava/artifactId
version${guava.version}/version
/dependency
dependency
groupIdorg.apache.derby/groupId
artifactIdderby/artifactId
version${derby.version}/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-core/artifactId
version${datanucleus.version}/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-rdbms/artifactId
version${datanucleus-rdbms.version}/version
/dependency
dependency
groupIdjavax.jdo/groupId
artifactIdjdo-api/artifactId
version3.0.1/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-api-jdo/artifactId
version${datanucleus.jdo.version}/version
exclusions
exclusion
groupIdjavax.jdo/groupId
artifactIdjdo2-api/artifactId
/exclusion
exclusion
groupIdjunit/groupId
artifactIdjunit/artifactId
/exclusion
exclusion
groupIdlog4j/groupId
artifactIdlog4j/artifactId
/exclusion
/exclusions
/dependency
!-- hadoop --
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-client/artifactId
version${hadoop.version}/version
/dependency
!-- hive --
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-common/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-serde/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-exec/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
[jira] [Created] (HIVE-6336) Issue is hive 12 datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
Nigel Savage created HIVE-6336:
--
Summary: Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
Key: HIVE-6336
URL: https://issues.apache.org/jira/browse/HIVE-6336
Project: Hive
Issue Type: Wish
Components: HiveServer2
Affects Versions: 0.12.0
Environment: Hadoop 2.2 local derby Meatastore embedded
Reporter: Nigel Savage
Priority: Blocker
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in thew hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following exception
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following exception
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not available, I have checked in the
datanucleus-core 3.0.2 jar and it is missing
upgrading to the first datanucleus above 3.0.2 that incluse the Map class
throws the ClassNotFoundException for RegexSerDe
what is the version of datanucleus I should be using with Hive 12 and Hadoop
2.2, thanks for your time and effort
here are the dependencies from the pom
dependency
groupIdjavax.jdo/groupId
artifactIdjdo-api/artifactId
version3.0.1/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-core/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-rdbms/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-api-jdo/artifactId
version${datanucleus.version}/version
/dependency
!-- hadoop --
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-client/artifactId
version${hadoop.version}/version
/dependency
!-- hive --
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-common/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-serde/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-exec/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-jdbc/artifactId
version${hive.version}/version
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-contrib/artifactId
version${hive.version}/version
/dependency
dependency
groupIdcom.jolbox/groupId
artifactIdbonecp/artifactId
version${bonecp.version}/version
/dependency
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
[jira] [Updated] (HIVE-6336) Issue is hive 12 datanucleus incompatability with org.apache.hadoop.hive.contrib.serde2.RegexSerDe
[
https://issues.apache.org/jira/browse/HIVE-6336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nigel Savage updated HIVE-6336:
---
Description:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in thew hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following exception
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following exception
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not available, I have checked in the datanucleus-core 3.0.2 jar and it
is missing
upgrading to the first datanucleus above 3.0.2 that incluse the Map class
throws the ClassNotFoundException for RegexSerDe
what is the version of datanucleus I should be using with Hive 12 and Hadoop
2.2, thanks for your time and effort
here are the dependencies from the pom
dependency
groupIdjavax.jdo/groupId
artifactIdjdo-api/artifactId
version3.0.1/version
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-core/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-rdbms/artifactId
version${datanucleus.version}/version
scoperuntime/scope
/dependency
dependency
groupIdorg.datanucleus/groupId
artifactIddatanucleus-api-jdo/artifactId
version${datanucleus.version}/version
/dependency
!-- hadoop --
dependency
groupIdorg.apache.hadoop/groupId
artifactIdhadoop-client/artifactId
version${hadoop.version}/version
/dependency
!-- hive --
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-common/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-serde/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-exec/artifactId
version${hive.version}/version
scopeprovided/scope
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-jdbc/artifactId
version${hive.version}/version
/dependency
dependency
groupIdorg.apache.hive/groupId
artifactIdhive-contrib/artifactId
version${hive.version}/version
/dependency
dependency
groupIdcom.jolbox/groupId
artifactIdbonecp/artifactId
version${bonecp.version}/version
/dependency
was:
Issue is hive 12 datanucleus incompatability with
org.apache.hadoop.hive.contrib.serde2.RegexSerDe
I have hadoop and hive running as a pseudo cluster local mode and derby as the
metastore
I have the following environment variables
HADOOP_HOME=/home/ubu/hadoop
JAVA_HOME=/usr/lib/jvm/java-7-oracle
I have the RegexSerDe declared in thew hive-site.xml
property
namehive.aux.jars.path/name
valuefile:///home/ubu/hadoop/lib/hive-contrib-0.12.0.jar /value
descriptionThis JAR file available to all users for
alljobs/description
/property
If I run with datanucleus.version3.0.2/datanucleus.version I get the
following exception
java.lang.ClassNotFoundException:
org.datanucleus.store.types.backed.Map
If I run with datanucleus.version3.2.0-release/datanucleus.version I get
the following exception
java.lang.ClassNotFoundException:
org/apache/hadoop/hive/contrib/serde2/RegexSerDe
basically the RegexSerDe class is picked up at run time but the datanucleus Map
class is not available, I have checked in the
datanucleus-core 3.0.2 jar and it is missing
upgrading to the first datanucleus above 3.0.2 that incluse the Map class
throws the ClassNotFoundException for RegexSerDe
what is the version of datanucleus I should be using with Hive 12 and Hadoop
2.2, thanks
