[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15930613#comment-15930613
]
Vaibhav Gumashta commented on HIVE-8678:
Removing fix version 1.2.2
> Pig fails to correctly load DATE fields using HCatalog
> --
>
> Key: HIVE-8678
> URL: https://issues.apache.org/jira/browse/HIVE-8678
> Project: Hive
> Issue Type: Bug
> Components: HCatalog
>Affects Versions: 0.13.1
>Reporter: Michael McLellan
>Assignee: Sushanth Sowmyan
>
> Using:
> Hadoop 2.5.0-cdh5.2.0
> Pig 0.12.0-cdh5.2.0
> Hive 0.13.1-cdh5.2.0
> When using pig -useHCatalog to load a Hive table that has a DATE field, when
> trying to DUMP the field, the following error occurs:
> {code}
> 2014-10-30 22:58:05,469 [main] ERROR
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
> org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
> converting read value to tuple
> at
> org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
> at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
> at
> org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
> at
> org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
> Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
> java.sql.Date
> at
> org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
> at
> org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
> at
> org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
> at
> org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
> 2014-10-30 22:58:05,469 [main] ERROR
> org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
> read value to tuple
> {code}
> It seems to be occuring here:
> https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
> and that it should be:
> {code}Date d = Date.valueOf(o);{code}
> instead of
> {code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14717588#comment-14717588
]
Sushanth Sowmyan commented on HIVE-8678:
Closed as Cannot reproduce
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Fix For: 1.2.2
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14652435#comment-14652435
]
Sushanth Sowmyan commented on HIVE-8678:
On digging further, my issues in the 0.13.1 vm were a different issue from the
one reported here, and was related to pig's jodatime being an older library
than needed. It was solved by adding a joda-time-2.1.jar to PIG_CLASSPATH, and
setting PIG_USER_CLASSPATH_FIRST so that it picked it up first. At this point,
I am not able to reproduce this issue with 0.13.1 either.
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14642976#comment-14642976
]
Sushanth Sowmyan commented on HIVE-8678:
Actually, after finding a 0.13.1 VM, I'm able to reproduce this. In 1.2,
however, I am not. So something changed along the way to fix this. I can dig
further to figure out what the problem was that made it not work in 0.13.1.
In addition, this problem exists with both orc and text formats in 0.13.1.
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14639129#comment-14639129
]
Sushanth Sowmyan commented on HIVE-8678:
No worries - sorry I didn't try experimenting on this earlier. :)
Hopefully this means that this bug was squished in the meanwhile somewhere
between 0.13.1 and 1.2 and does not exist any longer.
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637886#comment-14637886
]
Sushanth Sowmyan commented on HIVE-8678:
I'm currently unable to reproduce this issue on hive-1.2 and pig-0.14.0, where
I get the following:
In hive:
{noformat}
hive create table tdate(a string, b date) stored as orc;
OK
Time taken: 0.151 seconds
hive create table tsource(a string, b string) stored as orc;
OK
Time taken: 0.057 seconds
hive insert into table tsource values (abc, 2015-02-28);
...
OK
Time taken: 19.875 seconds
hive select * from tsource;
OK
abc 2015-02-28
Time taken: 0.143 seconds, Fetched: 1 row(s)
hive select a, cast(b as date) from tsource;
OK
abc 2015-02-28
Time taken: 0.092 seconds, Fetched: 1 row(s)
hive insert into table tdate select a, cast(b as date) from tsource;
...
OK
Time taken: 20.672 seconds
hive select * from tdate;
OK
abc 2015-02-28
Time taken: 0.051 seconds, Fetched: 1 row(s)
hive describe tdate;
OK
a string
b date
Time taken: 0.293 seconds, Fetched: 2 row(s)
{noformat}
In pig:
{noformat}
grunt A = load 'tdate' using org.apache.hive.hcatalog.pig.HCatLoader();
grunt describe A;
2015-07-22 15:42:26,367 [main] INFO
org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is
deprecated. Instead, use fs.defaultFS
A: {a: chararray,b: datetime}
grunt dump A;
...
(abc,2015-02-28T00:00:00.000-08:00)
grunt
{noformat}
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14637888#comment-14637888
]
Sushanth Sowmyan commented on HIVE-8678:
Also, unit tests exist since the introduction of DATE capability that have
tested date interop between hive and pig through HCatalog, and that still
succeeds for me when I try running them on hive 0.13.1.
Could you please show me what hive commands and pig commands you're running to
recreate this issue?
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14638012#comment-14638012
]
Michael McLellan commented on HIVE-8678:
I no longer have access to the system where this was an issue. This was about 9
months ago and we ended up working around it by just using Strings.
I don't remember anything more - sorry I didn't write down instructions to
reproduce when I created this.
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635548#comment-14635548
]
Sushanth Sowmyan commented on HIVE-8678:
What storage format are you using for the table in question? (i.e. is it Text,
RCFile, ORC, something else?)
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14635822#comment-14635822
]
Michael McLellan commented on HIVE-8678:
ORC
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-8678) Pig fails to correctly load DATE fields using HCatalog
[
https://issues.apache.org/jira/browse/HIVE-8678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14631906#comment-14631906
]
Sushanth Sowmyan commented on HIVE-8678:
Something seems weird here - looking at the code, it looks like the current
code, where it simply casts to Date should be the right way to do this, since
it should have called .getPrimitiveJavaObject() on the PrimitiveObjectInspector
to get this object, and DateObjectInspector.getPrimitiveJavaObject() should
have returned a Date. However, clearly, from your stack trace, you're getting a
string. I'll dig into this and update as I find more.
Pig fails to correctly load DATE fields using HCatalog
--
Key: HIVE-8678
URL: https://issues.apache.org/jira/browse/HIVE-8678
Project: Hive
Issue Type: Bug
Components: HCatalog
Affects Versions: 0.13.1
Reporter: Michael McLellan
Assignee: Sushanth Sowmyan
Using:
Hadoop 2.5.0-cdh5.2.0
Pig 0.12.0-cdh5.2.0
Hive 0.13.1-cdh5.2.0
When using pig -useHCatalog to load a Hive table that has a DATE field, when
trying to DUMP the field, the following error occurs:
{code}
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher -
org.apache.pig.backend.executionengine.ExecException: ERROR 6018: Error
converting read value to tuple
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:76)
at org.apache.hive.hcatalog.pig.HCatLoader.getNext(HCatLoader.java:58)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:211)
at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:553)
at
org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1614)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163)
Caused by: java.lang.ClassCastException: java.lang.String cannot be cast to
java.sql.Date
at
org.apache.hive.hcatalog.pig.PigHCatUtil.extractPigObject(PigHCatUtil.java:420)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:457)
at
org.apache.hive.hcatalog.pig.PigHCatUtil.transformToTuple(PigHCatUtil.java:375)
at
org.apache.hive.hcatalog.pig.HCatBaseLoader.getNext(HCatBaseLoader.java:64)
2014-10-30 22:58:05,469 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 6018: Error converting
read value to tuple
{code}
It seems to be occuring here:
https://github.com/apache/hive/blob/trunk/hcatalog/hcatalog-pig-adapter/src/main/java/org/apache/hive/hcatalog/pig/PigHCatUtil.java#L433
and that it should be:
{code}Date d = Date.valueOf(o);{code}
instead of
{code}Date d = (Date) o;{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
