[
https://issues.apache.org/jira/browse/PHOENIX-3240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15720932#comment-15720932
]
lordk edited comment on PHOENIX-3240 at 12/5/16 1:24 AM:
---------------------------------------------------------
I have a test under centos7.2 JDK1.8.0_112 apache-phoenix-4.9.0-HBase-1.2
pig-0.16.0. I still got same error :
Error: java.io.IOException: java.lang.ClassCastException:
org.joda.time.DateTime cannot be cast to
org.apache.phoenix.shaded.org.joda.time.DateTime at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.StoreFuncDecorator.putNext(StoreFuncDecorator.java:83)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:144)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:97)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:658)
at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:282)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:275)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:65)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:422) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by:
java.lang.ClassCastException: org.joda.time.DateTime cannot be cast to
org.apache.phoenix.shaded.org.joda.time.DateTime at
org.apache.phoenix.pig.util.TypeUtil.castPigTypeToPhoenix(TypeUtil.java:199) at
org.apache.phoenix.pig.PhoenixHBaseStorage.putNext(PhoenixHBaseStorage.java:189)
at
my sourcedata :
AA,CC,3.5,EE,GG,2015-08-30
BB,DD,3.5,FF,HH,2015-08-05
pig script:
AA = load '/sourcedata/farm-prod/complete3.txt' USING PigStorage(',') AS(
name:chararray,
address:chararray,
price:double,
unit:chararray,
info_source:chararray,
date:datetime
);
dump AA;
out put :
(AA,CC,3.5,EE,GG,2015-08-30T00:00:00.000-04:00)
(BB,DD,3.5,FF,HH,2015-08-05T00:00:00.000-04:00)
STORE AA into 'hbase://FARM_PRODUCT_PRICE' using
org.apache.phoenix.pig.PhoenixHBaseStorage('dev7,dev8,dev9','-batchSize 5000');
my phoenix table :
create table FARM_PRODUCT_PRICE
(
name varchar(30),
address varchar(80),
price double,
unit varchar(20),
info_source varchar(80),
date date not null CONSTRAINT pk PRIMARY KEY (name,address,date)
)VERSIONS=1,SALT_BUCKETS=3,COMPRESSION='snappy';
was (Author: lordk):
I have a test under centos7.2 JDK1.8.0_112 apache-phoenix-4.9.0-HBase-1.2
pig-0.16.0. I still got same error :
Error: java.io.IOException: java.lang.ClassCastException:
org.joda.time.DateTime cannot be cast to
org.apache.phoenix.shaded.org.joda.time.DateTime at
my sourcedata :
AA,CC,3.5,EE,GG,2015-08-30
BB,DD,3.5,FF,HH,2015-08-05
pig script:
AA = load '/sourcedata/farm-prod/complete3.txt' USING PigStorage(',') AS(
name:chararray,
address:chararray,
price:double,
unit:chararray,
info_source:chararray,
date:datetime
);
dump AA;
out put :
(AA,CC,3.5,EE,GG,2015-08-30T00:00:00.000-04:00)
(BB,DD,3.5,FF,HH,2015-08-05T00:00:00.000-04:00)
STORE AA into 'hbase://FARM_PRODUCT_PRICE' using
org.apache.phoenix.pig.PhoenixHBaseStorage('dev7,dev8,dev9','-batchSize 5000');
my phoenix table :
create table FARM_PRODUCT_PRICE
(
name varchar(30),
address varchar(80),
price double,
unit varchar(20),
info_source varchar(80),
date date not null CONSTRAINT pk PRIMARY KEY (name,address,date)
)VERSIONS=1,SALT_BUCKETS=3,COMPRESSION='snappy';
> ClassCastException from Pig loader
> ----------------------------------
>
> Key: PHOENIX-3240
> URL: https://issues.apache.org/jira/browse/PHOENIX-3240
> Project: Phoenix
> Issue Type: Bug
> Affects Versions: 4.8.0
> Reporter: YoungWoo Kim
> Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.2
>
>
> I'm loading data from Hive table to Phoenix table in using Phoenix-Pig
> integration. my pig script looks like following:
> {code}
> T = LOAD 'mydb.$TBL' USING org.apache.hive.hcatalog.pig.HCatLoader();
> STORE T into 'hbase://MYSCHEMA.$TBL' using
> org.apache.phoenix.pig.PhoenixHBaseStorage('i004,i005,i006','-batchSize
> 1000');
> {code}
> If the source table has timestamp, MapReduce job for Pig script does not work:
> {noformat}
> ERROR 0: java.lang.ClassCastException: org.joda.time.DateTime cannot be cast
> to org.apache.phoenix.shaded.org.joda.time.DateTime
> org.apache.pig.backend.executionengine.ExecException: ERROR 0:
> java.lang.ClassCastException: org.joda.time.DateTime cannot be cast to
> org.apache.phoenix.shaded.org.joda.time.DateTime
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.getStats(MapReduceLauncher.java:822)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:452)
> at
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.launchPig(HExecutionEngine.java:304)
> at org.apache.pig.PigServer.launchPlan(PigServer.java:1390)
> at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1375)
> at org.apache.pig.PigServer.execute(PigServer.java:1364)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:415)
> at org.apache.pig.PigServer.executeBatch(PigServer.java:398)
> at
> org.apache.pig.tools.grunt.GruntParser.executeBatch(GruntParser.java:171)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:234)
> at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205)
> at org.apache.pig.tools.grunt.Grunt.exec(Grunt.java:81)
> at org.apache.pig.Main.run(Main.java:502)
> at org.apache.pig.Main.main(Main.java:177)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
> Caused by: java.lang.ClassCastException: org.joda.time.DateTime cannot be
> cast to org.apache.phoenix.shaded.org.joda.time.DateTime
> at
> org.apache.phoenix.pig.util.TypeUtil.castPigTypeToPhoenix(TypeUtil.java:201)
> at
> org.apache.phoenix.pig.PhoenixHBaseStorage.putNext(PhoenixHBaseStorage.java:189)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:136)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:95)
> at
> org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:658)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:260)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
