getting class cast exception ( ERROR: Integer cannot be cast to String)

2014-04-22 Thread Prabhakar 
Hi,
I am getting very strange exception, i don't understand why i am getting
this exception. when i am dumping or storing the jndScoreAttrWithDevices
i am getting the class cast exception saying cannot convert Integer to
String.

but i am able to dump or store grpByScoreattributevalueFroMlg and
reqcolVehicleDevices.
 pls see the pig script in details statement and description.


grpByScoreattributevalueFroMlg= group fltrdScoreattributevalueForMlg BY
userid;
jndScoreAttrWithDevices= join grpByScoreattributevalueFroMlg by group,
reqcolVehicleDevices by fltrdVehicles.userid;

describe:
grpByScoreattributevalueFroMlg: {group:
chararray,fltrdScoreattributevalueForMlg: {(scoreattributevalueid:
chararray,scoreattributeid: int,userid: chararray,durationtypeid: int,slot:
chararray,durationslot: int,attributevalue: double,lasttimezone:
chararray,offset: int,createdat: chararray,updatedat: chararray,customerid:
int)}}

reqcolVehicleDevices: {fltrdVehicles::userid:
chararray,fltrdVehicles::customerid: int,deviceDays: long,device::offset:
int}

jndScoreAttrWithDevices: {grpByScoreattributevalueFroMlg::group:
chararray,grpByScoreattributevalueFroMlg::fltrdScoreattributevalueForMlg:
{(scoreattributevalueid: chararray,scoreattributeid: int,userid:
chararray,durationtypeid: int,slot: chararray,durationslot:
int,attributevalue: double,lasttimezone: chararray,offset: int,createdat:
chararray,updatedat: chararray,customerid:
int)},reqcolVehicleDevices::fltrdVehicles::userid:
chararray,reqcolVehicleDevices::fltrdVehicles::customerid:
int,reqcolVehicleDevices::deviceDays:
long,reqcolVehicleDevices::device::offset: int}

Exception:

Pig Stack Trace
---
ERROR 1066: Unable to open iterator for alias jndScoreAttrWithDevices.
Backend error : java.lang.Integer cannot be cast to java.lang.String

org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
open iterator for alias jndScoreAttrWithDevices. Backend error :
java.lang.Integer cannot be cast to java.lang.String
at org.apache.pig.PigServer.openIterator(PigServer.java:828)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
at
org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
at
org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
at org.apache.pig.Main.run(Main.java:538)
at org.apache.pig.Main.main(Main.java:157)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast
to java.lang.String
at
org.apache.pig.backend.hadoop.HDataType.getWritableComparableTypes(HDataType.java:76)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapReduce$Map.collect(PigGenericMapReduce.java:112)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:285)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:278)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:140)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672)



Thanks,
Prabhakar

Number of map task

2014-04-22 Thread Patcharee Thongtra 

Hi,

I wrote a custom InputFormat. When I ran the pig script Load function 
using this InputFormat, the number of InputSplit = 16, but there was 
only 2 map tasks handling these splits. Apparently the no. of map tasks 
= the no. of input files.


Does the number of Map task not correspond to the number of splits?

I think the job will be done quicker if there are more Map tasks?

Patcharee

Re: Number of map task

2014-04-22 Thread Pradeep Gollakota 
Pig is a little too smart when dealing with data. It has a feature called
split combination. If you set it to false, you should see more mappers.

SET pig.noSplitCombination true;



On Tue, Apr 22, 2014 at 12:14 PM, Patcharee Thongtra 
patcharee.thong...@uni.no wrote:

 Hi,

 I wrote a custom InputFormat. When I ran the pig script Load function
 using this InputFormat, the number of InputSplit = 16, but there was only 2
 map tasks handling these splits. Apparently the no. of map tasks = the no.
 of input files.

 Does the number of Map task not correspond to the number of splits?

 I think the job will be done quicker if there are more Map tasks?

 Patcharee