To give more context, the ISOToUnixInSecond return UnixTime in second. The
return value of this function is Long
75 @Override
76 public Long exec(Tuple input) throws IOException
77 {
78 if (input == null || input.size() < 1) {
79 return null;
80 }
81
82 // Set the time to default or the output is in UTC
83 DateTimeZone.setDefault(DateTimeZone.UTC);
84
85 DateTime result = new DateTime(input.get(0).toString());
86
87 return (result.getMillis() / 1000);
88 }
I think somehow pig is not converting the Long into the proper long type in
the pig script.
ISOToUnixInSecond('$STARTDATETIME') AS startTime:long
Hence during the comparion, it treat the Long as a string value ...
On Wed, Dec 15, 2010 at 4:28 PM, John Hui <[email protected]> wrote:
> This is actually, please ignore the code section below, Thanks!
>
> 7 eventData = FOREACH flattenData GENERATE B1::event AS event,
> B1::publication AS publication, B1::deviceType AS deviceType, B1::adID AS
> adID, B1::mcc AS mcc, B1::event_timestamp AS eventTimestamp:long,
> ISOToUnixInSecond('$STARTDATETIME') AS startTime:long;
> 8
> 9 eventData = FILTER eventData BY (event == 'adImpression') AND
> (eventTimesta mp <= startTime);
> 10
> 11 DESCRIBE eventData;
> 12
> 13 B = GROUP eventData BY (event, publication, deviceType, adID, mcc);
>
>
> On Wed, Dec 15, 2010 at 4:21 PM, John Hui <[email protected]> wrote:
>
>> I am having a hard time getting comparison to work. I am comparing from
>> two long values but I keep on getting a cast long to String error
>>
>> Backend error message
>> ---------------------
>> java.lang.ClassCastException: java.lang.Long cannot be cast to
>> java.lang.String
>> at java.lang.String.compareTo(String.java:92)
>> at
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr.doComparison(LTOrEqualToExpr.java:152)
>> at
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.LTOrEqualToExpr.getNext(LTOrEqualToExpr.java:114)
>> at
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.expressionOperators.POAnd.getNext(POAnd.java:83)
>> at
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POFilter.getNext(POFilter.java:148)
>> at
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.PhysicalOperator.processInput(PhysicalOperator.java:272)
>> at
>> org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POLocalRearrange.getNext(POLocalRearrange.java:255)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.runPipeline(PigMapBase.java:232)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:227)
>> at
>> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapBase.map(PigMapBase.java:52)
>> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:644)
>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:320)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1063)
>> at org.apache.hadoop.mapred.Child.main(Child.java:211
>>
>>
>> this is my pig code
>>
>> 7 eventData = FOREACH flattenData GENERATE B1::event AS event,
>> B1::publicatio n AS publication, B1::deviceType AS deviceType, B1::adID
>> AS adID, B1::mcc A S mcc, B1::event_timestamp AS eventTimestamp:long,
>> ISOToUnixInSecond('$STAR TDATETIME') AS startTime:long;
>> 8 9 ILLUSTRATE eventData;
>> 10 11 eventData = FILTER eventData BY (startTime <= startTime);
>> 12 --eventData = FILTER eventData BY (event == 'adImpression') AND
>> (eventTimes tamp <= startTime);
>> 13
>> 14 DESCRIBE eventData;
>>
>> Here is the output
>>
>> eventData: {event: chararray,publication: chararray,deviceType:
>> chararray,adID: chararray,mcc: chararray,eventTimestamp: long,startTime:
>> long}
>>
>>
>>
>
