Santhosh Srinivasan commented on PIG-1016:

Hc Busy, thanks for taking time to contribute the patch, explaining the details 
and especially for being patient. A few more questions and details have to be 
cleared up before we commit this patch.

IMHO, the right comparison should be along the lines of checking if o1 and o2 
are NullableBytesWritable followed by a check for PigNullableWritable and then 
followed by error handling code.

Alan, can you comment on this approach?

There is a more important semantic issue. If the map value types are strings 
and if the strings are numeric, then the value types for the maps will be of 
different types. In that case, the load function will break. In addition, 
conversion routines might fail when the compareTo method is invoked. An example 
to illustrate this issue.

Suppose, the records is ['key'#1234567890124567]. PIG-880 would treat the value 
as a string and there would be no problem. Now, with the changes reverted, the 
type is inferred as integer and the parsing will fail as the value is too big 
to fit into an integer

Secondly, assuming that the integer was small enough to be converted, the 
comparison method in DataType.java will return the wrong results when an 
integer and a string are compared. For example, if the records are:


The first value is treated as a string and the second value is treated as an 
integer. The compareTo method will return 1 to indicate that string > integer 
while in reality 123 > *$

Please correct me if the last statement is incorrect or let me know if it needs 
more explanation.

Thoughts/comments from other committers?

> Reading in map data seems broken
> --------------------------------
>                 Key: PIG-1016
>                 URL: https://issues.apache.org/jira/browse/PIG-1016
>             Project: Pig
>          Issue Type: Improvement
>          Components: data
>    Affects Versions: 0.4.0
>            Reporter: hc busy
>             Fix For: 0.5.0
>         Attachments: PIG-1016.patch
> Hi, I'm trying to load a map that has a tuple for value. The read fails in 
> 0.4.0 because of a misconfiguration in the parser. Where as in almost all 
> documentation it is stated that value of the map can be any time.
> I've attached a patch that allows us to read in complex objects as value as 
> documented. I've done simple verification of loading in maps with tuple/map 
> values and writing them back out using LOAD and STORE. All seems to work fine.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to