[ https://issues.apache.org/jira/browse/PIG-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771442#action_12771442 ]
Santhosh Srinivasan commented on PIG-1016: ------------------------------------------ Hc Busy, thanks for taking time to contribute the patch, explaining the details and especially for being patient. A few more questions and details have to be cleared up before we commit this patch. IMHO, the right comparison should be along the lines of checking if o1 and o2 are NullableBytesWritable followed by a check for PigNullableWritable and then followed by error handling code. Alan, can you comment on this approach? There is a more important semantic issue. If the map value types are strings and if the strings are numeric, then the value types for the maps will be of different types. In that case, the load function will break. In addition, conversion routines might fail when the compareTo method is invoked. An example to illustrate this issue. Suppose, the records is ['key'#1234567890124567]. PIG-880 would treat the value as a string and there would be no problem. Now, with the changes reverted, the type is inferred as integer and the parsing will fail as the value is too big to fit into an integer Secondly, assuming that the integer was small enough to be converted, the comparison method in DataType.java will return the wrong results when an integer and a string are compared. For example, if the records are: [key#*$] [key#123] The first value is treated as a string and the second value is treated as an integer. The compareTo method will return 1 to indicate that string > integer while in reality 123 > *$ Please correct me if the last statement is incorrect or let me know if it needs more explanation. Thoughts/comments from other committers? > Reading in map data seems broken > -------------------------------- > > Key: PIG-1016 > URL: https://issues.apache.org/jira/browse/PIG-1016 > Project: Pig > Issue Type: Improvement > Components: data > Affects Versions: 0.4.0 > Reporter: hc busy > Fix For: 0.5.0 > > Attachments: PIG-1016.patch > > > Hi, I'm trying to load a map that has a tuple for value. The read fails in > 0.4.0 because of a misconfiguration in the parser. Where as in almost all > documentation it is stated that value of the map can be any time. > I've attached a patch that allows us to read in complex objects as value as > documented. I've done simple verification of loading in maps with tuple/map > values and writing them back out using LOAD and STORE. All seems to work fine. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.