[
https://issues.apache.org/jira/browse/PIG-1016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771442#action_12771442
]
Santhosh Srinivasan commented on PIG-1016:
------------------------------------------
Hc Busy, thanks for taking time to contribute the patch, explaining the details
and especially for being patient. A few more questions and details have to be
cleared up before we commit this patch.
IMHO, the right comparison should be along the lines of checking if o1 and o2
are NullableBytesWritable followed by a check for PigNullableWritable and then
followed by error handling code.
Alan, can you comment on this approach?
There is a more important semantic issue. If the map value types are strings
and if the strings are numeric, then the value types for the maps will be of
different types. In that case, the load function will break. In addition,
conversion routines might fail when the compareTo method is invoked. An example
to illustrate this issue.
Suppose, the records is ['key'#1234567890124567]. PIG-880 would treat the value
as a string and there would be no problem. Now, with the changes reverted, the
type is inferred as integer and the parsing will fail as the value is too big
to fit into an integer
Secondly, assuming that the integer was small enough to be converted, the
comparison method in DataType.java will return the wrong results when an
integer and a string are compared. For example, if the records are:
[key#*$]
[key#123]
The first value is treated as a string and the second value is treated as an
integer. The compareTo method will return 1 to indicate that string > integer
while in reality 123 > *$
Please correct me if the last statement is incorrect or let me know if it needs
more explanation.
Thoughts/comments from other committers?
> Reading in map data seems broken
> --------------------------------
>
> Key: PIG-1016
> URL: https://issues.apache.org/jira/browse/PIG-1016
> Project: Pig
> Issue Type: Improvement
> Components: data
> Affects Versions: 0.4.0
> Reporter: hc busy
> Fix For: 0.5.0
>
> Attachments: PIG-1016.patch
>
>
> Hi, I'm trying to load a map that has a tuple for value. The read fails in
> 0.4.0 because of a misconfiguration in the parser. Where as in almost all
> documentation it is stated that value of the map can be any time.
> I've attached a patch that allows us to read in complex objects as value as
> documented. I've done simple verification of loading in maps with tuple/map
> values and writing them back out using LOAD and STORE. All seems to work fine.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
