[jira] Created: (PIG-1243) Passing Complex map types to and from streaming causes a problem

[Viraj Bhat (JIRA)]

Passing Complex map types to and from streaming causes a problem
----------------------------------------------------------------

                 Key: PIG-1243
                 URL: https://issues.apache.org/jira/browse/PIG-1243
             Project: Pig
          Issue Type: Bug
    Affects Versions: 0.6.0
            Reporter: Viraj Bhat
             Fix For: 0.7.0


I have a program which generates different types of Maps fields and stores it 
into PigStorage.
{code}
A = load '/user/viraj/three.txt' using PigStorage();

B = foreach A generate ['a'#'12'] as b:map[], ['b'#['c'#'12']] as c, 
['c'#{(['d'#'15']),(['e'#'16'])}] as d;

store B into '/user/viraj/pigtest' using PigStorage();
{code}

Now I test the previous output in the below script to make sure I have the 
right results. I also pass this data to a Perl script and I observe that the 
complex Map types I have generated, are lost when I get the result back.

{code}
DEFINE CMD `simple.pl` SHIP('simple.pl');

A = load '/user/viraj/pigtest' using PigStorage() as (simpleFields, mapFields, 
mapListFields);

B = foreach A generate $0, $1, $2;

dump B;

C = foreach A generate  (chararray)simpleFields#'a' as value, $0,$1,$2;

D = stream C through CMD as (a0:map[], a1:map[], a2:map[]);

dump D;
{code}


dumping B results in:

([a#12],[b#[c#12]],[c#{([d#15]),([e#16])}])
([a#12],[b#[c#12]],[c#{([d#15]),([e#16])}])
([a#12],[b#[c#12]],[c#{([d#15]),([e#16])}])

dumping D results in:

([a#12],,)
([a#12],,)
([a#12],,)

The Perl script used here is:
{code}
#!/usr/local/bin/perl

use warnings;

use strict;

while(<>) {

    my($bc,$s,$m,$l)=split/\t/;

    print("$s\t$m\t$l");

}
{code}

Is there an issue with handling of complex Map fields within streaming? How can 
I fix this to obtain the right result?

Viraj

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.