[
https://issues.apache.org/jira/browse/PIG-5048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15647088#comment-15647088
]
Nandor Kollar commented on PIG-5048:
------------------------------------
Thanks Daniel for the comments. I added these tests to the Junit test suite,
because it was easier to execute and verify, should I delete those which are
already covered in e2e tests, or it is fine? I'll change hive-contrib to test
dependency.
As for the collector, the UDTF exec was called twice: first from the normal
process, then for close, and I noticed that in the test, the output for the
process case was cleared in the close call. I just realized that this is
probably a test issue with mock Storage I used in the tests, putNext doesn't
make a deep copy from the tuples:
{code}
@Override
public void putNext(Tuple t) throws IOException {
mockRecordWriter.dataBeingWritten.add(TF.newTuple(t.getAll()));
}
{code}
Since in tests the tuple contains a bag, both the output bag of process and
close will point to the same bag instance. Will figure out how to test this
properly.
> HiveUDTF fail if it is the first expression in projection
> ---------------------------------------------------------
>
> Key: PIG-5048
> URL: https://issues.apache.org/jira/browse/PIG-5048
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Daniel Dai
> Assignee: Nandor Kollar
> Fix For: 0.17.0, 0.16.1
>
> Attachments: PIG-5048-1.patch, PIG-5048-2.patch, PIG-5048.patch
>
>
> The following script fail:
> {code}
> define explode HiveUDTF('explode');
> A = load 'bag.txt' as (a0:{(b0:chararray)});
> B = foreach A generate explode(a0);
> dump B;
> {code}
> Message: Unimplemented at
> org.apache.pig.data.UnlimitedNullTuple.size(UnlimitedNullTuple.java:31)
> If it is not the first projection, the script pass:
> {code}
> define explode HiveUDTF('explode');
> A = load 'bag.txt' as (a0:{(b0:chararray)});
> B = foreach A generate a0, explode(a0);
> dump B;
> {code}
> Thanks [~nkollar] reporting it!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
