[
https://issues.apache.org/jira/browse/PIG-934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12749042#action_12749042
]
Pradeep Kamath commented on PIG-934:
------------------------------------
The reason I thought a separate function with a "singleFile" in the name was
needed was because the current FileLocalizer.open() can handle directories and
hence returns a DataStorageInputStreamIterator which internally iterates over
the underlying multiple streams of the files in the directory. Keeping the same
name may give the impression that the same capability is present even for the
version which seeks to an offset. Seeking to an offset would only work for a
single file - hence maybe have a separate function where the name implies this
restriction might be cleaner.
> Merge join implementation currently does not seek to right point on the right
> side input based on the offset provided by the index
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: PIG-934
> URL: https://issues.apache.org/jira/browse/PIG-934
> Project: Pig
> Issue Type: Bug
> Affects Versions: 0.3.1
> Reporter: Pradeep Kamath
> Assignee: Ashutosh Chauhan
> Attachments: pig-934.patch
>
>
> We use POLoad to seek into right file which has the following code:
> {noformat}
> public void setUp() throws IOException{
> String filename = lFile.getFileName();
> loader =
> (LoadFunc)PigContext.instantiateFuncFromSpec(lFile.getFuncSpec());
> is = FileLocalizer.open(filename, pc);
> loader.bindTo(filename , new BufferedPositionedInputStream(is),
> this.offset, Long.MAX_VALUE);
> }
> {noformat}
> Between opening the stream and bindTo we do not seek to the right offset.
> bindTo itself does not perform any seek.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
