Ashutosh Chauhan updated PIG-934:

    Attachment: pig-934.patch

I think instead of creating a new method, better is to overload 
FileLocalizer.open() to have a consistent api. 
Currently, there is 
InputStream open(String fileSpec, PigContext pigContext)

Add an Overload method like this and within that fallback to previous method, 
if there is no need to seek.
InputStream open(String fileSpec, long offset, PigContext pigContext){
      if (offset == 0)
            return open(String fileSpec, PigContext pigContext);
      // first open, then seek and then return


This ensures minimum impact to clients of this api. 

Attaching a patch which does this.

> Merge join implementation currently does not seek to right point on the right 
> side input based on the offset provided by the index
> ----------------------------------------------------------------------------------------------------------------------------------
>                 Key: PIG-934
>                 URL: https://issues.apache.org/jira/browse/PIG-934
>             Project: Pig
>          Issue Type: Bug
>    Affects Versions: 0.3.1
>            Reporter: Pradeep Kamath
>         Attachments: pig-934.patch
> We use POLoad to seek into right file which has the following code: 
> {noformat}
>    public void setUp() throws IOException{
>         String filename = lFile.getFileName();
>         loader = 
> (LoadFunc)PigContext.instantiateFuncFromSpec(lFile.getFuncSpec());        
>         is = FileLocalizer.open(filename, pc);
>         loader.bindTo(filename , new BufferedPositionedInputStream(is), 
> this.offset, Long.MAX_VALUE);
>     }
> {noformat}
> Between opening the stream and bindTo we do not seek to the right offset. 
> bindTo itself does not perform any seek.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

Reply via email to