[
https://issues.apache.org/jira/browse/LUCENE-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14018025#comment-14018025
]
Uwe Schindler edited comment on LUCENE-5731 at 6/4/14 7:31 PM:
---------------------------------------------------------------
Thanks Robert. I was very busy today, so I had no time to look into it. But
from my first check it looks like our idea from the talk yesterday :-) I was
afraid to propose to implement this using an interface, thanks for doing it
that way. Otherwise we would have crazyness in ByteBufferIndexInput. The
interface hidden behind the randomAccessSlice() method just returning "slice()"
is wonderful.
{code:java}
@Override
public RandomAccessInput randomAccessSlice(long offset, long length) throws
IOException {
// note: technically we could even avoid the clone...
return slice(null, offset, length);
}
{code}
We can avoid the clone not in all cases, because we must duplicate the
ByteBuffer, if the offset is different. But for the simple case, if you request
the full IndexInput as slice (means offset==0L, length==this.length), we could
return "this".
EDIT: we cannot do this at the moment, because in the multi-mmap case, we
change the bytebuffers's position. So we always have to clone (otherwise the
random access slice would have side effects on file position of master slice).
was (Author: thetaphi):
Thanks Robert. I was very busy today, so I had no time to look into it. But
from my first check it looks like our idea from the talk yesterday :-) I was
afraid to propose to implement this using an interface, thanks for doing it
that way. Otherwise we would have crazyness in ByteBufferIndexInput. The
interface hidden behind the randomAccessSlice() method just returning "slice()"
is wonderful.
{code:java}
@Override
public RandomAccessInput randomAccessSlice(long offset, long length) throws
IOException {
// note: technically we could even avoid the clone...
return slice(null, offset, length);
}
{code}
We can avoid the clone not in all cases, because we must duplicate the
ByteBuffer, if the offset is different. But for the simple case, if you request
the full IndexInput as slice (means offset==0L, length==this.length), we could
return "this".
> split direct packed ints from in-ram ones
> -----------------------------------------
>
> Key: LUCENE-5731
> URL: https://issues.apache.org/jira/browse/LUCENE-5731
> Project: Lucene - Core
> Issue Type: Bug
> Reporter: Robert Muir
> Assignee: Robert Muir
> Fix For: 4.9, 5.0
>
> Attachments: LUCENE-5731.patch, LUCENE-5731.patch
>
>
> Currently there is an oversharing problem in packedints that imposes too many
> requirements on improving it:
> * every packed ints must be able to be loaded directly, or in ram, or
> iterated with.
> * things like filepointers are expected to be adjusted (this is especially
> stupid) in all cases
> * lots of unnecessary abstractions
> * versioning etc is complex
> None of this flexibility is needed or buys us anything, and it prevents
> performance improvements (e.g. i just want to add 3 bytes at the end of
> on-disk streams to reduce the number of bytebuffer calls and thats seriously
> impossible with the current situation).
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
