[
https://issues.apache.org/jira/browse/LUCENE-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12907493#action_12907493
]
Jason Rutherglen commented on LUCENE-2575:
------------------------------------------
I'm finally understanding the slice concept, basically we're
over-allocating space within the ByteBlockPool byte[]s for more
postings for a particular term, hence the levelSizeArray which
determines the length of each "slice" of a byte[] the postings
will use. They're probably not always filled in completely?
It's a bit tricky to follow by reading the code, which makes
figuring out how to make the RAM buffer concurrent challenging.
Especially in the newSlice method which rewrites the end of the
last slice with the forwarding index/address of the next slice.
It's very clever however maybe we can encapsulate it better with
methods delineating the various operations which right now are
operations directly on the assortment of arrays. In general we
can possibly get away with using copy-on-write to achieve
performant single-threaded write and multi-threaded reader
concurrency.
> Concurrent byte and int block implementations
> ---------------------------------------------
>
> Key: LUCENE-2575
> URL: https://issues.apache.org/jira/browse/LUCENE-2575
> Project: Lucene - Java
> Issue Type: Improvement
> Components: Index
> Affects Versions: Realtime Branch
> Reporter: Jason Rutherglen
> Fix For: Realtime Branch
>
>
> The current *BlockPool implementations aren't quite concurrent.
> We really need something that has a locking flush method, where
> flush is called at the end of adding a document. Once flushed,
> the newly written data would be available to all other reading
> threads (ie, postings etc). I'm not sure I understand the slices
> concept, it seems like it'd be easier to implement a seekable
> random access file like API. One'd seek to a given position,
> then read or write from there. The underlying management of byte
> arrays could then be hidden?
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
