[ https://issues.apache.org/jira/browse/LUCENE-4515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488217#comment-13488217 ]
Simon Willnauer commented on LUCENE-4515: ----------------------------------------- {quote} synchronizedAllocator looks unused? I guess you added that after removing all sync from RecyclingByteBlockAllocator ... but I think we can just add synchronizedAllocator back later if/when we need it? Separately can you call out that RecyclingByteBlockAllocator is not thread safe in its javadocs? {quote} regarding javadocs, I though I did this... will fix. Regardin sync. yeah lets drop it we can still add if needed, trivial though! {quote} I think you need the start ... because if you used more than one slice you won't know how to read "backwards" to get to the starting slice? {quote} the comment is on start but it says "end" I think given the fact that we know the freq we can read the slice without storing the end but we'd need to change SliceReader for it and I am not sure if that is worth the trouble we could get in. Yet, 4byte per term though. {quote} If we do that we have to make sure that allocator clears each int[] before returning it, in getIntBlock(). {quote} we really rely on this in ByteBlockPool already so which likely doesn't work at this time but we don't run into since we don't reuse in DWPT? I will add a test. {quote} The added MemoryIndex.reset method is sort of ... spooky? Like, do we really need/want to reuse a MemoryIndex? (I guess this is because we added passing in an allocator to the ctor ... so you want the byte[]'s returned to it ... but that also makes me nervous: should we really pass in an external allocator...?). {quote} I think reuse is a special usecase and I guess we should allow it. Yet, I totally agree this is risky. I suggest to make this possible if you subclass and expose this stuff via protected API so if you really really wanna do this you can if you subclass? > Make MemoryIndex more memory efficient > -------------------------------------- > > Key: LUCENE-4515 > URL: https://issues.apache.org/jira/browse/LUCENE-4515 > Project: Lucene - Core > Issue Type: Improvement > Components: modules/other > Affects Versions: 4.0, 4.1, 5.0 > Reporter: Simon Willnauer > Fix For: 4.1, 5.0 > > Attachments: LUCENE-4515.patch > > > Currently MemoryIndex uses BytesRef objects to represent terms and holds an > int[] per term per field to represent postings. For highlighting this creates > a ton of objects for each search that 1. need to be GCed and 2. can't be > reused. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org