Earwin, I used MMAP a lot, is quite nice, it has its place under the sun, but it is not a silver bullet, it has its quirks... the same goes for RAMDirectory.
bq. There is zero need for any such signal. ...non-existing file ... Why would IndexReader ever try to read non-existing file? IR is going to see its RAMDirectory point-in-time snapshot of an Index until you somehow try to reload updated Index image on disk. On Thu, Nov 25, 2010 at 6:00 PM, Earwin Burrfoot (JIRA) <j...@apache.org>wrote: > > [ > https://issues.apache.org/jira/browse/LUCENE-2691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12935794#action_12935794] > > Earwin Burrfoot commented on LUCENE-2691: > ----------------------------------------- > > {quote} > bq. You're still okay with an API that allows you to reopen IRs on > different directories? > Well, that's no good - we can catch this and throw an exc? > {quote} > I don't understand why should we bother with checking and throwing > exceptions, when we can prevent such things from compiling at all. > By using an API, that doesn't support reopening on anything different from > original source. > > bq. Really, there are two separate "things" open/reopen needs: > That's not true. Take a look at my WriterBackedReader above (or > DirectoryReader in trunk). It requires writer at least to call > deleteUnusedFiles(), nrtIsCurrent(). > So you can't easily reopen between Directory-backed and Writer-backed > readers without much switching and checking. > > bq. r_ram.reload(); //Here we want to reload from the FSDirecotory? > Use MMapDirectory? It's only a bit slower for searches, while not raping > your GC on big indexes. > Also check this out - https://gist.github.com/715617 , it is a > RAMDirectory offspring that wraps any other given directory and basically > does what you want (if I guessed right). > It doesn't use blocking for files, so file size limit is 2Gb, but this can > be easily fixed. On the up side - it reads file into memory only after the > size is known (unlike RAMDir), which allows you to use huge precisely-sized > blocks, lessening GC pressure. > I used it for a long time, but then my indexes grew, heaps followed, VM > exploded and I switched to MMapDirectory (with minor patches). > > bq. What is missing is a "signal" from IR.reload() to RAMdirectory to slurp > fresh information from FSDirecory? > There is zero need for any such signal. If a reader requests non-existing > file from RAMDirectory, it should check backing dir before throwing > exception. If backing dir does have the file - it is loaded and opened. > Why do you people love complicating things that much? :) > > > Consolidate Near Real Time and Reopen API semantics > > --------------------------------------------------- > > > > Key: LUCENE-2691 > > URL: https://issues.apache.org/jira/browse/LUCENE-2691 > > Project: Lucene - Java > > Issue Type: Improvement > > Reporter: Grant Ingersoll > > Assignee: Grant Ingersoll > > Priority: Minor > > Fix For: 4.0 > > > > Attachments: LUCENE-2691.patch, LUCENE-2691.patch > > > > > > We should consolidate the IndexWriter.getReader and the > IndexReader.reopen semantics, since most people are already using the > IR.reopen() method, we should simply add:: > > {code} > > IR.reopen(IndexWriter) > > {code} > > Initially, it could just call the IW.getReader(), but it probably should > switch to just using package private methods for sharing the internals > > -- > This message is automatically generated by JIRA. > - > You can reply to this email to add a comment to the issue online. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > >
