I just examined the field cache code. I don't think replacing FCs needs to be difficult. Lets make the CachedArray values variable volatile. values is already public.
On Tue, Aug 23, 2011 at 3:02 PM, Jason Rutherglen <[email protected]> wrote: >> Oh duh right -- you should be able to alloc too-large arrays and only >> realloc once you run out. So amortized cost is low but some reopens >> will take a hit to grow.... > > This hit will be minimal, ie, less than what we're doing now with > cloned deleted docs bit vectors, which feasibly happen on each > getReader() call. Growing the field cache will occur far less. > > Given there isn't a use case for the appendable field cache outside of > RT / LUCENE-2312. I may bake it in, it's hard to extract, it's hard > to maintain two patches. However the discussion was good. > >> DV is also provided by the IR (perDocValues method) so the RT reader > > Ok, it's not clear when / how DVs are used instead of field caches, > and why their access isn't merged together? > > On Tue, Aug 23, 2011 at 12:30 PM, Michael McCandless > <[email protected]> wrote: >> On Tue, Aug 23, 2011 at 12:09 AM, Jason Rutherglen >> <[email protected]> wrote: >>>> But, we are trying to move FC under IR control (Martijn has a patch), >>>> at which point an RT IR could have its own appending impl...? >>> >>> LUCENE-3360? It's placing the field cache into IR / SR. >> >> Yes! >> >>> The RAM >>> reader could return it's own impl where the underlying array can be >>> atomically replaced (when a larger sized array is needed). I agree >>> that is logical. >> >> Good. >> >>>> But then... FC still returns fixed arrays so you can't append until we fix >>>> that? >>> >>> I don't think anything should depend on the size of the field cache >>> array. If it does, it seems odd. Are you preferring moving field >>> cache access to method calls? What is the difference between that and >>> primitive array access? >> >> Oh duh right -- you should be able to alloc too-large arrays and only >> realloc once you run out. So amortized cost is low but some reopens >> will take a hit to grow.... >> >>> For now I will create an independent field cache implementation that >>> is appendable. It will only be associate-able with the DWPT / RAM >>> reader. Maybe somehow it can work with LUCENE-3360 and / or the >>> existing static FC access system. >> >> Sounds good. >> >>> Still not sure how doc values fits in. >> >> DV is also provided by the IR (perDocValues method) so the RT reader >> should be able to impl its own. Each lookup is a method call so it >> should be easier to back that w/ a more RT friendly data structure... >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: [email protected] >> For additional commands, e-mail: [email protected] >> >> > --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
