Hi Ian, To be clear the in memory index is purely ephemeral and is not meant to be persisted. It just compliments the persistent index to allow access to recently added/modified entries. So now to your queries
> How will you deal with JVM failure ? Do nothing. The index as explained is transient. Current AsyncIndex would anyway be performing the usual indexing and is resilient enough > How frequently will commits to the persisted index be performed ? This index lives separately. Persisted index managed by AsyncIndex works as is > I assume that switching to use ElasticSearch, which delivers NRT reliably in the 0.1s range has been rejected as an option ? No. The problem here is bit different. Lucene indexes are being used for all sort of indexing currently in Oak. In many cases its being used as purely property index. ES makes sense mostly for global fulltext index and would be an overkill for smaller more focused property index types of usecases. > If it has, you may find yourself implementing much of the core of ElasticSearch to make NTR work properly in a cluster. Again usecase here is not to support NTR as is. Current indexing would work as is and this transient index would compliment it. Chetan Mehrotra On Fri, Jul 24, 2015 at 1:01 PM, Ian Boston <[email protected]> wrote: > Hi Chetan, > > The overall approach looks ok. > > Some questions about indexing. > > How will you deal with JVM failure ? > and related. > How frequently will commits to the persisted index be performed ? > > I assume that switching to use ElasticSearch, which delivers NRT reliably > in the 0.1s range has been rejected as an option ? > > If it has, you may find yourself implementing much of the core of > ElasticSearch to make NTR work properly in a cluster. > > Best Regards > Ian > > > On 24 July 2015 at 08:09, Chetan Mehrotra <[email protected]> wrote: > >> On Fri, Jul 24, 2015 at 12:15 PM, Michael Marth <[email protected]> wrote: >> > From your description I am not sure how the indexing would be triggered >> for local changes. Probably not through the Async Indexer (this would not >> gain us much, right?). Would this be a Commit Hook? >> >> My thought was to use an Observor so as to not add cost to commit >> call. Observor would listen only for local changes and would invoke >> IndexUpdate on the diff >> >> Chetan Mehrotra >>
