My apology for irrelevant post there.
I don't wanna receive emails from @lucene.apache.org on this account as I have
another one for it.
I sent email to unsub@. But still receiving emails.
From: Ravikumar Govindarajan <ravikumar.govindara...@gmail.com>
Sent: Thursday, August 10, 2017 6:37 PM
Subject: Re: Storing external transaction log-ids in lucene...
Many thanks for the help...
Just one more clarification..
I also see that a sequence number is returned from IW on all mutations in
latest code including prepareCommit() [Apologize, as we are still on an old
version of lucene 4.6]
So the approach will be to co-relate the seq.no returned from individual
operations of IW with the external queue offset. Background commit thread
can then call prepareCommit() & it will exactly know what got through. Have
I understood it correctly?
Btw, we have a TrackingIndexWriter that returns a sequence number too, in
4.6. Is the new implementation something different from this?
On Thu, Aug 10, 2017 at 6:09 PM, Michael McCandless <
> IW.setCommitData (now .setLiveCommitData in 7.0) is the right way to store
> Mike McCandless
The Apache Lucene project will likely release its next major release, 7.0, in a
few months! Remember that Lucene developers generally try hard to backport new
> On Thu, Aug 10, 2017 at 6:57 AM, Ravikumar Govindarajan <
> ravikumar.govindara...@gmail.com> wrote:
> > Every mutation (Add/Update/Delete) has a transaction-id (incremental
> > assigned by our Messaging Queue (Kafka)
> > To index these mutations, an indexer thread pulls data from the queue,
> > & commits to IndexWriter, then updates the latest transaction-id in an
> > external system (ZooKeeper). During rollback/server-restarts, the threads
> > read the previous value from ZK & resume...
> > Have now moved to NRT implementation, where commit thread runs in the
> > background & am unable to find the latest transaction-id that got
> > Initially thought of adding transaction-id as a first-class field of the
> > index itself & then writing a custom-codec to harvest it, but this fails
> > when a mutation-set consists only of deletes [as there is no way to pass
> > these ids via IW delete APIs]
> > Is there a way to store this as part of segment-info?. Can
> > of IW help me here in any way?
> > --
> > Ravi