IW.setCommitData (now .setLiveCommitData in 7.0) is the right way to store this.
Advertising
Mike McCandless http://blog.mikemccandless.com On Thu, Aug 10, 2017 at 6:57 AM, Ravikumar Govindarajan < ravikumar.govindara...@gmail.com> wrote: > Every mutation (Add/Update/Delete) has a transaction-id (incremental long) > assigned by our Messaging Queue (Kafka) > > To index these mutations, an indexer thread pulls data from the queue, adds > & commits to IndexWriter, then updates the latest transaction-id in an > external system (ZooKeeper). During rollback/server-restarts, the threads > read the previous value from ZK & resume... > > Have now moved to NRT implementation, where commit thread runs in the > background & am unable to find the latest transaction-id that got committed > > Initially thought of adding transaction-id as a first-class field of the > index itself & then writing a custom-codec to harvest it, but this fails > when a mutation-set consists only of deletes [as there is no way to pass > these ids via IW delete APIs] > > Is there a way to store this as part of segment-info?. Can setCommitData() > of IW help me here in any way? > > -- > Ravi >
