> Patrick maybe you had an interesting use case in mind? I had one, but later on I found out that I don't necessarily use flush to achieve that so it's not really a valid use case that definitely need flush...
On Tue, Apr 25, 2023 at 7:26 PM Ishan Chattopadhyaya < ichattopadhy...@gmail.com> wrote: > I think Apache Solr could explore leveraging the returned sequence number > for its transaction logs. > > On Tue, 25 Apr 2023 at 18:36, Michael McCandless < > luc...@mikemccandless.com> wrote: > >> On Sun, Apr 23, 2023 at 6:19 AM Uwe Schindler <u...@thetaphi.de> wrote: >> >> Having the sequence number public in API does not bring any benefit, as >>> you cannot use it for anything. >>> >> >> Actually there are some interesting use cases for sequence numbers: >> >> They enable the caller to know the effective order of operations of >> concurrent indexing events. This can be useful for applications that might >> sometimes update the same document at the same time across threads to >> implement optimistic concurrency to re-index the same document if the order >> was not correct according to the applications external version tracking for >> out-of-order updates. OpenSearch has an array of locks to implement >> pessimistic concurrency (ensuring the that same id is never updated >> concurrently) but for cases where the conflicts are rare, the optimistic >> implementation based on Lucene's sequence numbers is likely more efficient. >> >> Another use case is precise indexing operation replay (e.g. from a >> Kinesis queue or transaction log or whatever) on recovering from a commit >> point: upon commit, you know which precise indexing event was captured in >> the commit, and on recovering you can resume indexing from precisely the >> next indexing event. This doesn't matter for idempotent updates, but, for >> other cases like append only, it is useful and performant. >> >> I also don't see why flush should return a sequence number -- it is not >> an externally visible event. Patrick maybe you had an interesting use case >> in mind? Note that commit also writes (and fsyncs) the next segments_N >> file, to light all the newly written/fsync'd segments for the next reader >> to open. >> >> Mike McCandless >> >> http://blog.mikemccandless.com >> >>
