On 12/02/2018 16:02, Brian Yee wrote:
I asked a question here about fast inventory updates last week and I was
recommended to use docValues with partial in-place updates. I think this will
work well, but there is a problem I can't think of a good solution for.
Consider this scenario:
InStock = 1 for a product.
InStock changes to 0 which triggers a fast in-place update with docValues.
But it also triggers a slow update that will rebuild the entire document. Let's
say that takes 10 minutes because we do updates in batches.
During that 5 minutes, InStock changes again to 1 which triggers a fast update
to solr. So in Solr InStock=1 which is correct.
The slow update finishes and overwrites InStock=0 which is incorrect.
How can we deal with this situation?
It's a slightly crazy idea, but in the past we've solved a similar
problem by building a custom Lucene codec that is backed by a Redis
database. You change the stock value in Redis and Lucene doesn't
actually notice and re-index.
http://www.flax.co.uk/blog/2012/06/22/updating-individual-fields-in-lucene-with-a-redis-backed-codec/
Not sure if this is a better way than DocValues, it was quite a while
ago and Lucene has moved on a bit since then....
Cheers
Charlie
--
Charlie Hull
Flax - Open Source Enterprise Search
tel/fax: +44 (0)8700 118334
mobile: +44 (0)7767 825828
web: www.flax.co.uk
