On Sun, Mar 2, 2025 at 7:21 AM Marc Davenport <madavenp...@cargurus.com.invalid> wrote:
> > @Michael - That second simpler architecture is very similar to what we are > considering; With the exception of a queue for announcing new > segments rather than a polling process. It is good to know that it's a > reasonable outline. You were very latency sensitive. Is there anything > you can share around the most important specs of your containers, pods, or > even nodes? While we run on these EC2 instances, there is a push to get us > into k8s. Did you have any issues as you migrated from one version of > lucene to another. I'm concerned that our current deployments only allow > one version of the software in production at any one time. > > >From what I remember, we did see a bit of noisy-neighbor behavior running two searchers in separate containers on the same instance. I don't think we ever found a concrete reason (at least while I was there). We gave up and ended up using smaller instances instead of putting multiple searchers on the same instance. For migration between different Lucene versions (or even versions of our software, which might change indexing behavior), we used the process described in https://careersatdoordash.com/blog/introducing-doordashs-in-house-search-engine/ in the section "Tenant Isolation and Search Stacks". Essentially, bring up a new indexer fleet, build a new index, then start bringing up new searchers that replicate from the new indexers.
