Hey everyone, I think it's a good time to talk about the future of the cross DC work that's been going on in the Solr Sandbox repo.
Mark (and others) have been doing amazing work there, and I think it's close to a state that we can bring it out of the sandbox and into Solr itself. It's been running successfully in Apple, and moving it into Solr would allow others to use and collaborate on it easier. There are two parts to this cross DC tool. - A cross-dc-producer <https://github.com/apache/solr-sandbox/tree/main/crossdc-producer>, which is a Solr plugin that has a Update-request-processor that adds documents to a Kafka Queue to be indexed asynchronously. - A cross-dc-consumer <https://github.com/apache/solr-sandbox/tree/main/crossdc-consumer>, which is a separate application that handles consuming documents from Kafka and sending it to Solr to be indexed. In my mind, I envision when we move the cross-dc-consumer into the Solr codebase, it would live much like the Prometheus Exporter. Where it's released as a part of the same binary package, but not in the slim distribution. As for the cross-dc-producer, that can live as a Solr module, much like the other modules that Solr ships with. Once again, this would not be included with the Slim distribution. This ties the cross-dc packages to specific Solr versions, much like other Modules or SolrJ. I think this is an acceptable thing, but others might want it to be more version agnostic or have a separate release schedule from Solr. Overall this thread is to just get the conversation started so that we can throw all the ideas out there before we make a decision on one. - Houston
