[
https://issues.apache.org/jira/browse/SOLR-17065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17842861#comment-17842861
]
Mark Robert Miller commented on SOLR-17065:
-------------------------------------------
https://github.com/markrmiller/solr/tree/cross-dc-cleanup-wip
I've addressed several test compilation issues and successfully resolved most
of the failing tests, with a few exceptions noted below which are currently
being ignored. Additionally, I've eliminated warnings, addressed forbidden
APIs, ensured compliance with spotless standards, and validated log calls.
However, I've encountered a challenge with test policy violations concerning
execute permissions for bytebuddy on the java executable. Despite several
attempts, including the following permissions settings, I haven't been able to
resolve the issue:
For Windows: permission java.io.FilePermission
"C:\\Users\\bitsy\\.jdks\\openjdk-19.0.1\\bin\\java.exe", "execute";
For Linux: permission java.io.FilePermission "<<ALL FILES>>/java", "execute";
*Ignored Test Methods:*
_DeleteByQueryToIdTest#testExpandDBQ_: This test is failing likely because the
Consumer is not batching deleteById updates to Solr as expected. The test
illustrates a deleteByQuery converting into multiple deleteByIds, with the
consumer sending these to Solr individually.
_SolrAndKafkaIntegrationTest#testMirroringUpdateProcessor_
_SolrAndKafkaMultiCollectionIntegrationTest:_ These tests seem to fail possibly
due to missing functionality that allows using collection properties to
configure crossdc settings.
On vacation for a week, so just pointing to the branch right now, which adds a
commit to Houston's work.
> Migrate Sandbox CrossDC functionality into Solr
> -----------------------------------------------
>
> Key: SOLR-17065
> URL: https://issues.apache.org/jira/browse/SOLR-17065
> Project: Solr
> Issue Type: Task
> Components: CrossDC, module - crossDC
> Reporter: Houston Putman
> Assignee: Houston Putman
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> As a part of [SIP-13: Cross Data Center
> Replication|https://cwiki.apache.org/confluence/display/SOLR/SIP-13%3A+Cross+Data+Center+Replication],
> the [solr-sandbox|https://github.com/apache/solr-sandbox] repository was
> created and a new Cross DC implementation has been developed.
> This CrossDC implementation relies on Kafka (for now, maybe other queues in
> the future), and has two parts.
> * A producer, which is a Solr plugin (An updateRequestProcessor), that sends
> documents to kafka after successfully indexing them.
> * A consumer, which is a standalone application, that reads documents for
> kafka and sends them to the mirrored SolrClouds.
> This has been in development for a year now and is used at multiple companies
> (even in production).
> We believe that the project has reached a level of maturity that it can be
> "graduated" from the sandbox module and be included in future Solr releases.
>
> As a part of the move I propose the following:
> * The Producer will be branded as the "cross-dc" Solr module, and be
> available just as any other module is.
> * The Consumer will be branded as the "cross-dc-manager" and be packaged
> just as the prometheus-exporter is, a standalone application that comes with
> Solr and uses many of the same Jars. The reason for the name change is that
> we might expand the role of this application to something beyond just
> "consuming" from the Kafka queue. In that case its much easier to rename it
> now than later.
>
> We will let this bake in the main branch for a while before backporting to
> 9.x.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
