Re: Bidirectional CDCR not working

[Nish Karve]

Arnold,

Have you copied the configuration from the Solr docs? The bi directional
cluster configuration (for cluster 1) has a malformed XML. It is missing
the closing tag for the updateLogSynchronizer under the request handler
configuration.

Please disregard if you have already considered that in your configuration.
I had a lot of issues trying to figure out the issue when I realized that
it was a documentation error.

Thanks
Nishant


On Thu, Mar 14, 2019, 2:54 PM Arnold Bronley  Configuration is almost identical for both clusters in terms of cdcr except
> for zkHost parameter configuration.
>
> On Thu, Mar 14, 2019 at 3:45 PM Arnold Bronley 
> wrote:
>
> > Exactly. I have it defined in both clusters. I am following the
> > instructions from here .
> >
> https://lucene.apache.org/solr/guide/7_7/cdcr-config.html#bi-directional-updates
> >
> > On Thu, Mar 14, 2019 at 3:40 PM Amrit Sarkar 
> > wrote:
> >
> >> Hi Arnold,
> >>
> >> You need "cdcr-processor-chain" definitions in solrconfig.xml on both
> >> clusters' collections. Both clusters need to act as source and target.
> >>
> >> Amrit Sarkar
> >> Search Engineer
> >> Lucidworks, Inc.
> >> 415-589-9269
> >> www.lucidworks.com
> >> Twitter http://twitter.com/lucidworks
> >> LinkedIn: https://www.linkedin.com/in/sarkaramrit2
> >> Medium: https://medium.com/@sarkaramrit2
> >>
> >>
> >> On Fri, Mar 15, 2019 at 1:03 AM Arnold Bronley  >
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I used unidirectional CDCR in SolrCloud (7.7.1) without any issues.
> But
> >> > after setting up bidirectional cdcr configuration, I am not able to
> >> index a
> >> > document.
> >> >
> >> > Following is the error that I am getting:
> >> >
> >> > Async exception during distributed update: Error from server at
> >> > http://host1:8983/solr/techproducts_shard2_replica_n6: Bad Request
> >> > request:
> >> > http://host1
> >> >
> >> >
> >>
> :8983/solr/techproducts_shard2_replica_n6/update?update.chain=cdcr-processor-chain=TOLEADER=
> >> >
> >>
> http://host2:8983/solr/techproducts_shard1_replica_n1=javabin=2
> >> > Remote error message: unknown UpdateRequestProcessorChain:
> >> > cdcr-processor-chain
> >> >
> >> > Do you know why I might be getting this error?
> >> >
> >>
> >
>

Re: Alternative for DIH

[Nish Karve]

If you absolutely want to use Kafka after trying other mechanisms, I would
suggest Kafka Connect. Jeremy Custenborder has a good Kafka connector as a
sink to SOLR. You can define your own avro schemas on the Kafka topic that
adhere to your SOLR schema to give you that degree of control.

We have used Lucidworks Spark Connector to index 500 million documents to
SOLR within 4 hours. We had around 70 fields per document. This is a very
good choice if you want to sync data from a DB to SOLR. Have an interim
step of using an ETL tool like Ab Initio that will perform the basic joins
on your table, extract the data in CSV for the Spark Connector. All the
hardwork of opening and managing the connections with SOLR is done in the
connector. Please note that this connector indexed data to a live SOLR
cluster unlike offline indexing using Map Reduce.

Thanks
Nishant

On Thu, Jan 31, 2019, 5:15 AM Srinivas Kashyap  Hello,
>
> As we all know DIH is single threaded and has it's own issues while
> indexing.
>
> Got to know that we can write our own API's to pull data from DB and push
> it into solr. One such I heard was Apache Kafka being used for the purpose.
>
> Can any of you send me the links and guides to use apache kafka to pull
> data from DB and push into solr?
>
> If there are any other alternatives please suggest.
>
> Thanks and Regards,
> Srinivas Kashyap
> 
> DISCLAIMER:
> E-mails and attachments from Bamboo Rose, LLC are confidential.
> If you are not the intended recipient, please notify the sender
> immediately by replying to the e-mail, and then delete it without making
> copies or using it in any way.
> No representation is made that this email or any attachments are free of
> viruses. Virus scanning is recommended and is the responsibility of the
> recipient.
>