Re: Solr Kafka DIH

2017-02-01 Thread Susheel Kumar
Hello Joel, This definitely seems like a good feature to add. In fact I was also looking to push data into Solr from Kafka and this can be good feature to have. I have created JIRA https://issues.apache.org/jira/browse/SOLR-10086 and can contribute as well. Thanks, Susheel On Tue, Jan 31,

Re: Solr Kafka DIH

2017-01-31 Thread John Bickerstaff
I wrote a simple java microservice that did this about a year ago... It was pretty simple - and the kafka topic served as a way to re-create my collection from scratch without hitting the database again in the event of the Solr servers going down. The code just read from Kafka topic one by one

Re: Solr Kafka DIH

2017-01-31 Thread Joel Bernstein
This would make a great Streaming Expression as well. If you're interested in working on this I'll help out along the way. Here is an example Stream that connects to a JDBC data source:

Re: Solr Kafka DIH

2017-01-31 Thread Mike Thomsen
Probably not, but writing your own little Java process to do it would be trivial with Kafka 0.9.X or 0.10.X. You can also look at the Confluent Platform as they have tons of connectors for Kafka to directly feed into other systems. On Mon, Jan 30, 2017 at 3:05 AM, Mahmoud Almokadem

Solr Kafka DIH

2017-01-30 Thread Mahmoud Almokadem
Hello, Is there a way to get SolrCloud to pull data from a topic in Kafak periodically using Dataimport Handler? Thanks Mahmoud