On Fri, Nov 21, 2014 at 10:50 PM, Dibyendu Bhattacharya < [email protected]> wrote:
> Hi, > > I have recently presented in LuceneSolrRevolution one of our Apache Blur > use case . This is about how we did Indexing Kafka Stream into Blur via > Spark Streaming. Here is the slides : > > > http://www.slideshare.net/lucidworks/near-real-time-indexing-kafka-messages-into-apache-blur-presented-by-dibyendu-bhattacharya-pearson-north-america > Thanks! > > Aaron, as I told you earlier, using Spark Streaming I can achieve very > high throughput of indexing, but I can see the IndexImporter is falling > behind to merge at the same pace indexing is happening. Is the latest fix > to faster index merger can solve this problem ? > We are making progress, there are some interesting read patterns during the merge that are causing some of the issues in performance. > > Regards, > Dibyendu > > > > > > On Mon, Sep 1, 2014 at 10:45 AM, Dibyendu Bhattacharya < > [email protected]> wrote: > >> Dear All, >> >> Here is the session details on "Near Real Time Indexing Kafka Messages >> into Apache Blur" which we will be presenting at LuceneSolr Revolution. >> >> http://lucenerevolution.org/presentations/ >> >> See under "What's Hot in OSS" category. >> >> I have written distributed Kafka Consumer using "Spark Streaming" and >> used this consumer to index messages into Apache Blur via Blur queuing API. >> Here is the github project for Kafka-Spark consumer : >> https://github.com/dibbhatt/kafka-spark-consumer >> >> Regards, >> Dibyendu >> >> >> >
