Maybe you can try or extend Sematext’s Redis parser: https://github.com/sematext/solr-redis <https://github.com/sematext/solr-redis>. Downside of this approach is another moving part - Redis.
HTH, Emir -- Monitoring - Log Management - Alerting - Anomaly Detection Solr & Elasticsearch Consulting Support Training - http://sematext.com/ > On 1 Feb 2018, at 19:55, Brian Yee <b...@wayfair.com> wrote: > > Hello, > > I want to use external file field to store frequently changing inventory and > price data. I got a proof of concept working with a mock text file and this > will suit my needs. > > What is the best way to keep this file updated in a fast way. Ideally I would > like to read changes from a Kafka queue and write to the file. But it seems > like I would have to open the whole file, read the whole file, find the line > I want to change, and write the whole file for every change. Is there a > better way to do that? That approach seems like it would be difficult/slow if > the file is several million lines long. > > Also, once I come up with a way to update the file quickly, what is the best > way to distribute the file to all the different solrcloud nodes in the > correct directory?
