Maybe you can try or extend Sematext’s Redis parser:
<https://github.com/sematext/solr-redis>. Downside of this approach is another
moving part - Redis.
Monitoring - Log Management - Alerting - Anomaly Detection
Solr & Elasticsearch Consulting Support Training - http://sematext.com/
> On 1 Feb 2018, at 19:55, Brian Yee <b...@wayfair.com> wrote:
> I want to use external file field to store frequently changing inventory and
> price data. I got a proof of concept working with a mock text file and this
> will suit my needs.
> What is the best way to keep this file updated in a fast way. Ideally I would
> like to read changes from a Kafka queue and write to the file. But it seems
> like I would have to open the whole file, read the whole file, find the line
> I want to change, and write the whole file for every change. Is there a
> better way to do that? That approach seems like it would be difficult/slow if
> the file is several million lines long.
> Also, once I come up with a way to update the file quickly, what is the best
> way to distribute the file to all the different solrcloud nodes in the
> correct directory?