On 01/02/2018 18:55, Brian Yee wrote:

I want to use external file field to store frequently changing inventory and 
price data. I got a proof of concept working with a mock text file and this 
will suit my needs.

What is the best way to keep this file updated in a fast way. Ideally I would 
like to read changes from a Kafka queue and write to the file. But it seems 
like I would have to open the whole file, read the whole file, find the line I 
want to change, and write the whole file for every change. Is there a better 
way to do that? That approach seems like it would be difficult/slow if the file 
is several million lines long.

Also, once I come up with a way to update the file quickly, what is the best 
way to distribute the file to all the different solrcloud nodes in the correct 

Another approach would be the XJoin plugin we wrote - if you wait a few days we should have an updated patch for Solr v6.5 and possibly v7. XJoin lets you filter/join/rank Solr results using an external data source.




Charlie Hull
Flax - Open Source Enterprise Search

tel/fax: +44 (0)8700 118334
mobile:  +44 (0)7767 825828
web: www.flax.co.uk

Reply via email to