On 01/02/2018 18:55, Brian Yee wrote:
Hello,
I want to use external file field to store frequently changing inventory and
price data. I got a proof of concept working with a mock text file and this
will suit my needs.
What is the best way to keep this file updated in a fast way. Ideally I would
like to read changes from a Kafka queue and write to the file. But it seems
like I would have to open the whole file, read the whole file, find the line I
want to change, and write the whole file for every change. Is there a better
way to do that? That approach seems like it would be difficult/slow if the file
is several million lines long.
Also, once I come up with a way to update the file quickly, what is the best
way to distribute the file to all the different solrcloud nodes in the correct
directory?
Another approach would be the XJoin plugin we wrote - if you wait a few
days we should have an updated patch for Solr v6.5 and possibly v7.
XJoin lets you filter/join/rank Solr results using an external data source.
http://www.flax.co.uk/blog/2016/01/25/xjoin-solr-part-1-filtering-using-price-discount-data/
http://www.flax.co.uk/blog/2016/01/29/xjoin-solr-part-2-click-example/
Cheers
Charlie
--
Charlie Hull
Flax - Open Source Enterprise Search
tel/fax: +44 (0)8700 118334
mobile: +44 (0)7767 825828
web: www.flax.co.uk
