The backing store doesn't matter much, in the sense that using it for real-time computation needs it to all end up in memory anyway. It can live wherever you want before that, like Solr. It's not going to be feasible to run anything in real-time off Solr or any other store. Yes the trick is to use Solr to figure out what has changed efficiently much like update files.
If you're using Hadoop, same answer mostly. It's going to read serially from wherever the data is and most stores are fine at listing out all data sequentially. On Thu, Aug 2, 2012 at 3:52 AM, Matt Mitchell <[email protected]> wrote: > Hi, > > The data I'm using to generate preferences happens to be in a solr > index. Would it be feasible, or make any sense, to write an adapter so > that I can use solr to store the preferences as well? The solr > instance could be embedded since this is all java, and would probably > end up being pretty quick. Our data is coming in fast, and I think > we'll outgrow the file based approach quickly. Thoughts? > > - Matt >
