James, In general, I think this is a reasonable approach (for some reasonable value of "real time"). A couple of questions.
Do you mean to use a standalone Solr instance, or the Yokozuna project (Solr integrated with Riak, in Riak 2.0)? (You can read about its usage here: http://docs.basho.com/riak/2.0.0pre11/dev/using/search/ )? I'm assuming you mean the latter, but just wanted to double-check. The approach of using Solr only for the indexing and querying (especially taking advantage of Solr's pagination abilities), and storing the data in Riak (and fetching via multiget) is a good one. The speed of the multiget will depend on what programming language (and which Riak client) you'll be using, but again, should be doable. Your best bet is to run some tests, and see if the speed is acceptable to your use case. On Tue, Mar 18, 2014 at 1:22 AM, James Hardy <[email protected]> wrote: > Hi Everyone > > I'm new to Riak and was hoping someone might be able to give me some > advice with how well suited Riak is to large data sets queried in real > time. My potential use case is similar to the idea of storing the json for > Twitter tweets in one bucket and then the json for the author of the tweet > in another. I'm hoping to use Solr (not Riak Search) for search. The idea > being I store a subset of the tweet in Solr with the ID of the author. > Then when a search is performed the author IDs are extracted from the > results and then fetched from Riak. I want to keep the Solr index to a > minimum and don't want to store all the author information in the index as > some of the values will change on a regular basis (think number of > followers etc). Will a Riak multi-fetch be fast enough to do this in real > time on a large data set? If not can anyone recommend another technology I > could either use as an intermediary (cache etc) or am I just using Riak in > the wrong way? I'd love to hear people's thoughts as I'm new to both Riak > and to a certain extent NoSQL as well and don't want to use the wrong tool > for the job. > > Many thanks > > James > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > >
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
