Us too. That's going to be huge for us! Michael Della Bitta
Applications Developer o: +1 646 532 3062 | c: +1 917 477 7906 appinions inc. "The Science of Influence Marketing" 18 East 41st Street New York, NY 10017 t: @appinions <https://twitter.com/Appinions> | g+: plus.google.com/appinions<https://plus.google.com/u/0/b/112002776285509593336/112002776285509593336/posts> w: appinions.com <http://www.appinions.com/> On Wed, Dec 18, 2013 at 9:55 AM, Mikhail Khludnev < mkhlud...@griddynamics.com> wrote: > Aha! SOLR-5244 is a particular case which I'm asking about. I wonder who > else consider it useful? > (I.m sorry if I hijacked the thread) > 18.12.2013 5:41 пользователь "Joel Bernstein" <joels...@gmail.com> > написал: > > > They are for different use cases. Hoss's approach, I believe, focuses on > > deep paging of ranked search results. SOLR-5244 focuses on the batch > export > > of an entire unranked search result in binary format. It's basically a > very > > efficient bulk extract for Solr. > > > > > > On Tue, Dec 17, 2013 at 6:51 PM, Otis Gospodnetic < > > otis.gospodne...@gmail.com> wrote: > > > > > Joel - can you please elaborate a bit on how this compares with Hoss' > > > approach? Complementary? > > > > > > Thanks, > > > Otis > > > -- > > > Performance Monitoring * Log Analytics * Search Analytics > > > Solr & Elasticsearch Support * http://sematext.com/ > > > > > > > > > On Tue, Dec 17, 2013 at 6:45 PM, Joel Bernstein <joels...@gmail.com> > > > wrote: > > > > > > > SOLR-5244 is also working in this direction. This focuses on > efficient > > > > binary extract of entire search results. > > > > > > > > > > > > On Tue, Dec 17, 2013 at 2:33 PM, Otis Gospodnetic < > > > > otis.gospodne...@gmail.com> wrote: > > > > > > > > > Hoss is working on it. Search for deep paging or cursor in JIRA. > > > > > > > > > > Otis > > > > > Solr & ElasticSearch Support > > > > > http://sematext.com/ > > > > > On Dec 17, 2013 12:30 PM, "Petersen, Robert" < > > > > > robert.peter...@mail.rakuten.com> wrote: > > > > > > > > > > > Hi solr users, > > > > > > > > > > > > We have a new use case where need to make a pile of data > available > > as > > > > XML > > > > > > to a client and I was thinking we could easily put all this data > > > into a > > > > > > solr collection and the client could just do a star search and > page > > > > > through > > > > > > all the results to obtain the data we need to give them. Then I > > > > > remembered > > > > > > we currently don't allow deep paging in our current search > indexes > > as > > > > > > performance declines the deeper you go. Is this still the case? > > > > > > > > > > > > If so, is there another approach to make all the data in a > > collection > > > > > > easily available for retrieval? The only thing I can think of is > > to > > > > > query > > > > > > our DB for all the unique IDs of all the documents in the > > collection > > > > and > > > > > > then pull out the documents out in small groups with successive > > > queries > > > > > > like 'UniqueIdField:(id1 OR id2 OR ... OR idn)' > > 'UniqueIdField:(idn+1 > > > > OR > > > > > > idn+2 OR ... etc)' which doesn't seem like a very good approach > > > because > > > > > the > > > > > > DB might have been updated with new data which hasn't been > indexed > > > yet > > > > > and > > > > > > so all the ids might not be in there (which may or may not > matter I > > > > > > suppose). > > > > > > > > > > > > Then I was thinking we could have a field with an incrementing > > > numeric > > > > > > value which could be used to perform range queries as a > substitute > > > for > > > > > > paging through everything. Ie queries like 'IncrementalField:[1 > TO > > > > 100]' > > > > > > 'IncrementalField:[101 TO 200]' but this would be difficult to > > > maintain > > > > > as > > > > > > we update the index unless we reindex the entire collection every > > > time > > > > we > > > > > > update any docs at all. > > > > > > > > > > > > Is this perhaps not a good use case for solr? Should I use > > something > > > > > else > > > > > > or is there another approach that would work here to allow a > client > > > to > > > > > pull > > > > > > groups of docs in a collection through the rest api until the > > client > > > > has > > > > > > gotten them all? > > > > > > > > > > > > Thanks > > > > > > Robi > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Joel Bernstein > > > > Search Engineer at Heliosearch > > > > > > > > > > > > > > > -- > > Joel Bernstein > > Search Engineer at Heliosearch > > >
