Parallel == multiple-threads across multiple-machines in the cluster? :-) By definition, temp views don't have no disk IO. It's map/reduce parallelized in memory directly served back over a TCP socket. Is that still not going to be fast enough?
K. On Fri, Apr 10, 2009 at 7:26 PM, Paul Davis <[email protected]> wrote: > On Fri, Apr 10, 2009 at 8:51 PM, kowsik <[email protected]> wrote: >> IMHO, the need for view intersections will go away once we have >> parallel map/reduce to the point where _temp_views's are fast! >> >> K. >> > > The lower bound for view generation is disk I/O. Temp views will never > be fast enough for production. > > HTH, > Paul Davis > >> On Fri, Apr 10, 2009 at 10:04 AM, Wout Mertens <[email protected]> >> wrote: >>> >>> On Apr 10, 2009, at 11:46 AM, Sho Fukamachi wrote: >>> >>>> the obvious followup question to those examples is "well, how do I find a >>>> document with all of (n) tags?". >>> >>> How about this algorithm. Needed: tagcount view and document-by-tag view >>> >>> - given a list of tags that the document should have >>> - find the tag that has the lowest document count with the tagcount view >>> - request all documents with that tag through the document-by-tag view >>> - filter manually on documents that match >>> >>> If that would mean too many documents, make a view that emits all >>> combinations of 2 tags a document has, that way you filter by that much >>> more. >>> >>> It would be neat if one could post a temporary view that runs against a >>> subset of the output of a real view. That way the viewserver farm could do >>> the filtering... >>> >>> Wout. >>> >> >
