On Fri, Mar 25, 2011 at 4:17 PM, Robert Collins <[email protected]> wrote: > On Sat, Mar 26, 2011 at 9:04 AM, Benji York <[email protected]> wrote: >> On Thu, Mar 24, 2011 at 10:11 PM, Robert Collins >> <[email protected]> wrote: >>> This is about optimising a particular part of the current webservice; >>> its still useful in an expand/filter world I think, because some >>> collections are massive. >> >> If I'm understanding the various moving parts correctly, the >> filter/expand mechanism would remove the need for this optimization. >> Not that we shouldn't do it for the short term gains (about which I have >> no opinion one way or the other). >> >> How would filter/expand help in this scenario? It will first return the >> membership of whatever set you specify and the client would then batch >> using that information. There isn't any need to do batch calculations >> on the server side, avoiding the expensive surrogate key calculations. > > the filter/expand system still runs into trouble in very large > batches; we don't know how large 'very large' is because we haven't > written it yet. A trivial example of where very large is too large: > ubuntu/allbugsever
Indeed, we can't be sure because we haven't built it yet, but I'm hopeful that we'll find a way. For example, 1 million bug URIs is 47 megabytes. Gzipped they take up 2.5 meg. If that's any indication, we might be able to avoid batching set membership altogether. -- Benji York _______________________________________________ Mailing list: https://launchpad.net/~launchpad-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~launchpad-dev More help : https://help.launchpad.net/ListHelp
