Nine times faster than O(n) is still O(n). If people take this on, I will be your biggest cheerleader. My point is to ask what problem this solves. After some thought I think it does NOT solve a technical problem, it solves a perception and adoption problem.
The problem a 9x speed up solves is, "I am prototyping something and my benchmarks are not impressive." Maybe that is enough. If CouchDB causes friction for adopters, then this would actually be a big priority. I see a Pareto distribution of data set sizes. Databases are either small, with all views building reasonably quickly; or very large (above 10GB) where no speedup will change the fundamental feeling of the view build: you kick it off, then you wait. And again, these speeds diminish in importance over a project life cycle. On Sun, Aug 18, 2013 at 11:51 AM, Thanos Vassilakis <[email protected]>wrote: > Build views performance gains: > Python 4-6 times faster + less memory > Erlang 7-9 times faster + much less memory. > > On Aug 16, 2013, at 18:54, Jan Lehnardt <[email protected]> wrote: > > > I think it is worth putting Jason’s and Jens’s viewpoints on a scale of > “learning to live with the pain” and “finding relief for the pain”, where > “pain” is any of View Build Time or View Index Time from Jason’s glossary. > > > > If living with the pain works for you, Jason’s point of view is very > valuable because all you need to is accept the current situation :) > > > > But I think it is also worth considering that there are scenarios where > living with the pain is just not a fun or viable thing to do and then > finding relief for the pain is a good strategy. > > > > This thread started a fork on dev@ with concrete engineering > suggestions on how to make this a reality in CouchDB, so please join us > there if you want to help out. > > > > Best > > Jan > > -- > > > > > > On Aug 16, 2013, at 17:24 , Jason Smith <[email protected]> wrote: > > > >> On Fri, Aug 16, 2013 at 10:16 PM, Jens Alfke <[email protected]> > wrote: > >> > >>> > >>> On Aug 15, 2013, at 10:14 AM, Jason Smith <[email protected]> wrote: > >>> > >>>> To restate my final sentence which you quoted: Accessing a view is an > >>> index > >>>> scan, it hardly matters what the total data size is; therefore after > the > >>>> building period, all views are basically always instantly available. > >>> > >>> You’re missing my point, Jason. Introducing an arbitrary “building > period” > >>> implies you don’t care about the latency between a database update and > when > >>> it becomes visible in view queries. As I said, that may be allowable in > >>> some types of applications, but definitely not all. To repeat my > example, > >>> an e-commerce customer is not going to tolerate any “building period” > >>> longer than a second or two in between their clicking “Add To Cart” > and the > >>> item showing up in their cart. > >>> > >> > >> I agree. But a hypothetical "Add to cart" would only be one or two > document > >> updates. That will be reflected in a view query pretty quickly. The HTTP > >> overhead is probably still the majority of the time spent; updating the > >> view index to reflect the new cart will be negligible. > >> > >> Do we agree there? It occurs to me that you and I may work in different > >> environments. Perhaps you are accustomed to much heavier update > frequency > >> than I. > >> > >> My own glossary: > >> > >> View build time: Building the views the first time you push a new design > >> document into an existing database. Can take a very long time, but can > be > >> done "offline." > >> > >> View update time: The time it takes couch to update a view to reflect > those > >> updates which happened since either the view build, or the view update > >> (whichever was more recent). > >> > >> To me, view update time is regrettable but for most applications > >> (read-heavy) it is not much of a problem. View build time is not a > concern > >> because you can run it at your leisure and "promote" it to the true > >> document ID using COPY. In most situations, native views solve a > >> non-problem (although I've cited a few cromulent exceptions.) > > >
