On Tue, Mar 17, 2009 at 4:07 PM, Daniel Friesen <[email protected]> wrote: > Dean Landolt wrote: >> >> On Mon, Mar 16, 2009 at 9:10 PM, Daniel Friesen >> <[email protected]>wrote: >> >> I can tell you it'd be a whole lot easier to just return every object for >> an >> entire page (if you store the root page with each document). It fits much >> nicer with the model of views -- and you can always filter on the client >> side. What kind of scale are we talking about though? Could there be >> thousands of widgets on a page? If so, some smaller logical grouping could >> be considered and you could just pull in what you need with a multi-key >> request. >> > > Widget is actually just a general word I threw out. We actually call them > "jits" in our app because quite simply they do everything. A "jit" ranges > from anything to a container for stuff, to a blob of text, and image tag, a > button, a calendar, an accordion, etc... And a very large number of them > appear on a page (go to a random webpage (I suppose not tooo complex), count > up the number of divs or whatever elements on the page, and then you'll get > a rough number of how many jits will be on a page). > I haven't bothered to benchmark how many SQL calls we're actually making on > every single pageview currently, but quite simply I already know that we're > making an extreme amount of them just because we have to query for data so > we can query for data dependent on it. > > Bradford Winfrey wrote: >> >> Just thinking aloud here, but storing each widget (if as Dean asked there >> are in fact "lots" of them) would be much better suited to be stored in its >> own document, should a widget change, you'd save a significant amount of >> space (since couch will keep a full copy of each revision until compaction) >> rather than saving the whole collection of them each time. >> Just a thought... >> Brad >> > > I intend to, that was just a structure example. I've looked at CouchDB a bit > more and decided the best idea was to store widgets in their own documents. > Not just because of space, but for writing (like the comments example on the > couchdb-joins article), and because jits don't really fit in the same > document. > > > Chris Anderson wrote: >>> >>> ... >>> >> >> Taking this approach would probably require that you store the full >> path(s) to the widget on the widget document itself. This isn't so bad >> but does complicate changing only a parent node say, halfway up the >> tree, as you have to touch all it's subnodes. >> > > Not really possible. There is no full path to the widget in the document. > Quite simply because widgets don't belong to a document (I have limited it > so that widgets can't cross sites) but a widget can belong to multiple > parent widgets and as a result can belong to multiple pages. > The ability to put the same Widgets on multiple pages is essential to the > app we're building so it can't really be worked around. >
This is possible, as long as you are willing to eat up a fair amount of disk space caching *all possible* paths to the jit. I guess I'm essentially talking about running your "walk" function and then memoizing the results of it [an array of paths] on the jit document itself. This should be possible as long as the graph of nested jits is acyclic. The queries from such a view would be pretty clean. For a given root page-id you'd be able to pull in all and only the jits on that page (sorted according to their nesting order) in a single request. > I actually drastically simplified the example. We actually have another > level, "references" which make it possible to have multiple instances of the > same jit instance. So something could actually end up like: > > page:A hasJit ref:AAA refersto jit:BBB hasJit ref:CCC refersto jit:DDD > ^^^^^^^ hasJit ref:EEE refersto jit:FFF > > page:B hasJit ref:GGG refersto jit:HHH hasJit ref:III refersto jit:FFF > > jit:FFF ...some > jit data... > > A jit reference normally stores only things like what jit it refers to, and > other things like sortkeys which don't belong to the jit, but rather belong > to it's individual existence on a page. While the Jit itself stores things > like what type of jit it is (container, text, calender, etc...), what jits > are inside of it (if the jit type can support stuff inside it; container, > accordion, etc...), jit level access control (a jit is editable if you can > edit the site, you can edit the page, or you can edit the jit itself), and a > jit "state" which is basically a key/value hash of data which the jit type > is responsible for managing and turning into the actual output. > > > The only possibilities I can think of, are a view with the ability to use > data from other documents it knows the id for. Or a plugin that makes it > possible to write a "walk" function similar to the code in a view (ie: JS). > So what it looks like, is I need a database engine with a good lightweight > scripting capability builtin for mangling data into query output to avoid > doing all that work in the app itself and killing both with the insane > amount of extra traffic. > > ~Daniel Friesen (Dantman, Nadir-Seen-Fire) [http://nadir-seen-fire.com] > -Nadir-Point & Wiki-Tools (http://nadir-point.com) (http://wiki-tools.com) > -MonkeyScript (http://monkeyscript.org) > -Animepedia (http://anime.wikia.com) > -Narutopedia (http://naruto.wikia.com) > -Soul Eater Wiki (http://souleater.wikia.com) > > -- Chris Anderson http://jchris.mfdz.com
