Hey, Jan asked me to post a discussion from IRC here or on the wiki, and I didn't see a wiki section that looked like it was a good place for a RandomIRCDiscussion page.
This was basically a discussion that went from bulk_docs fetch vs. http pipelining into a discussion about storing graphs in couchdb. Not sure which part jan wanted so here's all of it. Paul jan____ hmmm, bulk fetch.... 16:36 jan____ sounds sweet 16:37 davisp yah, though damienkatz said that it might be replaced by the pipelining which sounds like just sending a set of http requests and getting responses to them all at once 16:37 davisp I'm thinking that might require a custom client library in most languages though 16:38 jan____ need to read up on http pipelining... 16:39 dreid Not all at once. 16:39 dreid You send a series of requests, and the server responds to them in order. 16:39 davisp dreid: Sounds about right 16:39 davisp Totally didn't realize till just now it was part of the http/1.1 spec 16:40 dreid There is no promise that the responses will be received "all at once". 16:40 dreid I'm not entirely sure I like the idea, because it requires more logic on the client's part. 16:40 davisp dreid: How do you mean? Like, the server may decide to pause, or it may say "Hey fucker, to many requests, try a new connection"? 16:41 dreid davisp: A good example is two requests GET /ubuntu-8.04.iso HTTP/1.1 & GET /favicon.ico HTTP/1.1 16:41 davisp With you so far 16:41 dreid If those requests are pipelined, the server will not send /favicon.ico until it has sent all of /ubuntu-8.04.iso 16:42 dreid So the pipeline must be served in the order of the requests. 16:42 davisp well, there is only one pipe :D 16:43 dreid Another good example is a pipeline of a request of a couple of documents, and then a view (say tag counts) and then some more documents. 16:44 davisp Hmm. reading http://www.mozilla.org/projects/netlib/http/pipelining-faq.html makes me feel uncomfortable 16:44 davisp ie, the client needing more logic seems fairly accurate 16:48 davisp I think my tree view is working. Yay. 16:53 jan____ ncie 16:53 jan____ *speling 16:54 davisp I need to add children for each node. And for some of these, the multipel doc requests will probably start sucking. Although... I should look and see how deep the tree actually gets 17:02 plum Created: (COUCHDB-47) Building CouchDB against Spidermonkey source: https://issues.apache.org/jira/browse/COUCHDB-47 17:07 jan____ meh, RTFW :) 17:12 davisp jan____: Did you see the mailing list thread about the issues running behind a firewall? 17:12 jan____ davisp: glanced over that 17:12 davisp Where in the world did HttpConfig come from 17:13 jan____ ooooooold code 17:13 jan____ that's gone with mochiweb 17:13 davisp phew. I thought I was missing something 17:15 dreid davisp: Not to mention that there are very popular environments that give the actual client no control over wether or not pipelining is used. (i.e. XHR) 17:16 davisp dreid: Fair point. all I was looking for was something like a _bulk_docs get functionality too :( 17:18 jan____ davisp: that might be actually easy to add as a plug-in once we refactor the HTTP API code 17:18 dreid Well ... views are like that. 17:18 davisp dreid: This particular case I can't get the docs in the view 17:19 *** olegandreev joined #couchdb 17:19 dreid Granted, views are probably inefficient if all you want is, give me the entire body of the documents specified by this bunch of document IDs. 17:20 davisp dreid: Which is exactly what i'm looking for. Although, the list of ids comes from a view :D 17:20 davisp Everytime I use an emoticon, a fair dies. :( 17:20 davisp *fairy 17:21 dreid davisp: Are you trying to get the documents for a list of children? 17:21 davisp dreid: Trying to get the documents for a given node in a DAG so that I can build the entire tree up to the root as well as that node's immediate children 17:22 dreid I don't know what a DAG is. 17:22 davisp directed acyclicl graph 17:22 dreid I do know that you could do this with a view if every document had a reference to it's parent. 17:22 davisp Before inserting documents I make sure that each document has a reference to every one of it's descendants 17:23 davisp And a view like for i in descendants: map( doc.descendants[i], doc._id ) 17:23 davisp dreid: but if you can think of something better, I'm all ears 17:29 *** schapht left. 17:29 dreid Ok, so how about before inserting documents you make sure that each document has a reference to it's parent? 17:29 dreid Then you should be able to have a view that says "get me all documents whose parent is x" 17:30 *** binary42 quit (Read error: 110 (Connection timed out)) 17:31 davisp Well, that's the basic idea I'm using... kind of. But I'm more interested in "Give me all documents that have x as a descendant" which would be equivalent to "Give me X's parent, X's parent's parent, X's parent's parent's parent" etc. And it also allows for multiple paths from the root to X with finesse. Drawing the view with different edges for different relations will require data from the documents, but once you have the list of ids you just have to get the docs 17:33 davisp oh, actually, I don't need multiple docs. I can just map( descendant, doc ) can't I? 17:33 dreid I have to admit, CouchDB would not have been my first choice to store a digraph. 17:33 davisp and in an hour, that should be done reindexing 17:34 davisp How would you rather store it? 17:38 *** slightlyoff_afk is now known as slightlyoff 17:44 *** naderman quit (Nick collision from services.) 17:44 *** naderman_ joined #couchdb 17:45 *** bobesponja quit (Remote closed the connection) 17:45 *** bobesponja joined #couchdb 17:49 *** jan_____ joined #couchdb 17:49 jan_____ any replies to "How would you rather store it?" that I missed? 17:50 jan_____ (Apple, can you please make Airport work reliably with airport base stations?) 17:50 *** jan____ quit (Read error: 104 (Connection reset by peer)) 17:56 davisp jan_____: Nope 17:57 jan_____ ok, can you send results to the ML or put it into the wiki for future reference? gotta sleep now 17:57 davisp After I'm done reading this book I'm gonna have to spend some serious time with the indexing. 17:57 jan_____ :) 17:57 davisp jan_____: Results for what? 17:57 jan_____ you discussion with dreid 17:57 jan_____ *your 17:58 davisp will do 17:58 jan_____ thanks! 17:58 jan_____ night-o 17:58 davisp Awesome! 17:58 davisp Night man 17:58 jan_____ :) 17:58 *** jan_____ left.
