Adam Kocoloski wrote:
Hi Nitin, sure. I think Paul just meant that if we wrote it for
replication, anyone could use the same facility to build something
like what you're talking about.
Hi Adam, Paul,
Yes certainly agreed in theory but in practice I see two major
obstacles, one social and the other technical to this approach.
a) The number of people who can work on the replication code is a
handful and this hugely raises the bar and puts all this in the domain
of "someday-maybe" - "it will need to get on the release schedule" etc.
which is less attractive to me.
On the other hand doing it in userland via couchapp throws open the
gates for anyone to start experimenting with this and then the best
stuff will emerge and get traction, one hopes. I couldn't possibly make
any useful contribution at the replication level at my current level of
knowledge of Couch/Erlang/... but hacking at the couchapp level is not
out of the range of my capabilities. I suspect most users of Couch are
in the same boat as me.
So I deliberately focus on a REST based MIME-payload protocol that could
run in parallel with the REST based JSON-payload protocol.
At this level of abstraction it is easy to understand and talk about for
a much huger audience. Moreover the amount of work needed to bridge the
gap between the already working python code and a couchapp/javascript
impl of this is much smaller.
b) This will all also be possible in the replication layer but a
messaging system has additional requirements such as keeping cumulative
track of where the message came from - adding a MIME-header (JSON
attribute) i.e modifying the doc at each hop etc. These concerns are
not fundamental to point-point doc replication and need to be layered on
top/in addition to replication. Replication needs to faithfully
reproduce a doc from one endpoint to another without the kinds of
modification that happen to headers (metadata attributes) in message
transport.
So thinking of messaging as "merely" replication glosses over some
fundamental issues that distinguish message documents in motion, from
more general documents. Message documents have container level
attributes that get modified during the process of store-and-forward to
track the motion of the doc.
Generic documents in CouchDB have no such requirement and it may be
counterproductive to intermingle the requirements.
In fact it may be possible to make a stronger statement - Couch
replication should focus single mindedly on replication of documents in
a pt-to-pt fashion.
Messaging semantics can then be layered on top of replication, in an
upper layer if desired - or not. Injecting messaging concerns
(especially modification of container attributes) into replication is a
distraction of focus for replication, IMHO and possibly counter to the
requirements of faithful replication.
Hope this clarifies why I reacted somewhat strongly to conflating REST
based messaging with replication - they overlap but should not be
equated if we want to build a messaging system, != a peer to peer doc
sharing system.
After all, the replicator just speaks to CouchDB servers using the
same HTTP requests as everyone else.
I'm sure jchris knows better whether mimeparse would be suitable for
this. Cheers,
Mimeparse.js - looking at the code - seems focused on making the best
match of content-type to provide given an accept header so it may not be
the ...er best match for this.
Cheers,
Nitin
Adam
On Aug 6, 2009, at 5:26 PM, Nitin Borwankar wrote:
Hi Paul,
I never used the word replication - it should be possible to create a
REST based couchapp driven MIME transfer p2p web quiteindependent of
replication which is also cool. Front the couchapp with the usual
auth-proxy stuff for now so only auth'ed people can communicate with
you.
Just replace JSON with MIME in all the reference docs and make the
URL's point to a design doc that does the transformations.
On the way out it could be just _shows or an _list that takes
multiple objects and wraps them as a mime multipart.
On the way in set up some REST endpoints that take POST's, parse mime
multiparts ( jchris's mimeparser?) convert to Couch docs, manage
attachments and puts them in _attachments ... and we're off to the
races - free user controlled, MIME-and-Mail-as-aplatform driven
webmail apps for all.
Yay Couch!
Nitin
Paul Davis wrote:
Definitely some interesting points here. There have been discussions
on using multipart-mime messaging in the replication protocol which
could setup for some interesting prospects like this. I'm not sure on
specifics in terms of replication, but having an endpoint that allows
edits via multipart-mime could be a very fun thing to play with.
Also, AFAIK there's nothing that prevents an isomorphic
representation. As you point out, couchdb-python handles everything
just fine here.
Paul Davis
On Thu, Aug 6, 2009 at 4:46 PM, Nitin Borwankar<[email protected]>
wrote:
Hi guys,
I see that the python based dump/load uses MIME multipart docs as
an on-disk
serialisation format for couchdb databases.
An overall question then arises - can CouchDB be considered a MIME
database
which oh also happens to talk JSON?
So before that - is there a 1-1 strong correspondence between a
CouchDB
document and a MIME multipart, or are there things around the edges
that are
crufty - I would assume a strong correspondence since dump/load
uses it and
I haven't seen any caveast about document content that is not
dumpable.
So assuming the 1-1 correspondence - could one use some
"translation layer"
couchapp that accepts arbitrary content/type + multipart-mixed MIME
object
over HTTP and then transparenty serialise them to JSON underneath.
Given that dump/load already does this - it would see that there
are no
obvious glaring flaws in this logic - but I have been known to be
wrong,
once :-).
If this is indeed feasible - then each CouchDB + MIME-trans becomes
a web
mail node - and Couch begins to be the platform for a messaging
revolution
as well as an application revolution. I am thinking now not as
CouchDB for
backing up your email - but CouchDB as your mail client/server for
p2p MIME
based "email".
Permissions etc are important to avoid complete disaster of course
- but
private high quality communication that just reuses existing message
formats, with better storage and transport would seem like an idea
whose
time has come a long time ago and has been knocking at the door for a
decade.
Yes, yes, there's the issue of spam - so see the P.S.
Just a few idle thoughts,
Nitin
P.S. Back in 1998 I tried to convince Sybase to have MIME as a
native type
in the db and it even got speced out ( I have the spec with the
date on it!
) but got canned becous ethe VP of enginnering wanted to know "what
was the
market exactly for this kind of stuff". Other than that I was
granted a
patent for doing p2p discussions over email back in 2003 - I let it
expire
for multiple reasons. So I am somewhat non-naive about and aware
of the
issues and pitfalls around this sort of thinking. At the same time
I am of
the strong belief that when one looks at messages as data to be
moved around
between endpoints with well defined addressing schemes, and one
ignores the
protocols for a bit, then all sorts of fun things start to happen.
37% of all statistics are made up on the spot
-------------------------------------------------------------------------------------
Nitin Borwankar
[email protected]
