On Saturday 05 May 2012 12:21:24 Clint Byrum wrote: > Excerpts from Julian Edwards's message of Thu May 03 23:42:12 -0700 2012: > > On Friday 04 May 2012 18:12:44 Robert Collins wrote: > > > I would discourage, strongly discourage, any direct DB access from > > > pserv: our experience with LP with such access has been universally > > > bad. Let the appserver drive the DB exclusively, and offer appropriate > > > APIs for getting stuff from/to it. I think we glossed over this on > > > IRC; celery talking to postgresql might mean this needs some extra > > > glue for celery, or something. > > > > For read access only, can you elaborate why this is bad? > > I've not been involved with Launchpad, but I have done a few multi-tiered > architectures. > > There are a few reasons: > > * The database used is an implementation detail. Putting a lightweight > layer of indirection between the DB and the other pieces of the app means > being able to swap out the DB for the cases that matter. With the hyper > scale requirement, this is likely to happen as it becomes clear which > tables just cannot be served through a purely relational model. API's > map intentions rather than implementations. > > * API's can be used as layers of control. The postgres and mysql > protocols both make proxying a real chore, and so, its hard to control > the number of threads. pgbouncer seems pretty good, but it then requires > a dedicated proxy just for pgsql, which ties you further into pg. An API > call, however, can be extended to provide needed metrics, and then be > an intelligent choke point or pressure-release for a limited resource > like the database. > > * Intelligence in the pipeline. This makes it easier to cache > intelligently, easier to route/shard/etc. The layer of indirection used > to be just in code, but you really need it in the network separation > so that the pieces can be scaled individually and whole parts can be > refactored without touching every place that might access that place.
Thanks Clint, that's well elaborated. For the record, I was playing Devil's Advocate to some extent since we'd be insulated through Django's ORM, but the points are well understood. > Put more succinctly, API changes are easier than schema changes. I'd argue the opposite if you're using lazr.restful :) J -- Mailing list: https://launchpad.net/~maas-devel Post to : [email protected] Unsubscribe : https://launchpad.net/~maas-devel More help : https://help.launchpad.net/ListHelp
