Steve, another though re. your original post;
On 01/09/11 22:38, SteveC wrote:
Could we accept the edit traffic? No, far too much. Could we provide a good user experience, clearly no. Could they help us scale? No they would be viewed as taking over on any kind of timescale they needed. Could they host us? Again no, it would be too slow of a process and it'd be a takeover and the community would probably reject it.
I think something that's entirely feasible and in no way a "takeover" would be setting up a number of very fast read-only mirrors that fully support all the API read calls - most importantly the "map" call but also the changeset download, full way/full relation download, and individual object access. (Along the lines of what Nic said, but it doesn't necessarily require special software.)
The technology is there - albeit a bit clumsy: consume minutely diffs with Osmosis and run a read-only rails port on the resulting database. Scaling is unlimited - set up 100 of these boxes with a load balancer in front, maybe add an internal HTTP proxy so you don't request the minutely diffs 100 times but only once - easy.
The only thing we'd have to change to make use of such an infrastructure is that editors would have to learn to read from one server and write to another. That's a very small change.
The risk of conflicts during upload would be increased but that increas would not come from this technology - the technology would introduce a minimal extra delay that is negligible compared to the average length of an edit session; the increase would come from more people editing at the same time and it would be just the same with any other method of scaling.
So, in addition to providing a lot of iron for the read access caching, those who want to turbo-charge OSM would perhaps have to look into improving conflict handling with the major editors, or implement some kind of conflict resolution in the API, but again that's not really a database scaling issue...
Bye Frederik _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
