On 11 Feb 2009, at 08:52, 80n wrote:
On Wed, Feb 11, 2009 at 2:32 AM, Brett Henderson <[email protected]>
wrote:
Iván Sánchez Ortega wrote:
> El Miércoles, 11 de Febrero de 2009, Shaun McDonald escribió:
>
>> Having a slightly less efficient import that ensures consistency
is a
>> great thing, as it means that the database doesn't grow too
quickly.
>>
>
> I fail to see why slower growth is a good thing?
>
Slower (perhaps "steady" is a better word) growth has its advantages
for
replication. Osmosis data extraction is fairly efficient and can
extract data many times faster than it is currently added, but
downstream systems can't always process it fast enough. So while it
would be great to be able to import data faster, there probably should
be an upper limit to import speed allow downstream systems to keep up.
This is not a valid reason.
Speaking as one of the downstream consumers of the Osmosis diffs we
can take it as fast as you can make it.
Thinking about it for a bit longer, wouldn't you like to be able to
predict when you need to increase the storage capacity, or change an
algorithm to be able to cope with the increased data. Or what about
being able to predict when we need to upscale the main database server
or run another donation drive for more servers?
That's why I wouldn't want to suddenly see a week of the API being
hammered with maximal data imports.
Shaun
_______________________________________________
dev mailing list
[email protected]
http://lists.openstreetmap.org/listinfo/dev
