On 7 September 2012 17:28, Roland Olbricht <[email protected]> wrote:
> The planet file is necessary for the first startup. Afterwards, it can > work forever solely on minute diffs. And new instances can be cloned from > the exisiting instance over a public interface, without a planet file. > > This initial startup has usually the following timing: > > 2 hours to download planet.osm.bz2 > > 12-24 hours to import planet.osm.bz2 into the database. This contains > various optimizations that are fine tuned on the properties of the XML > planet structure and maybe gets worse with a differently organised file > format. > > 4-30 hours to catch up by applying the minute diffs. This depends on how > many days the planet file is old (always at least two days, one for the > planet file itself, a second because we have done the above import > procedure). > > Altogether, there is not much point in "just improving the download time", > because it has a diminishing share of the total time needed. It would be by > far more important to get the import step again fast. > > I doubt that converting the PBF into a XML is done in the saved half an > hour on the download, so the simplest solution to convert after download is > surely a slowdown. > If overall duration is an issue, is it worth patching a planet using hourly/minutely diffs before importing the planet itself? That way you can begin importing a planet file that is no more than a few hours old rather than using a freshly downloaded file that is already old due to the time taken to create a planet. If you do this you'll have less diffs to apply to your database to catch up after completing the import. You could convert a PBF planet to XML file as part of the same patching step. You could stream the XML output directly into your import job saving more time. Brett
_______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
