Am 25.01.2020 um 22:19 schrieb marc marc: > Le 25.01.20 à 21:43, Simon Poole a écrit : >> Am 25.01.2020 um 21:12 schrieb marc marc: >>> is there a convenient way to update a pbf to exactly the same diff as >>> osm.org? and/or add the state.txt of the pbf produced on osm.org? >>> this would allow to have the same pbf mirrored on other sites without >>> having to download and/or encode the same thing several times >> The answer is likely no, because I don't believe there is a guarantee >> that the planet dump contains exactly the commits of a certain set of >> diffs, > the planet dump is done from the db itselft and not using by a cascade > of diffs (min->hourly>daily>weekly) ? > or maybe a special diff between planet dump.
Yes the dump works via a completely different mechanism. The problem is actually fixable (not expecting a lot of enthusiasm for it though), we could post creating the dump run an update from a specific diff prior to when the dump started on the dump before publishing. Doing this wouldn't guarantee that the binaries of a parallel updated planet file are the same, but they would have the same contents. >> But that doesn't matter in a practical sense > it avoids having different files between mirrors, which is quite > annoying if a bug occurs on one variant and not on another depending > on the "mirror". See above. > >> In any case there is no argument to be made for the ~1'000 downloads >> that start shortly every week after the plant dump has been produced, > I agree. > maybe a rate limit "1st full speed" or "full speed for mirror" AFAIK that is already the case. Simon >> and of which for weird reasons 25% are actually >> downloading the compressed XML file (which is 30% larger). > there are probably some tools that don't support pbf (for example > osmfilter (I don't mind too much, I do an osmconvert before, but I guess > this extra step is not done and/or not available and/or not known by > other people). > >> Simon >> >>> Le 25.01.20 à 18:34, marc tobias a écrit : >>>> Wolfram, who asked about the bandwidth, is already running a mirror >>>> https://download.bbbike.org/osm/planet/ >>>> (listed on https://wiki.openstreetmap.org/wiki/Planet.osm#Downloading) >>>> so downloading it every week is likely trying to update his mirror. >>>> >>>> marc tobias >>>> >>>>> Message: 2 >>>>> Date: Sat, 25 Jan 2020 11:46:20 +0100 >>>>> From: Simon Poole <si...@poole.ch> >>>>> To: dev@openstreetmap.org >>>>> Subject: Re: [OSM-dev] planet.osm.org is verry slow >>>>> Message-ID: <4f5c3f2c-f3c9-0545-0dd2-98df390e4...@poole.ch> >>>>> Content-Type: text/plain; charset="utf-8" >>>>> >>>>> Yes, there are now bandwidth restrictions on connections to >>>>> planet.openstreetmap.org because the usage had skyrocketed and outgoing >>>>> bandwidth was completely saturated. >>>>> >>>>> But the obvious question is: why are you even trying to download the >>>>> full planet twice within a fortnight? Particularly given the file can >>>>> easily be updated in situ. >>>>> >>>>> Simon >>>> _______________________________________________ >>>> dev mailing list >>>> dev@openstreetmap.org >>>> https://lists.openstreetmap.org/listinfo/dev >>>> >>> _______________________________________________ >>> dev mailing list >>> dev@openstreetmap.org >>> https://lists.openstreetmap.org/listinfo/dev > _______________________________________________ > dev mailing list > dev@openstreetmap.org > https://lists.openstreetmap.org/listinfo/dev
signature.asc
Description: OpenPGP digital signature
_______________________________________________ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
