Hello, meanwhile osmupdate has been changed to download replication diffs only.
But there is still an issue I cannot explain... I thought that the "normal" daily diffs would lack of some data which were provided around midnight, but there seem data to vanish from the middle of the day too: # normal diff $ zcat 20111103-20111104.osc.gz |grep -c "timestamp=\"2011-11-03T12:" 58968 # replication diff $ cat 1103-1104.osc |grep -c "timestamp=\"2011-11-03T12:" 59068 And yes, I thought on cumulating the version in the second file before I started counting with grep. Of course, it could easily be that I overlooked something - it's late in Germany. :-) Does anyone have an idea? Markus -------- Original-Nachricht -------- > Datum: Sun, 06 Nov 2011 00:32:49 +0100 > Von: mar...@gmx.eu > An: Frederik Ramm <frede...@remote.org>, dev@openstreetmap.org > Betreff: Re: [OSM-dev] Incomplete diffs? > Hi Frederik, > > thanks for the explanation! > > The _replication_ diffs are the right choice if you want to update a full > history file. > > Most people who update their OSM files on a regular basis do not need > replication diffs, they are satisfied with the newest version of each object > which has been changed. > > > So, if you want to use daily diffs but avoid the danger of missing > > edits, use the replication diff. > > Very good advice. > > Until today I chose NOT to use the diffs in the > planet.openstreetmap.org/history/ directory because they are outdated. They > usually come with a delay > of 25 hours. Du you know if the creation process could be accelerated > somehow? > > Now I will attend to osmupdate and try to change from daily normal diffs > to daily replication diffs. Seems to be better than loosing objects once in > a while. > > Meanwhile people can use Osmosis, or run osmupdate with the --hourly > option which will restrict the program to replication diffs. > > Markus > > -------- Original-Nachricht -------- > > Datum: Sat, 05 Nov 2011 22:56:01 +0100 > > Von: Frederik Ramm <frede...@remote.org> > > An: dev@openstreetmap.org > > Betreff: Re: [OSM-dev] Incomplete diffs? > > > Hi, > > > > On 11/05/2011 06:58 PM, mar...@gmx.eu wrote: > > > Meanwhile I found out that this node simply did not appear in the > daily > > diffs: > > > http://www.openstreetmap.org/browse/node/1470178889 > > > > > > It was crated at 2011-10-16T23:58Z by a large changeset along with > > 23.000 other nodes. > > > Neither the 16/17 nor the 17/18 daily diff contain this node whereas > the > > hourly diff from October 17 01:00 does. > > > > There are two types of diffs; "replication diffs" and normal diffs. A > > replication diff contains everything that happened between two > > timestamps, including multiple changes of the same object, whereas a > > normal diff only contains the information required to get from state 1 > > to state 2. > > > > Also, replication diffs are created in a relatively fail-safe process > > with Osmosis whereas the normal diffs can miss changes in some cases > > when a long-running database transaction that was created before 0:00 > > extends past the time when the diff is created. (There was a time when > > we had only "normal" diffs, and it was near impossible to make sure the > > minutely/hourly ones did not miss anything.) > > > > For minutely and hourly diffs, we only offer replication diffs these > > days. For daily diffs, we have the normal ones under > > planet.openstreetmap.org/daily, as well as the replication diffs under > > planet.openstreetmap.org/history. > > > > The normal diff indeed lacks the node in question, but the daily > > replication diff under history/2011/1016-1017.osc.gz has it. > > > > So, if you want to use daily diffs but avoid the danger of missing > > edits, use the replication diff. > > > > Frankly I don't know why the normal daily diffs are still created at > > all; if one really wanted to offer a reduced-traffic version of the > > replication diffs then it would indeed make sense to simply deflate the > > replication diff using Osmosis' --simplify-change task. > > > > Bye > > Frederik > > > > -- > > Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" > E008°23'33" > > > > _______________________________________________ > > dev mailing list > > dev@openstreetmap.org > > http://lists.openstreetmap.org/listinfo/dev > > _______________________________________________ > dev mailing list > dev@openstreetmap.org > http://lists.openstreetmap.org/listinfo/dev _______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
