Hi Paul, On 15 September 2012 18:37, Paul Norman <[email protected]> wrote:
> I am using the --write-pgsql-dump in preparation for importing the planet > into a pgsnapshot database and have noticed some oddities with ways. > > I am reading from the latest .pbf and using enableBboxBuilder=no > enableLinestringBuilder=yes nodeLocationStoreType=TempFile. I tried > InMemory > but 16G of ram is not sufficient with the latest planet. > > My --log-progress task reported nodes being processed at about 250k > nodes/second. It has now moved to the ways and the reports are interesting. > > It was reporting approximately 150-200 objects/second but when I manually > take compute the ways/second from the way numbers I get figures in the > realm > of 5k+ objects/second. I know there may be some differences from deleted > ways skewing the numbers, but this seems beyond what that would account > for. > > It has taken approximately 20 minutes to reach way 4.5m, about 4k > ways/second. > > iotop reports java reading at 5-10 M/s and writing at 150-400 K/s > iostat reports the array with the .pbf and tempfile at 300 r/s 28 w/s 6000 > rkB/s 330 wkB/s, 90 %util. > > Although the progress seems okay (day or so to finish) I'm stumped by the > objects/sec number. > > As an aside, about a year ago it took 5 hours to do it with InMemory but > I'd > have to go up to 24 GB RAM to do that again. > The numbers do seem a little odd, but I can't explain it. The logic is the same for all object types so if nodes are calculated correctly then ways should also be calculated correctly. Brett
_______________________________________________ osmosis-dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/osmosis-dev
