On Mon, 2008-03-10 at 23:40 +0100, Robert Vollmert wrote:
> On Mar 10, 2008, at 22:43, Jon Burgess wrote:
> > The original multipolygons created by the conversion above all had the
> > same tags and no defined roles.
>
> Does osm2pgsql really require the same tags on all ways? The comments
> in the code seem to say it's collecting tags from all member ways, in
> which case removing tags from the inner ways shouldn't break things.
The problem code is elsewhere. When osm2pgsql is streaming the ways
through the code it has to make a decision whether to emit the way
immediately to postgres or whether to hold it in RAM until it has read
the relations which are later on in the file.
If we hold all the ways in RAM until we have read all the relations then
we can always do the right thing, but this costs several GB of ram. This
is what the code used to do until the planet file got too large and lots
of people were unable to run the code on reasonably spec'd hardware.
Currently the code looks at the keys on the way and if it is marked as
being for a likely polygon then it will keep in ram, else it goes
straight out to postgres. In this way we can avoid holding all the ways
with highway=<something> in memory when they are not going to be in a
multipolygon. Unfortunately all these new inner ways with 0 tags are
just going to get discarded (they won't be held in RAM or written to
Postgres).
For sure we could implement it differently, we could read the whole file
to extract the relations and then go back to re-read the ways again but
this adds significant time to the import.
> > As I mentioned before, I am not against change. I'd be really happy to
> > see this rationalisation occur. It will eventually result in a cleaner
> > data model and less code in osm2pgsql. The transition period does risk
> > breaking what we have already and this is what I'm trying to avoid.
>
> I should confess at this point that before the discussion resumed
> this afternoon I "fixed" multipolygon relations with id < 1000. So my
> question above will be answered soon enough...
The only visible effect will be areas (or holes within areas)
disappearing from the map.
> Would it be difficult to handle relations differently based on
> whether or not there are members with non-empty roles?
The fundamental issue is that we get to read the ways before we have
seen the relations in the planet dump. I mentioned in an email on the
dev list last week that we may need to do something clever with the
planet dump to allow tools to get the relation information before the
nodes and ways.
Lots of solutions are possible but it will take time to implement and
test the different approaches. Some of the mess with the present
multipolygons is because they were added with an ill-defined
specification.
Jon
_______________________________________________
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
