[osmosis-dev] Closed ways - polygons?
I'm attempting to use OSM data, as imported to PostGIS via Osmosis, as layers in a project in some GIS software; I've had some success crafting queries to pull highways into a separate table, and am now working on hydrologic features. My current roadblock is: how/where can I convert closed OSM ways into polygons? And then of course there will be multipolygons. Is this something better done in PostGIS? Is this a feature of osmosis that has a different name than I'm looking for? Thanks. (I'd also be interested in discussing/sharing queries with people who are similarly using OSM data in a PostgreSQL/PostGIS server backend for GIS software.) ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Closed ways - polygons?
On Mon, Dec 21, 2009 at 10:25:52PM +0100, Frederik Ramm wrote: Are you aware of osm2pgsql, a tool which has nothing to do with Osmosis but does roughly what you want? I found that osm2pgsql omitted several attributes that I require. Also, the schema created by osmosis is more suitable to my needs. If nothing else it appears that I could write a query using PostGIS functions that will do what I want: http://postgis.refractions.net/documentation/manual-1.3/ch06.html#IsClosed ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Closed ways - polygons?
Hi, Lennard wrote: David Carmean wrote: I found that osm2pgsql omitted several attributes that I require. Also, the schema created by osmosis is more suitable to my needs. If these attributes are the user, uid, version and timestamp, check out http://trac.openstreetmap.org/changeset/19144 And if they are other attributes, then it is very likely that you will be able to modify the osm2pgsql style file to suit your needs. This is probably going to be easier than trying to build proper multipolygons from data imported by osmosis, unless you are a real SQL wizard. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Closed ways - polygons?
Hi, David Carmean wrote: If nothing else it appears that I could write a query using PostGIS functions that will do what I want: http://postgis.refractions.net/documentation/manual-1.3/ch06.html#IsClosed Be advised that multipolygons may consist of any number of outer and inner ways which do not have to be closed themselves, they only have to form a closed ring. It will certainly be hard (but certainly not impossible) to do this with SQL alone. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] Changeset files (was Removing Minutely and Hourly Changesets)
On Mon, 2009-12-21 at 01:08 -0500, Anthony wrote:
Cool. If anyone familiar with the planet dumper tool is listening...
In
http://svn.openstreetmap.org/applications/utils/planet.osm/C/output_osm.c
} else if ((*in = 0) (*in 32)) {
escape_tmp[len] = '?';
len++;
should be something like
} else if ((*in 0) (*in 32)) {
len+=sprintf(escape_tmp[len], #%d;, *in);
Something like as in I haven't even checked if that compiles :).
Most of the control characters are not allowed in a valid XML file. It
makes no difference whether they are present as an ASCII character or as
the equivalent entity.
$ echo foo#01;/foo | xmllint -format -
-:1: parser error : xmlParseCharRef: invalid xmlChar value 1
foo#01;/foo
Of course, another thing to consider is that 1024 bytes isn't enough
for the truly pathological cases. I think you need like 1531 or
something to handle that. Fixing this might be enough to properly
process the current db, though.
How do you arrive at the 1531 number?
Any chance of adding num_changes?
The current output reflects the same information as the /changeset API
call. Do you think it should be there too?
Jon
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Changeset files (was Removing Minutely and Hourly Changesets)
On Mon, Dec 21, 2009 at 11:44, Jon Burgess jburgess...@googlemail.com wrote:
On Mon, 2009-12-21 at 01:08 -0500, Anthony wrote:
Cool. If anyone familiar with the planet dumper tool is listening...
In
http://svn.openstreetmap.org/applications/utils/planet.osm/C/output_osm.c
} else if ((*in = 0) (*in 32)) {
escape_tmp[len] = '?';
len++;
should be something like
} else if ((*in 0) (*in 32)) {
len+=sprintf(escape_tmp[len], #%d;, *in);
Something like as in I haven't even checked if that compiles :).
Most of the control characters are not allowed in a valid XML file. It
makes no difference whether they are present as an ASCII character or as
the equivalent entity.
$ echo foo#01;/foo | xmllint -format -
-:1: parser error : xmlParseCharRef: invalid xmlChar value 1
foo#01;/foo
Indeed. These were all created by Potlatch while it still wasn't doing
proper input validation (http://trac.openstreetmap.org/ticket/2072).
Would anything break if these values were just cleaned up in the
database so we don't have to deal with them in other tools in the
future?
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Changeset files (was Removing Minutely and Hourly Changesets)
On Mon, Dec 21, 2009 at 6:44 AM, Jon Burgess jburgess...@googlemail.comwrote:
On Mon, 2009-12-21 at 01:08 -0500, Anthony wrote:
Cool. If anyone familiar with the planet dumper tool is listening...
In
http://svn.openstreetmap.org/applications/utils/planet.osm/C/output_osm.c
} else if ((*in = 0) (*in 32)) {
escape_tmp[len] = '?';
len++;
should be something like
} else if ((*in 0) (*in 32)) {
len+=sprintf(escape_tmp[len], #%d;, *in);
Something like as in I haven't even checked if that compiles :).
Most of the control characters are not allowed in a valid XML file. It
makes no difference whether they are present as an ASCII character or as
the equivalent entity.
Ah yes. Hmm. That said, most of the characters actually in the database
are carriage returns, which along with tabs and line feeds (also in the db)
are valid in XML. Other characters are present - for instance ASCII 3 in
http://www.openstreetmap.org/browse/changeset/1325382 - those will be more
of a problem.
Hopefully the database can be cleaned of the rest of the characters, because
I'd imagine each dumper is going to have a slightly different way of dealing
with them. Until that's done, I guess there's no right answer.
Of course, another thing to consider is that 1024 bytes isn't enough
for the truly pathological cases. I think you need like 1531 or
something to handle that. Fixing this might be enough to properly
process the current db, though.
How do you arrive at the 1531 number?
strlen(quot;)*255+1
Not sure if that's the absolute longest encoded string. But 255 quotes
makes a valid key/value, and the planet dumper would truncate it, right?
Any chance of adding num_changes?
The current output reflects the same information as the /changeset API
call. Do you think it should be there too?
Not as a bug, but as a feature request, I guess so. It's more useful in the
dumps than the API (you can use it to make sure you've got everything
downloaded), but it'd be useful in the API as well, I suppose. It seems to
be in the DB, so there shouldn't be a performance impact, right?
I see it's mentioned on http://wiki.openstreetmap.org/wiki/.osm
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Changeset files (was Removing Minutely and Hourly Changesets)
On Mon, Dec 21, 2009 at 7:06 AM, Ævar Arnfjörð Bjarmason ava...@gmail.comwrote: Would anything break if these values were just cleaned up in the database so we don't have to deal with them in other tools in the future? I can't see it breaking unless the cleanup causes a unique key violation (or empty string?). But I can't say I know. It always could break something which was relying on broken behavior, but that's unavoidable. Would you be deleting the characters altogether? That seems to be what the API does (http://www.openstreetmap.org/api/0.6/changeset/1325382), along with quoting the valid ones ( http://www.openstreetmap.org/api/0.6/changeset/2062776 http://www.openstreetmap.org/api/0.6/changeset/1662850 http://www.openstreetmap.org/api/0.6/changeset/966232). Maybe that's what the planet dumper should do in the mean time as well, so at least everything (?) is consistent. If you're going to clean up the db, then also put an integrity constraint on it to help ensure they don't get back in there if there are other bugs in the future. Just k's and v's, right? Also a tighter constraint on display_name, which also disallows certain other characters (I don't know of any db corruptions there yet, though). Oh yeah, and member_role too. Again, don't know of any current db corruptions, but that should be constrained as well. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Changeset files (was Removing Minutely and Hourly Changesets)
Ah yes. Hmm. That said, most of the characters actually in the database are carriage returns, which along with tabs and line feeds (also in the db) are valid in XML. Other characters are present - for instance ASCII 3 in http://www.openstreetmap.org/browse/changeset/1325382 - those will be more of a problem. The history dumper just skips over those chars[1]. The regex is actually from Matt Amos who also wrote the planet exporter. So I'd guess that he would have no objections to change the .c program to the same behavior as the history export tool. Hopefully the database can be cleaned of the rest of the characters, because I'd imagine each dumper is going to have a slightly different way of dealing with them. Until that's done, I guess there's no right answer. Any chance of adding num_changes? The current output reflects the same information as the /changeset API call. Do you think it should be there too? Not as a bug, but as a feature request, I guess so. It's more useful in the dumps than the API (you can use it to make sure you've got everything downloaded), but it'd be useful in the API as well, I suppose. It seems to be in the DB, so there shouldn't be a performance impact, right? The planet export program even queries the value from the database and discards the information so it should be trivial to add the information. Cheers, Lars [1] http://bitbucket.org/lfrancke/historydump/src/tip/src/main/java/org/openstreetmap/util/Dumper.java#cl-21 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Questions on Nominatim and administrative boundaries + osm2pgsql
Hello Jon, Thank you for the detailed reply. We have made significant progress with your explanations. Regards, david -- for MapOSMatic team ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Changeset files (was Removing Minutely and Hourly Changesets)
On Tue, Dec 22, 2009 at 1:56 AM, Lars Francke lars.fran...@gmail.comwrote: Ah yes. Hmm. That said, most of the characters actually in the database are carriage returns, which along with tabs and line feeds (also in the db) are valid in XML. Other characters are present - for instance ASCII 3 in http://www.openstreetmap.org/browse/changeset/1325382 - those will be more of a problem. The history dumper just skips over those chars[1]. The regex is actually from Matt Amos who also wrote the planet exporter. So I'd guess that he would have no objections to change the .c program to the same behavior as the history export tool. To complete the conversation, Osmosis does the same thing as the history dumper if I'm reading the history dumper regex properly. It ignores all characters between 0x00 and 0x1F with the exception of 0x09, 0x0A and 0x0D which it leaves unchanged. It doesn't write '?' characters for illegal characters, it simply drops them. Brett ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[josm-dev] Can't commit any more - SVN issue?
Hi, when I try to commit I get svn: Commit failed (details follow): svn: MKACTIVITY of '/svn/!svn/act/c6d4c5bb-97c2-d343-806f-fa8346091637': authorization failed: Could not authenticate to server: rejected Basic challenge (https://josm.openstreetmap.de) The URL seems fine (it's the https URL) and sources on the net (i.e. http://jira.codehaus.org/browse/HAUS-1867) indicate that this could be a problem at the server end. Could somebody with access to the server please have a look? Thanks and regards Karl ___ josm-dev mailing list josm-...@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Can't commit any more - SVN issue?
On Mon, 21 Dec 2009, Karl Guggisberg wrote: Could somebody with access to the server please have a look? The server has load problems. This should be temporary, so please retry. Ciao -- http://www.dstoecker.eu/ (PGP key available) ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Can't commit any more - SVN issue?
Dirk Stöcker openstreet...@dstoecker.de writes: On Mon, 21 Dec 2009, Karl Guggisberg wrote: Could somebody with access to the server please have a look? The server has load problems. This should be temporary, so please retry. It just worked for me. Matthias ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
