[osmosis-dev] Problem with --node-key-value
Hi, i am new here so sorry if i mail to the wrong place. I wanted an osm file with all places=* so city, town etc. But i am getting an empty out.osm. What am i donig wrong? With ways (--way-key-value) this works great. terminal command: osmosis --read-xml in.osm --node-key-value keyValueList=place.city,place.town,place.village,place.hamlet,place.suburb --used-node idTrackerType=BitSet --write-xml out.osm out.osm: ?xml version='1.0' encoding='UTF-8'? osm version=0.6 generator=Osmosis 0.36 bound box=49.39089,7.76757,51.66252,10.24398 origin=0.37-SNAPSHOT/ /osm regards faulerflex -- GMX DSL Doppel-Flat ab 19,99 euro;/mtl.! Jetzt auch mit gratis Notebook-Flat! http://portal.gmx.net/de/go/dsl ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Osmosis 0.37 Released
On Sat, Oct 16, 2010 at 9:41 PM, Jochen Topf joc...@remote.org wrote: On Sat, Oct 16, 2010 at 05:33:37PM +1100, Brett Henderson wrote: - Updated the pgsql schema (now version 6) to move all tags into hstore columns, add a ways.nodes column and CLUSTER the nodes and ways tables. Significant performance increases. Is there some magic involved or do you just call CLUSTER after the import? When I tested CLUSTER it helped with reads, but creating the cluster was very expensive. No magic unfortunately. The CLUSTER step is the most expensive part of the process because it changes the location on disk of every record in the database. It took several days for a full planet when I tested it out. I was performing bbox queries (similar ROMA or TRAPI but with my implementation that uses true way geometries instead of just node locations) and 1x1 degree queries for heavily populated areas (eg. Munich, London) were taking up to an hour to perform. After removing tags tables and performing CLUSTER statements the time reduced to somewhere between 5 and 10 minutes due to much reduced disk seeking. The database is kept up to date with hourly diffs so the CLUSTER step only has to be performed once (or once every few months perhaps) but the read benefits are ongoing. If you're performing regular full imports it may not make sense. If the overhead of performing the CLUSTER step outweighs the read benefits (depends on the lifetime of your database) then you can skip the CLUSTER step. It makes no functional difference that I'm aware of. Brett ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] lost relations while clipping
Hi Maxim, 2010/10/12 Maxim Dubinin s...@gis-lab.info Hi osmosis-devs, On our regional extraction system we've ran into what seems to be a bug. Relations, that are members by themselves are dropped while extracting. Example of such relation: http://www.openstreetmap.org/browse/relation/358215 Code and example to reproduce this behavior is attached. Thanks for the detailed example, it makes it much easier to diagnose your issue. The reason for the dropped relation is a limitation of how Osmosis performs bounding-box/bounding-polygon processing. Osmosis processes all data in a stream. First it processes all nodes one by one and keeps track of which nodes are in the bounding box. It then processes each way one by one and uses the node list to determine which ways are in the bounding box. Then it processes the relations one by one and uses the list of nodes, ways, and already selected relations to determine if a relation is in the bounding box. In your example, relation 795 is processed first and contains relation 792. But 792 hasn't been processed (in your example, the two ids are out of order) yet so Osmosis doesn't know that it will be inside the bounding box and ignores it. It then processes 792 which has two ways inside the bounding box so it includes that relation. But it never goes back to include relation 795 because it has forgotten about it. In your example, if you switched the order of relation 792 and 795 then 795 *would* get included. The only way to avoid this limitation is to replace your existing --bp step: --bp file=test.poly with --bp file=test.poly completeWays=yes completeRelations=yes This makes Osmosis run *much* slower and might not be acceptable. With those options enabled, it builds temporary stores of nodes, ways and relations so that it can go back and randomly access data. But the temporary data stores are quite slow which has a very large impact on performance. Hope that helps. Brett ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Problem with --node-key-value
I've just taken a look at the code (I didn't write these tasks) for the --node-key-value task and it works slightly differently to the --way-key-value task. The --way-key-value task passes through node and relation objects unchanged, but the --node-key-value task drops them. That means that when you have a --node-key-value task followed by a --used-node task you are going to drop everything. Note that it doesn't make sense to use the --used-node task in this case anyway because place nodes will typically not be part of a way. So two suggestions: 1. Remove the --used-node task. 2. Use the --tag-filter task instead which is much more flexible. Cheers, Brett On Sun, Oct 17, 2010 at 1:15 AM, Flex Fauler faulerf...@gmx.de wrote: Hi, i am new here so sorry if i mail to the wrong place. I wanted an osm file with all places=* so city, town etc. But i am getting an empty out.osm. What am i donig wrong? With ways (--way-key-value) this works great. terminal command: osmosis --read-xml in.osm --node-key-value keyValueList=place.city,place.town,place.village,place.hamlet,place.suburb --used-node idTrackerType=BitSet --write-xml out.osm out.osm: ?xml version='1.0' encoding='UTF-8'? osm version=0.6 generator=Osmosis 0.36 bound box=49.39089,7.76757,51.66252,10.24398 origin=0.37-SNAPSHOT/ /osm regards faulerflex -- GMX DSL Doppel-Flat ab 19,99 euro;/mtl.! Jetzt auch mit gratis Notebook-Flat! http://portal.gmx.net/de/go/dsl ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] osmosis
It looks like your input file contains duplicate nodes. It is possible that multiple versions of the same node exist. This has come up a number of times on the mailing list. Does anybody have the list of steps for fixing this handy? Brett On Fri, Oct 15, 2010 at 3:21 AM, Jose Cid don...@gmail.com wrote: Hello All, I've been trying to import osm files (bz2) into postGres DB using osmosis (version 0.36 - built from svn). The command I used is the following: ./osmosis --read-xml-0.6 file=north_america.osm.bz2 --write-apidb-0.6 populateCurrentTables=yes host=localhost database=openstreetmap user=osm password=osm1 validateSchemaVersion=no The result being the following: SEVERE: Thread for task 1-read-xml-0.6 failed org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to bulk insert nodes into the database. at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.flushNodes(ApidbWriter.java:577) at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.process(ApidbWriter.java:1108) at org.openstreetmap.osmosis.core.container.v0_6.NodeContainer.process(NodeContainer.java:58) at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.process(ApidbWriter.java:1082) at org.openstreetmap.osmosis.xml.v0_6.impl.NodeElementProcessor.end(NodeElementProcessor.java:117) at org.openstreetmap.osmosis.xml.v0_6.impl.OsmHandler.endElement(OsmHandler.java:107) at org.apache.xerces.parsers.AbstractSAXParser.endElement(Unknown Source) at org.apache.xerces.parsers.AbstractXMLDocumentParser.emptyElement(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanStartElement(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source) at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source) at org.apache.xerces.parsers.XMLParser.parse(Unknown Source) at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(Unknown Source) at org.apache.xerces.jaxp.SAXParserImpl.parse(Unknown Source) at javax.xml.parsers.SAXParser.parse(SAXParser.java:195) at org.openstreetmap.osmosis.xml.v0_6.XmlReader.run(XmlReader.java:108) at java.lang.Thread.run(Thread.java:636) Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint nodes_pkey at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2062) at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1795) at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257) at org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367) at org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:321) at org.openstreetmap.osmosis.apidb.v0_6.ApidbWriter.flushNodes(ApidbWriter.java:575) ... 19 more Oct 14, 2010 12:14:21 PM org.openstreetmap.osmosis.core.Osmosis main SEVERE: Execution aborted. org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more tasks failed. at org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146) at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:92) at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:37) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352) at org.codehaus.classworlds.Launcher.main(Launcher.java:31) This has been the case for several other OSM files that I attempted to import. Has anyone run into the above issue? Any solutions? ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org
Re: [OSM-dev] OSM binary format (pbf) 1.0 is in osmosis trunk.
I've released version 0.37 of Osmosis with this included. I haven't had a chance to update the wiki either. I'll try to do so over the next few days if nobody beats me to it. If anybody sees any issues with the binary support, please let Scott and I know. I'm now building Osmosis via an automated Hudson process so pushing out new builds should be fairly quick once a fix is identified. On Wed, Oct 13, 2010 at 10:53 AM, Scott Crosby scro...@cs.rice.edu wrote: On Thu, Sep 16, 2010 at 10:23 AM, Scott Crosby scro...@cs.rice.eduwrote: For now, for simplicity, I'm going to revert to the same metadata as the XML format. Just a BBox and source field. I'll make them both optional, making it easier to upgrade metadata features in the future. When/if there is a consensus for additional metadata fields, support for them can be added then. I'll be releasing an rc2 at some point. This has been done. RC2 is in osmosis trunk. Changes are almost exclusively to the underlying osmbin.jar with no format incompatibilities. Changes include: Sorry its taken so long. Personal reasons have kept me away from this work. I have committed the 1.0 version of the 'osmbin' jar to osmosis trunk. I have also increased the maximum size of a header or fileblock to 64kb and 32mb respectively. (These limits are used to detect corrupt files.) I believe I have also fixed the two reported bugs, Frederick's bug with reporting the wrong error message and the negative UserID bug. The only thing left is to rename 'osmbin' to 'osmpbf' to match the name of the format, and put a copy of the source code into OSM's SVN server and to find a good home for the jar. (Any suggestions?) For now the jar lives in osmosis's SVN repository and the source is on github. In osmosis, the important change is that the tasks have been renamed to match the *.pbf file extension and are now --write-pbf and --read-pbf. I am leaving behind the old task names --read-bin and --write-bin so that existing scripts will work, but please fix your scripts. I also made one small API change. The timestamps metadata field should have been an int64, not an int32. This is not a format-compatability change, but it may require minor changes to code using the protobuf definitions. I am not sure when I will have time to update the wiki with the documentation of the pbf tasks. For now, I am attaching a description of all of the options. Scott /// // --write-pbf Arguments: file=filename Currently '-' representing stdout is not supported. compress=deflate (default) Use deflate compression on each block compress=none Disable compression. About twice as fast to write and twice the size. batchlimit=8000 Block size used when compressing. This is a reasonable default. Batchlimits that are too big may cause files to exceed the defined filesize limits. granularity=100 The granularity or precision used to store coordinates. The default of 100 nanodegrees is the highest precision used by OSM, corresponding to about 1.1cm at the equator. In the current osmosis implementation, the granularity must be a multiple of 100. If map data is going to be exported to software that does not need the full precision, increasing the granularity to 1 nanodegrees can save about 10% of the file size, while still having 1.1m precision. omitmetadata=false (default) omitmetadata=true Omit non-geographic metadata on OSM entities. This includes version number and timestamp of the last edit to the entity as well as the user name and id of the last modifier. Omitting this metadata can save 15% filesize when exporting to software that does not need this data. usedense=true (default) Nodes can be represented in a regular format or a dense format. The dense format is about 30% smaller, but more complex. To make it easier to interoperate with (future) software that chooses to not implement the dense format, the dense format may be disabled. // --read-pbf Arguments: file=filename Currently '-' representing stdin is not supported. // Usage tips: The default options for reading and writing are the safe options and work efficiently and quickly. Buffering can improve performance. The binary format processes data in batches, entities are queued until a limit is reached, then that batch is serialized and compressed. This serialization can run concurrently with other osmosis processing. With more than one core, writing throughput can be increased by about 60% by placing a buffer in the processing pipeline just before writing. Similarily, a buffer placed in the pipelilne immediatelly after parsing can likewise improve read concurrency. Eg: osmosis --read-pbf file=XXX --b bufferCapacity=12000 OR osmosis --b bufferCapacity=12000 --write-pbf file=XXX ... When generating data for export to other applications, I suggest considerring --omitmetadata and
Re: [OSM-dev] OSM binary format (pbf) 1.0 is in osmosis trunk.
On Sat, 16 Oct 2010, Scott Crosby wrote: Also, can you give me the URL for the source code for pbf2osm to check that a few edge-cases are correctly handled? http://git.openstreetmap.nl/index.cgi/pbf2osm.git/ Stefan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM binary format (pbf) 1.0 is in osmosis trunk.
Hi, On 10/16/2010 03:25 PM, Stefan de Konink wrote: Also, can you give me the URL for the source code for pbf2osm to check that a few edge-cases are correctly handled? http://git.openstreetmap.nl/index.cgi/pbf2osm.git/ I'm hoping to take that, fix it up a bit (so that the resulting binary is not called test any more etc ;) and put in in the OSM SVN together with a set of Debian scripts that automatically build the protobuf stuff and produce a statically linked Debian package with little no dependencies. I think that should further reduce the hurdle to using it. Bye Frederik ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Disallowing certain characters in tag keys
Hi, what is the problem with escaping problematic characters? There should be build in functions for most languages, like uri_escape in Perl and URLEncode.encode in Java. This proposal [1] moves values into the key to describe conditions. (Although you could argue, it should be done like that anyway...) [1] http://wiki.openstreetmap.org/wiki/Proposed_features/Extended_conditions_for_access_tags, Sebastian Jochen Topf wrote: Hi! I am currently fighting some issues where tags with strange characters in them need to be represented in a URL for Taginfo. Lots of other websites probably will have similar issues. Characters like /, ?, , etc. have special meaning in URLs so if they appear in tags I can't have those tags in URLs. Sometimes escaping characters as %XX helps, sometimes not. And those problems are not confined to web pages and URLs only. Special characters that need escaping are often a problem. We can't really do anything about that with regard to tag values, they must be allowed to contain all those characters. But it would help at least a little if we knew those characters can never appear in tag keys. And I can't really see a legitimate reason why we need those characters in keys. Looking at the database almost all cases where they appear in keys are obvious errors. Out of the about 2 different keys, there are only about 190 keys with problematic characters in them (another about 800 with whitespace). Really the only case that I can't immediately rule out as errors or see an alternative tagging are tag keys like maxspeed:weight7.5. And with those you can already see the problems: Some of them have gt; instead of the . So I'd like us to think about whether we can disallow a few characters from appearing in tag keys. Technically this would mean changing the API to check for those characters, removing any that are already in the database (can be done with normal manual edits because there are so few cases) and adding checks to the editors so that they can give meaningful error messages. Shouldn't be too hard. So, what characters am I talking about? I haven't drawn up a complete list and we certainly would need to discuss this further. Here is a preliminary list: Whitespace Should use '_' instead of whitespace in keys, whitespace are also very confusing for users, especially at beginning and end of a text. /+?#;%' Special characters in XML, HTML and/or URLs. \' Characters often used for quoting. =Because its used in many places as the separation character between tag key and tag value. If we disallow this, we can always treat one string like foo=bar as k:foo, v:bar without any ambiguities. This is a small list of special characters, all other characters should still be allowed. That means tag keys can still be in Chinese or whatever. We'd just disallow a few characters of which we know that they will make problems again and again. And to emphasize this again: I am only talking about tag keys. Tag values must be allowed to contain the full Unicode set of characters. Jochen ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Disallowing certain characters in tag keys
I agree with whitespace - this can be very confusing = To add: Make keys lowercase (or even remove diacritics), because keys are always simple names. Am 16.10.10 20:44, schrieb Jochen Topf: Hi! I am currently fighting some issues where tags with strange characters in them need to be represented in a URL for Taginfo. Lots of other websites probably will have similar issues. Characters like /, ?,, etc. have special meaning in URLs so if they appear in tags I can't have those tags in URLs. Sometimes escaping characters as %XX helps, sometimes not. And those problems are not confined to web pages and URLs only. Special characters that need escaping are often a problem. We can't really do anything about that with regard to tag values, they must be allowed to contain all those characters. But it would help at least a little if we knew those characters can never appear in tag keys. And I can't really see a legitimate reason why we need those characters in keys. Looking at the database almost all cases where they appear in keys are obvious errors. Out of the about 2 different keys, there are only about 190 keys with problematic characters in them (another about 800 with whitespace). Really the only case that I can't immediately rule out as errors or see an alternative tagging are tag keys like maxspeed:weight7.5. And with those you can already see the problems: Some of them have gt; instead of the . So I'd like us to think about whether we can disallow a few characters from appearing in tag keys. Technically this would mean changing the API to check for those characters, removing any that are already in the database (can be done with normal manual edits because there are so few cases) and adding checks to the editors so that they can give meaningful error messages. Shouldn't be too hard. So, what characters am I talking about? I haven't drawn up a complete list and we certainly would need to discuss this further. Here is a preliminary list: Whitespace Should use '_' instead of whitespace in keys, whitespace are also very confusing for users, especially at beginning and end of a text. /+?#;%' Special characters in XML, HTML and/or URLs. \' Characters often used for quoting. =Because its used in many places as the separation character between tag key and tag value. If we disallow this, we can always treat one string like foo=bar as k:foo, v:bar without any ambiguities. This is a small list of special characters, all other characters should still be allowed. That means tag keys can still be in Chinese or whatever. We'd just disallow a few characters of which we know that they will make problems again and again. And to emphasize this again: I am only talking about tag keys. Tag values must be allowed to contain the full Unicode set of characters. Jochen ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Disallowing certain characters in tag keys
On Sat, Oct 16, 2010 at 09:55:41PM +0200, Sebastian Klein wrote: what is the problem with escaping problematic characters? There should be build in functions for most languages, like uri_escape in Perl and URLEncode.encode in Java. The problem is that its very hard to get this right. I have just spend an hour debugging a problem where the semi-colon (;) character in a URL was mis-handled by Apache. The mod_proxy module I use in Taginfo silently cuts of everything after an ; in the URL even if its escaped. Thats probably because the ; is handled specially for some reason in the RFC defining URLs. I found an option to fix this, but its doesn't work with mod_rewrite, so that had to be worked around, too. I got it to work, but I don't want to know what the next problem will be. It just goes to show that even software like Apache has problems dealing with these things, not to speak of some scripts somebody just hacked together. I already mentioned the keys in the database with gt; or so in there, probably from some software escaping once too often. Special characters must be escaped exactly once. If they are not escaped or escaped more than once, you get broken results. And on the other side you have to de-escape exactly once. This is difficult to get right. And the penalty for not getting it right might just be an SQL injection or a cross-site-scripting attack vector. Not allowing those characters in the first place, makes software easier to write and understand and more robust. And it even makes it friendlier for humans, because if they use those characters you can immediately give them an error message instead of creating broken data. And all of that without any cost, really. I don't see that we ever need those characters in tag keys. Of course if we do need those characters than we have to get all of this right and right every time. This proposal [1] moves values into the key to describe conditions. (Although you could argue, it should be done like that anyway...) [1] http://wiki.openstreetmap.org/wiki/Proposed_features/Extended_conditions_for_access_tags, I think thats a misguided use of tag keys probably invented by people who have never actually tried to write code that tries to interpret OSM tags. Jochen -- Jochen Topf joc...@remote.org http://www.remote.org/jochen/ +49-721-388298 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Disallowing certain characters in tag keys
On Sat, Oct 16, 2010 at 10:31:47PM +0200, Andreas Kalsch wrote: I agree with whitespace - this can be very confusing = To add: Make keys lowercase (or even remove diacritics), because keys are always simple names. Thats a different issue. I agree that keys normally should be lowercase. But thats just a nice convention. There might be good reasons for uppercase keys, for instance when the key name was used in upper case in some other system where data was imported from. There is no need to force people into a convention here. Thats different from the issue I have been talking about where there are real problems with some characters. Jochen Am 16.10.10 20:44, schrieb Jochen Topf: Hi! I am currently fighting some issues where tags with strange characters in them need to be represented in a URL for Taginfo. Lots of other websites probably will have similar issues. Characters like /, ?,, etc. have special meaning in URLs so if they appear in tags I can't have those tags in URLs. Sometimes escaping characters as %XX helps, sometimes not. And those problems are not confined to web pages and URLs only. Special characters that need escaping are often a problem. We can't really do anything about that with regard to tag values, they must be allowed to contain all those characters. But it would help at least a little if we knew those characters can never appear in tag keys. And I can't really see a legitimate reason why we need those characters in keys. Looking at the database almost all cases where they appear in keys are obvious errors. Out of the about 2 different keys, there are only about 190 keys with problematic characters in them (another about 800 with whitespace). Really the only case that I can't immediately rule out as errors or see an alternative tagging are tag keys like maxspeed:weight7.5. And with those you can already see the problems: Some of them have gt; instead of the . So I'd like us to think about whether we can disallow a few characters from appearing in tag keys. Technically this would mean changing the API to check for those characters, removing any that are already in the database (can be done with normal manual edits because there are so few cases) and adding checks to the editors so that they can give meaningful error messages. Shouldn't be too hard. So, what characters am I talking about? I haven't drawn up a complete list and we certainly would need to discuss this further. Here is a preliminary list: Whitespace Should use '_' instead of whitespace in keys, whitespace are also very confusing for users, especially at beginning and end of a text. /+?#;%' Special characters in XML, HTML and/or URLs. \' Characters often used for quoting. =Because its used in many places as the separation character between tag key and tag value. If we disallow this, we can always treat one string like foo=bar as k:foo, v:bar without any ambiguities. This is a small list of special characters, all other characters should still be allowed. That means tag keys can still be in Chinese or whatever. We'd just disallow a few characters of which we know that they will make problems again and again. And to emphasize this again: I am only talking about tag keys. Tag values must be allowed to contain the full Unicode set of characters. Jochen ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- Jochen Topf joc...@remote.org http://www.remote.org/jochen/ +49-721-388298 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Disallowing certain characters in tag keys
+1. much sanity ensues. On Sat, Oct 16, 2010 at 7:44 PM, Jochen Topf joc...@remote.org wrote: So I'd like us to think about whether we can disallow a few characters from appearing in tag keys. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Disallowing certain characters in tag keys
Jochen Topf wrote: This proposal [1] moves values into the key to describe conditions. (Although you could argue, it should be done like that anyway...) [1] http://wiki.openstreetmap.org/wiki/Proposed_features/Extended_conditions_for_access_tags, I think thats a misguided use of tag keys probably invented by people who have never actually tried to write code that tries to interpret OSM tags. No speculation required, I'm the one who is to blame for that proposal. Before I finished the text that's still in the wiki, however, I /did/ write an experimental implementation* for this syntax, as well as an alternative syntax, to find out whether the ideas could work from a developer's perspective. I didn't encounter any significant problems. In retrospect ... well, maybe it wasn't the best of ideas to write that implementation based on the GraphView plugin for JOSM. After all, I figure that working on a comparably small in-memory dataset makes a significant difference for the way you would deal with keys. Add to this that I didn't have to deal with any web issues, or in fact any interface between programs at all (- no encoding issues), and it probably wasn't remotely a representative development experience. Unfortunately, it seems this will produce exactly the outcome I didn't intend at all: more variety in tagging. People will continue to use those parts of the proposal that don't require special characters (maxspeed:wet, :forward and the like), so we will use completely different solutions for simple and for more complex cases. Well, I'll file this as failed attempt of overzealous newbie to build tagging cathedral. Tobias Knerr * I ended up never publishing the implementation, though - not due to condition handling itself, but because I never got around to implement a proper opening_hours parser. Turns out that's actually more work than the entire rest of the syntax. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM binary format (pbf) 1.0 is in osmosis trunk.
On Sat, Oct 16, 2010 at 1:39 AM, Brett Henderson br...@bretth.com wrote: I've released version 0.37 of Osmosis with this included. I haven't had a chance to update the wiki either. I'll try to do so over the next few days if nobody beats me to it. I just updated the Wiki. Scott ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[josm-dev] JOSM session
Hello, I made good progress implementing ticket #4029 (JOSM sessions) but I am stuck at some point. To be able to reorder the layers correctly, set their visibilities and so on, I need to create those layers and get the resulting layer object. Since layers are created in the background I sometimes get null pointers instead of the actual layer or when I try to re-order them, they have not been added yet to the main view. How can I make sure that I get called AFTER the layer is created and added to the main view. Thanks, N. ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
