Re: [osmosis-dev] Exception in Osmosis 0.36
Hi! Am 02.08.2010 14:50, schrieb Brett Henderson: I'm pretty sure I've found and fixed the problem. Can you try the new build I've just created? http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-0.37-SNAPSHOT.zip http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-0.37-SNAPSHOT.tgz Works like a charm. bye Nop ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [OSM-dev] New OSM binary fileformat implementation.
On Sat, Jul 31, 2010 at 11:26 AM, Frederik Ramm frede...@remote.org wrote: Scott, others, Scott Crosby wrote: I would like to announce code implementing a binary OSM format that supports the full semantics of the OSM XML. [...] The changes to osmosis are just some new tasks to handle reading and writing the binary format. [...] This was 3 months ago. What's the status of this project? Are people actively using it? Is it still being developed? Can the Osmosis tasks be used in the new Osmosis code architecture (see over on osmosis-dev) that Brett has introduced with 0.36? I'm using it personally. I know of no other users, except that Nolan Darilek is interested in whether the format can be expanded with geographic indexing information. I have a few minor tweaks that I've been intending to make before declaring the format final. Basically, definining some optional fileformat fields (eg, is the file sorted? And on what paramater.) There's no infrastructure using these fields, however. How much interest is there in this code and format? Scott ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [Routing] [MoNav] Fast exact ( mobile ) routing
On Tue, Aug 3, 2010 at 2:28 PM, feverzsj fever...@hotmail.com wrote: hi VeaaC, Very impressive job, according to what it describes. I've investigated lots navi tool/lib based on osm. But, they are all osm specified, which means you can't import your own vector data. Will MoNav Preprocessor be able to process other vector data(e.g. ogr supported file with navi data inside, and import by preprocessor using some kind rule)? MoNav is build very modular, all it takes to import another kind of data set is for you to write a new importer plugin. The importer plugin generates a normalized representation of the data set making all other plugin types independent of the input format. Nevertheless, the current rendering plugins rely either on Mapnik or the OpenStreetMap tile server. To visualize your own data format you would either have to write your own Mapnik stylesheet or wait until a vector rendering plugin is available. If you can give me some good reasons to support specific input formats I will consider implementing an input plugin for them. Christian Vetter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] McNav vector rendering? (was: Fast exact ( mobile ) routing)
VeaaC FDIRCT wrote: Nevertheless, the current rendering plugins rely either on Mapnik or the OpenStreetMap tile server. To visualize your own data format you would either have to write your own Mapnik stylesheet or wait until a vector rendering plugin is available. Do you have plans to write a vector rendering plugin to remove the dependency on local or remote tiles? I'm working on an amateur radio application called APRSISCE that runs on Windows Mobile and Win32 and would love to be able to query for XML and do local map rendering rather than fetching OSM tiles as I do now. With a vector rendering solution, I can do arbitrary scaling instead of being restricted to the 2x per zoom level that OSM tiles provide. Lynn (D) - KJ4ERJ - Author of APRSISCE for Windows Mobile and Win32 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] McNav vector rendering? (was: Fast exact ( mobile ) routing)
On Tue, Aug 3, 2010 at 2:58 PM, Lynn W. Deffenbaugh (Mr) ldeff...@homeside.to wrote: Do you have plans to write a vector rendering plugin to remove the dependency on local or remote tiles? I'm working on an amateur radio application called APRSISCE that runs on Windows Mobile and Win32 and would love to be able to query for XML and do local map rendering rather than fetching OSM tiles as I do now. With a vector rendering solution, I can do arbitrary scaling instead of being restricted to the 2x per zoom level that OSM tiles provide. While I do plan on implementing a vector rendering plugin, it will most likely not feature the kind of detail the prerendered Mapnik tiles do. I am thinking of only displaying route-able ways, their names as well as place labels. This would make the vector rendering plugin work with any kind of data set, not just OpenStreetMap data. Christian Vetter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New OSM binary fileformat implementation.
On Sun, Aug 1, 2010 at 6:39 AM, Brett Henderson br...@bretth.com wrote: I'll help incorporate this into the rest of Osmosis. There's a few things to work through though. I don't have a lot of time to work with this, but I can split up my working branch (which includes several unrelated changes) into separate orthogonal pieces. Git is *VERY* good at this. That would simplify integration. Is there a demand for the binary format in its current incantation? I'm not keen to incorporate it if nobody will use it. I think it would be used in the mkgmap splitter, if available. Can the code be managed in the main OSM Subversion repo instead of GIT? Yes. I use git personally, but there's very good SVN integration. Is any code reuse between Osmosis and other applications required? Yes. The *.proto files must be shared with other applications that use the binary format, including C/Java/Python/.net/ I wrote some java parser code that is intended to be shared across the other Java osmosis applications. (Eg, I'm using it in my splitter changes.) in crosby/binary/file and crosby/binary/*.java I suggest that all of this be put in a separate library along with jamesmikedupont's C/C++ code. Scott ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [Routing] [MoNav] Fast exact ( mobile ) routing
On Tue, Aug 3, 2010 at 3:26 PM, Tom Müller tmerl...@web.de wrote: Hi, interesting project! But how do I get the preprocessor? When I try to start the moNavC.exe it crahes with No data-directory found: plugins.ini missing. So what else do I need? Thanks! Tom For now you will have to compile the preprocessor yourself. Some pointers are given at: http://code.google.com/p/monav/wiki/CompileGuide I will try to provide precompiled binaries for the preprocessor in a few days. While I will not upload Linux binaries it is actually much easier to compile the Linux binaries for your distribution than it is to compile the Windows binaries. If you just want to route using the standard car / speed profile you can download some of the data packages at: http://code.google.com/p/monav/wiki/Downloads#OSM_Renderer_Packages OSM Renderer data packages need an active Internet connection to fetch the prerendered tiles form the OpenStreetMap server. If you download the matching Mapnik Renderer package the tiles are fetched from those files instead. This is the recommended method for mobile devices. Christian Vetter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New full history dump available
Am 02.08.2010 21:40, schrieb Stefan de Konink: Could someone also put this out as a torrent? I'm not sure if it would help anybody. As only a little number of users will download it, it will get well distributed in the torrent network. With a small number of seeds a torrent download will take a lot longer then the http download -- and nobody will use it. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New full history dump available
On Tue, Aug 3, 2010 at 2:06 PM, Peter Körner osm-li...@mazdermind.de wrote: Am 02.08.2010 21:40, schrieb Stefan de Konink: Could someone also put this out as a torrent? I'm not sure if it would help anybody. As only a little number of users will download it, it will get well distributed in the torrent network. With a small number of seeds a torrent download will take a lot longer then the http download -- and nobody will use it. Is there still no support in torrent for http seeding? ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New full history dump available
Am 03.08.2010 20:13, schrieb Anthony: Is there still no support in torrent for http seeding? I think there is in some clients, but eg. not in my Synology DiskStation. Also: as long as an http seed is available, why should users take the torrent way when they can't take advantage of a big pool of peers? Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New full history dump available
On Tue, Aug 3, 2010 at 2:16 PM, Peter Körner osm-li...@mazdermind.de wrote: Am 03.08.2010 20:13, schrieb Anthony: Is there still no support in torrent for http seeding? I think there is in some clients, but eg. not in my Synology DiskStation. Also: as long as an http seed is available, why should users take the torrent way when they can't take advantage of a big pool of peers? Because a small pool of peers is better than no pool of peers? ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] How to clean up Negative IDs
This is in reference to the USGS OSMCP project - not the real OSM... When we imported our chunk of data initially (not me - the guy responsible is on walkabout in the Rockies), we followed the convention of using negative IDs in the .OSM file. But osmosis was used to load the data into the database and now all of our data has negative IDs. This seems to have a really nasty effect on the API - every time something is edited, a new copy is created with positive IDs and the old version with the negative IDs persists. I assume there is something in the API that says negative IDs == BAD. I've been trying to test that theory but keep hitting stumbling blocks. Postgres doesn't seem to want to let me defer integrity constraints, so my efforts to change a few IDs to positive values keeps failing. Maybe I've lost my SQL chops (or maybe I just can't do that as the openstreetmap database user). Am I barking up the right tree? Should I just go ahead and destroy the database and repopulate it using bulk_upload.py instead of osmosis? -Eric Wolf -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf 720-334-7734 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to clean up Negative IDs
On Aug 3, 2010, at 12:28 PM, Eric Wolf wrote: This is in reference to the USGS OSMCP project - not the real OSM... When we imported our chunk of data initially (not me - the guy responsible is on walkabout in the Rockies), we followed the convention of using negative IDs in the .OSM file. But osmosis was used to load the data into the database and now all of our data has negative IDs. This seems to have a really nasty effect on the API - every time something is edited, a new copy is created with positive IDs and the old version with the negative IDs persists. I assume there is something in the API that says negative IDs == BAD. I've been trying to test that theory but keep hitting stumbling blocks. Postgres doesn't seem to want to let me defer integrity constraints, so my efforts to change a few IDs to positive values keeps failing. Maybe I've lost my SQL chops (or maybe I just can't do that as the openstreetmap database user). Am I barking up the right tree? Should I just go ahead and destroy the database and repopulate it using bulk_upload.py instead of osmosis? I'd do that -Eric Wolf -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf 720-334-7734 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev Steve stevecoast.com ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to clean up Negative IDs
On Tue, Aug 3, 2010 at 7:28 PM, Eric Wolf ebw...@gmail.com wrote: I assume there is something in the API that says negative IDs == BAD. I've been trying to test that theory but keep hitting stumbling blocks. Postgres doesn't seem to want to let me defer integrity constraints, so my efforts to change a few IDs to positive values keeps failing. Maybe I've lost my SQL chops (or maybe I just can't do that as the openstreetmap database user). Am I barking up the right tree? Should I just go ahead and destroy the database and repopulate it using bulk_upload.py instead of osmosis? all IDs in the database should be positive - negative IDs are only used on the client side to indicate elements which need to be created by the server (so that they can be referenced by other elements in the file). if you have negative IDs then you should use an uploader against an API, rather than directly inserting the data into the database with osmosis. a further complication is that postgres allocates new IDs from a sequence, which is separate from the table and needs to be kept up-to-date after any direct-to-database import (not sure if osmosis does this?), otherwise postgres may attempt to allocate duplicate primary key IDs... uploading using bulk_upload.py is probably safer, but much slower, unfortunately... cheers, matt ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [josm-dev] Change to changeset comment handling, RfD
Hi, ce-test, qualified testing bv - Gert Gremmen wrote: I suggest we try to JOSM add some comments by itself: Merkaartor did that, or perhaps does it still. These auto-generated comments are next to worthless. They cannot replace one human being telling another human being in a few words what has been done and why. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ josm-dev mailing list josm-...@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [OSM-dev] How to clean up Negative IDs
On 3 August 2010 20:28, Eric Wolf ebw...@gmail.com wrote: This is in reference to the USGS OSMCP project - not the real OSM... When we imported our chunk of data initially (not me - the guy responsible is on walkabout in the Rockies), we followed the convention of using negative IDs in the .OSM file. But osmosis was used to load the data into the database and now all of our data has negative IDs. This seems to have a really nasty effect on the API - every time something is edited, a new copy is created with positive IDs and the old version with the negative IDs persists. I assume there is something in the API that says negative IDs == BAD. I've been trying to test that theory but keep hitting stumbling blocks. Postgres doesn't seem to want to let me defer integrity constraints, so my efforts to change a few IDs to positive values keeps failing. Maybe I've lost my SQL chops (or maybe I just can't do that as the openstreetmap database user). Am I barking up the right tree? Should I just go ahead and destroy the database and repopulate it using bulk_upload.py instead of osmosis? If there's no way disable the postgres contraints (I'm sure there is.. but I'm a sql noob), I'd filter your .osm file through sed removing the '-' in 'ref=-' and 'id=-' and reimport with osmosis, or modify your conversion script. Using bulk_upload.py and the API will take ages. Cheers ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New full history dump available
So here you are: http://toolserver.org/~mazder/temp/full-planet-100801.osm.bz2.torrent I'm seeding for the next months via my 640 kBit/s uplink. I added the webseed to the torrent. It would be better if those torrents would be created automatically and published in an rss feed. Peter Am 03.08.2010 20:21, schrieb Anthony: On Tue, Aug 3, 2010 at 2:16 PM, Peter Körnerosm-li...@mazdermind.de wrote: Am 03.08.2010 20:13, schrieb Anthony: Is there still no support in torrent for http seeding? I think there is in some clients, but eg. not in my Synology DiskStation. Also: as long as an http seed is available, why should users take the torrent way when they can't take advantage of a big pool of peers? Because a small pool of peers is better than no pool of peers? ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [josm-dev] Change to changeset comment handling, RfD
Frederik Ramm frede...@remote.org writes: Hi, ce-test, qualified testing bv - Gert Gremmen wrote: I suggest we try to JOSM add some comments by itself: Merkaartor did that, or perhaps does it still. These auto-generated comments are next to worthless. They cannot replace one human being telling another human being in a few words what has been done and why. Especially the why part would be hard to guess for any program. Matthias ___ josm-dev mailing list josm-...@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [OSM-dev] How to clean up Negative IDs
Just how slow is bulk_upload.py? I am loading a 177MB .osm file into an empty database on a quad 3.6Ghz Xeon with 6GB RAM and 700GB of RAID5. The machine is basically idle except for this load. It's already taken almost an hour. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf 720-334-7734 On Tue, Aug 3, 2010 at 12:48 PM, andrzej zaborowski balr...@gmail.comwrote: On 3 August 2010 20:28, Eric Wolf ebw...@gmail.com wrote: This is in reference to the USGS OSMCP project - not the real OSM... When we imported our chunk of data initially (not me - the guy responsible is on walkabout in the Rockies), we followed the convention of using negative IDs in the .OSM file. But osmosis was used to load the data into the database and now all of our data has negative IDs. This seems to have a really nasty effect on the API - every time something is edited, a new copy is created with positive IDs and the old version with the negative IDs persists. I assume there is something in the API that says negative IDs == BAD. I've been trying to test that theory but keep hitting stumbling blocks. Postgres doesn't seem to want to let me defer integrity constraints, so my efforts to change a few IDs to positive values keeps failing. Maybe I've lost my SQL chops (or maybe I just can't do that as the openstreetmap database user). Am I barking up the right tree? Should I just go ahead and destroy the database and repopulate it using bulk_upload.py instead of osmosis? If there's no way disable the postgres contraints (I'm sure there is.. but I'm a sql noob), I'd filter your .osm file through sed removing the '-' in 'ref=-' and 'id=-' and reimport with osmosis, or modify your conversion script. Using bulk_upload.py and the API will take ages. Cheers ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to clean up Negative IDs
I imagine the bottleneck is the Railsport doing precondition checks for everything as it's going in. I don't think I could give an educated guess for time remaining, but on the api.osm.org server it usually takes 4+ hours to send in a 50k-change diff file (around 25MB?). Based on that I'd say you have at least half a day of waiting left. On Tue, Aug 3, 2010 at 3:46 PM, Eric Wolf ebw...@gmail.com wrote: Just how slow is bulk_upload.py? I am loading a 177MB .osm file into an empty database on a quad 3.6Ghz Xeon with 6GB RAM and 700GB of RAID5. The machine is basically idle except for this load. It's already taken almost an hour. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf 720-334-7734 On Tue, Aug 3, 2010 at 12:48 PM, andrzej zaborowski balr...@gmail.comwrote: On 3 August 2010 20:28, Eric Wolf ebw...@gmail.com wrote: This is in reference to the USGS OSMCP project - not the real OSM... When we imported our chunk of data initially (not me - the guy responsible is on walkabout in the Rockies), we followed the convention of using negative IDs in the .OSM file. But osmosis was used to load the data into the database and now all of our data has negative IDs. This seems to have a really nasty effect on the API - every time something is edited, a new copy is created with positive IDs and the old version with the negative IDs persists. I assume there is something in the API that says negative IDs == BAD. I've been trying to test that theory but keep hitting stumbling blocks. Postgres doesn't seem to want to let me defer integrity constraints, so my efforts to change a few IDs to positive values keeps failing. Maybe I've lost my SQL chops (or maybe I just can't do that as the openstreetmap database user). Am I barking up the right tree? Should I just go ahead and destroy the database and repopulate it using bulk_upload.py instead of osmosis? If there's no way disable the postgres contraints (I'm sure there is.. but I'm a sql noob), I'd filter your .osm file through sed removing the '-' in 'ref=-' and 'id=-' and reimport with osmosis, or modify your conversion script. Using bulk_upload.py and the API will take ages. Cheers ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to clean up Negative IDs
Just killed the bulk_upload.py job, dropped database and recreated it. Used sed to fix the negative numbers. osmosis took 427263 milliseconds. Yes. I did update the ID sequences in postgres. Things are much happier without all that negativism. It's still very slow in Potlatch. At least part of the problem is the insane complexity of the features (yes, that straight road segment needs 82 nodes!) -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf 720-334-7734 On Tue, Aug 3, 2010 at 2:55 PM, Ian Dees ian.d...@gmail.com wrote: I imagine the bottleneck is the Railsport doing precondition checks for everything as it's going in. I don't think I could give an educated guess for time remaining, but on the api.osm.org server it usually takes 4+ hours to send in a 50k-change diff file (around 25MB?). Based on that I'd say you have at least half a day of waiting left. On Tue, Aug 3, 2010 at 3:46 PM, Eric Wolf ebw...@gmail.com wrote: Just how slow is bulk_upload.py? I am loading a 177MB .osm file into an empty database on a quad 3.6Ghz Xeon with 6GB RAM and 700GB of RAID5. The machine is basically idle except for this load. It's already taken almost an hour. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf 720-334-7734 On Tue, Aug 3, 2010 at 12:48 PM, andrzej zaborowski balr...@gmail.comwrote: On 3 August 2010 20:28, Eric Wolf ebw...@gmail.com wrote: This is in reference to the USGS OSMCP project - not the real OSM... When we imported our chunk of data initially (not me - the guy responsible is on walkabout in the Rockies), we followed the convention of using negative IDs in the .OSM file. But osmosis was used to load the data into the database and now all of our data has negative IDs. This seems to have a really nasty effect on the API - every time something is edited, a new copy is created with positive IDs and the old version with the negative IDs persists. I assume there is something in the API that says negative IDs == BAD. I've been trying to test that theory but keep hitting stumbling blocks. Postgres doesn't seem to want to let me defer integrity constraints, so my efforts to change a few IDs to positive values keeps failing. Maybe I've lost my SQL chops (or maybe I just can't do that as the openstreetmap database user). Am I barking up the right tree? Should I just go ahead and destroy the database and repopulate it using bulk_upload.py instead of osmosis? If there's no way disable the postgres contraints (I'm sure there is.. but I'm a sql noob), I'd filter your .osm file through sed removing the '-' in 'ref=-' and 'id=-' and reimport with osmosis, or modify your conversion script. Using bulk_upload.py and the API will take ages. Cheers ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Period In Key Names: Will It Cause A Problem?
Will a period in the key name for a tag cause a problem? It is proposed in: http://wiki.openstreetmap.org/wiki/Proposed_features/Tag_structures I would like to use it for some custom tags I am using, namely name.prefix, and name.full (see below for context). The idea is that .prefix and .full is closely tied to the primary name key, and not some other forms of name such as alt_name. If the properties applied to the alt_name key than alt_name.prefix will be used instead, etc. However, if the period will cause any sort of problem I will just use underscore in its place. The underscore doesn't make the relation as obvious, but its not a big deal. -- Forwarded message -- Date: Tue, 3 Aug 2010 16:01:46 -0500 From: Ian Dees ian.d...@gmail.com To: Richard Weait rich...@weait.com Cc: Kevin Atkinson ke...@atkinson.dhs.org, talk...@openstreetmap.org Subject: Re: [Talk-us] Would Like To Clean Salt Lake City Street Names On Tue, Aug 3, 2010 at 3:57 PM, Richard Weait rich...@weait.com wrote: On Tue, Aug 3, 2010 at 4:05 PM, Kevin Atkinson ke...@atkinson.dhs.org wrote: I'm almost done with this script. It's not a full bot, but instead modifies an osm file which I will read back into JOSM and upload the changed parts (or if that doesn't work use one of the upload scripts). Changes in what It will do noted below. [ ... ] I'm not going to do this. Instead I am going to simply remove variants on name (for example all other names in the 900 East example). And than leave all other name_* and *_name alone (i.e. name_1, alt_name, etc). When the alternate name is a numbered street, it will get the .prefix and .full tags. For example: name: Lorna Circle name_1: 3805 South name_1.prefix: W name_1.full: West 3805 South Whoa. Are you considering adding a period . to the key? Might that mess up postgres? From http://www.postgresql.org/docs/8.2/static/sql-syntax-lexical.html The period (.) is used in numeric constants, and to separate schema, table, and column names. Perhaps bounce some of these ideas around on dev? I imagine the Ruby code is doing the proper escaping for text like this, but you're right: definitely worth a message to the dev list. Not that it matters too much to the majority of the OSM community, but a period in the key name is invalid for most NoSQL-based systems (MongoDB in particular). ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [josm-dev] Change to changeset comment handling, RfD
On Tue, Aug 3, 2010 at 6:52 PM, Frederik Ramm frede...@remote.org wrote: But I think it is the obligation of the editor to make it perfectly clear to the user that their edits *will* be viewed as less cooperative by a large majority of the community if they don't carry a proper changeset comment - if not at least an *effort* is made. I think the obligation is on you to show that this is the case. There's a world of difference between good changeset comments are helpful, useful, valuable (which is pretty much true by definition, if the comment wasn't helpful, useful, valuable, it wouldn't be good) and a large majority of the community *will* view people's edits as 'less cooperative' if they don't provide a good changeset comment (with what, every edit, 10% of edits, 5% of edits?). That's what the discussion on *this* list is about - not whether changeset comments are good or bad, or whether Frederik is an evil dictator who wants the project to cater to his personal taste, or (as was suggested to me in a personal email) whether or not Frederik uses meaningful changeset comments himself. I asked you for your username, because I wanted to see what you consider to be a good changeset comment. It never struck me that you might not actually consider your own changeset comments to be good. ___ josm-dev mailing list josm-...@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [OSM-dev] Period In Key Names: Will It Cause A Problem?
Will a period in the key name for a tag cause a problem? Generally: No. You were asking about database problems: No problems at all in regards to PostgreSQL. MongoDB is the only database I know that has problems with dots in keys but you'd have to escape everything anyway, so no problem there either. It is proposed in: http://wiki.openstreetmap.org/wiki/Proposed_features/Tag_structures I'd disregard that page...but that's just my opinion. I would like to use it for some custom tags I am using, namely name.prefix, and name.full (see below for context). The idea is that .prefix and .full is closely tied to the primary name key, and not some other forms of name such as alt_name. If the properties applied to the alt_name key than alt_name.prefix will be used instead, etc. I'd use name:full, name:prefix etc. for it. As far as I can tell it's pretty much established practice and you'd make _at least_ my life easier :) Cheers, Lars ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Period In Key Names: Will It Cause A Problem?
On Wed, 4 Aug 2010, Lars Francke wrote: Will a period in the key name for a tag cause a problem? Generally: No. You were asking about database problems: No problems at all in regards to PostgreSQL. MongoDB is the only database I know that has problems with dots in keys but you'd have to escape everything anyway, so no problem there either. Yeah, that is what I was asking. It is proposed in: http://wiki.openstreetmap.org/wiki/Proposed_features/Tag_structures I'd disregard that page...but that's just my opinion. I would like to use it for some custom tags I am using, namely name.prefix, and name.full (see below for context). The idea is that .prefix and .full is closely tied to the primary name key, and not some other forms of name such as alt_name. If the properties applied to the alt_name key than alt_name.prefix will be used instead, etc. I'd use name:full, name:prefix etc. for it. As far as I can tell it's pretty much established practice and you'd make _at least_ my life easier :) How? Anyway, you don't think the distinction between Parameter and a Property is important? I like the . better, but again I don't care that much. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New full history dump available
On Mon, Aug 2, 2010 at 9:29 PM, Lars Francke lars.fran...@gmail.com wrote: Hi guys, Matt Amos was so nice to run the history export again. The result is available here: http://planet.openstreetmap.org/full-experimental/full-planet-100801.osm.bz2 and it's grown from 13 GB in February to 17 GB. The regular planet has grown from 8 to 10 GB in the same time. Have fun and it would always be interesting to know if you're using it, having problems with it and what you're using it for. This is a non issue, but interesting. I ran bzip2recover on it and every two bz2 blocks are just ~100bytes.. I'm guessing this is an artifact from parallel bzip2, but that it could be so extreme was new to me. Size in bytes for bz2 blocks in full planet history file: 36972 rec00614fullplanet.bz2 152 rec00615fullplanet.bz2 This is from the ordinary planet dump: 107188 rec00025plane.bz2 32877 rec00026planet.bz2 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Period In Key Names: Will It Cause A Problem?
I'd use name:full, name:prefix etc. for it. As far as I can tell it's pretty much established practice and you'd make _at least_ my life easier :) How? I've just rewritten parts of OSMdoc.com to support : in keys[1] because they were the only delimiter in widespread use. So I naturally hope no other delimiters gain any traction and I think they are unnecessary. Anyway, you don't think the distinction between Parameter and a Property is important? I like the . better, but again I don't care that much. Do you have a good example where this distinction would be relevant (i.e. any examples where you'd like to use . _and_ :)? I don't but again I might be wrong and I don't care that much either way. Cheers, Lars [1] http://osmdoc.blogspot.com/2010/01/full-writeup-of-how-i-parse-tags.html ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] New full history dump available
Well spotted. Different releases of pbzip2 were used. We'll fix this in future releases. But unlikely to be more frequent then once every 2 months. / Grant On 8/4/10, Erik Johansson e...@kth.se wrote: On Mon, Aug 2, 2010 at 9:29 PM, Lars Francke lars.fran...@gmail.com wrote: Hi guys, Matt Amos was so nice to run the history export again. The result is available here: http://planet.openstreetmap.org/full-experimental/full-planet-100801.osm.bz2 and it's grown from 13 GB in February to 17 GB. The regular planet has grown from 8 to 10 GB in the same time. Have fun and it would always be interesting to know if you're using it, having problems with it and what you're using it for. This is a non issue, but interesting. I ran bzip2recover on it and every two bz2 blocks are just ~100bytes.. I'm guessing this is an artifact from parallel bzip2, but that it could be so extreme was new to me. Size in bytes for bz2 blocks in full planet history file: 36972 rec00614fullplanet.bz2 152 rec00615fullplanet.bz2 This is from the ordinary planet dump: 107188 rec00025plane.bz2 32877 rec00026planet.bz2 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- Sent from my mobile device ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] How to clean up Negative IDs
On that note, I think there's a lot of scope for improving import speed in Osmosis. Currently it does it all with multi-line SQL inserts. PostgreSQL JDBC drivers now have COPY support and I have tried it out in the --fast-write-pgsql task. It works well and is much faster. There's a bit of work to add it to the apidb tasks though so I'm unlikely to attempt it any time soon. As for negative ids, I always intended to add the ability to add new OSM data directly to a database but never got around to it. Using the bulk uploader scripts are certainly the safest option on the production database. On Wed, Aug 4, 2010 at 8:06 AM, Eric Wolf ebw...@gmail.com wrote: Just killed the bulk_upload.py job, dropped database and recreated it. Used sed to fix the negative numbers. osmosis took 427263 milliseconds. Yes. I did update the ID sequences in postgres. Things are much happier without all that negativism. It's still very slow in Potlatch. At least part of the problem is the insane complexity of the features (yes, that straight road segment needs 82 nodes!) -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf 720-334-7734 On Tue, Aug 3, 2010 at 2:55 PM, Ian Dees ian.d...@gmail.com wrote: I imagine the bottleneck is the Railsport doing precondition checks for everything as it's going in. I don't think I could give an educated guess for time remaining, but on the api.osm.org server it usually takes 4+ hours to send in a 50k-change diff file (around 25MB?). Based on that I'd say you have at least half a day of waiting left. On Tue, Aug 3, 2010 at 3:46 PM, Eric Wolf ebw...@gmail.com wrote: Just how slow is bulk_upload.py? I am loading a 177MB .osm file into an empty database on a quad 3.6Ghz Xeon with 6GB RAM and 700GB of RAID5. The machine is basically idle except for this load. It's already taken almost an hour. -Eric -=--=---===---=--=-=--=---==---=--=-=- Eric B. Wolf 720-334-7734 On Tue, Aug 3, 2010 at 12:48 PM, andrzej zaborowski balr...@gmail.comwrote: On 3 August 2010 20:28, Eric Wolf ebw...@gmail.com wrote: This is in reference to the USGS OSMCP project - not the real OSM... When we imported our chunk of data initially (not me - the guy responsible is on walkabout in the Rockies), we followed the convention of using negative IDs in the .OSM file. But osmosis was used to load the data into the database and now all of our data has negative IDs. This seems to have a really nasty effect on the API - every time something is edited, a new copy is created with positive IDs and the old version with the negative IDs persists. I assume there is something in the API that says negative IDs == BAD. I've been trying to test that theory but keep hitting stumbling blocks. Postgres doesn't seem to want to let me defer integrity constraints, so my efforts to change a few IDs to positive values keeps failing. Maybe I've lost my SQL chops (or maybe I just can't do that as the openstreetmap database user). Am I barking up the right tree? Should I just go ahead and destroy the database and repopulate it using bulk_upload.py instead of osmosis? If there's no way disable the postgres contraints (I'm sure there is.. but I'm a sql noob), I'd filter your .osm file through sed removing the '-' in 'ref=-' and 'id=-' and reimport with osmosis, or modify your conversion script. Using bulk_upload.py and the API will take ages. Cheers ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Period In Key Names: Will It Cause A Problem?
On Wed, 4 Aug 2010, Lars Francke wrote: I'd use name:full, name:prefix etc. for it. As far as I can tell it's pretty much established practice and you'd make _at least_ my life easier :) How? I've just rewritten parts of OSMdoc.com to support : in keys[1] because they were the only delimiter in widespread use. So I naturally hope no other delimiters gain any traction and I think they are unnecessary. Anyway, you don't think the distinction between Parameter and a Property is important? I like the . better, but again I don't care that much. Do you have a good example where this distinction would be relevant (i.e. any examples where you'd like to use . _and_ :)? I don't but again I might be wrong and I don't care that much either way. name.full:en name.full:fr But I am just going to use : for now. It will be very easy to fix latter if necessary. Personally I think : is a bit overloaded, but that's another story. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [josm-dev] Change to changeset comment handling, RfD
Hi, Ulf Lamping wrote: Sorry, you overemphasize the importance of changeset comments. I think you underestimate the importance of changeset comments. Please keep in mind, that a changeset comment is *optional*, and there is no misbehaviour if not added. If a user don't want to add comments it is his right to do so. Just because everybody has the right to be a prick it doesn't mean that that is socially accepted. Bye Frederik ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
Hi, Sebastian Klein wrote: Personally I only check a don't show this message again-box if I'm 100% sure I understood the message and I really, really don't want it to show up again We have had users on talk-de who had accidentally clicked a don't show this message again box and were desperately looking for a way to undo that. Bye Frederik ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
Hi, Julien Balas wrote: I prefere some new data and no comment rather than... nothing. See, that's the difference between the data is more important and the community is more important view. I think that the people count more than the data they contribute, and if their attitude towards the community is I have better things to do than explain my mapping to you geeks, then I'd rather not have their data. Of course not everybody who doesn't use changeset comments is arrogant and selfish - to be arrogant and selfish you must first know that comments are important for the community and *then* ignore that wish, and until now, it seems, not everybody knows that. Bye Frederik ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
I suggest we try to JOSM add some comments by itself:
Region - Closest town - Major changes ( name of highway changed, added
forests, mainly buildings, POI)
based on simple statistics, then add a semicolon for the user to add a
reason or other comment.
Based on the type of edits a type of comment could be:
{ Added / Modified / Deleted} XXX POI Parking near Paris France
{ Added / Modified / Deleted} XXX nodes of Highway Piccadilly Circus
near London UK
{ Added / Modified / Deleted} XXX nodes of type Waterway near Venice
Italy
{ Added / Modified / Deleted} XXX names of type Highway near Newtown
Iceland
Based on the largest number of common property in the changeset.
Of course some edits will show a mix of all kind of edits, places and
POI and
a comments will show nonsense... But for users like me (and you?) whose
edit are around some kind of theme, or geographically limited,
statistics
can help creating some meaningful start of comment
Alternately the server may add such in a second line in a retrofit
way...
I am not sure how this will work out in practice, but someone may give
it a try..
Not me.. I cannot code.
Just 2 cents that may inspire someone
Gert
-Oorspronkelijk bericht-
Van: josm-dev-boun...@openstreetmap.org
[mailto:josm-dev-boun...@openstreetmap.org] Namens Julien Balas
Verzonden: dinsdag 3 augustus 2010 16:33
Aan: Russ Nelson
CC: josm-dev@openstreetmap.org
Onderwerp: Re: [josm-dev] Change to changeset comment handling, RfD
i use to work on a system where comments where mandatory to validate
an
order.
most of the people use X as a comment.
then the IT guys add a rule forbidding X
then users use .
then IT add a rule at least 3 chars
then user use bla as comments
and so on, and so on.
as far as i don't like empty comments, i think that if a user quit OSM
because a changeset comment is mandatory it's too bad.
I prefere some new data and no comment rather than... nothing.
--
JB
2010/8/3 Russ Nelson nel...@crynwr.com
Frederik Ramm writes:
Hi,
Ulf Lamping wrote:
Sorry, you overemphasize the importance of changeset comments.
I think you underestimate the importance of changeset comments.
Trouble is that you can't force people to provide good comments. You
can force them to provide comments, but short of introducing
artificial intelligence or actual human moderation into comment
submission, you have to accept what people give.
Anything you try to force people to do, they will rebel against.
--
--my blog is athttp://blog.russnelson.com
Crynwr supports open source software
521 Pleasant Valley Rd. | +1 315-600-8815
Potsdam, NY 13676-3213 | Sheepdog
___
josm-dev mailing list
josm-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev
___
josm-dev mailing list
josm-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev
___
josm-dev mailing list
josm-dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
Frederik Ramm wrote: Hi, ce-test, qualified testing bv - Gert Gremmen wrote: I suggest we try to JOSM add some comments by itself: Merkaartor did that, or perhaps does it still. These auto-generated comments are next to worthless. They cannot replace one human being telling another human being in a few words what has been done and why. Yes, it is pretty useless. However adding it as another changeset tag, e.g. auto_comment=* would not do any harm. Sebastian ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
On Tue, Aug 3, 2010 at 5:22 PM, Ulf Lamping ulf.lamp...@googlemail.com wrote: Am 03.08.2010 18:12, schrieb Anthony: On Tue, Aug 3, 2010 at 4:51 AM, Ulf Lampingulf.lamp...@googlemail.com wrote: I don't think that the change to 10 chars is a good idea. E.g. if I only add a tag to a node, the comment tag added is IMO sufficient but still rings the bell. I'd say a comment of tag added isn't any better than no comment at all. But if I'll add a much longer comment, I'll spend more time to add a meaningful comment than the time to do the actual change. A lot more than the 1% of mapping time that Frederik mentioned earlier. *nod*. Understood. I was more arguing for allowing no comments than for requiring long detailed ones. This isn't like code, where you're usually going to spend hours on a single commit (and where you can reasonably test your change before you commit it). I usually use Potlatch, and I save very often just to avoid losing my work when my browser crashes or when Potlatch decides to corrupt my edit. As such, I usually don't even know what's in a changeset. The changeset generally isn't at a high enough level to merit a comment. ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
Ulf, Is it really your strategy to offend anyone not sharing your point of view and build a community on top of that? Please count the number of people who have participated in the discussion on talk, and see how many of them thought that changeset comments are important. If I had the impression that this was just a personal itch of mine that nobody shared, I'd long since gone quiet. Of course, even though a large majority of contributors think that changesets are important, you can still contribute to the project without doing what they'd like you to do. The question here, on this mailing list, should be: What is the role of an editor in general, or JOSM in particular, in questions such as this? There is absolutely no doubt that good changeset comments are helpful, useful, valuable. (I'll not argue about that, and nobody in their right mind would claim otherwise.) Of course there may be users who still choose not to enter such a comment, and perhaps I was unnecessarily polarising when I said they were selfish or misbehaving; there might be other reasons, e.g. they are very pressed for time, only making a tiny edit which they wouldn't have the time to make if they had to think of a sentence that describes it; or they might (mistakenly) think that they have to write in English and don't find the right words; or their edit was so complex that they feel they can't do justice to it in a changeset comment, or whatever. So yes, there might be reasons why someone would decline to enter a comment. But how far should an editor go in encouraging them, even nagging them, to reconsider? We have seen in the past, when Potlatch was young and buggy, that lots of people flamed against Potlatch - *especially* on talk-de, I guess you'll remember. That was because the Potlatch editor let people do some things which the majority of the community did not like; often, Potlatch users weren't even aware that in the eyes of the community they misbehaved. I don't have to explain that to you. An editor that makes it too easy - or, let's take a placatively extreme example, an editor that doesn't even ask for a changeset comment - would surely be viewed as problematic by the community, and its use not recommended. On the lists, people would say: Oh, another guy using the XYZ editor, so no changeset comment of course.. I am adamant that JOSM should not make it too easy for people to not enter a changeset comment. I think it should be possible to omit it if one really wants to (so the 3-character rule we had until now was perhaps not really good). But I think it is the obligation of the editor to make it perfectly clear to the user that their edits *will* be viewed as less cooperative by a large majority of the community if they don't carry a proper changeset comment - if not at least an *effort* is made. That's what the discussion on *this* list is about - not whether changeset comments are good or bad, or whether Frederik is an evil dictator who wants the project to cater to his personal taste, or (as was suggested to me in a personal email) whether or not Frederik uses meaningful changeset comments himself. This discussion is about the question how much an editor should try to influence the user's behaviour vis-a-vis the community. There is absolutely no doubt in my mind that a contribution *without* a changeset comment is inferior to the exact same contribution *with* a changeset comment (unless of course the comment is ... or fuck you). But how far should an editor go in trying to keep users from making such inferior contributions (or help them making them better)? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
Hi, Anthony wrote: I asked you for your username, because I wanted to see what you consider to be a good changeset comment. It never struck me that you might not actually consider your own changeset comments to be good. I have given a number of examples that I consider good in the discussion over on talk; but I also agree with the examples others have posted there. An ideal changeset comment, in my opinion, would convey the what, where, and why of an edit. The quality of my own changeset comments is absolutely irrelevant in this discussion; let's assume, if it gives you pleasure, that they are all just That might discredit the messenger, but not change anything about the message. I think that it is important to keep the two separate, the message and the messenger. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
On Tue, Aug 3, 2010 at 7:34 PM, Frederik Ramm frede...@remote.org wrote: Hi, Anthony wrote: I asked you for your username, because I wanted to see what you consider to be a good changeset comment. It never struck me that you might not actually consider your own changeset comments to be good. I have given a number of examples that I consider good in the discussion over on talk; but I also agree with the examples others have posted there. An ideal changeset comment, in my opinion, would convey the what, where, and why of an edit. The quality of my own changeset comments is absolutely irrelevant in this discussion; let's assume, if it gives you pleasure, that they are all just That might discredit the messenger, but not change anything about the message. I think that it is important to keep the two separate, the message and the messenger. minor haiti geometry repair fix source typo fix self-intersecting boundary move lake from one relation to other remove forest self-intersection Is that you? ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
Am 04.08.2010 01:19, schrieb Frederik Ramm: Ulf, P.S: Yes, this isn't a theoretical situation. While riding my motorbike, I frequently have a lot of such tiny changes spread over a huge area. Perhaps we can try to be a bit more reasonable about this. That's what I'm trying to do. I have the impression that those who oppose changeset comments, including you, think that they are expected to write an essay. In software development, I've written anything from shit! to a 1000 chars essay what's going wrong - or simply nothing. That's the thing: It *really* depends on the situation ... The way I like to think of this is: If you speak to another mapper later that day, and they ask you what you've been up to? and you answer them in one sentence - that's what you should put as a changeset comment. Sounds like a good rule of thumb how to add comments - a thing we just still miss. I agree that if you were to add 100 POIs and upload each in their own changeset, it would be tedious to actually put an *individual* changeset comment to each. Ideally, you would keep a changeset open and have them all in one changeset, but if you must have lots of mini-changesets with only one or two changes each, then that's a case where I'd just re-use the same changeset comment. After all, if you *had* put them all in one changeset then the comment would have been added POIs from motorcycle survey. I've tried to add all of the changes in one rush. This too often resulted in an OutOfMemoryException. Not a situation I want to upload something. So my habit becomes to upload (very) tiny locations. I think we should find the best way to make as many users as possible understand why changeset comments are useful, and supply them as good as they can when they use JOSM. I'll perfectly agree. If the best way to do that turns out to actually drop any user nannying in JOSM and just put a decent explanation in the user manual or so, I'm game. My idea is that JOSM should explain (or links to help) why those comments are useful and help the user how those could look like (in a guideful but neutral language). But if the user turns out to ignore those info (for whatever reason), it's not JOSMs job to enforce those comments. Regards, ULFL ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
Re: [josm-dev] Change to changeset comment handling, RfD
On Tue, Aug 03, 2010 at 08:08:56PM -0400, Anthony wrote: The quality of my own changeset comments is absolutely irrelevant in this discussion; let's assume, if it gives you pleasure, that they are all just That might discredit the messenger, but not change anything about the message. I think that it is important to keep the two separate, the message and the messenger. minor haiti geometry repair fix source typo fix self-intersecting boundary move lake from one relation to other remove forest self-intersection Is that you? Being a programmer and a daily user of version control systems, I share Frederik's view that some effort should be made to write good changeset comments. They can be useful later, say, after several months or years. Of the above comments, I think that the first one is questionable, almost as bad as the fixes or adjustments regularly written by some long-time contributors. The remaining ones are descriptive, if the changesets contain just that (e.g., replace source=lndsat with source=Landsat, fix polygons or multipolygons). Nobody is perfect, at least not all the time. :-) Marko ___ josm-dev mailing list josm-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/josm-dev
