Re: [Potlatch-dev] [OpenStreetMap] #4626: Potlatch2 toolbox disappears
#4626: Potlatch2 toolbox disappears --+ Reporter: SomeoneElse | Owner: potlatch-dev@… Type: defect | Status: new Priority: minor| Milestone: Component: potlatch2|Version: Resolution: | Keywords: --+ Comment (by Max1234ita@…): Replying to [comment:1 SomeoneElse]: Grr. bottom-left and corner should of course read bottom-right-hand corner. Confirmed on Windows XP SP3 too. Regards, MAx - Italy -- Ticket URL: https://trac.openstreetmap.org/ticket/4626#comment:2 OpenStreetMap http://www.openstreetmap.org/ OpenStreetMap is a free editable map of the whole world ___ Potlatch-dev mailing list Potlatch-dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/potlatch-dev
Re: [OSM-dev] History statistics
Hi no it was a mistake. Basicly atm there is one huge nested map, that holds all required information: nodemap[Objekt-ID][Timestamp] = (lat, lon) lat/lon is atm stored as double. The suggested memory layout is as follows: 1. An Array with a 32bit Pointer for each Node-ID 2. A Series of Timestamp-Lat-Lon Triplets, each stored as Integers, all together on one row for all versions, ending with a 0-byte Memory-Usage would be 8*(max. node-id) for 1. 4*(number of used node-ids) + 12*(number of used node-version) for 2. Maximum memory usage for a complete planet would be ~40GB. With a normal Array for 1. the minimum memory usage would be ~10GB. This could be reduced by using a sparsetable http://google-sparsehash.googlecode.com/svn/trunk/doc/sparsetable.html Peter Am 16.10.2012 10:50, schrieb Ab_fab: Thanks for your answer Peter. You did not answer to the list, was it intentional ? I forwarded your reply to the French list thread [1], where the initial discussion with Aurelien and others took place, back in july. I totally miss the skills required to implement anything ... Anyway, it could be of interest for some people, in particular for stats purpose on the evolution of the database contents (not only the number of buildings from cadastre ;-) ). French community has also (I think) enough material ressources to setup such low activity database, once this RAM issue for the import is solved. Regards Fabien [1] http://lists.openstreetmap.org/pipermail/talk-fr/2012-July/045337.html http://lists.openstreetmap.org/pipermail/talk-fr/2012-October/049849.html 2012/10/16 Peter Körner pe...@mazdermind.de mailto:pe...@mazdermind.de Hi Well, osm-history-renderer is - after all - a proof of concept. It's impossible atm. to import a full country without having tons of RAM. If you are interested in optimizing the memory usage I could provide you with plans about a memory layout frederik ramm and I once talked about; I never came around to implement it, while the necessary interfaces are already in place. Regards, Peter Am 15.10.2012 15:08, schrieb Ab_fab: I think the memory problems occured during the osm-history-importer [1] process Some people were successfull in building the database for limited areas, but France is extract is huge in comparison. Is there a way to limit RAM usage for such large files during import ? [1] https://github.com/MaZderMind/__osm-history-renderer https://github.com/MaZderMind/osm-history-renderer 2012/7/16 Jochen Topf joc...@remote.org mailto:joc...@remote.org mailto:joc...@remote.org mailto:joc...@remote.org On Mon, Jul 16, 2012 at 01:28:50PM +0200, Aurélien FILEZ wrote: I'm searching history statistics of the french territory contributions in order to make some graphics about : - Number of named roads - Numbers of house numbers - Number of kilometers of different types of roads (highway, primary, secondary, tertiary, residential). I tried OSMIUM with the last avalable france.osh.pbf, but data are too many data and cause bad allocs errors on a 8 Gb memory server. What exactly did you do? If you use the RangeFromHistory handler to filter out one specific date in the history and then the Statistics handler, you should be able to generate some statistics, though not exactly the kind of statistics you are asking for. But you can use the Statistics handler as a model on how to write your own. You'd have to tun the thing several times, each time filtering out a different date with RangeFromHistory. Or you write your own handler that directyl works on history data, but thats a bit more tricky. In any case it should not need 8GB for all of this. You might have used the MmapAnon node storage instead of SparseTable handler. This explains the different options: http://wiki.openstreetmap.org/__wiki/Osmium/Storing_Node___Positions http://wiki.openstreetmap.org/wiki/Osmium/Storing_Node_Positions Jochen -- Jochen Topf joc...@remote.org mailto:joc...@remote.org mailto:joc...@remote.org mailto:joc...@remote.org http://www.remote.org/jochen/ +49-721-388298 tel:%2B49-721-388298 tel:%2B49-721-388298 _ dev mailing list dev@openstreetmap.org mailto:dev@openstreetmap.org
Re: [OSM-dev] OSM Wishlist
Hi Am 12.10.2012 16:14, schrieb Serge Wroclawski: I made a second issue at the same time regarding gathering histrorical data around way geometry. Right now, due to the way OSM stores its geometry data, the only way to know about a way's geometry history is to call the history call for the way, then make a history call for each and every node that was ever in the way. That's something I ussed about 4 years ago when I stared working with the history and someone pointed me to an important fact: The API is to support mappers. Using it for small tools/experiments is ok, as long as they dont't put noticable load on the api. For everything else, there are planet/history dumps. With today's tools it's quite simple to extract your area of interest from a history dump and work on that. In fact it's much much easier to use those dumps to fetch data then to crawl the api, once ypou get the algorithms in place. Talking with Matt about this a month ago on IRC, he suggested a new call could be made, one that would do the above work in a single call. This would allow the same work to be done with less server overhead, and greater resource management. Tom closed it, saying he didn't like expensive calls. I don't like expensive calls *on the main api* either. But why not build a history api? It could feed itsself from the history dumps and kept up to date from the replication diffs. It's all there, just assemble it. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
On Mon, Oct 15, 2012 at 07:48:24PM -0700, John Firebaugh wrote: On Mon, Oct 15, 2012 at 4:17 PM, Tom Hughes t...@compton.nu wrote: Exporting raw data was never the primary purpose of the export tab - it was really added to address the constant request for the ability to export images of an area. The data export things was something I threw in because I could, and for most purposes people should be looking elsewhere for raw data. Should they? The general consensus at the BoF was that some (many?) people do first look to the export tab for raw data, and leave confused by its interface and limitations. It's unfortunate that for a project whose guiding principle is open data, actually getting that data proves to be a challenge for those not familiar with the various OSM satellite sites. And so the thought was that the export tab is due for a redesign and a rethink, and it should guide people interested in exports at a particular size to the appropriate place, if not offer a download directly. The challenge, of course, is to characterize the various use cases, determine what the appropriate source and format is for each case, and see how and if it is possible to build something that can satisfy many different needs in a way that's not so complex that it deters just as many interested people as the current interface does. OSM has created a great ecosystem of people, projects and companies exporting OSM data in many different formats for many different purposes. There are huge differences in the ways the data is formatted and each of thoses formats has it's uses, but also its drawbacks. You have to know quite a lot about OSM to make informed decisions about which kind of export you want for your specific needs. I don't think we can bring all of this into a single export tab. In fact, if we reduce all of this into a single export tab we risk alienating people who try out whats offered, find it lacking and then go away not understanding that they tried the wrong approach. I think instead of the export tab the approach should be some kind of wizard. You click on the big button that says Use OpenStreetMap or so. That leads to a series of choices where we can ask people what they want and offer solutions for the most common use cases such as printed map, static map on a web site, dynamic map on a web site, web map with routing for 'how to find us' page etc. The most common (and easier to fullfill) choices lead to places similar to the current export tab where end-users can get what they want. For the cases where we don't have simple solutions the wizard ends in a message telling the user that what he wants is complicated and links to relevant wiki pages. Jochen -- Jochen Topf joc...@remote.org http://www.remote.org/jochen/ +49-721-388298 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
On Tue, Oct 16, 2012, at 11:09, Frederik Ramm wrote: - How could the Mapnik style be more open to improvements? We definitely need to get it on GitHub. I don't think the version control system used makes a difference - I doubt the 400+ issues filed against the current style would be solved quicker with git. The problem is not that this uses SVN (...) I think there is some kind of social coding positive effect when things are on Github. I can't explain it but it just is. As for source control - I can't stand working with SVN anymore, Git just spoiled me with cheap local branching and such. In Wikipedia, all user-to-user messages are automatically public (unless both parties agree to take it to email). Maybe food for thought - do we need private messages at all? There seem to be quite a number of bullies in OSM who send self-important messages to lesser experienced mappers explaining their rules... that could be stopped if all messages were public. That is a very interesting idea and I like it. Encouraging people to use public means of communication (wall-style messages, forums etc.) is definitely the way to go. Once there is some kind of public messaging (wall, activity stream or otherwise) integrated into osm.org I don't see the benefit of having private messages anymore - that is what e-mail should be for. I'm writing down your idea to bring it up at some point in the future. Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
On 16/10/12 05:42, Serge Wroclawski wrote: On Mon, Oct 15, 2012 at 7:17 PM, Tom Hughes t...@compton.nu wrote: Exporting raw data was never the primary purpose of the export tab - it was really added to address the constant request for the ability to export images of an area. Do we have statics (web logs) that show which functionality people use? We could do something, but it would have to be done from the rails logs so it would be rather complicated - because the relevant call is a POST we can't tell what format people asked for from the apache logs which would otherwise be a lot easier to parse. The data export things was something I threw in because I could, and for most purposes people should be looking elsewhere for raw data. I can see your concern about resources here, but is the export tab currently a source of problems? Huh? I don't think I mentioned resources? The issue isn't resources, it's about the suggestion that we somehow direct people to somewhere where they can download a relevant extract when they want a larger area. The problem I see is, other than pointing to planet.openstreetmap.org I'm not sure where else we would send them? While there are lots of other places to get things like regional extracts they are unofficial third party sites so I don't think that we should be referring people to them from the main site. If so, then I can see a good argument for providing something like the gui I made for XAPI but integrated with the OSM site (but hitting an xapi or overpass backend). Well I did try and get an official tiled data server for supporting large area downloads some time ago but OWG didn't want to do it. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] multipolygon relation and non-closed ways
Well, I care about the mapper who is interested in an agreed final result and doesn't want to spend his time reading the opinion of individual persons. Just for this there's a talk page for each wiki page. When interested you can subscribe to the talk page separately. If the talk page isn't enough additional pages can be easily created. Willi On Monday, October 15, 2012 10:43 AM Pedro Larroy [mailto:pedro.larroy.li...@gmail.com] wrote: Hi Willi. Sorry to read your negative opinion. I think a better and more detailed wiki page helps osm, irrespectively of edit history. I don't see why editing and refining a page is bad. Sure things can always be done better, but the beauty of wikis is that you can update them fast without going through a comitee. Regards. On Oct 14, 2012 4:39 AM, Willi wil...@gmx.de wrote: Imho it's not only bad behavior to change a wiki page 19 times on the same day it's harming OSM. Having the discussion on the OSM-dev list makes this even worse. http://wiki.openstreetmap.org/w/index.php?title=Relation:multipolygonaction =history All changes except minor should be discussed on the Talk page first. Even if you are the expert and are 200% sure that what you are writing is correct it might be misleading or even not understandable to non experts. And imho that's just the case for the new additions. Asking to be involved in discussions but changing the page without discussion is topping this. http://wiki.openstreetmap.org/wiki/Talk:Relation:multipolygon#Recent_change_ about_validity_of_a_multipolygon_relation I'm afraid more mapper will turn away from Wiki pages when they neither can follow nor understand such frequent changes. That's harming OSM. Willi ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Tue, Oct 16, 2012, at 01:03, sly (sylvain letuffe) wrote: If you propose to change it by creating a community-driven (instead of admin-driven as you put it) wishlist, by any means - do it. The operative word being do. I'll be glad to do so, and will start it. Unless the MapBox team (and tom) doesn't want to look at such a process (in the end gathering wish never read is just going to spend my time) In my opinion this is a broader topic than the Mapbox wishlist. I think some kind of user-voice coordinator or driver or however it would be (informally, of course) called would be extremely useful. Such person would need to keep up with the current development activity (Github, Trac, dev@, rails-dev@ and others) and represent user interests. I think this would be A LOT of work and probably not for only one person but I for one would love to see end-users more involved during development phase - testing, feedback, incremental improvements and all that. So the question for me is - who would be interested in doing such work... Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Older version of OWL alive?
Matt (or others who might know) - Is there a staging (or some other kind of) version of OWL alive that I could look at? Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Older version of OWL alive?
On Tue, 2012-10-16 at 09:32 -0400, Alex Barth wrote: Matt (or others who might know) - Is there a staging (or some other kind of) version of OWL alive that I could look at? sadly not. the version on dev (which was crushing it) died under its own weight a while ago. the situation back in May [1] is pretty much the same situation we're in now, despite great work by Ian Dees on trying to get the front-end working. cheers, matt [1] http://lists.openstreetmap.org/pipermail/talk/2012-May/062959.html ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
fr == Frederik Ramm frede...@remote.org writes: fr In Wikipedia, all user-to-user messages are automatically public fr (unless both parties agree to take it to email). Maybe food for fr thought - do we need private messages at all? There seem to be fr quite a number of bullies in OSM who send self-important messages fr to lesser experienced mappers explaining their rules... that fr could be stopped if all messages were public. Are you talking about the OSMF DWG? -- Eric Marsden ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
fr == Frederik Ramm frede...@remote.org writes: - Should SVG export be replaced with something else, or improved in Mapnik enough to be worthwhile? fr Definitely a question that I am often asked - keep in mind that there fr are two kinds of SVG, that which conforms to the standard and that which fr is readable by Adobe Illustrator, and most people actually need the fr second. I always tell people to use Maperitive for SVG export but it's fr not open source and only works on limited areas. Mapnik's SVG output is not very usable in an editor such as Inkscape, because objects are not logically grouped. It would be fantastic if this could be improved, and if landuse/roads/buildings/public transport/labels etc. were split into separate SVG layers (likewise for PDF output, in fact). I guess this would require dedicated backends, instead of benefiting from Cairo's nice multi-output features. -- Eric Marsden ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
The problem I see is, other than pointing to planet.openstreetmap.org I'm not sure where else we would send them? While there are lots of other places to get things like regional extracts they are unofficial third party sites so I don't think that we should be referring people to them from the main site. I think pointing people to jXAPI could work, or one thing. And, the whole 'unofficial third party sources' bit isn't that strong; planet.osm.orgpoints to third-party sources, and so does the wiki. In short, they exist and are useful, and I think it makes sense to reduce the number of logical leaps you need to take to get from A to B, because I'd assume a ton of people are getting lost along the way. On Tue, Oct 16, 2012 at 11:32 AM, Eric Marsden eric.mars...@free.fr wrote: fr == Frederik Ramm frede...@remote.org writes: fr In Wikipedia, all user-to-user messages are automatically public fr (unless both parties agree to take it to email). Maybe food for fr thought - do we need private messages at all? There seem to be fr quite a number of bullies in OSM who send self-important messages fr to lesser experienced mappers explaining their rules... that fr could be stopped if all messages were public. Are you talking about the OSMF DWG? -- Eric Marsden ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] missing nodes
Hi All, I'm relatively new to processing the OSM datasets and trying to extract interesting data. What I'm trying to do is build a dataset of all UK roads. I first used osmosis with a way filter based on the highway tag, which worked reasonably well. However, when I delved into the data, I noticed that a lot of my ways had node refs to missing node entities in the file I'm extracting data from (a recent england dump). Now, I expect some roads to get chopped off at borders and so on but I've done it manually via a java program that parses the full england dump and I've counted ~1.3million ways which reference nodes that I could not find in the same XML file. This seems excessive to me, unless I'm misunderstanding some aspect of the data model. Any ideas from the old hands? Many thanks, David ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
On 16/10/12 16:40, Tom MacWright wrote: The problem I see is, other than pointing to planet.openstreetmap.org http://planet.openstreetmap.org/ I'm not sure where else we would send them? While there are lots of other places to get things like regional extracts they are unofficial third party sites so I don't think that we should be referring people to them from the main site. I think pointing people to jXAPI could work, or one thing. And, the whole 'unofficial third party sources' bit isn't that strong; planet.osm.org http://planet.osm.org points to third-party sources, and so does the wiki. Well when jxapi.osm.org is stable then maybe we would, though I'm not entirely clear if this is really what it is designed for or how likely it would be to stand lots of large area downloads. As to other sites, the problems are several fold - how do we decide to bless one particular site over another? not to mention the fact that we might end up driving more traffic to such sites than they could handle. As a general rule we have always tried to avoid relying on, or directing people to, third party sites from the main site. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
Tom Hughes wrote: As to other sites, the problems are several fold - how do we decide to bless one particular site over another? not to mention the fact that we might end up driving more traffic to such sites than they could handle. From a purely end-user perspective, having lots of third-party sites is both a strength and a weakness - it's great that there's lots of choice, but also confusing to newbies. Take for example the perennial question how do I get an OSM map on my Garmin Satnav?. The answers (e.g. on help.osm.org) tend to be (pointing at the wiki) there at lots of different maps all displaying slightly different information, and (pointing at the wiki again) you can even make your own if you want to! But (pointing at http://garmin.openstreetmap.nl/) here's what you _actually_ want, at least the first time you want to try it. It's similar with download extracts - there are lots of places that you could download extracts from, but, as http://planet.osm.org/ currently does, there are two where it makes sense to look first. There are lots of cases where it makes sense to have a cornucopia of different third-party options, but it's difficult to argue the benefit of variety for something simple like planet extracts. Obviously there are already a few officially blessed third-party sites around - (e.g. 3 of the 4 osm.org sets of map tiles). Cheers, Andy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
On Tue, Oct 16, 2012 at 8:47 AM, Tom Hughes t...@compton.nu wrote: On 16/10/12 16:40, Tom MacWright wrote: The problem I see is, other than pointing to planet.openstreetmap.org http://planet.openstreetmap.**org/ http://planet.openstreetmap.org/ I'm not sure where else we would send them? While there are lots of other places to get things like regional extracts they are unofficial third party sites so I don't think that we should be referring people to them from the main site. I think pointing people to jXAPI could work, or one thing. And, the whole 'unofficial third party sources' bit isn't that strong; planet.osm.org http://planet.osm.org points to third-party sources, and so does the wiki. Well when jxapi.osm.org is stable then maybe we would, though I'm not entirely clear if this is really what it is designed for or how likely it would be to stand lots of large area downloads. It was indeed designed to handle larger areas like this, but I think it might be more interesting to restart the thread about cached, tiled data again to make these large area queries faster. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] OpenStreetBugs API in integration
Hi The current OSB integration work into osm.org, will this include the OSB API? http://wiki.openstreetmap.org/wiki/OpenStreetBugs/API#Editing_bugs Users of OSM data on their websites would be interested to integrate bug reporting directly into their application. Thanks -Mikel * Mikel Maron * +14152835207 @mikel s:mikelmaron___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] missing nodes
Hi David, did you find out exactly where in your toolchain these nodes get lost? Maybe it's the filter process with Osmosis? I wouldn't assume a bug, I think it's more likely a misunderstanding in how the program options should be used. I'm not sure about the official API definition but it might be perfectly normal that not every noderef has a corresponding node in the file. If you do cheap extracts (e.g. with osmconvert without --complete-ways option) the way objects will not be modified. Each way object will stay complete, even if all of its nodes (but one) has been removed from the file. Regards Markus Original-Nachricht Datum: Tue, 16 Oct 2012 16:40:18 +0100 Von: David Prime da...@primefarm.co.uk An: dev@openstreetmap.org Betreff: [OSM-dev] missing nodes Hi All, I'm relatively new to processing the OSM datasets and trying to extract interesting data. What I'm trying to do is build a dataset of all UK roads. I first used osmosis with a way filter based on the highway tag, which worked reasonably well. However, when I delved into the data, I noticed that a lot of my ways had node refs to missing node entities in the file I'm extracting data from (a recent england dump). Now, I expect some roads to get chopped off at borders and so on but I've done it manually via a java program that parses the full england dump and I've counted ~1.3million ways which reference nodes that I could not find in the same XML file. This seems excessive to me, unless I'm misunderstanding some aspect of the data model. Any ideas from the old hands? Many thanks, David ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] missing nodes
Hi Markus, I ditched osmosis because I wanted to know where they were being dropped so wrote my own filter tool which gave the same results. Looks like it's just how it is, but I'm wondering what the point of a way with broken noderefs is. Is it corruption or a feature? David On Tue, Oct 16, 2012 at 5:33 PM, mar...@gmx.eu wrote: Hi David, did you find out exactly where in your toolchain these nodes get lost? Maybe it's the filter process with Osmosis? I wouldn't assume a bug, I think it's more likely a misunderstanding in how the program options should be used. I'm not sure about the official API definition but it might be perfectly normal that not every noderef has a corresponding node in the file. If you do cheap extracts (e.g. with osmconvert without --complete-ways option) the way objects will not be modified. Each way object will stay complete, even if all of its nodes (but one) has been removed from the file. Regards Markus Original-Nachricht Datum: Tue, 16 Oct 2012 16:40:18 +0100 Von: David Prime da...@primefarm.co.uk An: dev@openstreetmap.org Betreff: [OSM-dev] missing nodes Hi All, I'm relatively new to processing the OSM datasets and trying to extract interesting data. What I'm trying to do is build a dataset of all UK roads. I first used osmosis with a way filter based on the highway tag, which worked reasonably well. However, when I delved into the data, I noticed that a lot of my ways had node refs to missing node entities in the file I'm extracting data from (a recent england dump). Now, I expect some roads to get chopped off at borders and so on but I've done it manually via a java program that parses the full england dump and I've counted ~1.3million ways which reference nodes that I could not find in the same XML file. This seems excessive to me, unless I'm misunderstanding some aspect of the data model. Any ideas from the old hands? Many thanks, David ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
Hi, On 16.10.2012 17:32, Eric Marsden wrote: fr thought - do we need private messages at all? There seem to be fr quite a number of bullies in OSM who send self-important messages fr to lesser experienced mappers explaining their rules... that fr could be stopped if all messages were public. Are you talking about the OSMF DWG? All of DWG's messages are already public; either because they're user block messages, or because they're posted to talk-fr by their recipients, or both ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] missing nodes
Hi, On 16.10.2012 18:34, David Prime wrote: I ditched osmosis because I wanted to know where they were being dropped so wrote my own filter tool which gave the same results. Looks like it's just how it is, but I'm wondering what the point of a way with broken noderefs is. Is it corruption or a feature? The database has no way with broken node refs. If you make an extract with Osmosis, and use neither clipIncompleteEntities nor completeWays, than you will end up with those ways that continue outside your polygon or bbox pointing to non-included nodes. If you have downloaded an England dump where 1.3 million nodes are referenced but not present then that dump is certainly broken. Can you make it available for inspection? Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
As to other sites, the problems are several fold - how do we decide to bless one particular site over another? Please be honest. You, I, and everbody else also set preferences on at best educated guesses, often more trust in what or whom we know or don't know. We use OpenLayers and not Leaflet on the main page, we don't have routing on the main page although several services are realiable. We have certain tile sources and others not. Some services have an address redirect off .osm.org, others not, for no particular reason. 1. The simplest way would be a public voting. Ask - whether somebody does volunteer to do the administration even longterm - let the community give comments on the project and site and person 2. A more elaborate way would be to design a benchmark for the task and test how the service in question performs. Then publish and discuss the test results. 3. An even more elabortae way would to have a test installation, announce the service as beta, observe how usage develops and give a report after some months. I encounter 1. easily possible, and 2. with some enthusiasm. I don't think 3. is necessary, but it may be worth considering. In any case, the produced material and documents suffice for a diligent decision. not to mention the fact that we might end up driving more traffic to such sites than they could handle. Could you give figures what amount of traffic you expect? 1 GB per day, 1 GB per hour, 1 GB per minute, or even more? Cheers, Roland ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] missing nodes
Hi David, David Prime writes: wrote my own filter tool which gave the same results. Looks like it's just how it is, but I'm wondering what the point of a way with broken noderefs is. Is it corruption or a feature? There shouldn't be that many missings refs at all. What is the source of your England dump? Is it contained there? I read your sentence below that it is already missing in your input file. to get chopped off at borders and so on but I've done it manually via a java program that parses the full england dump and I've counted ~1.3million ways which reference nodes that I could not find in the same XML file. Stephan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] missing nodes
There shouldn't be that many missings refs at all. What is the source of your England dump? Is it contained there? I read your sentence below that it is already missing in your input file. Stephan It's just your standard dump from http://download.geofabrik.de/openstreetmap/europe/great_britain/ Now I know it's corruption (potentially on my part), I'm going to look very carefully at how I used osmosis and how I wrote my app and see if I'm doing anything daft. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
On Mon, Oct 15, 2012 at 8:48 PM, John Firebaugh john.fireba...@gmail.com wrote: On Mon, Oct 15, 2012 at 4:17 PM, Tom Hughes t...@compton.nu wrote: Exporting raw data was never the primary purpose of the export tab - it was really added to address the constant request for the ability to export images of an area. The data export things was something I threw in because I could, and for most purposes people should be looking elsewhere for raw data. Should they? The general consensus at the BoF was that some (many?) people do first look to the export tab for raw data, and leave confused by its interface and limitations. It's unfortunate that for a project whose guiding principle is open data, actually getting that data proves to be a challenge for those not familiar with the various OSM satellite sites. And so the thought was that the export tab is due for a redesign and a rethink, and it should guide people interested in exports at a particular size to the appropriate place, if not offer a download directly. The challenge, of course, is to characterize the various use cases, determine what the appropriate source and format is for each case, and see how and if it is possible to build something that can satisfy many different needs in a way that's not so complex that it deters just as many interested people as the current interface does. I'm really sorry to have missed that session at the conference, but I'm happy to see that it gets carried forward to the list quickly. I agree with John that there is a need for a clear path for folks to the various data consumption options. I believe the Export tab should really be the Data tab, with options for * immediately downloading the current map extent as XML, full history XML, PBF and whatever other formats people want, * viewing the data as an overlay (now in the edit menu) * and a link to more data consumption options which could be something like https://docs.google.com/document/d/1dASGp5U0jRNZ3Kzb-LtzeOtUACqlLVh9GZdiKTftTOQ/edit (feel free to edit). The latter should not be a separate page but rather a div popup or something like that. -- martijn van exel http://oegeo.wordpress.com ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
On Oct 16, 2012, at 2:48 PM, Martijn van Exel m...@rtijn.org wrote: I agree with John that there is a need for a clear path for folks to the various data consumption options. I believe the Export tab should really be the Data tab, with options for * immediately downloading the current map extent as XML, full history XML, PBF and whatever other formats people want, * viewing the data as an overlay (now in the edit menu) * and a link to more data consumption options which could be something like https://docs.google.com/document/d/1dASGp5U0jRNZ3Kzb-LtzeOtUACqlLVh9GZdiKTftTOQ/edit (feel free to edit). The latter should not be a separate page but rather a div popup or something like that. I'm liking this direction and also renaming from 'export' to 'data' as this is clearer. Simple and based on a couple of common use cases. I'd add: - Move embed functionality to a clean control on the front page (that's not really 'export' or 'data') - Possibly do the same with image export (e. g. make embedding as slippy map or as image optional) This would mean folding http://planet.osm.org/ into a future data tab, no? How do the planet osm maintainers see this? -- martijn van exel http://oegeo.wordpress.com ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
2012/10/16 Tom Hughes t...@compton.nu: The problem I see is, other than pointing to planet.openstreetmap.org I'm not sure where else we would send them? You could point them here: http://wiki.openstreetmap.org/wiki/Downloading_data I think what is really missing are some short explanations, e.g. that the export tab is using ressources on the server and therefore only small areas can be downloaded and that there are alternative downloading options (and then point them to the wiki page above). It is all stuff that might seem obvious to you and to most people involved in OSM and maybe also for CS-people not so involved in general, but it isn't for someone occasionally looking at our page and clicking on the export tab. cheers, Martin ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
Wow, that page does look a lot like what I concocted this morning - see earlier in the thread or here is the link again: https://docs.google.com/document/d/1dASGp5U0jRNZ3Kzb-LtzeOtUACqlLVh9GZdiKTftTOQ/edit My major concern with that page, now that I know it's there, is that there is no explanation of the APIs that are more suitable for data consumption than the main API (XAPI, Overpass). If I get some time I will add some info. But I think, for the main web site, we want something that is curated a little more tightly than a wiki page. Martijn On Tue, Oct 16, 2012 at 3:27 PM, Martin Koppenhoefer dieterdre...@gmail.com wrote: 2012/10/16 Tom Hughes t...@compton.nu: The problem I see is, other than pointing to planet.openstreetmap.org I'm not sure where else we would send them? You could point them here: http://wiki.openstreetmap.org/wiki/Downloading_data I think what is really missing are some short explanations, e.g. that the export tab is using ressources on the server and therefore only small areas can be downloaded and that there are alternative downloading options (and then point them to the wiki page above). It is all stuff that might seem obvious to you and to most people involved in OSM and maybe also for CS-people not so involved in general, but it isn't for someone occasionally looking at our page and clicking on the export tab. cheers, Martin ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- martijn van exel http://oegeo.wordpress.com ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Why are so many changeset so large?
I really like how activity streams shows easy-to-understand changes on the map using changemonger [1,2]. At the same time it creates an alternative break down of changes that is more granular than changesets. This diverts attention from _comments on changesets_. This is not ideal in my mind - these comments on changesets have great potential to become an even more important communication channel in the future. I understand activity streams / changemonger suggests a broken up view of data changes because many changesets are so large that they are effectively not meaningful. I'd like to understand better why these changesets are so large. Unscientifically digging back on the history of today, I'm seeing many many changesets that seem like they could be just as well much smaller - both in the sense of geographic extent and number of elements - I don't want to call anybody out here, but this is what I found: - http://www.openstreetmap.org/browse/changeset/13514072 - http://www.openstreetmap.org/browse/changeset/13523015 - http://www.openstreetmap.org/browse/changeset/13508818 I understand that there will always be cases where a large changeset makes sense (e. g. bot changes), but it seems that we have many unnecessarily large changesets that make changesets a not very useful granularity for looking at data history. My questions - What are the recommendations for change set sizes? - Are there technical reasons why changesets should tend to be large? Are they expensive on some level? - Could editors encourage users to do more and smaller changesets? - What else could be done to encourage smaller changesets with meaningful comments? [1] http://lists.openstreetmap.org/pipermail/rails-dev/2012-October/001086.html [2] Click on 'activity' here http://suncobalt.dyndns.org:8081/?lat=51.61lon=22.44zoom=7layers=M Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Why are so many changeset so large?
On 17/10/12 00:04, Alex Barth wrote: - What are the recommendations for change set sizes? Personally I tend to put everything that is logically grouped together in one changeset where possible. But by that I mean that I'll spend a few hours out collecting data in a small area and then probably upload that in one changeset - sometimes more than one if I take a break while editing and it times out. - Are there technical reasons why changesets should tend to be large? Are they expensive on some level? I believe it's entirely because we've got so many people doing mechanical or semi-mechanical edits. That includes bots but also things like people using xapi or overpass to download all objects matching some set of tags, then change those tags and reupload. - Could editors encourage users to do more and smaller changesets? - What else could be done to encourage smaller changesets with meaningful comments? Encouraging people to go out and do actual local survey based mapping instead of trying to enforce their tagging ideas on the whole world with mass edits. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Why are so many changeset so large?
Hi Alex, What do you mean by large? Do you mean changesets that span a large area (spanning whole continents)? Or changesets that have a lot of objects modified (perhaps more than 1000)? Based on the examples you provided, it seems you mean the former. Is this correct? Eugene On Wed, Oct 17, 2012 at 7:04 AM, Alex Barth a...@mapbox.com wrote: I really like how activity streams shows easy-to-understand changes on the map using changemonger [1,2]. At the same time it creates an alternative break down of changes that is more granular than changesets. This diverts attention from _comments on changesets_. This is not ideal in my mind - these comments on changesets have great potential to become an even more important communication channel in the future. I understand activity streams / changemonger suggests a broken up view of data changes because many changesets are so large that they are effectively not meaningful. I'd like to understand better why these changesets are so large. Unscientifically digging back on the history of today, I'm seeing many many changesets that seem like they could be just as well much smaller - both in the sense of geographic extent and number of elements - I don't want to call anybody out here, but this is what I found: - http://www.openstreetmap.org/browse/changeset/13514072 - http://www.openstreetmap.org/browse/changeset/13523015 - http://www.openstreetmap.org/browse/changeset/13508818 I understand that there will always be cases where a large changeset makes sense (e. g. bot changes), but it seems that we have many unnecessarily large changesets that make changesets a not very useful granularity for looking at data history. My questions - What are the recommendations for change set sizes? - Are there technical reasons why changesets should tend to be large? Are they expensive on some level? - Could editors encourage users to do more and smaller changesets? - What else could be done to encourage smaller changesets with meaningful comments? [1] http://lists.openstreetmap.org/pipermail/rails-dev/2012-October/001086.html [2] Click on 'activity' here http://suncobalt.dyndns.org:8081/?lat=51.61lon=22.44zoom=7layers=M Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Blank Tiles at z23 and Deeper
On Wed, Oct 10, 2012 at 4:15 PM, Jon Burgess jburgess...@gmail.com wrote: On Wed, 2012-10-10 at 11:08 -0400, jpk wrote: The hash directory thing is good, and I can get that sorted (yay, progress!), but it doesn't explain all the sea-colored tiles at =z23 for tiles that aren't yet cached. From what I gather looking at mod_tile/renderd source, I see mapnik render the tile first, based on the z/x/y coordinates, and then the hashing and caching stuff happens after. Is that right? If so, it would seem that the problem lies somewhere in there. Any pointers as I start looking in that direction? At zoom 23 the world dimensions are 2^23 * 256 pixels, or 2^31. Any calculation done on these pixel values with signed 32 bit arithmetic has a risk of overflowing or changing sign by accident. I cannot tell you where this limitation occurs. If you really want to fix it then you could use a debugger to see where things go wrong. Or you could try compiling mod_tile and Mapnik with the GCC the -ftrapv option which should trap any signed integer overflow. Jon Ah, the number 23 seemed odd to me until now, but this makes sense. Relatedly, I've been playing with Mapnik and so far haven't run into problems getting toy stuff to render down to z30, so I suspect the interesting stuff related to my blank tiles is happening in mod_tile/renderd. Anyway, I plan to dig into that next chance I get. If I get something working and want to share patches, should I post them here on dev@ or is there a better place? -jpk -- john p. kiffmeyer ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Why are so many changeset so large?
Eugene - right, I mean changesets that are geographically large. On Oct 16, 2012, at 8:03 PM, Eugene Alvin Villar sea...@gmail.com wrote: Hi Alex, What do you mean by large? Do you mean changesets that span a large area (spanning whole continents)? Or changesets that have a lot of objects modified (perhaps more than 1000)? Based on the examples you provided, it seems you mean the former. Is this correct? Eugene On Wed, Oct 17, 2012 at 7:04 AM, Alex Barth a...@mapbox.com wrote: I really like how activity streams shows easy-to-understand changes on the map using changemonger [1,2]. At the same time it creates an alternative break down of changes that is more granular than changesets. This diverts attention from _comments on changesets_. This is not ideal in my mind - these comments on changesets have great potential to become an even more important communication channel in the future. I understand activity streams / changemonger suggests a broken up view of data changes because many changesets are so large that they are effectively not meaningful. I'd like to understand better why these changesets are so large. Unscientifically digging back on the history of today, I'm seeing many many changesets that seem like they could be just as well much smaller - both in the sense of geographic extent and number of elements - I don't want to call anybody out here, but this is what I found: - http://www.openstreetmap.org/browse/changeset/13514072 - http://www.openstreetmap.org/browse/changeset/13523015 - http://www.openstreetmap.org/browse/changeset/13508818 I understand that there will always be cases where a large changeset makes sense (e. g. bot changes), but it seems that we have many unnecessarily large changesets that make changesets a not very useful granularity for looking at data history. My questions - What are the recommendations for change set sizes? - Are there technical reasons why changesets should tend to be large? Are they expensive on some level? - Could editors encourage users to do more and smaller changesets? - What else could be done to encourage smaller changesets with meaningful comments? [1] http://lists.openstreetmap.org/pipermail/rails-dev/2012-October/001086.html [2] Click on 'activity' here http://suncobalt.dyndns.org:8081/?lat=51.61lon=22.44zoom=7layers=M Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Notes - OSM improvements BoF at SOTM PDX
This would mean folding http://planet.osm.org/ into a future data tab, no? How do the planet osm maintainers see this? http://planet.openstreetmap.org is the interface from the main DB to all kind of data consuming services, including the renderers, Nominatim, Overpass API and several others. It has roughly the same importance than the main API and it is the place where minute and other replication streams are distributed in an efficient way along with the Planet.osm. As this is the data hub with appropriate user interface for the technical folks, I suggest rather not to bend it into something of minor importance like an Export tab on a website, even if that one easily visible to newbies. On the other hand, for somebody who is not yet familiar with the OSM data format (XML, PBF or both), the planet.osm.org is unlikely to be useful, so both are services with quite distinct audiences. Cheers, Roland ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Why are so many changeset so large?
On 10/17/2012 01:04 AM, Alex Barth wrote: I really like how activity streams shows easy-to-understand changes on the map using changemonger [1,2]. At the same time it creates an alternative break down of changes that is more granular than changesets. This diverts attention from _comments on changesets_. This is not ideal in my mind - these comments on changesets have great potential to become an even more important communication channel in the future. I agree. I will add changeset comments to changeset descriptions on the demo instance and let's see how this turns out. One challenge I see with that is the fact that some (most?) people don't add relevant information to their changesets. But perhaps seeing their changesets as activities would change that behavior and they would use changeset comments as a communication channel, not as a required field in an editor. I understand activity streams / changemonger suggests a broken up view of data changes because many changesets are so large that they are effectively not meaningful. I'd like to understand better why these changesets are so large. One thing that became immediately apparent once I managed to get the whole thing up and running is the fact that changesets really do come in all shapes and sizes. Is that a problem? I thought about it and my conclusion is that it's just another thing that the social/activity stream view could help with. While I agree with Tom's comment about encouraging people to go out and survey instead of writing edit bots, I think we should accept and embrace all changes when thinking about improvements to the site. Specifically, I thought about adding things like: 1. Changeset size (number of changes) indicator on a single activity view. 2. Changeset size (in terms of bounding box) indicator on a single activity view. 3. Simple filtering features for (1) and (2). Right now the Activity Server holds multipoint geometry for every changeset so it's possible to implement filtering like that (as opposed to considering only the bounding box of a changeset) Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Why are so many changeset so large?
On 10/17/2012 01:04 AM, Alex Barth wrote: - http://www.openstreetmap.org/browse/changeset/13514072 wheelmap_visitor is sort of a bot, it uploads changes made to the wheelchair=* accessibility tags by anonymous users on http://wheelmap.org/ It only touches that one tag. It generates a new change set every few hours. There is no clear pattern so i assume that it uploads each individual change when it happens and a new changeset is started whenever the previous one timed out. Putting each single change to a wheelmap=* tag into a changeset of its own doesn't seem to make much sense here. See also http://www.openstreetmap.org/user/wheelmap_visitor - http://www.openstreetmap.org/browse/changeset/13523015 Seems to be related to http://lima.schaaltreinen.nl/remap/ that checks for ways with implausible angles between segments These are probably manual edits based on the suggestions from that site, and span a large area as these suggestions were not ordered by region - http://www.openstreetmap.org/browse/changeset/13508818 This one only covers a few objects (three nodes, one way, three areas), all of them uranium mines or related to those. There are not that many uranium mines on the planet so anything touching more than one of them is going to produce a large changeset area. Putting each name change in a changeset of its own wouldn't have made much sense in this case though IMHO -- hartmut ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
