Re: [OSM-dev] OSM Wishlist (community wishlist built experimentation)
On 10/18/2012 05:04 AM, sly (sylvain letuffe) wrote: I do, but as you said, alone isn't gone be an easy task, that's why I'll soon be proposing this on talk, with the hope to get the help of everyone. I have just no ideas if something usefull could get out of this, if that won't just turn to chaos, but I guess I'll have to try to find out. It's great that you are driving this. One potential use of such list would be defining the Top Ten Tasks list. Year is coming to an end, I imagine a revised list is in order soon. User community feedback could be one of the factors when deciding what to put on this list. As a starting point, I have cleaned-up http://wiki.openstreetmap.org/wiki/API_v0.7 into what I consider ideas for development ranging from good to improvable with a good idea behind. I've also turned it into less technical and, hopefully, understandable to long standing non-dev mappers. However, it is not limited anymore to what people think should go into the next API 0.7 version (I think it's even worse when you ask non-techies people to find a solution themself) but to something gathering ideas of wishes they find usefull for their work as mappers, what they miss more while editing (may it be external tools, websites, software or, of course API improvements). I'm really starting to think that this page (API 0.7) should be taken apart... things like Make the web interface accessible have not much to do with the API and are simply getting drowned in all the techy ideas for low-level API improvements. That's why I think an effort like yours - to clean it up and make it human-readable is needed. here it is, self-explained : http://wiki.openstreetmap.org/wiki/Contributors_functionalities_wishlist If anyone here wants to give feedbacks before it goes to talk, please do. The page looks good. In addition I would consider couple of things: * From my experience form of presentation matters when trying to explain complex things to people. Right now there is a lenghty first paragraph, there is no Table of Contents. Small things like that can be easily fixed and they make content more presentable. Of course there is always the question if we really care about people who have so short attention span that they can't get through a large chunk of text but that's another discussion... For now I would suggest splitting the content into sections as much as possible (especially the first paragraphs into something like Introduction or Rationale or What is it?) and putting Table of Contents on the page. * Similar to above point - page title. I would suggest something that rolls off the tongue which Contributors functionalities wishlist does not do exactly :-) Perhaps something like Community wishlist or similar? * I would toy with the idea of changing how specific wishlist items are presented - I think having some structure like a table with description, discussion and status and perhaps use colors to mark what is being worked on etc. This would further improve readability. Ironically, none other than the Top Ten Tasks list has a good example of this, see template: http://wiki.openstreetmap.org/wiki/Template:TopTenTask I would think about making similar template but more community-focused so no technology list but something more useful to non-developers. From my side I would love to see someone from the community get involved in current development. For example, right now we are on the brink of having Gravatar support merged into the osm.org website: https://github.com/openstreetmap/openstreetmap-website/pull/131 There are some very important privacy questions asked in that discussion but I don't think anyone from the end-user community even read those... What would be great if someone would create a blog or a diary dedicated specifically to reporting on OSM development activity - something like OSM This Week list that already exists. On such blog people could get easily involved without having to create a Github account etc. and the blog author would feed back the comments to developers. Keep up the good work - this effort is really useful. Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist (community wishlist built experimentation)
On jeudi 18 octobre 2012, Paweł Paprota wrote: It's great that you are driving this. One potential use of such list would be defining the Top Ten Tasks list. A list with that name allready exists, therefore I don't want to say that this list would define the TTT However, any admin is of course welcome to feed a part of the TTT with idea from that new list. Hi Pawel, * From my experience form of presentation matters when trying to explain complex things to people. Right now there is a lenghty first paragraph, there is no Table of Contents. I do admit the first paragraph is huge, but it serves the purpose of scaring away un-serious people. And I find the wiki Table of content format problematic in such case, see : http://wiki.openstreetmap.org/wiki/API_v0.7 You first read At this point, we're brainstorming new/changing features for API version 0.7. and then you jump to real ideas, forgeting to read the Brainstorming importante part. But why not trying after all, that shouldn't be a big deal. Of course there is always the question if we really care about people who have so short attention span that they can't get through a large chunk of text but that's another discussion... That the people I want to contribute ;-) * Similar to above point - page title. I would suggest something that rolls off the tongue Does a page named TTT has better audience than one named CFW ? I don't know, and don't really care as long as the title express what it is, but if someone finds a title fullfeeling both objectiv, I'll be glad to change ! Perhaps something like Community wishlist or similar? I've been thinking about that, and, well, why not. The 1st sentence should clear the fact it is not about distributing T-Shirts to mappers Ironically, none other than the Top Ten Tasks list has a good example of this, see template: http://wiki.openstreetmap.org/wiki/Template:TopTenTask Complex..., I don't want to attract wiki experts but contributors. I would think about making similar template but more community-focused so no technology list but something more useful to non-developers. Unless if you come with something nice ? (I'm a wiki noob and hate spending time to understand complex templates items) -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist (community wishlist built experimentation)
On 10/17/2012 09:04 PM, sly (sylvain letuffe) wrote: However, it is not limited anymore to what people think should go into the next API 0.7 version (I think it's even worse when you ask non-techies people to find a solution themself) but to something gathering ideas of wishes they find usefull for their work as mappers, what they miss more while editing (may it be external tools, websites, software or, of course API improvements). here it is, self-explained : http://wiki.openstreetmap.org/wiki/Contributors_functionalities_wishlist If anyone here wants to give feedbacks before it goes to talk, please do. I would suggest putting it on help.openstreetmap.org rather than on the wiki. I know that is a bit of an abuse of help, but the builtin voting system and reordering according to votes can be useful to help filter the list of wishes and suggestions. It should make the whole thing cleaner and more manageable. Kai ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist (community wishlist built experimentation)
On jeudi 18 octobre 2012, Kai Krueger wrote: I would suggest putting it on help.openstreetmap.org rather than on the wiki. For sure, a wiki for voting purpose is to be classified as one of the worst tool to do it, but it was easy to copy the first wishes from the same syntax. Using help even if better suited as a tool, would be a terrible abuse of the help.openstreetmap.org instance. And in the end, we need a bit of freedom to modify things all around without pain about users rights. I'll switch if that goes out of control -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist (community wishlist built experimentation)
On 18 October 2012 16:27, Kai Krueger kakrue...@gmail.com wrote: I would suggest putting it on help.openstreetmap.org rather than on the wiki. I know that is a bit of an abuse of help, but the builtin voting system and reordering according to votes can be useful to help filter the list of wishes and suggestions. It should make the whole thing cleaner and more manageable. I'd suggest not putting it on help, since it's an abuse of the help system. If you really want to have lots of ideas and let them vote on them, then just re-activate the uservoice thing. http://osm.uservoice.com/forums/41047-general It's already full of suggestions. I suspect this new effort will achieve similar results. Cheers, Andy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist (community wishlist built experimentation)
Sweet - I think such a list is great and useful for further informing priorities in OSM. I'd love to see this framed larger than the Knight funded work though. I am really worried about expectations. OpenStreetMap won't cover larger improvement needs with 500+k, OpenStreetMap need a lot more resources :) At the same time I think such a list should be encouraging people and other orgs to get started working on improving OSM. This is why I just took off the reference to MapBox and the Knight Grant from this wishlist. I've tried to tinker with the phrasing, but in the end I feel as long as it says 'mapbox' on there is an expectation that we'll be able to follow through on key things people ask for within the next months. All that said, I am really interested in what people will come up with, I think the list as it stands right now is already very useful, thank you for working on this. On Oct 18, 2012, at 11:27 AM, Kai Krueger kakrue...@gmail.com wrote: On 10/17/2012 09:04 PM, sly (sylvain letuffe) wrote: However, it is not limited anymore to what people think should go into the next API 0.7 version (I think it's even worse when you ask non-techies people to find a solution themself) but to something gathering ideas of wishes they find usefull for their work as mappers, what they miss more while editing (may it be external tools, websites, software or, of course API improvements). here it is, self-explained : http://wiki.openstreetmap.org/wiki/Contributors_functionalities_wishlist If anyone here wants to give feedbacks before it goes to talk, please do. I would suggest putting it on help.openstreetmap.org rather than on the wiki. I know that is a bit of an abuse of help, but the builtin voting system and reordering according to votes can be useful to help filter the list of wishes and suggestions. It should make the whole thing cleaner and more manageable. Kai ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist (community wishlist built experimentation)
Le mardi 16 octobre 2012 14:29:08, Paweł Paprota a écrit : I think this would be A LOT of work and probably not for only one person but I for one would love to see end-users more involved during development phase - testing, feedback, incremental improvements and all that. So the question for me is - who would be interested in doing such work... I do, but as you said, alone isn't gone be an easy task, that's why I'll soon be proposing this on talk, with the hope to get the help of everyone. I have just no ideas if something usefull could get out of this, if that won't just turn to chaos, but I guess I'll have to try to find out. As a starting point, I have cleaned-up http://wiki.openstreetmap.org/wiki/API_v0.7 into what I consider ideas for development ranging from good to improvable with a good idea behind. I've also turned it into less technical and, hopefully, understandable to long standing non-dev mappers. However, it is not limited anymore to what people think should go into the next API 0.7 version (I think it's even worse when you ask non-techies people to find a solution themself) but to something gathering ideas of wishes they find usefull for their work as mappers, what they miss more while editing (may it be external tools, websites, software or, of course API improvements). here it is, self-explained : http://wiki.openstreetmap.org/wiki/Contributors_functionalities_wishlist If anyone here wants to give feedbacks before it goes to talk, please do. -- sly (sylvain letuffe) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hi Am 12.10.2012 16:14, schrieb Serge Wroclawski: I made a second issue at the same time regarding gathering histrorical data around way geometry. Right now, due to the way OSM stores its geometry data, the only way to know about a way's geometry history is to call the history call for the way, then make a history call for each and every node that was ever in the way. That's something I ussed about 4 years ago when I stared working with the history and someone pointed me to an important fact: The API is to support mappers. Using it for small tools/experiments is ok, as long as they dont't put noticable load on the api. For everything else, there are planet/history dumps. With today's tools it's quite simple to extract your area of interest from a history dump and work on that. In fact it's much much easier to use those dumps to fetch data then to crawl the api, once ypou get the algorithms in place. Talking with Matt about this a month ago on IRC, he suggested a new call could be made, one that would do the above work in a single call. This would allow the same work to be done with less server overhead, and greater resource management. Tom closed it, saying he didn't like expensive calls. I don't like expensive calls *on the main api* either. But why not build a history api? It could feed itsself from the history dumps and kept up to date from the replication diffs. It's all there, just assemble it. Peter ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Tue, Oct 16, 2012, at 01:03, sly (sylvain letuffe) wrote: If you propose to change it by creating a community-driven (instead of admin-driven as you put it) wishlist, by any means - do it. The operative word being do. I'll be glad to do so, and will start it. Unless the MapBox team (and tom) doesn't want to look at such a process (in the end gathering wish never read is just going to spend my time) In my opinion this is a broader topic than the Mapbox wishlist. I think some kind of user-voice coordinator or driver or however it would be (informally, of course) called would be extremely useful. Such person would need to keep up with the current development activity (Github, Trac, dev@, rails-dev@ and others) and represent user interests. I think this would be A LOT of work and probably not for only one person but I for one would love to see end-users more involved during development phase - testing, feedback, incremental improvements and all that. So the question for me is - who would be interested in doing such work... Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On vendredi 12 octobre 2012, Tom MacWright wrote: Hey all (or, well, those subscribed to dev@ - my flamewar shields are at 50% so I'm not risking an email to talk), This only is my opinion, but wishlists shouldn't be reduced to those subscribed to dev@ Maybe start on dev, but I think a wiki page should better be suited for a summary of the ~3/5 tasks to focus on. What do you want to see happen with OSM's software this year? So many things on my side that I don't see how we can decide this on a mailing list in one thread. Better list, then short list, then vote (or kind of) and focus on few tasks. -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hey Sly, This only is my opinion, but wishlists shouldn't be reduced to those subscribed to dev@ Maybe start on dev, but I think a wiki page should better be suited for a summary of the ~3/5 tasks to focus on. Yes and no. Everything is a trade between being totally open (announce it on IRC and see what the crowd thinks!) and being totally productive (hole up and just do it until you're done and then realize it's duplicated effort). For the 'we have tasks' stage, I'd like to handle this in GitHub issues, because they are focused on doing things. There are existing wiki pages for improvements (top ten tasks, api 0.7, improving openstreetmap) which have shown at best mixed success of staying updated and being good for the 'actual doing things' collaboration phase. Tom On Mon, Oct 15, 2012 at 6:08 AM, sly (sylvain letuffe) li...@letuffe.orgwrote: On vendredi 12 octobre 2012, Tom MacWright wrote: Hey all (or, well, those subscribed to dev@ - my flamewar shields are at 50% so I'm not risking an email to talk), This only is my opinion, but wishlists shouldn't be reduced to those subscribed to dev@ Maybe start on dev, but I think a wiki page should better be suited for a summary of the ~3/5 tasks to focus on. What do you want to see happen with OSM's software this year? So many things on my side that I don't see how we can decide this on a mailing list in one thread. Better list, then short list, then vote (or kind of) and focus on few tasks. -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On lundi 15 octobre 2012, Tom MacWright wrote: Hey Sly, Hi tom, Yes and no. Everything is a trade between being totally open (announce it on IRC and see what the crowd thinks!) and being totally productive (hole up and just do it until you're done and then realize it's duplicated effort). That's why I'm proposing to build a short task list of wishlist first (between those who don't want to offer free ponies [1]) and then propose it to wider audience. For the 'we have tasks' stage, I'd like to handle this in GitHub issues, My opinion is that the wiki would be the good place for the we decide wich tasks, and after that, okay, you'r the one doing the job, so your choice. There are existing wiki pages for improvements (top ten tasks, api 0.7, improving openstreetmap) which have shown at best mixed success of staying updated and being good for the 'actual doing things' collaboration phase. Is the wiki tool in cause ? or the rules for building those pages ? top ten tasks is These are the Top Ten Tasks that the OSM System Administrators What about the community ? This only is a todo list by the admins, for the admins coded by the admins. So far so good, but that's not a wishlist, or, at least, not a wishlist of the community. [1] api 0.7 page is a brainstorming page open to every one (no moderators, no short list, no voting sytem) and, ultimetly, no one to ever do what's writen here because this is not building tasks lists, it's gathering ideas. improving openstreetmap is a page I've heard for the first time, so I guess it is not geared toward the community whishlist If the thread you've launched here named OSM Wishlist implies that OSM is it's community, then my suggestion whould be to create a wiki page explaining it's goal, asking on the dev list to build a list of say 15 to 20 non-utopic/troll/ponies tasks then ask the community to choose 5 out of them. -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
top ten tasks is These are the Top Ten Tasks that the OSM System Administrators What about the community ? This only is a todo list by the admins, for the admins coded by the admins. So far so good, but that's not a wishlist, or, at least, not a wishlist of the community. Disclaimer: I have just started working on OSM a few weeks ago so I may be wrong with my impression about how things work. Generally I would say that OSM works like a typical open source project - people who do the actual programming work choose what they want to work on. That's OK since this characteristic is the main attraction to open source for programmers around the world - they can work on what they like, instead of working on what their boss or a customer order them to work on. I think every open source project, including the big ones has some challenges with user voice being heard or at least that's the impression. If you propose to change it by creating a community-driven (instead of admin-driven as you put it) wishlist, by any means - do it. The operative word being do. Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Mon, 2012-10-15 at 18:13 +0200, Paweł Paprota wrote: top ten tasks is These are the Top Ten Tasks that the OSM System Administrators What about the community ? This only is a todo list by the admins, for the admins coded by the admins. So far so good, but that's not a wishlist, or, at least, not a wishlist of the community. the page says: These are the Top Ten Tasks that the OSM System Administrators would really like your development help on. so i think it's unfair to say it's a list by the admins, for the admins. the important part of the sentence is ... would really like your development help on. we (EWG) formulated this list as a curated selection of tasks that we thought were generally important so that people who were looking for something to do would have some idea of what the most important[1] items were, and where their efforts would be most appreciated. Disclaimer: I have just started working on OSM a few weeks ago so I may be wrong with my impression about how things work. Generally I would say that OSM works like a typical open source project - people who do the actual programming work choose what they want to work on. That's OK since this characteristic is the main attraction to open source for programmers around the world - they can work on what they like, instead of working on what their boss or a customer order them to work on. exactly - there are no restrictions on what you should work on. the data is open, the software is open, and you can work on whatever is interesting to you. and if, surrounded by this huge number of different things to work on, you want to work on something that some people who are knowledgeable about OSM think is important enough to have short-listed; that's the Top Ten Tasks[1]. I think every open source project, including the big ones has some challenges with user voice being heard or at least that's the impression. If you propose to change it by creating a community-driven (instead of admin-driven as you put it) wishlist, by any means - do it. The operative word being do. one problem (which probably started this thread) is that a wishlist is potentially infinite. and, even treating it as an ideas-gathering forum, the value becomes diluted with the quantity of ideas presented. the hardest part of making the process useful is curating the list of wishes and doing the work of turning it into a list of achievable and practical items. and, as you rightly point out, the operative word is do. cheers, matt [1] this is not to say that other things aren't important. when the Top Ten Tasks were written, our crystal ball was out of order so we sadly couldn't predict the future. things change, and ten is much too small a number to get everything that's important onto the list, so the TTTs are a just a suggestion - they're not gospel. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
I'd love to get back to the topic of what's next. At least my at-the-moment thought for why dev@ has been more efficient than the mailing list is something pretty simple: when people post on here with some bit of knowledge - like saying that something is already implemented, or that there are potential performance problems, it's easy to know who said it, and to ask for further information. This has been vital so far, since this is a serious learning process as far as prior art. If we sign every post on the Wiki (wiki-discussion style with ~~~), that might constitute some improvement, but there's still no easy way of messaging as well as writing there. On Mon, Oct 15, 2012 at 9:37 AM, Matt Amos zerebub...@gmail.com wrote: On Mon, 2012-10-15 at 18:13 +0200, Paweł Paprota wrote: top ten tasks is These are the Top Ten Tasks that the OSM System Administrators What about the community ? This only is a todo list by the admins, for the admins coded by the admins. So far so good, but that's not a wishlist, or, at least, not a wishlist of the community. the page says: These are the Top Ten Tasks that the OSM System Administrators would really like your development help on. so i think it's unfair to say it's a list by the admins, for the admins. the important part of the sentence is ... would really like your development help on. we (EWG) formulated this list as a curated selection of tasks that we thought were generally important so that people who were looking for something to do would have some idea of what the most important[1] items were, and where their efforts would be most appreciated. Disclaimer: I have just started working on OSM a few weeks ago so I may be wrong with my impression about how things work. Generally I would say that OSM works like a typical open source project - people who do the actual programming work choose what they want to work on. That's OK since this characteristic is the main attraction to open source for programmers around the world - they can work on what they like, instead of working on what their boss or a customer order them to work on. exactly - there are no restrictions on what you should work on. the data is open, the software is open, and you can work on whatever is interesting to you. and if, surrounded by this huge number of different things to work on, you want to work on something that some people who are knowledgeable about OSM think is important enough to have short-listed; that's the Top Ten Tasks[1]. I think every open source project, including the big ones has some challenges with user voice being heard or at least that's the impression. If you propose to change it by creating a community-driven (instead of admin-driven as you put it) wishlist, by any means - do it. The operative word being do. one problem (which probably started this thread) is that a wishlist is potentially infinite. and, even treating it as an ideas-gathering forum, the value becomes diluted with the quantity of ideas presented. the hardest part of making the process useful is curating the list of wishes and doing the work of turning it into a list of achievable and practical items. and, as you rightly point out, the operative word is do. cheers, matt [1] this is not to say that other things aren't important. when the Top Ten Tasks were written, our crystal ball was out of order so we sadly couldn't predict the future. things change, and ten is much too small a number to get everything that's important onto the list, so the TTTs are a just a suggestion - they're not gospel. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Le lundi 15 octobre 2012 18:13:34, Paweł Paprota a écrit : - people who do the actual programming work choose what they want to work on. I think this is a little different here (this thread) than usually is (even with what usually happen with OSM core development) Tom started this thread with : So, along with the big 'kicking off' blog post on MapBox[1] (...) What are the tasks which everyone agrees on, but nobody has had the time to tackle? [1]: http://mapbox.com/blog/kicking-off-knight-work/ By everyone I assume he meant everyone on the dev list (or everyone of the osm community) which is another way no to say What task do I want to work on And reading the linked blog about the big 'kicking off' what I understand is that he is not saying the MapBox team is going to work on what they want but Build in the open : to allow anybody in the OpenStreetMap community to engage and to facilitate a transparent process In the end, what I understand of his OSM Wishlist thread, is that he wants to start work at some intentionally broad, but it aims to: 1. Improve editing of OpenStreetMap data 2. Make the OpenStreetMap experience more social 3. Make it easier to get data out of OpenStreetMap And since this is really broad, he is willing to gather tasks, agreed by the OSM community. they can work on what they like, instead of working on what their boss or a customer order them to work on. Maybe we are in between on this thread ;-) If you propose to change it by creating a community-driven (instead of admin-driven as you put it) wishlist, by any means - do it. The operative word being do. I'll be glad to do so, and will start it. Unless the MapBox team (and tom) doesn't want to look at such a process (in the end gathering wish never read is just going to spend my time) -- sly (sylvain letuffe) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hey Sly, The simple answer is that MapBox (including myself) is finding ways to improve OpenStreetMap. Some of these are pretty obvious (design!), and some of them require a very high level of knowledge (API changes). By posting here, I'm trying to get a good idea of what exists, what experts think about problems and priorities, and what other issues need addressing. What this is, is neither voting on community suggestions nor is it rule from your friendly overlord. Life is full of middles. That said, let's talk less about talking and your personal suspicions and more about actual substance; aka un-derail this thread. Tom On Mon, Oct 15, 2012 at 4:03 PM, sly (sylvain letuffe) li...@letuffe.orgwrote: Le lundi 15 octobre 2012 18:13:34, Paweł Paprota a écrit : - people who do the actual programming work choose what they want to work on. I think this is a little different here (this thread) than usually is (even with what usually happen with OSM core development) Tom started this thread with : So, along with the big 'kicking off' blog post on MapBox[1] (...) What are the tasks which everyone agrees on, but nobody has had the time to tackle? [1]: http://mapbox.com/blog/kicking-off-knight-work/ By everyone I assume he meant everyone on the dev list (or everyone of the osm community) which is another way no to say What task do I want to work on And reading the linked blog about the big 'kicking off' what I understand is that he is not saying the MapBox team is going to work on what they want but Build in the open : to allow anybody in the OpenStreetMap community to engage and to facilitate a transparent process In the end, what I understand of his OSM Wishlist thread, is that he wants to start work at some intentionally broad, but it aims to: 1. Improve editing of OpenStreetMap data 2. Make the OpenStreetMap experience more social 3. Make it easier to get data out of OpenStreetMap And since this is really broad, he is willing to gather tasks, agreed by the OSM community. they can work on what they like, instead of working on what their boss or a customer order them to work on. Maybe we are in between on this thread ;-) If you propose to change it by creating a community-driven (instead of admin-driven as you put it) wishlist, by any means - do it. The operative word being do. I'll be glad to do so, and will start it. Unless the MapBox team (and tom) doesn't want to look at such a process (in the end gathering wish never read is just going to spend my time) -- sly (sylvain letuffe) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
the page says: These are the Top Ten Tasks that the OSM System Administrators would really like your development help on. so i think it's unfair to say it's a list by the admins, for the admins. I admit I was unfair. I shortcuted it too much. But still, that's a list by the admins. But I haven't problems with that. What I'm asking tom is : do you wish to code at admins or community requests ? I did heard his flameware shields are not operational enough for a mail on talk but it might be interpreted as he doesn't want to spent too much time reading people's requests about free ponies or chinese tags, but it could also be interpreted as he wants OSM admins to assign MapBox team tasks. Or in-between : Anyone of the osm community who knows a bit what he is talking about and be able to express reasonable requests about what he needs in his day to day mapping work exactly - there are no restrictions on what you should work on. the data is open, the software is open, and you can work on whatever is interesting to you. I know that. The question is more : On what should the MapBox team work on, and who should decide what is this what -- sly (sylvain letuffe) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hey sly - I'm very interested in people weighing in on a wishlist thread like that because it's super useful for understanding the problem space around improving OSM in the areas editors, social experience, data export better. In the end of the day, just like you or anyone else Tom and I and others in our team will have to decide ourselves what we'd like to work on, and we would be dumb if we worked on something that noone wants to see happening. So this wishlist also serves as a way to gauge to see where there's momentum. That said, what improvements do you think should we be pushing on? On Oct 15, 2012, at 7:18 PM, sly (sylvain letuffe) li...@letuffe.org wrote: the page says: These are the Top Ten Tasks that the OSM System Administrators would really like your development help on. so i think it's unfair to say it's a list by the admins, for the admins. I admit I was unfair. I shortcuted it too much. But still, that's a list by the admins. But I haven't problems with that. What I'm asking tom is : do you wish to code at admins or community requests ? I did heard his flameware shields are not operational enough for a mail on talk but it might be interpreted as he doesn't want to spent too much time reading people's requests about free ponies or chinese tags, but it could also be interpreted as he wants OSM admins to assign MapBox team tasks. Or in-between : Anyone of the osm community who knows a bit what he is talking about and be able to express reasonable requests about what he needs in his day to day mapping work exactly - there are no restrictions on what you should work on. the data is open, the software is open, and you can work on whatever is interesting to you. I know that. The question is more : On what should the MapBox team work on, and who should decide what is this what -- sly (sylvain letuffe) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hi, That said, let's talk less about talking and your personal suspicions and more about actual substance; Is that substance : http://wiki.openstreetmap.org/wiki/API_v0.7 ? aka un-derail this thread. Got it, sorry. -- sly (sylvain letuffe) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Oct 15, 2012, at 7:40 PM, sly (sylvain letuffe) li...@letuffe.org wrote: Hi, That said, let's talk less about talking and your personal suspicions and more about actual substance; Is that substance : http://wiki.openstreetmap.org/wiki/API_v0.7 I'd say it is! ? aka un-derail this thread. Got it, sorry. -- sly (sylvain letuffe) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 16/10/12 00:40, sly (sylvain letuffe) wrote: That said, let's talk less about talking and your personal suspicions and more about actual substance; Is that substance : http://wiki.openstreetmap.org/wiki/API_v0.7 ? Well the problem with that page is that there are a huge number of things there and they range widely from completely sensible via misguided to barking mad. It would be very easy for somebody taking that page as a source of ideas to go off on a complete wild goose chase... Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 12-Oct-12, at 3:50 PM, Iván Sánchez Ortega wrote: Also. ogr3osm. I would love to have the time and resources (or paid time, nudgenudgewinkwink) to redo ogr2osm; adding a backtracking-like algorithm to minimise the amount of geometries' shared nodes (and their bounding boxes) in memory, in order to be able to convert datasets with gazillions of geometries into .osm format. Backtracking-like in the sense that the data processing would be done in a tree-like fashion, walking through overlapping geometries, processing only geometries which have all their nodes already into the list of generated node IDs, writing to file and destroying from memory nodes that won't appear again because all the overlapping geometries have been processed. Yes, it sounds like a mouthful. I have a bunch of napkin notes with the algorithm written down, though :-) I hate to steal your consulting work but I already redid the node merging in ogr2osm :) I have it check if there is an existing node on its list before creating one - this turns out to be way quicker than checking after the fact for nodes in common and removing one of them. I haven't profiled it in awhile but the slowest step is now writing the XML out with SimpleXMLWriter. I intend to evaluate switching to a different library to get more performance. It didn't appear to be disk bound, but spending all of its time in SimpleXMLWriter. I'm giving a talk tomorrow on ogr2osm and might expand what I'm saying about the node merging. I ran some statistics on my in-progress NHD translation. This is a fairly complex translation with involved logic, but it does drop a few smaller layers. For a 600 MB .mdb (400 MB .shp) resultng in a 540 MB .osm it takes 12 minutes on my home server. It's CPU bound and single-threaded. I think it uses about 6-7 gigs of ram for that. I may be able to get that down substantially, I haven't really attacked the RAM usage yet. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Sunday, 14 de October de 2012 10:05:31 Paul Norman escribió: I'm giving a talk tomorrow on ogr2osm [...] For a 600 MB .mdb (400 MB .shp) [...] it uses about 6-7 gigs of ram for that. I may be able to get that down substantially, I haven't really attacked the RAM usage yet. Well, that's the problem I want to tackle down (and it's possible to tackle it down). You just need some mechanism to free nodes from memory when you're sure all the overlapping geometries have been proccessed. Please do let me know how the ogr2osm talk goes. Best, -- -- Iván Sánchez Ortega i...@sanchezortega.es i...@geonerd.org Aviso: Este e-mail es confidencial y no debería ser usado por nadie que no sea el destinatario original. No se permite la reproducción mediante fotocopia, walkie-talkie, emisora de radioaficionado, satélite, televisión por cable, proyector, señales de humo, código morse, braille, lenguaje de signos, taquigrafía o cualquier otro medio. Bajo ningún concepto debe traducirse al francés este e-mail. Este e-mail no puede ser ridiculizado, parodiado, juzgado en una competición, o leído en voz alta con un acento gracioso llevando un bigote falso y/o cualquier tipo de sombrero, incluyendo pero no limitándose a pañuelos. No inciten ni provoquen a este e-mail. Si está medicándose, puede experimentar nauseas, desorientación, histeria, vómitos, pérdida temporal de la memoria a corto plazo y malestar general al leer este e-mail. Consulte a su médico o farmacéutico antes de leer este e-mail. Todas las modelos descritas en este e-mail son mayores de 18 años. Este e-mail se reserva el derecho de admisión. Si ha recibido este e-mail por error es probablemente porque estaba borracho cuando escribí la dirección del destinatario. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 14-Oct-12, at 5:31 AM, Iván Sánchez Ortega wrote: On Sunday, 14 de October de 2012 10:05:31 Paul Norman escribió: I'm giving a talk tomorrow on ogr2osm [...] For a 600 MB .mdb (400 MB .shp) [...] it uses about 6-7 gigs of ram for that. I may be able to get that down substantially, I haven't really attacked the RAM usage yet. Well, that's the problem I want to tackle down (and it's possible to tackle it down). You just need some mechanism to free nodes from memory when you're sure all the overlapping geometries have been proccessed. Please do let me know how the ogr2osm talk goes. My target is to be able to create 2G .osm files without hitting swap on my home server (16G ram). Past that you run into problems with handling the output file even if ogr2osm can handle . I suspect there's a lot of low-hanging fruit for the memory usage. Right now I don't believe *any* memory is freed up so it's keeping around everything even when it's done with it. By default ogr2osm creates an in-memory copy of the data source to allow it to be changed (necessary for some translations) and using the command-line option to disable that would reduce the memory usage. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Am 12.10.2012 01:58, schrieb Tom MacWright: What are the tasks which everyone agrees on, but nobody has had the time to tackle? there is still a lot of information missing to compete with commercial data for (car) navigation system (e.g. from Navteq and Tele-Atlas): * have house numbers/addresses everywhere * have turn restrictions everywhere * have lane guiding informations everywhere * have maxspeed information everywhere * for 3d navigation, buildings being present everywhere would be nice Support for mapping these informations is appreciated but not the most important issue in my point of view. From my point of view e.g. in Germany we are moving in a lot of areas already from mapping the information to keep the information up to date. I would rate that we don't have methods neither tools to check the data. So this is the most open issue from my point of view. I'm aware of tools like OSM-Inspector/keeprigt and osmbugs, but that's not the same as monitor and update the data. An example for this: in Germany the drugstore brand Schlecker went into bankruptcy. While checking/removing all Schlecker shops in my town I notived that 2 out of 5 shops had been closed long time ago but still being present in the OSM data/map. The problem: if an area is considered completely mapped, nobody is going out to check the data for changes... Best regards, Michael. PS: there has been a discussion about adding some kind of last checked-tag to each object. But to me and a lot of other persons this doesn't seem to be an appropriate method. Maybe this could be stored in a seperate data base linked to the main db (API 0.7?). ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Imagine there is list of schools in a city. This list is maintained by a school district office with updated website, phone numbers, and office opening times. They maintain this list through a simple PHP form on their website that allows them to make updates to each listing (node). If any node on their list is changed by anyone but them or the people listed as able to edit that form on their own website (with their OSM Ids) then they are notified and can correct or confirm acceptance of the change. They do not need to be the last editors of the node to know that the information contained therein is approved by them. Nobody can block somebody else from editing something. If you mean approve in this direction, it would pretty never happen. However, the school district can fork the Planet into an own database and move newer edits as act of approvement to its database. This is a typical use case of the idea of federated databases. No work has been done so far, but the point * Tools to move, merge, and diff objects between different OSM databases is surely a good point for the tools list. The federated databases will also solve most issues around imports, including patching an updated import without damaging exisiting data. The second approach is to get a concise report of all updates that happened. It is then up to the stuff of the school district to rollback or accept the edits. Concering tools to watch updates, a lot of things have happened very recently. However, none of these tools work perfectly for the school district case yet, but this will significatly improve in the next weeks. Roland ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hi, On 10/12/2012 02:14 AM, Andy Allan wrote: As for all-new things, I'd like to explore how to integrate all the various QA feeds into a combined overview, rather than having to hunt around different sites. That would be a great thing to have, and one that has been eluding us for quite a while. I remember SteveC bringing that up years ago - the bug platform to end all bug platforms, if you will. Jochen and I had a little discussion about this with the MapQuest guys when their involvement in OSM started, with the aim of potentially building such a platform for them, but that didn't lead anywhere. As the operator of OSM Inspector, I am occasionally contacted about adding third-party layers to it, and OSMI meanwhile has not only the bugs found by our code but also a routing layer that consists partly of Pascal Neis' and partly of Dennis Luxen's work. I would be totally open towards the idea of merging OSMI stuff into a common platform. I do believe though that we can't take the easy route and build a new centralised QA platform; we'll have to build a meta platform into which everyone can feed their input, so that we can continue to harness the creativity of OSM hackers around the globe who come up with new QA measures all the time (and whom we don't want to run all their queries on our central database, even if it were suitable). Building such a meta platform is not an easy task because bugs are much more than just popup markers on the map - bugs can also be a way, or a collection of ways, or a convex hull around a broken polygon; a good platform will also allow the user to flag those bugs which are false positives. All this in an environment where a multitude of QA providers feed hundreds of thousands of check results into the platform every day. Maybe all that thinking is why I never got around to build something - perhaps it is time to take a step back and build the simplest thing that could possibly work. Then again, if it doesn't offer an advantage over what's already there, it won't reach the goal of concentrating QA measures. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 12.10.2012 01:58, Tom MacWright wrote: What do you want to see happen with OSM's software this year? What are the tasks which everyone agrees on, but nobody has had the time to tackle? There are some ideas around but I'm still missing a tool that can graphically show the diffs on an area/changelist. Let's start with geometry modifications. Typical situation: I look at the map in my area and I could bet there was a road at a specific point. Now I want to know: Was there a road in the past? I want to open my history-browser, select the area of interest and then tick a check-box to view all deleted nodes,roads (within a defined timeframe, maybe a slider). If I identify a changeset which did delete that way I want to know what else that changeset did. So I want to have it display the state of the map before the edit and the geometry changes of that changeset in a different colorset. To make it even more convenient intelligent filters can remove changes from the display where for example a node was deleted but in the same position is an area which has the tags as well. This should filter conversion of POI nodes into POI areas. Deleted roads could be filtered if the road network contains a new road connecting the end-nodes so no gap was created. This is a more complex filter as it could be multiple edits involved. So I want a tool that makes it possible to do a quality control by checking diffs like it's possible in wikipedia for years. The problem is that our data is a lot harder to diff. Stephan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Fri, 12 Oct 2012, Andy Allan wrote: On 12 October 2012 00:58, Tom MacWright t...@macwright.org wrote: What do you want to see happen with OSM's software this year? What are the tasks which everyone agrees on, but nobody has had the time to tackle? snip As for all-new things, I'd like to explore how to integrate all the various QA feeds into a combined overview, rather than having to hunt around different sites. And there are so many of those - both global and local! In addition, I would like to see something new here too. It was up as a project for Google Summer of Code, but the project fell through. The idea was to create an engine that can flag suspicious changesets. I would love to write this myself, but I've (sadly) zero time for it. You can find the general idea at: http://wiki.openstreetmap.org/wiki/GSoC_Project_Ideas_2012#Anomalies_Detection_Engine cheers, Derick ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hi, On 10/12/2012 12:08 PM, Derick Rethans wrote: And there are so many of those - both global and local! In addition, I would like to see something new here too. It was up as a project for Google Summer of Code, but the project fell through. The idea was to create an engine that can flag suspicious changesets. There's some working code on this (which is actually in production I believe) by Paul Norman of the Data Working Group, here: https://github.com/pnorman/osm-weirdness Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 10/12/2012 3:27 AM, Stephan Knauss wrote: So I want a tool that makes it possible to do a quality control by checking diffs like it's possible in wikipedia for years. The problem is that our data is a lot harder to diff. +1 for a wishlist: visual changeset diff? ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Okay, so visual changeset tools, better history tools, and osmbugs. OSMBugs or something like it will definitely get some love - early on we've just been battling confusion about wtf OSMBugs is, with all of the versions. That's mostly cleared up now. Anything else? On Fri, Oct 12, 2012 at 6:25 AM, Mike N nice...@att.net wrote: On 10/12/2012 3:27 AM, Stephan Knauss wrote: So I want a tool that makes it possible to do a quality control by checking diffs like it's possible in wikipedia for years. The problem is that our data is a lot harder to diff. +1 for a wishlist: visual changeset diff? __**_ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.**org/listinfo/devhttp://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Okay, so visual changeset tools, better history tools, and osmbugs. OSMBugs or something like it will definitely get some love - early on we've just been battling confusion about wtf OSMBugs is, with all of the versions. That's mostly cleared up now. Anything else? Have you read my e-mail to the list? :-) http://lists.openstreetmap.org/pipermail/dev/2012-October/025751.html I'd say these are more important than history or QA tools but that's just my opinion... Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
@Pawel: totally, I think those are awesome. I was mostly prodding this thread since it was getting into the details of changeset/history improvements and don't want it to be laser-focused on the technical details of one thing :) On Fri, Oct 12, 2012 at 8:38 AM, Paweł Paprota ppa...@fastmail.fm wrote: Okay, so visual changeset tools, better history tools, and osmbugs. OSMBugs or something like it will definitely get some love - early on we've just been battling confusion about wtf OSMBugs is, with all of the versions. That's mostly cleared up now. Anything else? Have you read my e-mail to the list? :-) http://lists.openstreetmap.org/pipermail/dev/2012-October/025751.html I'd say these are more important than history or QA tools but that's just my opinion... Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 12/10/12 00:58, Tom MacWright wrote: So, along with the big 'kicking off' blog post on MapBox[1], I posted three basic issues in the openstreetmap-website tracker - JSON support, filtering on the main api, and a tag api. The three of these were closed by Tom Hughes, which is fine - they might be all bad ideas, and if we're treating the issue tracker as Stuff we all agree is definitely good, then they don't belong there: they're relatively undiscussed ideas, since I had just posted them. But that's beside the point of this email: That's not strictly accurate - the JSON API one did not get closed. Personally I'm not a fan of using bug trackers for things that aren't either actual bugs, or concrete proposals with patches attached. What I call wishlist type tickets tend to just hang around forever because in open source people generally work on things they are interested in, not things other people have asked for. This is obviously a little different because you guys are planning to work on these things, but issue trackers that are acquiring things at a faster rate than things are being closed have an unfortunate effect on my brain where I start trying to figure out how to get rid of some things in order to get things under control again. As it happens I also wasn't aware that you wouldn't be able to reopen the issue in the way a reporter can in trac. My personal opinion is that, as Andy has said, the rails-dev list is probably the best place for early stage discussions, but I've reopened the tickets which did get closed for now and if people prefer to do the discussion there then fine. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Friday 12 October 2012, Tom MacWright wrote: Hey all (or, well, those subscribed to dev@ - my flamewar shields are at 50% so I'm not risking an email to talk), So, along with the big 'kicking off' blog post on MapBox[1], I posted three basic issues in the openstreetmap-website tracker - JSON support, filtering on the main api, and a tag api. The three of these were closed by Tom Hughes, which is fine - they might be all bad ideas, and if we're treating the issue tracker as Stuff we all agree is definitely good, then they don't belong there: they're relatively undiscussed ideas, since I had just posted them. But that's beside the point of this email: What do you want to see happen with OSM's software this year? What are the tasks which everyone agrees on, but nobody has had the time to tackle? Tom [1]: http://mapbox.com/blog/kicking-off-knight-work/ Something I would find interesting personally is an API interface to diary entries (for reading at least). Namely a geolocated diary entries within bbox call (I'm not sure there's even a spatial index over diary entries atm). This would allow easy (easy) implementation of a map-based diary entry browser a la http://wiki.openstreetmap.org/wiki/User:Robert#Diary_entries and generally go towards creating better use of diaries. robert. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Building such a meta platform is not an easy task because bugs are much more than just popup markers on the map - bugs can also be a way, or a collection of ways, or a convex hull around a broken polygon; a good platform will also allow the user to flag those bugs which are false positives. All this in an environment where a multitude of QA providers feed hundreds of thousands of check results into the platform every day. For your information, osmose http://wiki.openstreetmap.org/wiki/Osmose (free software) developed for and by the french community is doing kind of exactly that. A meta plateforme taking inputs from distant plugins as xml files, false positive management, plugin framework (optionnal), redistibution API to embeded devices, a JOSM plugin, allready an OpenstreetBugs layer and a quick editor. It lacks however the convex hull around buggy areas. We don't really have the server ressources (yet) to make it available world wide, but the software beeing GPL, if I remember correctly, could be re-used and upgraded to run at larger scale. -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Fri, Oct 12, 2012 at 9:12 AM, Tom Hughes t...@compton.nu wrote: That's not strictly accurate - the JSON API one did not get closed. That's true, but my two bugs did get closed. More on this below. Personally I'm not a fan of using bug trackers for things that aren't either actual bugs, or concrete proposals with patches attached. I added two issues. First was one about caching of .../:element/:id/:version calls. Looking at the code, we set no cache headers, even though OSM elements of a certain version are immutable; that seems like a mere oversight. And I suggested that we could make a redirect of ../:element/:id calls to the object's version url. Code would be forthcoming, by me, in the next two weeks. Not today, probably, because I'm flying to SOTM US. The reason for this code is that with Changemonger (the project I'm presenting on during Sunday's sessions), I make a lot of individual object calls, and I've been thinking about caching in general. This issue was closed by Tom with little/no explanation. I made a second issue at the same time regarding gathering histrorical data around way geometry. Right now, due to the way OSM stores its geometry data, the only way to know about a way's geometry history is to call the history call for the way, then make a history call for each and every node that was ever in the way. This is expensive. Talking with Matt about this a month ago on IRC, he suggested a new call could be made, one that would do the above work in a single call. This would allow the same work to be done with less server overhead, and greater resource management. Tom closed it, saying he didn't like expensive calls. This is despite the fact that other people/peojects are already either using the above method (which is more expensive) or continuing to use the undocumented h This is obviously a little different because you guys are planning to work on these things, but issue trackers that are acquiring things at a faster rate than things are being closed have an unfortunate effect on my brain where I start trying to figure out how to get rid of some things in order to get things under control again. If you're suggesting that the problem is These tickets are not marked as wishlist and not given to anyone - then I agree. I should have assigned the first one to me, at the very least. The second one, I thought needed a little more thinking through. As it happens I also wasn't aware that you wouldn't be able to reopen the issue in the way a reporter can in trac. My personal opinion is that, as Andy has said, the rails-dev list is probably the best place for early stage discussions, but I've reopened the tickets which did get closed for now and if people prefer to do the discussion there then fine. That's fine, but it's very frustrating to have a ticket closed without discussion. It's really disheartening. - Serge ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 12/10/12 15:14, Serge Wroclawski wrote: On Fri, Oct 12, 2012 at 9:12 AM, Tom Hughes t...@compton.nu wrote: That's not strictly accurate - the JSON API one did not get closed. That's true, but my two bugs did get closed. In case you haven't noticed they have been reopened now. More on this below. Personally I'm not a fan of using bug trackers for things that aren't either actual bugs, or concrete proposals with patches attached. I added two issues. First was one about caching of .../:element/:id/:version calls. Looking at the code, we set no cache headers, even though OSM elements of a certain version are immutable; that seems like a mere oversight. And I suggested that we could make a redirect of ../:element/:id calls to the object's version url. I have no problem setting the cache headers, but I am concerned about the redirect idea because those URLs are almost certainly used far more often than the version specific ones and that change would mean that every one of those requests would wind up getting redirected, hence turning one request into two. I made a second issue at the same time regarding gathering histrorical data around way geometry. Right now, due to the way OSM stores its geometry data, the only way to know about a way's geometry history is to call the history call for the way, then make a history call for each and every node that was ever in the way. This is expensive. Talking with Matt about this a month ago on IRC, he suggested a new call could be made, one that would do the above work in a single call. This would allow the same work to be done with less server overhead, and greater resource management. We have been planning a history call, but that was more along the lines of a history version of the map call rather than individual objects. That is really aimed at supporting P1 style rollback rather than changeset analysis, which should ideally be done offline without having to hit the API if possible. The question of how far to go in recursively resolving objects, and the history of objects is a tricky one, because the cost of the call can quickly escalate as you do this. What people who ask for the history of a way really tend to want is not a full history of every node, but rather they want to know how it's geometry has changed over time, which leads to the kind of fake version creation the P1 call does. Maybe instead we should just return full history for the nodes and leave the client to figure out how to show a view of the sequence of changes to tags and geometry by examining both way and node history? Recursing into relations is a whole other issue, because that can recurse many times and lead to massive inflation of work and size of response. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
From: Frederik Ramm [mailto:frede...@remote.org] Sent: Friday, October 12, 2012 3:21 AM To: dev@openstreetmap.org Subject: Re: [OSM-dev] OSM Wishlist Hi, On 10/12/2012 12:08 PM, Derick Rethans wrote: And there are so many of those - both global and local! In addition, I would like to see something new here too. It was up as a project for Google Summer of Code, but the project fell through. The idea was to create an engine that can flag suspicious changesets. There's some working code on this (which is actually in production I believe) by Paul Norman of the Data Working Group, here: https://github.com/pnorman/osm-weirdness For some value of production. It produces cryptic output, needs to be run with python -i and kicked by running the detect() function every now and then and only works with statistics based on the number of nodes/ways/relations added/deleted/modified. It points out some interesting stuff but also points out some trivial stuff. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Serge Wroclawski writes: This is expensive. does this matter? Is there a requirement to have it running against the latest r/w api? I fetch my osm data for editing from the osm ro-mirror (overpass). This is a lot faster than waiting for main API. Works fine as well. Why not add another (or a 3rd, 4th, ...) ro-mirror for these special tasks. load can be distributed quite easy onto different servers. Could even be a database made specifically for these history calls. Just preprocess the geometry. And if it must be working with the latest revision because the data is newer than a few minutes then the API could query this service and in case it's not available there only then doing the expensive calls. I once thought of implementing something like this. I think Peter already has a prototype doing similar. https://github.com/MaZderMind/OpenStreetMap-History-API Stephan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Oct 12, 2012 10:24 AM, Tom Hughes t...@compton.nu wrote: On 12/10/12 15:14, Serge Wroclawski wrote: On Fri, Oct 12, 2012 at 9:12 AM, Tom Hughes t...@compton.nu wrote: That's true, but my two bugs did get closed. In case you haven't noticed they have been reopened now. Hadn't noticed yet... Been getting ready for the conference and github doesn't notify on reopen. More on this below. Personally I'm not a fan of using bug trackers for things that aren't either actual bugs, or concrete proposals with patches attached. I added two issues. First was one about caching of .../:element/:id/:version calls. Looking at the code, we set no cache headers, even though OSM elements of a certain version are immutable; that seems like a mere oversight. And I suggested that we could make a redirect of ../:element/:id calls to the object's version url. I have no problem setting the cache headers Okay I'll just do that for now. We have been planning a history call, but that was more along the lines of a history version of the map call rather than individual objects. That is really aimed at supporting P1 style rollback rather than changeset analysis, which should ideally be done offline without having to hit the API if possible. Change analysis (changeset or otherwise) needs to touch some history resource... Don't see a way around that. The question of how far to go in recursively resolving objects, and the history of objects is a tricky one, because the cost of the call can quickly escalate as you do this. What people who ask for the history of a way really tend to want is not a full history of every node, but rather they want to know how it's geometry has changed over time, which leads to the kind of fake version creation the P1 call does. Right.. So if that information can be provided another way, I'll take it. Maybe instead we should just return full history for the nodes and leave the client to figure out how to show a view of the sequence of changes to tags and geometry by examining both way and node history? That was the plan, and yes it's a pain in the butt for the client, but I was writing the client :-) Recursing into relations is a whole other issue, because that can recurse many times and lead to massive inflation of work and size of response. Yeah.. I agree it's a hairy mess and I'm willing to put it aside for now. Also I hope to announce a public beta of Changemonger so we should get some real world data on its api call patterns in the real world. Up until today it's been testers looking to feed it huge changesets to break it. - Serge ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 12.10.2012 01:58, Tom MacWright wrote: What do you want to see happen with OSM's software this year? What are the tasks which everyone agrees on, but nobody has had the time to tackle? reading some of the answers it also seems that appropriate server hardware is an issue for many of the smaller projects. We have developers with great ideas but often the hardware is not powerful enough to provide their services world-wide. If the funding limited to software development? Or could it also be used to provide a technical infrastructure for the various projects that are already around? Bringing a prototype project to a server also has different problems. Clear technical documentation can help to deploy a service better or use the existing tools more efficient. Writing good technical documentation is a rare skill, having support in this area can help boosting our existing developer activities and bring up new great projects. Stephan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Viernes, 12 de octubre de 2012 01:58:23 Tom MacWright escribió: What do you want to see happen with OSM's software this year? What are the tasks which everyone agrees on, but nobody has had the time to tackle? OK, here's mine: A diff/patch tool for imported data. Let me explain. Assume I imported a dataset, (e.g. Spanish powerplants 2010) a few months (or years) ago. I survive the flamewars in imports@, I manage to clean up the data, I manage to upload everything without too many validator errors in JOSM. So far, so good. Now, assume a newer version of the same dataset appears on the wild (e.g. Spanish powerplants 2012). I want to be able to: a) Know which data the provider added or deleted, and have those additions and deletions uploaded into OSM (Has the authoritative source added more powerplants?) b) The data provider wants to know what changes the OSM community has done, related to the data they provided (have they changed the boundary of this old polygon Igforgot to check?) I don't know if the way to tackle this problem down is to keep a registry of imported data, in order to check newer versions of external datasets with the registered older versions of already-imported external datasets, or rely on source= tags and version numbers. Seriously, guys from NMAs would *love* this. Also. ogr3osm. I would love to have the time and resources (or paid time, nudgenudgewinkwink) to redo ogr2osm; adding a backtracking-like algorithm to minimise the amount of geometries' shared nodes (and their bounding boxes) in memory, in order to be able to convert datasets with gazillions of geometries into .osm format. Backtracking-like in the sense that the data processing would be done in a tree-like fashion, walking through overlapping geometries, processing only geometries which have all their nodes already into the list of generated node IDs, writing to file and destroying from memory nodes that won't appear again because all the overlapping geometries have been processed. Yes, it sounds like a mouthful. I have a bunch of napkin notes with the algorithm written down, though :-) Also. Right now, the HOT would greatly appreciate a seeding GUI for MapProxy. Some python+openlayers+javascript hacking should do in a couple of weeks. The goal being that a user would be able to seed the caches through the development server web interface, without needing to edit config files. Bonus points for a cache configuration GUI. Cheers, -- Iván Sánchez Ortega i...@sanchezortega.es i...@geonerd.org ¿Donde está el enchufe para el COM4:? ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Fri, Oct 12, 2012 at 11:50 PM, Iván Sánchez Ortega i...@sanchezortega.es wrote: A diff/patch tool for imported data. opengeo presented at SOTM about their plans for geogit [1]. from what i understood one of their aims is interoperability between separately-maintained OGC model datasets and OSM. i think i'm right in saying that there's a fair way to go before it's a finished product, but i'm sure they'd welcome any support. cheers, matt [1] http://blog.opengeo.org/tag/geogit/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Imagine there is list of schools in a city. This list is maintained by a school district office with updated website, phone numbers, and office opening times. They maintain this list through a simple PHP form on their website that allows them to make updates to each listing (node). If any node on their list is changed by anyone but them or the people listed as able to edit that form on their own website (with their OSM Ids) then they are notified and can correct or confirm acceptance of the change. They do not need to be the last editors of the node to know that the information contained therein is approved by them. Whatever makes this possible, that's what I want, please. Pretty please. tnx :) alex ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On 12 October 2012 00:58, Tom MacWright t...@macwright.org wrote: Hey all (or, well, those subscribed to dev@ - my flamewar shields are at 50% so I'm not risking an email to talk), Bear in mind that technical (generally software) stuff is often better on dev@ anyway - and there's lots of developers on this list who aren't subscribed to the general flamewar^Wtalk@ list. The three of these were closed by Tom Hughes, which is fine - they might be all bad ideas, and if we're treating the issue tracker as Stuff we all agree is definitely good, then they don't belong there: they're relatively undiscussed ideas, since I had just posted them. We have rails-dev@ was the place to discuss matters relating to the api/website, and here for matters with a wider impact. Does discussing new ideas on github bring advantages over mailing lists that I'm missing? It's not something that I'm used to. What do you want to see happen with OSM's software this year? What are the tasks which everyone agrees on, but nobody has had the time to tackle? Ah, good stuff. There's a lot of stuff on Potlatch2 that I'd like to see, but that I don't have time to work on. There's a junction editor that needs writing, an in-editor tutorial framework that needs finishing off, and a lot of half-done stuff in translations, UI and unit testing. I haven't seen anything specifically rulling it out, but would I be right in saying that mapbox don't have plans to contribute to P2? Beyond that, I'd like to see the issues around the deleted-item-map-call sorted out, and OWL (or equivalent) integrated into the rails port to power the history tab. As for all-new things, I'd like to explore how to integrate all the various QA feeds into a combined overview, rather than having to hunt around different sites. Cheers, Andy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hey, I haven't seen anything specifically rulling it out, but would I be right in saying that mapbox don't have plans to contribute to P2? No current plans, though that's more for lack of any experience in Flash Flex than anything set in stone or technical. Given that we currently don't plan to have some kind of P2 beater that takes its place in the edit tab, it might make a lot of sense to take a closer look at doing some targeted improvements of it. Thanks for your input! I had also totally forgotten about the deleted map item call - that should totally be on the list. Tom On Thu, Oct 11, 2012 at 8:14 PM, Andy Allan gravityst...@gmail.com wrote: On 12 October 2012 00:58, Tom MacWright t...@macwright.org wrote: Hey all (or, well, those subscribed to dev@ - my flamewar shields are at 50% so I'm not risking an email to talk), Bear in mind that technical (generally software) stuff is often better on dev@ anyway - and there's lots of developers on this list who aren't subscribed to the general flamewar^Wtalk@ list. The three of these were closed by Tom Hughes, which is fine - they might be all bad ideas, and if we're treating the issue tracker as Stuff we all agree is definitely good, then they don't belong there: they're relatively undiscussed ideas, since I had just posted them. We have rails-dev@ was the place to discuss matters relating to the api/website, and here for matters with a wider impact. Does discussing new ideas on github bring advantages over mailing lists that I'm missing? It's not something that I'm used to. What do you want to see happen with OSM's software this year? What are the tasks which everyone agrees on, but nobody has had the time to tackle? Ah, good stuff. There's a lot of stuff on Potlatch2 that I'd like to see, but that I don't have time to work on. There's a junction editor that needs writing, an in-editor tutorial framework that needs finishing off, and a lot of half-done stuff in translations, UI and unit testing. I haven't seen anything specifically rulling it out, but would I be right in saying that mapbox don't have plans to contribute to P2? Beyond that, I'd like to see the issues around the deleted-item-map-call sorted out, and OWL (or equivalent) integrated into the rails port to power the history tab. As for all-new things, I'd like to explore how to integrate all the various QA feeds into a combined overview, rather than having to hunt around different sites. Cheers, Andy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
Hi Tom, So, along with the big 'kicking off' blog post on MapBox[1], I posted three basic issues in the openstreetmap-website tracker - JSON support, filtering on the main api, and a tag api. I think these are important, but there are far more important things. And also, these issues seem like an infrastructure issues and I would agree a little bit (is there such a thing?) with the pushback. I would say - build something that needs JSON support, filtering and tag API... What do you want to see happen with OSM's software this year? What are the tasks which everyone agrees on, but nobody has had the time to tackle? ...which takes me to the point: 1. Since this is a wishlist I can do this: PLEASE PLEASE PLEASE sprinkle your Mapbox goodness on the osm.org map style! I respect that the current one's been around for whateverlongtime and it works but come on... let's face it, it's not modern enough to serve as a face for the main entry point to this project. And the style at zoom levels 0-5 should be considered torture. Also there is a lot of 3rd party map styles that show that OSM data can be visualized in a very pretty way. There are over 400 open issues [1] for this stuff. At this point it may be easier to start from scratch, I don't care. Just please work it out with people who have the power (I heard it's somewhere in SVN authored by someone with Windows) to modify it and improve it. a) Yes, there are multiple angles to be considered - osm.org main style is not about showing one single aspect of OSM. b) Yes, there are concerns about not displaying everything. c) Yes, I am sure there are infrastructure concerns about changing the style since every single tile would have to be rendered. If OSM people successfully handled redaction period, I have no doubt that c) can be worked out. As for the other two - YOU ARE MAPBOX - beautiful maps and all that - who else could do a good job on this? OK... let me calm down for a second... (1) is so far ahead of everything else that I need a minute to compose... 2. Basic web-based (JS/HTML5) editor. Perhaps continue what's been already started [2]. No question this area can be a major barrier for newcomers, especially potential converts from places like Google MapMaker and such. 3. Social OSM - I've been working on this a lot lately [3] and it's good that we are in contact. Let's cooperate and make this happen. 4. Usability/design improvements to osm.org - in short: better face = more mappers = better data. Another area of interest of mine. I've found some existing work that is very good: [4], [5] that can potentially inspire such efforts. Again, let's make this happen. I think these areas (especially (1)...) need a lot of love and you are in a unique position to help give it to them... [1] https://trac.openstreetmap.org/query?status=!closedcomponent=mapnik [2] http://www.geowiki.com/ [3] https://github.com/ppawel/osm-activity-server [4] http://forum.openstreetmap.org/viewtopic.php?pid=281564 [5] http://wiki.openstreetmap.org/wiki/User:Pumbur/Re Paweł ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OSM Wishlist
On Fri, Oct 12, 2012 at 2:14 AM, Andy Allan gravityst...@gmail.com wrote: Does discussing new ideas on github bring advantages over mailing lists that I'm missing? It's not something that I'm used to. github is pretty good for discussions, it integrates into mails. -- James Michael DuPont Member of Free Libre Open Source Software Kosova http://flossk.org Saving wikipedia(tm) articles from deletion http://SpeedyDeletion.wikia.com Contributor FOSM, the CC-BY-SA map of the world http://fosm.org Mozilla Rep https://reps.mozilla.org/u/h4ck3rm1k3 Free Software Foundation Europe Fellow http://fsfe.org/support/?h4ck3rm1k3 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
