Re: [OSM-talk] Import guidelines proposal update
Paul Norman wrote: From: Lester Caine [mailto:les...@lsces.co.uk] Subject: Re: [OSM-talk] Import guidelines proposal update who last edited an object! ). Where the import HAS nice unique object identifiers things are a lot easier, but raw vector data like the French import, and I think the Spanish data you are talking about CAN still be 'diffed' against earlier imports, and result in perhaps new data that can simply be imported, or perhaps an overlay that identifies conflicts that need a human eye. Isn't it better to spend time working out a GOOD way of using the data going forward rather than having to manually merge the whole lot again in a couple of years time ... and every couple of years. My thoughts on how to handle this for data with persistent unique identifiers without adding those as tags is to ** a. Record the correspondence between source ID and temporary pre-upload negative OSM ID b. Record the correspondence between pre-upload negative OSM ID and OSM ID c. Combine for a correspondence between source ID and OSM ID, and save this ** EXCEPT - that requires ALL the data from the external import to be loaded in order to create the OSM ID which may not be a bad thing? ... BUT Part of the 'preprocessing' before ever uploading the import would be to identify which objects are going to be uploaded and which not, so you need to create an 'id' initially related to the data source? That is providing that the data source is actually identifiable data. What I had not considered up until now is if the data source is simply a raw vector file with version of a paper map, then while the individual lines could be 'imported' the data is almost useless until it has been 'identified'? You may just as well simply trace? But even here all is not lost since one can still pre-process the data and provide the link back as to which lines have been copied and which not. In which case the OSM ID provides additional data back to the source, but I doubt that there is any value simply importing millions of lines segments directly into the main database? This has to be a secondary staging area to handle that data? d. When updating, identify objects that have changed or been added to the source e. For changed or deleted objects if the OSM object was last edited by the importer's import account, upload a new version reflecting the changes. Objects that have been edited by a person will require manual intervention, like now f. Handle new objects like before g. Identify objects deleted in OSM and check these, then submit corrections to the source. The one case this doesn't handle very well is POIs that have been changed from a node into a way. I'm going to be working on implementing this in a limited way for updating addresses locally. Addresses are different because the address should be unique in the city. While the UK 'address database' can't be uploaded freely yet, I have been slowly importing data manually, and it just irritates that every building has duplicates of much of this data. I know a few attempts have been made at relations and the like to group stuff, but as I have said in the past, isn't now the time to provide a mechanism that uses 'lookup tables' for some of this which will automatically simplify what is stored in the tags against each object? An 'address' in the UK only needs to be the 'property id' - house/flat number or name - and the 'postcode'. Everything else can be provided by a 'lookup' on the postcode reference. Now this ACTUALLY does not work simply because the 'postcode' has too many edge cases where you need additional information to provide 'street'. That is why the nlpg data does not use it and simply provides a reference to it deep inside. It provides a street gazetteer with a clean reference number for each street ( and in theory a 'way' for the physical location, but in most cases this is just a couple of 'end points' :( ) This is the sort of process that could simplify a LOT of the micro/macro mapping problems that are now building up, since a world wide 'street gazetteer' is the base for all of the routing programs? And a top level map in its own right? All of the problems of 'turns relations' would be managed in the 'street gazetteer', while the underlying map can display all of the pretty stuff such as the grass verges, footpaths and the like? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
From: Lester Caine [mailto:les...@lsces.co.uk] Sent: Friday, September 21, 2012 11:47 PM To: 'OSM' Subject: Re: [OSM-talk] Import guidelines proposal update Paul Norman wrote: From: Lester Caine [mailto:les...@lsces.co.uk] Subject: Re: [OSM-talk] Import guidelines proposal update who last edited an object! ). Where the import HAS nice unique object identifiers things are a lot easier, but raw vector data like the French import, and I think the Spanish data you are talking about CAN still be 'diffed' against earlier imports, and result in perhaps new data that can simply be imported, or perhaps an overlay that identifies conflicts that need a human eye. Isn't it better to spend time working out a GOOD way of using the data going forward rather than having to manually merge the whole lot again in a couple of years time ... and every couple of years. My thoughts on how to handle this for data with persistent unique identifiers without adding those as tags is to ** a. Record the correspondence between source ID and temporary pre-upload negative OSM ID b. Record the correspondence between pre-upload negative OSM ID and OSM ID c. Combine for a correspondence between source ID and OSM ID, and save this ** EXCEPT - that requires ALL the data from the external import to be loaded in order to create the OSM ID which may not be a bad thing? ... BUT Part of the 'preprocessing' before ever uploading the import would be to identify which objects are going to be uploaded and which not, so you need to create an 'id' initially related to the data source? That is providing that the data source is actually identifiable data. Well, you don't need to create an ID related to the data source - this is for the case of data with persistent unique identifiers. If decisions were made to not upload parts of the data with the first upload this could easily be captured with the fact that there is no pre-upload negative ID corresponding to a particular source ID. What I had not considered up until now is if the data source is simply a raw vector file with version of a paper map, then while the individual lines could be 'imported' the data is almost useless until it has been 'identified'? You may just as well simply trace? But even here all is not lost since one can still pre-process the data and provide the link back as to which lines have been copied and which not. In which case the OSM ID provides additional data back to the source, but I doubt that there is any value simply importing millions of lines segments directly into the main database? This has to be a secondary staging area to handle that data? Data that is purely vectors with absolutely no information that can be turned into OSM tags is basically useless. For the case where objects do not have a unique ID you'll have to use spatial matching, likely in PostGIS. This may run into problems if the geometry in the source substantially changes for the same object on the ground but this is an inherent limitation of the lack of persistent IDs. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Paul Norman wrote: From: Lester Caine [mailto:les...@lsces.co.uk] Sent: Friday, September 21, 2012 11:47 PM To: 'OSM' Subject: Re: [OSM-talk] Import guidelines proposal update Paul Norman wrote: From: Lester Caine [mailto:les...@lsces.co.uk] Subject: Re: [OSM-talk] Import guidelines proposal update who last edited an object! ). Where the import HAS nice unique object identifiers things are a lot easier, but raw vector data like the French import, and I think the Spanish data you are talking about CAN still be 'diffed' against earlier imports, and result in perhaps new data that can simply be imported, or perhaps an overlay that identifies conflicts that need a human eye. Isn't it better to spend time working out a GOOD way of using the data going forward rather than having to manually merge the whole lot again in a couple of years time ... and every couple of years. My thoughts on how to handle this for data with persistent unique identifiers without adding those as tags is to ** a. Record the correspondence between source ID and temporary pre-upload negative OSM ID b. Record the correspondence between pre-upload negative OSM ID and OSM ID c. Combine for a correspondence between source ID and OSM ID, and save this ** EXCEPT - that requires ALL the data from the external import to be loaded in order to create the OSM ID which may not be a bad thing? ... BUT Part of the 'preprocessing' before ever uploading the import would be to identify which objects are going to be uploaded and which not, so you need to create an 'id' initially related to the data source? That is providing that the data source is actually identifiable data. Well, you don't need to create an ID related to the data source - this is for the case of data with persistent unique identifiers. If decisions were made to not upload parts of the data with the first upload this could easily be captured with the fact that there is no pre-upload negative ID corresponding to a particular source ID. BUT you may still need to identify the the un-merged data when processing in later upload cycles ... see below. What I had not considered up until now is if the data source is simply a raw vector file with version of a paper map, then while the individual lines could be 'imported' the data is almost useless until it has been 'identified'? You may just as well simply trace? But even here all is not lost since one can still pre-process the data and provide the link back as to which lines have been copied and which not. In which case the OSM ID provides additional data back to the source, but I doubt that there is any value simply importing millions of lines segments directly into the main database? This has to be a secondary staging area to handle that data? Data that is purely vectors with absolutely no information that can be turned into OSM tags is basically useless. For the case where objects do not have a unique ID you'll have to use spatial matching, likely in PostGIS. This may run into problems if the geometry in the source substantially changes for the same object on the ground but this is an inherent limitation of the lack of persistent IDs. I beg to differ here, although I did originally think the same! We need some feedback from users who have access to this type of data, and I am wondering based on the comments about the French data if THAT is of this style? And I am STILL looking at a staging layer anyway! Raw vector data like this - if it is all that is available - has to be traced and tagged, but why shouldn't it be provided as a layer from which line segments can be simply selected rather than having to trace them? click,click,click,click, close(to join into area), identify. The processed lines can then be hidden and you move onto the next set ... your comment on 'stability' of the coordinates between imports is valid, and needs managing but I can see a case for 'tracing' say a street element, tagging it, but NOT including it in the later 'import'. You just need to make that information persistent without using OSM id's. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
2012/9/20 Lester Caine les...@lsces.co.uk: My own interest here is more historic than current and I was looking for the development of areas relating to my family tree, but there seems to be a general consensus that once an object ceases to exist it should be deleted from the database. there is not this general consensus in the community for completely removing former objects (but it might be a majority who doesn't want them, not sure). Have a look at abandoned and disused features, some historic features and also objects to be expected in the future (proposed and construction). Just a few days ago someone proposed on the German ML to agree on a standard way for tagging these by applying the status as a prefix (e.g. disused:amenity=pub). There is some well established objects that work differently though (railway=abandoned, etc.) cheers, Martin ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Martin Koppenhoefer wrote: My own interest here is more historic than current and I was looking for the development of areas relating to my family tree, but there seems to be a general consensus that once an object ceases to exist it should be deleted from the database. there is not this general consensus in the community for completely removing former objects (but it might be a majority who doesn't want them, not sure). Have a look at abandoned and disused features, some historic features and also objects to be expected in the future (proposed and construction). Just a few days ago someone proposed on the German ML to agree on a standard way for tagging these by applying the status as a prefix (e.g. disused:amenity=pub). There is some well established objects that work differently though (railway=abandoned, etc.) I am thinking that a second database 'layer' is the best way of handling some of this. I think that this also marries up with other historic data such as 'imports' which can then be retained in a compatible manor and used to process a new 'import' to provide difference reports at least. railway=abandoned is something of a grey area. The abandoned line over the road from here is still classified as active, so a bridge had to be built to accommodate electric running. Sounds silly perhaps, but the local steam preservation group have extended the line to Broadway, and have an option to extend to Honeybourne to connect to the 'main line' so had the 'abbandoned line' not been marked ... But the one thing I lobby for very strongly is that the correct 'start_date' is attached to an object. This is the one aspect of the recent confusion that has probably been overlooked. Having deleted existing buildings, the fact that they already exist has been lost, so there is no way to identify them from new buildings that only appear in the latest import ... With all the historic mapping now available we have the option to add historic dates to objects as well ... without interfering with anybody else? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On jeudi 20 septembre 2012, Frederik Ramm wrote: Hi, Hi, If the negative effects however affect other/different people - perhaps because they are using the API outside of specifications, or causing more work for people elsewhere in the project - then they can't. I can only fully agree with that. But it don't think the changes to the guidelines I'm proposing are not abeying to that obvious pre-requist. They are just not expressing it clearly because I'm only proposing to change the Mandatory dedicated account for imports and not the guidelines to constuct local guidelines which could be wrote in another paragraph. And this allready exist on chapter 8, 9 and 10 of the http://wiki.openstreetmap.org/wiki/Import/Guidelines Which I am not proposing to change because those rules your are refering to should still be applied (and enforced) to all type of imports, even if some local community proposes otherwise, in which case, contact should me made with that community to tell it it is proposing invalid guidelines conflicting with general ones. Real life exemple : during the redaction bot, any types of import have been forbiden, and one french contributor got blocked because importing in this period even if the information was made available globally and locally by all means we (french community) could. The block is then perfectly justified, although we (french community) would have prefered to block him ourself, and sent him a french language message to explain why. There are very simple technical things. For example, assume that there was a French DWG dealing only with French cases; we don't have the means to set things up in a way that the French DWG can only block French users. imho, and without any offense, your view on that is too technicaly narrowed and based on a lack of trust. The technical feature blocking a user exists (no doubt about this ;-) ), what you are probably saying is that there is no technical way to narrow that right to block to a region ? Yet, the DWG group has this right ? why would you, and wouldn't I (or cquest) ? Because of this very lack of trust ! You don't have the technical mean to restrict, but you still have the means to do it : trust. Just ask him to restrict himself at blocking cadastre imports, in France, and I'm sure he will respect that. (Here I'm betting that by French users you meant users operating in France, because if not, we are in a deep disagreeing) We don't even have a proper definition of local communities Not proper, and not for all doesn't mean we don't have a far enough definition for at least the french community and we are discussing here. French community = people subscribe to talk-fr, participating in our forum or web site or members of our local fondation. So, what if a Toulouse mapper comes to OSMF and complains that OSMF-FR is unfairly suppressing Languedoc self-determination? What if local communities decide stuff that is considered harmful to the project as a whole by someone on the other side of the world? Who would adjudicate such a conflict? Can the world-wide community be called to a vote that is binding for France? Can the French community make a binding rule for Toulouse? How many is a community, anyway? Do they have to be incorporated? Do they have to be democratic? What if a national community - as has been the case in the past with some Eastern European national communities - takes a very liberal attitude towards copyright (the government web page says private use only but they never prosecuted anyone...)? Can a national community make a deal with a sponsor and allow the sponsor to carry the OSM logo? In here, you are only expressing a fear from the futur, from things that haven't yet happen and that might never happen. Come on ! Have trust in the future and our (the world community) ability to solve problems as they come. (And even more when my question, here, was returning back to rules that have been there for a long time. They might have prooved not enough, but not as bad as it is now for the french community) I think that your suggestion is too much like case law: There's a rule that leads to a result you don't like, and then you amend it with a little extra rule specifically for that purpose. You seam to forgot that, in the first place, someone from the DWG did exactly that : changed a recommandation into a law that the same group is enforcing because a result was what it liked What I'm doing, is proposing something that is between (therefore the amendment of a special rule) what was, and what DWG wants. Maybe because I'm used to it in France, but I suspect the case is the same elsewhere. Laws are made of a 1st version, and then dozen of amendments to counter it's bad effects we discover later, don't you think there is something good in it ? (In your case, you have built a regional limited import special rule into the separate import
Re: [OSM-talk] Import guidelines proposal update
Am 20.09.2012 13:43, schrieb sly (sylvain letuffe): which isolated processes ? You are the guy that requests that the French community is doing things different than the rest of the OSM-word. So you must answer your question by yourself... Best regards, Michael. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On jeudi 20 septembre 2012, Lester Caine wrote: sly (sylvain letuffe) wrote: The 'mechanisms' that we use MUST be managed centrally, What are you talking about ? What mechanisms are you refering to ? Simply the methods by which data is added to the database. There are several methods in play to make data includes in the database, and not only I think we don't MUST but we can't, that's pure utopia. My moving mouse clicing in JOSM is part of my method to enter data in the database, do you wish to manage my moving mouse centrally ? Sorry to play dumb, but if we want to discuss mechanisms in one and only rule : that won't work, so let's return to the one special class of mechanisms I was refering to : semi-manual imports, done with JOSM by one contributor, on a smaller scale than a country for wish the community of that country has described guidelines and has allowed not to use a dedicated account And all I am trying to understand now is why if we HAVE digital data to work with for which further versions will be provided over the coming decades someone has to manually check every line every year or so? I think this is off topic, even if related, but I'll be glad to explain for the case I know if you wish. Although, it was partially explained for the special case of the french import in the thread Import guidelines OSMF/DWG governance for wich that question is also off topic. You can start a new topic like why are french importing from cadastre if every line as to me checked every year or more general for any import like : why are people importing data from other sources and I'll be glad to answer No, I'm not asking you to go away because you are bothering me, it just looks like you want more informations on what we do and why we do it with our french cadastre. But since this thread is allready a bit confused, I'll try to keep it focused on : Import guidelines proposal update This data was in the database, so only the changes needed to be posted, but a mistake was made. We learn from mistakes and so what I am trying to learn is if we could have HELPED by reducing the chance of the mistake? By providing tools that take advantage of the data and process it in a way that it is more useful ... in a format that is compatible with later importing to OSM. Every thing can be improved right ? To stay in topic, my point is that letting the local community have some final decision, and possibility to contact in his language, the owner of the offending changeset, not only will the error be better understood by the community, but the author is also less likely to close the OSM dors with frustration. Like it might well happen with this user after beeing blocked : http://www.openstreetmap.org/user/dd40cestmoi He went on our forum to present himself, to talk, to say he will ask questions, then was sent an english email and then blocked by pnorman and that guy never appeared again even to ask us why he couldn't edit. (maybe it's only temporary, but that guy let a terrible mess behing that he couldn't correct because he was blocked) I'll be contacting him soon to try to keep him (are we not a community ?) and aked him if he could clean the mess, or understand why it led to that mess. (For the rest, this is slightly off topic here imho, and this more a matter of how technically improve imports, and I guess the dev list is best suited) -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Hi, On 09/21/12 14:12, sly (sylvain letuffe) wrote: No problems, let's discuss. But while we do talk about a future rule, the previous one should (I mean must) still apply until the new one is ready to replace it. This is not about one rule. This is about the whole question of rules and authority. No need to say what was the previous rule right ? You mean the previous rule as in yesterday? Half a year ago? Two years ago? Or back when we had nodes and segments in our data model ;) The current situation is that DWG does their job as they see fit and defines rules they think are necessary. For example: We do not have a rule in OSM that says you must use a changeset comment, and we don't have a rule that says you must reply when other mappers send you messages. It's good style to do it but there's no rule that you *must*. Creating rules for these situations would be awkward - it would raise all kinds of questions like what exactly counts as a reply and so on. And it would also sound like contributing to OSM was a major problem because there are so many rules. So we don't have any. However, every once in a while DWG gets a complaint about a particular user making lots of edits that are questionable. Not outright vandalism or edit warring, but something exotic enough to make other mappers in the area uneasy. The other mappers watch the user in question but it is hard to watch him because all his changeset comments are just small fixes. The other mappers try to contact the user but he never replies. In cases like this, I have occasionally told the mapper in question that OSM is a teamwork project, and that he must be a teamplayer and communicate with his peers, else we cannot use his work even if it is good. I have occasionally had to put a block on people like that in order to get them to reply at all. Now there's no written rule for this. If the guy started a thread on the talk list about where is it written that you need to respond to emails? I would not even be able to point to a wiki page - it's simply something that we take for granted. The separate account rule is just such a rule, that DWG has created to do their job. I will not continue discussing this: As long as DWG have to clean up the mess they will make the rules governing imports and mechanical edits. Exceptions from the rules can be negotiated with DWG in advance if someone thinks they really need one. I say as long as... because the subsidiarity I mentioned in my post is a real possibility; if the French community has a couple of willing and capable people maybe we could experiment with setting up a sub-DWG responsible for France only. Maybe we should just try it out and see if it improves the situation. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Fri, Sep 21, 2012 at 3:40 PM, Frederik Ramm frede...@remote.org wrote: As long as DWG have to clean up the mess they will make the rules governing imports and mechanical edits. Exceptions from the rules can be negotiated with DWG in advance if someone thinks they really need one. Thanks Frederik for this clear statement. This is also different from the previous replies from the DWG where it was just said The DWG follows the guidelines defined by the community.. It is also clear that the French community never went to the DWG asking them to clean up the French cadastre mess. This is something we already did ourselves in the past. As already said, we still need the DWG to block someone when required. And this is also something we did in the past and will continue to do. I say as long as... because the subsidiarity I mentioned in my post is a real possibility; if the French community has a couple of willing and capable people maybe we could experiment with setting up a sub-DWG responsible for France only. Maybe we should just try it out and see if it improves the situation. Very happy to see some progress here. We have Christian who already applied for this role but we don't know if he received some answer so far. Another thread showed that recruiting a DWG member works by co-optation. Perhaps for some unknow reasons, you don't want him for this try in which case, you could select another one or we can call for other applicants. Pieren ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 20 September 2012 08:02, Greg Troxel g...@ir.bbn.com wrote: I'm mostly a lurker in these discussions, and generally more pro-import than many who participate in import decisions. But I find the 'separate account for import' to be an utterly reasonable (along with the rest of the guidlines), easy to follow rule, and I am boggled by the objections to it. I haven't followed all of this thread, but here's my experience with this rule or recommendation. First of all setting the username through which you're uploading your edit is such a small issue that it doesn't really matter for the person uploading. But then I don't see it as solving any problem compared to source= tagging either on objects being uploaded, or changeset (often the granularity provided by tagging entire changesets is completely unpractical and would result in more than 50% false positives). Secondly I don't see it as an overwhelming trend currently in OSM. Thirdly it introduces the problem of how many import accounts to use, what to name them and potential anonymity of the person uploading the changes if the account name doesn't contain their nickname. In the Spanish community there has been a strong will to follow all the import guidelines when the Corine Land Cover dataset was being discussed, analysed and prepared for importing. The import guidelines wiki page gave everyone the idea that it would be best to use a single collective account with the same login details used by all the people participating. It's now obvious that this wasn't a good idea because it was difficult to contact the person who did the actual work in case there was a need for discussion, on top of that there's the practical problem of sharing login details. As with most imports there's days or weeks (sometimes months) of manual processing that needs to be done before data is ready for upload to OSM, and this is done by a real person. I think the whole point of having accounts in OSM is for the people uploading their work to be easily contactable. Fast forward two years and the current (lasting for about a year now) Spanish cadastre discussions and import attempts have an even stronger push to follow all the import guidelines because the DWG has blocked these import attempts on various occasions (which from my point of view is continuing to damage OSM in Spain because mappers are left in a limbo -- there's no point drawing building outlines in their towns from imagery if they have a better source at hand). Well, this time a single import account has been registered per province with a single person coordinating the (potential) imports in each province. The assignments have been documented on the wiki. This is better but the account names are still not directly linked with real people, and the division by provinces is artificial because the data was supposed to be uploaded by users only for the areas they know personally, which may be on village level for example. Cheers ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On vendredi 21 septembre 2012, Frederik Ramm wrote: Hi, Hi again, This is not about one rule. This is about the whole question of rules and authority. No problems, let's also talk about rules and authority. But we (french community) are facing one problem right now, not problems, one problem, and this problem appeared one month ago. Are you asking us to let go with the only reason that this will probably, one day, be solved by a new document we are secretly discussing so please wait ? and accept, that, during all this discussion M. Norman (from who I have much respect for his volounteer work of tracking and stopping vandalism) is still blocking users of our community : http://www.openstreetmap.org/user_blocks/248 because that poor guy doesn't read english, was following what we've always done. With, I admit an terrible error that will also be detected by our own radars and that I have starting to discuss the issues with him ? No.. maybe too late.. that guy might be gone because the DWG doesn't played it nice with him. Have we lost a vandal ? have we lost a member ? I couldn't tell, nor the DWG can. Is that building communities ? No need to say what was the previous rule right ? You mean the previous rule as in yesterday? Half a year ago? Two years ago? Or back when we had nodes and segments in our data model ;) Nice try ;-) But I said previous and rule. We now have a de facto rule about a mandatory second import account right ? because if we don't comply, we are blocked, I call this a rule. What we had to that *previous* rule about mandatory second import account ? Nothing, we had nothing because it wasn't a rule but a strong recommandation. Therefore, the previous rule to that is no rules. No matter how far you go back, dinosaures didn't have any special rules about mandatory import accounts. The current situation is that DWG does their job as they see fit and defines rules they think are necessary. I've seen that. And I do think I understand why it is so : volonteers, lack of time, no time for talking and lack of massive complaints. Therefore, a quick law was created so that vandals won't argue they haven't been warn and make it faster for quick unarguing bans with as a result, limited numbers of imports, wich means less radar alerts over the xk nodes barrier. That it is enough as a first step if complaints are low, because you are probably facing a vandal arguing for himself. We are not in this case, we have found what we considere a little flaw in this procedure and a large part of a community has confirmed that. Your missing volonteers ? we have As it been proven that accepting the amendment I'm proposing causes harm to the project ? no what's left ? I have no other clue than maybe pride forcing the DWG not to accept a step back. So we don't have any. (rules) Forcing someone to do something arguing there is a text somewhere that says so is a rule, or we have missunderstandings... And I don't argue against all rules, of course not ! we need rules. Blocking a mass good data deleting user is good, and should be written as a rule, and enforce as such. Now there's no written rule for this. If the guy started a thread on the talk list about where is it written that you need to respond to emails? I would not even be able to point to a wiki page - it's simply something that we take for granted. I do agree that we need flexibilty above rules (that's what judges do), and accept that it may have some collateral dommage sometimes. But what if 50 people comes to you saying that responding to emails is not writent anywhere ? will you still ignore them and continue answering and loosing time ? Well I guess no, time for rules. Accepted rules of course. The separate account rule is just such a rule, that DWG has created to do their job. I will not continue discussing this: As long as DWG have to clean up the mess they will make the rules governing imports and mechanical edits. Exceptions from the rules can be negotiated with DWG in advance if someone thinks they really need one. You don't want to discuss that ? If the core of the problem is that the DWG has no time to clean up the mess, therefore is creating rules. That's where we should work. What mess ? the data mess ? I say as long as... because the subsidiarity I mentioned in my post is a real possibility; if the French community has a couple of willing and capable people maybe we could experiment with setting up a sub-DWG responsible for France only. Maybe we should just try it out and see if it improves the situation. Good ! let's try. What we have to lose by trying ? some complaining users having been blocked because they have done bad imports without respect of general guidelines and france's cadastre special guidelines ? That shouldn't change much from now ;-) beside understanding why they've been blocked... -- sly qui suis-je : http://sly.letuffe.org email
Re: [OSM-talk] Import guidelines proposal update
sly (sylvain letuffe) wrote: http://www.openstreetmap.org/user_blocks/248 because that poor guy doesn't read english, was following what we've always done. Hang on - they've been editing since 5th September, it's just over two weeks later; their changeset 13180810 contains 21976 nodes and they've imported some buildings in error 4 times? Surely they're exactly the sort of person who needs to be told whoa horsey! and given a suggestion that they have a chat with someone in the local community? Cheers, Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
2012-09-20 Frederik Ramm However, every once in a while DWG gets a complaint about a particular user making lots of edits that are questionable. Not outright vandalism or edit warring, but something exotic enough to make other mappers in the area uneasy. The other mappers watch the user in question but it is hard to watch him because all his changeset comments are just small fixes. The other mappers try to contact the user but he never replies. In cases like this, I have occasionally told the mapper in question that OSM is a teamwork project, and that he must be a teamplayer and communicate with his peers, else we cannot use his work even if it is good. I have occasionally had to put a block on people like that in order to get them to reply at all. This is not only a question of guidelines. And the DWG role is more of last intervention when the community was not able to discuss with mappers and correct the situation. The DWG work would be facilitated if communications were developped with local communities and first contacts made by these local communities. This would also contribute to develop more experienced and responsible mappers. To my point of view, it is essential to favorize development of local communities, to empower these communities with tools adapted to them. Pierre De : Frederik Ramm frede...@remote.org À : talk@openstreetmap.org Envoyé le : Vendredi 21 septembre 2012 9h40 Objet : Re: [OSM-talk] Import guidelines proposal update Hi, On 09/21/12 14:12, sly (sylvain letuffe) wrote: No problems, let's discuss. But while we do talk about a future rule, the previous one should (I mean must) still apply until the new one is ready to replace it. This is not about one rule. This is about the whole question of rules and authority. No need to say what was the previous rule right ? You mean the previous rule as in yesterday? Half a year ago? Two years ago? Or back when we had nodes and segments in our data model ;) The current situation is that DWG does their job as they see fit and defines rules they think are necessary. For example: We do not have a rule in OSM that says you must use a changeset comment, and we don't have a rule that says you must reply when other mappers send you messages. It's good style to do it but there's no rule that you *must*. Creating rules for these situations would be awkward - it would raise all kinds of questions like what exactly counts as a reply and so on. And it would also sound like contributing to OSM was a major problem because there are so many rules. So we don't have any. However, every once in a while DWG gets a complaint about a particular user making lots of edits that are questionable. Not outright vandalism or edit warring, but something exotic enough to make other mappers in the area uneasy. The other mappers watch the user in question but it is hard to watch him because all his changeset comments are just small fixes. The other mappers try to contact the user but he never replies. In cases like this, I have occasionally told the mapper in question that OSM is a teamwork project, and that he must be a teamplayer and communicate with his peers, else we cannot use his work even if it is good. I have occasionally had to put a block on people like that in order to get them to reply at all. Now there's no written rule for this. If the guy started a thread on the talk list about where is it written that you need to respond to emails? I would not even be able to point to a wiki page - it's simply something that we take for granted. The separate account rule is just such a rule, that DWG has created to do their job. I will not continue discussing this: As long as DWG have to clean up the mess they will make the rules governing imports and mechanical edits. Exceptions from the rules can be negotiated with DWG in advance if someone thinks they really need one. I say as long as... because the subsidiarity I mentioned in my post is a real possibility; if the French community has a couple of willing and capable people maybe we could experiment with setting up a sub-DWG responsible for France only. Maybe we should just try it out and see if it improves the situation. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On vendredi 21 septembre 2012, SomeoneElse wrote: sly (sylvain letuffe) wrote: http://www.openstreetmap.org/user_blocks/248 because that poor guy doesn't read english, was following what we've always done. Surely they're exactly the sort of person who needs to be told whoa horsey! and given a suggestion that they have a chat with someone in the local community? I totally agree, and that's what I've done today (In a language I'm sure he understand because we allready spoke on our forum) But more than that, I've asked him why he has done it this way, what exactly happen, what procedure he has followed and if he needs help to sort the mess. Not only will this, hopefully, convince him not to leave but will also help our community to improve our documentation, our guidelines, our integration procedure... or to sharpen our blades (kind of joking as it needs to be the last resort) -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
andrzej zaborowski wrote: Well, this time a single import account has been registered per province with a single person coordinating the (potential) imports in each province. The assignments have been documented on the wiki. This is better but the account names are still not directly linked with real people, and the division by provinces is artificial because the data was supposed to be uploaded by users only for the areas they know personally, which may be on village level for example. To my eyes that provides a perfect base to work from, but if you have not been following the thread ... What I have been asking is how we can manage on-going imports of a dataset that is being updated regularly. This is probably on 'off-line' function, and could well be managed by the 'local chapter' on their own computers. This is the 'process' I'm looking to be developed, so that the raw import data is held in a format that later imports can be compared against, and only differences then get further procession. Breaking this process down into provinces, and importing the pre-processed RAW data via an import account gives us a clean base which mappers can then work against and improve the data ... and changes to the 'imported' data would then be mirrored back to the staging process. Seeing that an element is version 1 by the import user immediately tells you that it may need additional local information adding ( we need to be able to see who last edited an object! ). Where the import HAS nice unique object identifiers things are a lot easier, but raw vector data like the French import, and I think the Spanish data you are talking about CAN still be 'diffed' against earlier imports, and result in perhaps new data that can simply be imported, or perhaps an overlay that identifies conflicts that need a human eye. Isn't it better to spend time working out a GOOD way of using the data going forward rather than having to manually merge the whole lot again in a couple of years time ... and every couple of years. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
From: sly (sylvain letuffe) [mailto:li...@letuffe.org] Sent: Friday, September 21, 2012 7:41 AM To: talk@openstreetmap.org Subject: Re: [OSM-talk] Import guidelines proposal update On vendredi 21 septembre 2012, Frederik Ramm wrote: Hi, Hi again, This is not about one rule. This is about the whole question of rules and authority. No problems, let's also talk about rules and authority. But we (french community) are facing one problem right now, not problems, one problem, and this problem appeared one month ago. Are you asking us to let go with the only reason that this will probably, one day, be solved by a new document we are secretly discussing so please wait ? and accept, that, during all this discussion M. Norman (from who I have much respect for his volounteer work of tracking and stopping vandalism) is still blocking users of our community : http://www.openstreetmap.org/user_blocks/248 because that poor guy doesn't read english, was following what we've always done. With, I admit an terrible error that will also be detected by our own radars and that I have starting to discuss the issues with him ? A 0-hour block is largely placed to make sure that the message is read. They go away when someone logs in and reads the message. If you can suggest an email that I can cc initial messages to for follow up I could do so. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
2012-09-21 Lester Caine What I have been asking is how we can manage on-going imports of a dataset that is being updated regularly. This is probably on 'off-line' function, and could well be managed by the 'local chapter' on their own computers. This is the 'process' I'm looking to be developed, so that the raw import data is held in a format that later imports can be compared against, and only differences then get further procession. Breaking this process down into provinces, and importing the pre-processed RAW data via an import account gives us a clean base which mappers can then work against and improve the data ... and changes to the 'imported' data would then be mirrored back to the staging process. Seeing that an element is version 1 by the import user immediately tells you that it may need additional local information adding ( we need to be able to see who last edited an object! ). Where the import HAS nice unique object identifiers things are a lot easier, but raw vector data like the French import, and I think the Spanish data you are talking about CAN still be 'diffed' against earlier imports, and result in perhaps new data that can simply be imported, or perhaps an overlay that identifies conflicts that need a human eye. Isn't it better to spend time working out a GOOD way of using the data going forward rather than having to manually merge the whole lot again in a couple of years time ... and every couple of years. In Canada, Natural Ressources Canada, the national mapping agency is collaborating with OSM, producing OSM import files from is topographic database Canvec. The OSM collaborators are following a procedure to carefully integrate this data into OSM. NRCan compared recently Osm and Canvec data for planning road network update field work for Canvec. They also provided this helpful information to the OSM community with detected differences. see http://lists.openstreetmap.org/pipermail/talk-ca/2012-July/004934.html I think that this shows that even without an unique ID, it is possible to develop monitoring tools of imports. The fixme attribute is used to monitor differences between the two databases. The Fixme Highlight Warnings style, in JOSM, offers the possibility to monitor database discrepancies. Pierre De : Lester Caine les...@lsces.co.uk À : OSM talk@openstreetmap.org Envoyé le : Vendredi 21 septembre 2012 17h33 Objet : Re: [OSM-talk] Import guidelines proposal update andrzej zaborowski wrote: Well, this time a single import account has been registered per province with a single person coordinating the (potential) imports in each province. The assignments have been documented on the wiki. This is better but the account names are still not directly linked with real people, and the division by provinces is artificial because the data was supposed to be uploaded by users only for the areas they know personally, which may be on village level for example. To my eyes that provides a perfect base to work from, but if you have not been following the thread ... What I have been asking is how we can manage on-going imports of a dataset that is being updated regularly. This is probably on 'off-line' function, and could well be managed by the 'local chapter' on their own computers. This is the 'process' I'm looking to be developed, so that the raw import data is held in a format that later imports can be compared against, and only differences then get further procession. Breaking this process down into provinces, and importing the pre-processed RAW data via an import account gives us a clean base which mappers can then work against and improve the data ... and changes to the 'imported' data would then be mirrored back to the staging process. Seeing that an element is version 1 by the import user immediately tells you that it may need additional local information adding ( we need to be able to see who last edited an object! ). Where the import HAS nice unique object identifiers things are a lot easier, but raw vector data like the French import, and I think the Spanish data you are talking about CAN still be 'diffed' against earlier imports, and result in perhaps new data that can simply be imported, or perhaps an overlay that identifies conflicts that need a human eye. Isn't it better to spend time working out a GOOD way of using the data going forward rather than having to manually merge the whole lot again in a couple of years time ... and every couple of years. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http
Re: [OSM-talk] Import guidelines proposal update
From: Lester Caine [mailto:les...@lsces.co.uk] Subject: Re: [OSM-talk] Import guidelines proposal update who last edited an object! ). Where the import HAS nice unique object identifiers things are a lot easier, but raw vector data like the French import, and I think the Spanish data you are talking about CAN still be 'diffed' against earlier imports, and result in perhaps new data that can simply be imported, or perhaps an overlay that identifies conflicts that need a human eye. Isn't it better to spend time working out a GOOD way of using the data going forward rather than having to manually merge the whole lot again in a couple of years time ... and every couple of years. My thoughts on how to handle this for data with persistent unique identifiers without adding those as tags is to a. Record the correspondence between source ID and temporary pre-upload negative OSM ID b. Record the correspondence between pre-upload negative OSM ID and OSM ID c. Combine for a correspondence between source ID and OSM ID, and save this d. When updating, identify objects that have changed or been added to the source e. For changed or deleted objects if the OSM object was last edited by the importer's import account, upload a new version reflecting the changes. Objects that have been edited by a person will require manual intervention, like now f. Handle new objects like before g. Identify objects deleted in OSM and check these, then submit corrections to the source. The one case this doesn't handle very well is POIs that have been changed from a node into a way. I'm going to be working on implementing this in a limited way for updating addresses locally. Addresses are different because the address should be unique in the city. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 09/19/2012 10:55 PM, Richard Weait wrote: [1] Of course, I don't mean you personally, Jean-Marc. I have no idea of your OSM screen name, if you are a Cadastre importer or if you use an import account. I mean those who have been knowingly ignoring the import guidelines. I was not integrating Cadastre data, until a couple of days ago when I did a couple of villages just to make sure that I understand the problem well enough and confirm that it does take manual work to do it right. One of the possible outcomes of the current debates is that the current rules are just fine as they are. But even if things turn out that way, the occurrence of this debate shows that not everyone was convinced. That is why I offered to clarify the consensual reasons for the rules, so that the rules become evident to all - including the current skeptics; or that they are modified to fit those consensual reasons. In your message, you explain in detail the profile of some of the dissenting editors and some of the explanations given by those who just find the rules inconvenient. Indeed some of the dissent is antisocial and must be controlled, and most of the inconvenience explanations can be addressed technically. But that still does not make clear for everyone the reasons for those rules. Many people on this list have been collaborating for long enough to have mutually adjusted and internalized a set of implicit values that represent OSM culture. But that culture is not entirely homogeneous, especially since linguistic barriers weaken the links with large clumps of users. So we need debate, to resolve the differences or find that there are no differences after all. We cannot do that with implicit values : we have to make them explicit. Today the rules are explicit, but not the reasons that underlie them. Expliciting can be tedious and provoke annoying nitpicking, but I fear that dissent will recur as long as the dissenters don't feel that they have been made part of an inclusive process that grounds the rules in mutually agreed values. So let's enumerate the whys of the hows. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Hi, On 09/19/2012 04:24 PM, sly (sylvain letuffe) wrote: I've read the rather long thread Import guidelines OSMF/DWG governance and I'd like to propose a change on the wiki page : http://wiki.openstreetmap.org/wiki/Import/Guidelines I think that imports, or all automated edits, have multiple aspects. Some of them can be covered by local or national policies, others cannot. For example, I think that a local or national agreement would usually have the last word about what data is imported (maybe except in some extreme cases where the whole community suffers because someone imports every single cobblestone in a city but that's theoretical). But besides the content aspect, there's also the technical or procedural aspect - things like where and how to document your import, or whether or not you need a separate import account, or whether it is acceptable to do large-scale imports with an account the name of which signals disdain for the project. I don't think these should be decided locally. In the coming years we'll hopefully develop a good subsidiary structure with local chapters in all major communities, and I would expect that this will also bring a healthy discussion about what the local chapters can decide by themselves and what not. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Frederik Ramm wrote: But besides the content aspect, there's also the technical or procedural aspect - things like where and how to document your import, or whether or not you need a separate import account, or whether it is acceptable to do large-scale imports with an account the name of which signals disdain for the project. I don't think these should be decided locally. Seconded There are perhaps three separate discussions here ... 1 - How fine a detail should be allowed? 2 - Is the style of raw imported data acceptable to OSM? 3 - Do we need to be able to identify a raw import? The first is more of a problem than the other two? People mapping at a macro level only using the same was as the road, boundary, edge of building, and so on make it difficult for those of us who are now adding the footpaths between that road and building. And some will always oppose adding some types of data - such as building. I have no problem with adding the coble stones but as an area tagged such, which may actually be the road! The automatic reduction of that area to a way for routing is another matter? The second links to the first when we import a course dataset and it needs to be reworked to fit into the OSM 'guidelines'. It may be preferable NOT to import the raw data, but provide it as an overlay for tracing? Or rework the raw data prior to import to a suitable format. The third then becomes a matter of 'is this the same data that as provided by the raw import'. Personally I think that identifying an element against a unique_id from the source data SHOULD be the standard, so that hopefully in years to come we can simply automatically scan a new dataset and flag everything that has changed? That includes objects that have disappeared! We then need to be able to identify those items that have not been modified (point 3) and update them if necessary. And those that have (point 2) so we can provide a 'manual merge' list. The 'separate account' was a crude attempt to provide a short term fix for 2/3 until a proper solution was put in place, and currently is still the best way to identify things until a more rugged solution is provided - centrally! -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 10:55 PM, Richard Weait rich...@weait.com wrote: I've combined their responses and made them generic. I take my turn to combine your arguments: - Too hard to register with another email. I say use username+osmimp...@yourisp.com if they support it. Alternatively, those concerned in the French community can surely offer their members additional email accounts to support their community. - Uploading other contributor's work is probably breaking another guideline. Using a single separate user account or a proxy user for all users has been already suggested. It would comply with the import guideline but we don't want that. - I don't want to change account settings in JOSM. I say start josm with alternate josm.home directory with your saved credentials. Like: java -Xmx2038M -Djosm.home=/home/username/import -jar /home/username/bin/josm-latest.jar - in Europe, the trend is to open more and more public geodata. It's usual to find contributors uploading external data from 2, 3 or 4 different sources. Each will require a different user account, a different email address and a different JOSM preference file. Each time you change something in your preference, you will have to repeat it in all your homes. - Cadastre is not an import. Cadastre is an import. Could you do the same thing if there were no Cadastre to import? No, - Untrue. The cadastre is also available as WMS. We started by tracing manually over raster images. I guess what UK users are doing today with OS buildings, we did it in the past until we were able to retrieve the vector data. - Cadastre is different; I am careful before I upload. All mappers are careful don't insult the rest of the community. :-) Fixing and reconciling data before upload is the obligation you have when contributing. Cadastre is still an import. - I agree. Cadastre is an import, but not a blind, automated, large scale import done after conflation on a GIS application. - No. I want credit for all of my mapping statistics all in one place. Simplest to fix. UserStat now allows you to combine stats from a group of your accounts. - nobody came here for statistics. Pieren ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Hi, De : Michael Kugelmann On 19.09.2012 23:45, Vincent de Chateau-Thierry wrote: The only criteria for removing French Cadastre data will be the value of the source tag. That's a bad idea: if someone for what ever reason just decides to remove (or change) the source=cadastre tag of a object (and don't change anything else) you can't identify the object any more. Or to be more precise: you need to use a lot of effort and check all versions of an object (this means: the whole planet) whether it once had the source=cadastre tag. But thats a lot of work to do. Much (!) more easy to identify all the object is if you can take all object created by a special account = just check the changesets. Checking all versions of all objects is the thing we just went through with a lot of pain and effort: creating and using the redaction bot. And we are all aware that this was necessary but not nice and a lot of unporoductive effort. So let's learn from the past and avoid possible issues in the future that can be done easily with very small aditional effort in the presence. That means that a separate ccount is a way of identifying contributions depending on their source. I can understand that such way is a good practice when a given source is strongly linked to a way of dealing with it : massive upload or single edit. But the problem with french cadastre remains the same since it is both used in massive _and_ single object uploads. In my previous mail I pointed out a way : http://www.openstreetmap.org/browse/way/181674272 This way has one of its nodes which stand for a amenity=cafe : http://www.openstreetmap.org/browse/node/1920879534 Both geometry of the way and tags for the node come from the same changeset, since I added them at the same time. But node information does not come from the Cadastre, it is my own survey. With the separate-account-by-source suggestion, I would have : 1- started JOSM with my account-for-import (which does not exist yet :-) ) 2- drawn the way, tagged with building=yes and source=Cadastre 3- uploaded it 4- exit from JOSM and restart with my regular account 5- edited the node tags 6- uploaded Wow I don't think such process is productive. It is artificialy time consuming with basically no gain at all. vincent Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ? Je crée ma boîte mail www.laposte.net ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
I'd like to propose a change on the wiki page : http://wiki.openstreetmap.org/wiki/Import/Guidelines I think that imports, or all automated edits, have multiple aspects. Up to that point, we fully agree. there's also the technical or procedural aspect (...) I don't think these should be decided locally. And that's where we disagree. Your are not accepting any distinction about all different cases in your statement, and it seams you are implicitly denying the ability of the local community to decide wisely. What I miss is trust, I don't think we can build a world community of local communities if no trust is transfered to local communities (or is it a local chapter ? I have no clues about what differences there are) There is a big diplomatic difference between : We don't belive your local community is wise enough, so we decide of technical and procedural aspects for you and block your users if they don't follow this guideline and We trust your self governance, here is the key to block your own members, we are here to back you up in case of emmergency, please designate x representative of your community we can talk to, and here are the guideline we wish you enforce respect to your members In the coming years we'll hopefully (...) discussion about what the local chapters can decide by themselves and what not. Any reasons to wait for years ? That's exactly the discussion we are having now about a real case need, we have sent a representative of our community we trust, I have a proposal for the first rule to be discussed and agreed upon. -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
sly (sylvain letuffe) wrote: there's also the technical or procedural aspect (...) I don't think these should be decided locally. And that's where we disagree. Your are not accepting any distinction about all different cases in your statement, and it seams you are implicitly denying the ability of the local community to decide wisely. sly - take a step back. The 'mechanisms' that we use MUST be managed centrally, along with the core software, and it is this mechanism that is currently BROKEN when handling imported data? We do not have a robust process in place world wide so we don't want groups running creating their own isolated processes? Now I have no doubt there are some clever people in every local workgroup who can take their own data sources and manipulate them in a way that can then be imported into the main database. There are no objections to that. Some imports will be geo-referencing new raster layers and there is no dispute about that process, but when it comes to 'importing' raw data there are big holes in the process world wide which still need plugging rather than local groups plouging on down their own 'agenda'. Now if there is no interest in supporting a central mechanism to work towards AUTOMATICALLY importing LOCALLY processed data then so be it. Go on wiping and reloading every time the source data is updated and manually merging everything. I happen to think that is the wrong way of doing it, but in the case of the French data I don't have the information to suggest anything else :( In the case of the UK data we know how, we are just not allowed to yet double :( -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
The 'mechanisms' that we use MUST be managed centrally, What are you talking about ? What mechanisms are you refering to ? and it is this mechanism that is currently BROKEN when handling imported data? Are you talking about the mechanism that the dwg is blocking users not using a dedicated account for any third changeset over 10k objects wich looks like an import to them ? Well, I won't use such a word as broken since it has proven usefull for several cases to prevent and detect vandalism, but I'll be glad to use the world not optimal and to be improved We do not have a robust process in place world wide so we don't want groups running creating their own isolated processes? Sorry to say it again, but I don't understand you, could you be more precise with a specific example ? what robust process are you talking about ? which isolated processes ? -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 20/09/2012 13:18, Lester Caine wrote: Go on wiping and reloading every time the source data is updated and manually merging everything. Sounds ugly doesn't it ? Because it is. Wouldn't it be much better if each building from the cadastre had a UUID that could be traced so that differential imports could be performed with little disturbance and little manual work ? Yes. But sadly that is not how the French cadastre works : it is just a bunch of georeferenced images. So the user of cadastral data has to repair the buildings split where a cadastral plot limit is drawn across, check for proper geographic referencing using GPS traces, imagery and geodesic reference points, expunge the data that describes buildings that are already in OSM, check the general sanity of the data, remove the occasional artefacts... I don't like it either - it is a lousy cadastre but that's the only one we have. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Jean-Marc Liotier wrote: On 20/09/2012 13:18, Lester Caine wrote: Go on wiping and reloading every time the source data is updated and manually merging everything. Sounds ugly doesn't it ? Because it is. Wouldn't it be much better if each building from the cadastre had a UUID that could be traced so that differential imports could be performed with little disturbance and little manual work ? Yes. But sadly that is not how the French cadastre works : it is just a bunch of georeferenced images. So the user of cadastral data has to repair the buildings split where a cadastral plot limit is drawn across, check for proper geographic referencing using GPS traces, imagery and geodesic reference points, expunge the data that describes buildings that are already in OSM, check the general sanity of the data, remove the occasional artefacts... I don't like it either - it is a lousy cadastre but that's the only one we have. So would it not be better to provide it as an raster overlay instead? And trace from that. But I was assuming that this was vector data? So it can be processed into a database? I am sure that from version to version they are not going to be changing the coordinates of the majority of buildings? All that raw data can be imported as a layer in OSM, but in addition it can be compared with a previous import and identical elements ignored? That just leaves the changes between versions to be processed, and you end up with a better version of the cadastre data than the government ;) And reference it to the rest of the OSM data. Some of us are playing similar 'tricks' with the UK OS data ... -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
sly (sylvain letuffe) wrote: The 'mechanisms' that we use MUST be managed centrally, What are you talking about ? What mechanisms are you refering to ? Simply the methods by which data is added to the database. And all I am trying to understand now is why if we HAVE digital data to work with for which further versions will be provided over the coming decades someone has to manually check every line every year or so? This data was in the database, so only the changes needed to be posted, but a mistake was made. We learn from mistakes and so what I am trying to learn is if we could have HELPED by reducing the chance of the mistake? By providing tools that take advantage of the data and process it in a way that it is more useful ... in a format that is compatible with later importing to OSM. I know there is something of a 'cultural' thing here and that has some involvement in the recent problems, but at the end of the day we all just want to help, and 'diving for the shelter' does not help. Fresh eyes and computing power can provide an alternative view ... but it would still be nice if we had a core mechanism that said 'this is a raw import from xxx and it's id is yyy'? It's the 'id is yyy' that seems to be the stumbling block with some people? But I currently see no way to develop an automated update process without. I see no reason that even if the raw data has no internal id we can't add that via the import process? I do it all the time with the raw data I'm being supplied, and now the sources are using my id's to improve their end. Unfortunately not usable mapping stuff though. -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
De : Lester Caine So would it not be better to provide it as an raster overlay instead? And trace from that. But I was assuming that this was vector data? French cadastre is vector data in about 70-80% of the 36.000 municipalities. The rest is made of old paper maps turned into pixels and delivered as raster data. Both are available as raster layer in JOSM thanks to the cadastre-fr plugin. For the vector data, it is also available as raw .osm files, split into thematic layers : mainly administrative boundaries and buildings. So it can be processed into a database? I am sure that from version to version they are not going to be changing the coordinates of the majority of buildings? All that raw data can be imported as a layer in OSM, but in addition it can be compared with a previous import and identical elements ignored? That just leaves the changes between versions to be processed, and you end up with a better version of the cadastre data than the government ;) And reference it to the rest of the OSM data. Sure it can be processed. Change detection for buildings is a topic discussed on talk-fr but there is no real tool vailable yet to deal with it. And as said by Jean-Marc, buildings taken from the cadastre as vector parts don't have any ID at all. vincent Une messagerie gratuite, garantie à vie et des services en plus, ça vous tente ? Je crée ma boîte mail www.laposte.net ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Vincent de Chateau-Thierry wrote: For the vector data, it is also available as raw .osm files, split into thematic layers : mainly administrative boundaries and buildings. Sure it can be processed. Change detection for buildings is a topic discussed on talk-fr but there is no real tool vailable yet to deal with it. And as said by Jean-Marc, buildings taken from the cadastre as vector parts don't have any ID at all. My own interest here is more historic than current and I was looking for the development of areas relating to my family tree, but there seems to be a general consensus that once an object ceases to exist it should be deleted from the database. So we need a home to put that data. You have data from 2009? and 2012 for France, so it would be nice to retain all this history as well. This is were 'local' archives may play a roll, and additional servers provide additional layers such as the historic data that has been purged ... or older versions of imports. In specific relation to vector imports, I presume that the cadastre data is 'simply' individual lines? Rather than shapes? So every item currently has it's own ID even if it's only a line number on a list, and comparing the 2009 data with 2012 will produce a list of lines deleted and lines added? 'Hopefully'! Some cleaver-clogs could probably put together a bit of code that links lines where their ends touch, but if you just manually select lines and 'link' them and then add a house number etc. Now we have an ID for those set of lines in the import database and we only touch them again if the raw data changes ... hopefully the geo-referencing has not changed between versions! This is the sort of development we can all go around duplicating or club together and come up with core code that only needs some local filtering to work with a particular import? Isn't it better where we HAVE vector data to make the best use of it, and then spend our time enhancing the details ... like adding road names and house numbers? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Hi, On 20.09.2012 12:29, sly (sylvain letuffe) wrote: And that's where we disagree. Your are not accepting any distinction about all different cases in your statement, and it seams you are implicitly denying the ability of the local community to decide wisely. What I wanted to say is: If the negative effects of a bad decision only, or mostly, affect the local community, then they can be trusted to take these decisions themselves, because they will learn from their mistakes. If the negative effects however affect other/different people - perhaps because they are using the API outside of specifications, or causing more work for people elsewhere in the project - then they can't. Of course, this does not mean that one could not have a system where rules are made centrally but execution of these rules is entrusted to local communities (and only if that doesn't work, someone else will step in). In the coming years we'll hopefully (...) discussion about what the local chapters can decide by themselves and what not. Any reasons to wait for years? I don't think we should *wait* for years, I just believe that it will take a long, long time to get this worked out properly. There are tons of things that need to be at least thought about on the way to a federated OSM project. There are very simple technical things. For example, assume that there was a French DWG dealing only with French cases; we don't have the means to set things up in a way that the French DWG can only block French users. We don't even have a proper definition of local communities and who is entitled to whatever privileges we grant local communities - for example, we recently had an issue in the Crimea which is part of Ukraine but where local mappers would rather not be governed by decisions made by the wider Ukrainian community. So, what if a Toulouse mapper comes to OSMF and complains that OSMF-FR is unfairly suppressing Languedoc self-determination? What if local communities decide stuff that is considered harmful to the project as a whole by someone on the other side of the world? Who would adjudicate such a conflict? Can the world-wide community be called to a vote that is binding for France? Can the French community make a binding rule for Toulouse? How many is a community, anyway? Do they have to be incorporated? Do they have to be democratic? What if a national community - as has been the case in the past with some Eastern European national communities - takes a very liberal attitude towards copyright (the government web page says private use only but they never prosecuted anyone...)? Can a national community make a deal with a sponsor and allow the sponsor to carry the OSM logo? All this has been discussed for years, on and off, when we talked about local chapters. And I expect that it will be another couple of years until we have a structure that works. That's exactly the discussion we are having now about a real case need, we have sent a representative of our community we trust, I have a proposal for the first rule to be discussed and agreed upon. Personally, I don't think you can disregard all the questions I mentioned above and simply make a rule that says a few nice things about a national community which might or might not be well defined in any particular case. I think that your suggestion is too much like case law: There's a rule that leads to a result you don't like, and then you amend it with a little extra rule specifically for that purpose. (In your case, you have built a regional limited import special rule into the separate import account rule, but what if tomorrow the French community decides that they would like to be exempt from something else...?) I think that we need to take quite a few steps back and stop discussing about oh god oh god a respected French OSMer was blocked by evil DWG, where on earth did they get that authority to block him and how can we take it away from them. We should be discussing what rules we need at all, where we don't need any rules, who makes these rules, how local communities come into play there, and all that. This, I believe, takes a lot of time, and real, committed, long-time work by a few individuals who really want to move the project forward, rather than just a quick fix for a particular problem. (Technically, and in the very-long-run, my cloud-nine astronaut vapourware vision is of regional communities operating their own databases and them all to be in some kind of federated system. But that's not something we can decide by a quick wiki poll tomorrow ;) Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09 E008°23'33 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
I believe that dedicated accounts are generally better for imports than using mixed ones which are also used for original data. This really helps a lot in sorting data according to its intellectual properties holders. The only exception I could possibly see is data that doesn't come with strings attached (PD/cc0). Please note that even if you don't simply copy the data from another dataset but do some redactional work on it, it will still remain derived from the original source. cheers, Martin ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Martin Koppenhoefer wrote: I believe that dedicated accounts are generally better for imports than using mixed ones which are also used for original data. This really helps a lot in sorting data according to its intellectual properties holders. Yes, absolutely. The really obvious example of this is the Polish UMP data, which was licensed CC-BY-SA and could not be kept post-licence change. If dedicated accounts had been used, removing this data would have been relatively easy; in reality, it has been (and continues to be) a nightmare. :( So although I understand the motivation behind sly's suggestion that In the case of regionaly limited imports (inside a country), it is highly recommanded to get in touch with the local community to discuss your planned import and ask them if you should, shouldn't or must use a dedicated account, this approach has proved problematic in the past and I would caution against repeating the same mistake. cheers Richard -- View this message in context: http://gis.19327.n5.nabble.com/Import-guidelines-proposal-update-tp5726210p5726241.html Sent from the General Discussion mailing list archive at Nabble.com. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 5:41 PM, Richard Fairhurst rich...@systemed.net wrote: I would caution against repeating the same mistake. I thought that such issue is not possible anymore with ODbl. Pieren ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Pieren wrote: I thought that such issue is not possible anymore with ODbl. No, the Contributor Terms simply say You are indicating that, as far as You know, You have the right to authorize OSMF to use and distribute those Contents under our current licence terms (1a). If the licence changes to one which is incompatible with the import, OSMF may remove Your contributions from the Project (1b)... and that rather requires being able to identify what these incompatible contributions are. cheers Richard -- View this message in context: http://gis.19327.n5.nabble.com/Import-guidelines-proposal-update-tp5726210p5726248.html Sent from the General Discussion mailing list archive at Nabble.com. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 6:06 PM, Richard Fairhurst rich...@systemed.net wrote: Pieren wrote: If the licence changes to one which is incompatible with the import, OSMF may remove Your contributions from the Project (1b)... and that rather requires being able to identify what these incompatible contributions are. So, theoretically, we might have the same issue when tracing from Bing for instance. Should we use a different account for Bing imagery contributions as well, just in case we move later to a licence incompatible with Bing ? Pieren ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 19/09/2012 17:26, Pieren wrote: So, theoretically, we might have the same issue when tracing from Bing for instance. Should we use a different account for Bing imagery contributions as well, just in case we move later to a licence incompatible with Bing ? No, because tracing over Bing isn't the same as importing data. It's producing data from scratch by interpreting an image. There are still legal arguments about this, but there are heavy hints that, providing the TCs of the imagery provider don't forbid it (i.e. Bing doesn't, Google does), no copyright or licence carries across from the imagery to what's traced from it. There are similar issues with using imagery to create data as importing it, but they're related to accuracy, not licensing. -- Jonathan (Jonobennett) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
2012/9/19 Pieren pier...@gmail.com: So, theoretically, we might have the same issue when tracing from Bing for instance. The only obligation when creating data with the help of Bing aerial imagery is that is has to be a non-commercial editor and that you have to contribute back to openstreetmaps.org (which fortunately seems to redirect to openstreetmap.org). Another example would be the PCN in Italy, who gave us (OSM) the right to derive information from their aerial imagery, but they don't require any attribution or similar, so there is not an issue: the data can be used in OSM under whatever license OSM uses. Don't confuse copying data with creating it (with the help of imagery). cheers, Martin ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
The really obvious example of this is the Polish UMP data, which was licensed CC-BY-SA and could not be kept post-licence change. If dedicated accounts had been used, removing this data would have been relatively easy; in reality, it has been (and continues to be) a nightmare. :( Although I agree we shouldn't forget the past to avoid repeating the same mistakes but since every cases beeing different, I'm proposing to let local communities decide what they think is good for them. Still, I believe every local community is happily discussing it internationaly to know and learn from other (The french community was, is, and I believe will), but I wish to correct those guidelines to reflect the entrusted power we let in local community's hands. The french community wants a bit of autonomy and be able to block it's own abusing members regarding the cadastre import and following it's own guidelines without interferance for third party that we have considered harmfull both to our community and to the data in the end. this approach has proved problematic in the past and I would caution against repeating the same mistake. I hear you well, this is good to remember, we have thought about all that, but still, we have decided otherwise. Since I'm not a native british speaker, and although I had access on this very list to a special what brit says, brits might think otherwise could you translate me your I would caution against into something related to my proposed change ? - Are you agreeing to self determination of local communities about imports ? - Are you agreeing to the above change of the wiki while still thinking communities should really really use a dedicated account, but, ultimetly could choose not to ? Or, to make it even clearer, can I commit my change to the wiki without starting an edit war with anyone, and if no, with what type of wording change could it be commited ? -- sly qui suis-je : http://sly.letuffe.org email perso : sylvain chez letuffe un point org ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 12:06 PM, Richard Fairhurst rich...@systemed.net wrote: Pieren wrote: I thought that such issue is not possible anymore with ODbl. No, the Contributor Terms simply say You are indicating that, as far as You know, You have the right to authorize OSMF to use and distribute those Contents under our current licence terms (1a). If the licence changes to one which is incompatible with the import, OSMF may remove Your contributions from the Project (1b)... and that rather requires being able to identify what these incompatible contributions are. More likely, and more often, what happens is that a well intentioned mapper uses a source for which he believes he has permission. Imports, then finds out that he didn't have (sufficient) permission for the current license. This has happened many times. Volunteer to discuss this if you've done it yourself, but I'm reluctant to point out your mistakes unless provoked. :-) What has also happened, when cleaning up the inappropriate import, is that the user mixed the import with their regular account. If time has passed since the import, they often have failed memory of the changesets involved, etc. it becomes more of a mess to clean up. So use a separate account for imports. And follow the other guidelines as well. The guidelines make for a better OSM, even if it is slightly less convenient. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 19/09/12 at 16:24 +0200, sly (sylvain letuffe) wrote: Hi, I've read the rather long thread Import guidelines OSMF/DWG governance and ^^ Note that the use of the term guidelines is problematic by itself. Either they are *guidelines*, that is, things that people SHOULD follow, but it's OK (but not recommended) not to follow them. Or they are *rules*, that is, things that people MUST follow. If the DWG blocks accounts based on *guidelines*, I think that they should be renamed to *rules*. Lucas ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Sep 19, 2012, at 12:06 PM, Richard Fairhurst rich...@systemed.net wrote: Pieren wrote: I thought that such issue is not possible anymore with ODbl. No, the Contributor Terms simply say You are indicating that, as far as You know, You have the right to authorize OSMF to use and distribute those Contents under our current licence terms (1a). If the licence changes to one which is incompatible with the import, OSMF may remove Your contributions from the Project (1b)... and that rather requires being able to identify what these incompatible contributions are. I am much in favor of clarifying this passage. It's not even clear what's compatible with ODbL means. We should flat out only allow data to be contributed that - is your own - someone else entrusted to you to be contributed (i. e. a public express permission to contribute someone else's data that is otherwise licensed differently) - only requires attribution but is otherwise open (i.e.: no share-alike data). Public domain data would be a good example. These are btw the principles I personally follow and that many mappers that I talk to recommend. Everything else is grey area and potentially puts OSM into an extremely unflexible position in regards to future adjustments to licensing terms. cheers Richard -- View this message in context: http://gis.19327.n5.nabble.com/Import-guidelines-proposal-update-tp5726210p5726248.html Sent from the General Discussion mailing list archive at Nabble.com. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk Alex Barth http://twitter.com/lxbarth tel (+1) 202 250 3633 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 6:41 PM, Martin Koppenhoefer dieterdre...@gmail.com wrote: The only obligation when creating data with the help of Bing aerial imagery is that is has to be a non-commercial editor and that you have to contribute back to openstreetmaps.org The only obligation to use cadastre data is to provide credits and the year of the cadastre data (you can use older versions if you like). The cadastre data can be used in OSM under whatever license OSM uses. Pieren ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Richard Weait wrote: So use a separate account for imports. And follow the other guidelines as well. The guidelines make for a better OSM, even if it is slightly less convenient. I suppose if a mapper is ONLY using import data then they only need one account? As long as it is flagged as being an account with data based on a single import source? I know that it was identifying where data came from that initiated the use of a separate identified account for that data, so 'Marcussacapuces91' failing to identify that he IS importing data is still a problem in my eyes ... giving the local community some of their own autonomy should not allow them to rubber stamp changes that remove this simply courtesy to other mappers to identify 'importing' over 'mapping'? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 6:51 PM, Richard Weait rich...@weait.com wrote: requires being able to identify what these incompatible contributions are. : What has also happened, when cleaning up the inappropriate import, is that the user mixed the import with their regular account. I agree with that. We can specify in the guideline : Provide by any means the possibility to separate imported data and regular contributions. Of course, it is better if the identification can be automated (required for a massive deletion in case of licence change) in which case a standard source tag is more appropriate than a vague credit in a plain text user account description. Pieren ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Before we go further with policy edits, perhaps we should make sure that everyone understands the goals and that there is a consensus about them... That will make the resulting rules or guidelines more acceptable. Let's focus on the item that triggered the current debate : the requirement for a separate account when imports are involved. From what I have understood in the recent threads (please correct - or lets put that in some wiki space), the reasons for requiring a separate account when imports are involved are as follow : - Identify the import as an import (itself a sub-class of mass edits) to let moderators focus on that class of potentially widely damaging changes. - Provide context about the import, such as a link to a page clarifying license, attribution, quality and methodology issues specific to the source of data. - Let moderators easily identify all edits derived from a particular source of data, so that they can be processed or reverted if grievous license issues occur with that source. - Let moderators easily identify all edits by a particular contributor, so that they can be processed or reverted if grievous quality issues occur with that contributor. Are there other goals that I have missed ? Once there is agreement on the goals, I believe that we'll find it easier to agree on the means. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 20 September 2012 00:41, Richard Fairhurst rich...@systemed.net wrote: Martin Koppenhoefer wrote: I believe that dedicated accounts are generally better for imports than using mixed ones which are also used for original data. This really helps a lot in sorting data according to its intellectual properties holders. Yes, absolutely. The really obvious example of this is the Polish UMP data, which was licensed CC-BY-SA and could not be kept post-licence change. If dedicated accounts had been used, removing this data would have been relatively easy; in reality, it has been (and continues to be) a nightmare. :( I think this is a counter example as it is well known which data is imported, and it is still a nightmare. The imported data is marked in a way that was standard for imports, and continues to be used for local TIGER 2011 imports for example. Cheers ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Sending to imports@ and cc'ing talk@ as that's more widely read than the wiki talk pages. From: sly (sylvain letuffe) [mailto:li...@letuffe.org] Subject: Re: [OSM-talk] Import guidelines proposal update Or, to make it even clearer, can I commit my change to the wiki without starting an edit war with anyone, and if no, with what type of wording change could it be commited ? There definitely is not general agreement at this time that this passage should be changed. At this point any changes to the guidelines may be superseded by the current work to combine the various bulk edit documentation into an OSMF bulk edit policy. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 1:57 PM, Jean-Marc Liotier j...@liotier.org wrote: Before we go further with policy edits, perhaps we should make sure that everyone understands the goals and that there is a consensus about them... That will make the resulting rules or guidelines more acceptable. Since you[1] are trying to revise guidelines that are found to be acceptable across the community you'll also want to be clear about how that is a benefit to the community at large. :-) I'm all for clearing up the multiple documents. But let's be clear on the facts at hand. A group of importers decided that they weren't going to follow the guidelines. Then one failed to respond when approached about a specific guideline. And now a group is upset to find that their self-declared-being-above-the-other-mappers is not widely supported. It would be easier to accept your sudden interest in clarifying guidelines if you'd shown any intention to follow them. Do you know who else asks for exceptions to the OSM guidelines when DWG halts them for non-compliance? - Politically motivated editors. Those who insist that their language must be first or only or default language on a node or road or other object without concern for the on the ground rule or other mappers. - The ignorant. They want OSM to match colors on their favourite device or something so they map all roads as highway=motorway and all grass as leisure=park. - Edit war participants. But it isn't an edit war because they are clearly 'right'. - Motivated capitalists. But I have to remove that competing grocery store in my neighbourhood. I need the business. They all got caught with their hands in the cookie jar. Back to the requirement for an import account. In private conversations, I've heard many explanations about why mappers haven't / hadn't created an import account. Only a few have claimed not to know of the import account requirement. I've combined their responses and made them generic. - Too hard to register with another email. I say use username+osmimp...@yourisp.com if they support it. Alternatively, those concerned in the French community can surely offer their members additional email accounts to support their community. - I don't want to change account settings in JOSM. I say start josm with alternate josm.home directory with your saved credentials. Like: java -Xmx2038M -Djosm.home=/home/username/import -jar /home/username/bin/josm-latest.jar - Cadastre is not an import. Cadastre is an import. Could you do the same thing if there were no Cadastre to import? No, you couldn't. Cadastre is an import. - Cadastre is different; I am careful before I upload. All mappers are careful don't insult the rest of the community. :-) Fixing and reconciling data before upload is the obligation you have when contributing. Cadastre is still an import. - No. I want credit for all of my mapping statistics all in one place. Simplest to fix. UserStat now allows you to combine stats from a group of your accounts. [1] Of course, I don't mean you personally, Jean-Marc. I have no idea of your OSM screen name, if you are a Cadastre importer or if you use an import account. I mean those who have been knowingly ignoring the import guidelines. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 19.09.2012 17:03, Martin Koppenhoefer wrote: I believe that dedicated accounts are generally better for imports a very clear +1from my side. Best regards, Michael. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 19.09.2012 18:51, Richard Weait wrote: More likely, and more often, what happens is that a well intentioned mapper uses a source for which he believes he has permission. Imports, then finds out that he didn't have (sufficient) permission for the current license. This has happened many times. Verry good point. Thanks Richard! Best regards, Michael. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 19.09.2012 18:42, sly (sylvain letuffe) wrote: Although I agree we shouldn't forget the past to avoid repeating the same mistakes but since every cases beeing different, I'm proposing to let local communities decide what they think is good for them. But only if it is not completely opposed to the consensus and well accpeted practice used since long term by teh rest of the worldwide community. Because there is not a OSM France project and a OSM Germany project and but ONE (!) worldwide OSM project. So there must be some (not a complete) concord, otherwise OSM would not work. Still, I believe every local community is happily discussing it internationaly to know and learn from other From my very personal opinion I always try to follow the international community and not do some special German behavior. There are some very special issues where this can't be avoided. But these are not on basic issues like using seperate accounts but special points (e.g. as already mentioned the use of motorroad == yes) . And that's one reason why I do not only read the talk.de ML but also the international talk ML. And that's also the reason I like to join international events like SOTM where the international community meets. Best regards, Michael. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
rw == Richard Weait rich...@weait.com writes: rw the facts at hand. A group of importers decided that they weren't rw going to follow the guidelines. Then one failed to respond when rw approached about a specific guideline. And now a group is upset to rw find that their self-declared-being-above-the-other-mappers is not rw widely supported. You are being unnecessarily inflammatory, Richard. Noone before you has talked of some mappers being above other mappers; shame on you for framing things in that way. One constructive question which has emerged from the debate (see messages by Frederik Ramm, Richard Fairhurst and Jean-Marc Liotier) is as follows: certain local communities have established, over many years, tools, specialized guidelines and monitoring tools for the use of specific data sources. In some cases these may deviate from the general, broad guidelines for imports/mechanical edits. What criteria should be used to distinguish between - local customs which have no negative impact on the project and allow the project to represent geographically/culturally specific features (eg. motorway=yes in Germany) - local customs which endanger the project globally (such as integrating data from a source whose copyright status is unclear/debatable) with a whole range of intermediate issues, such as the separate-account-for-imports suggestion, where French community consensus is that (given all the points already discussed) the inconvenience and burden on contributors outweigh the claimed benefits. -- Eric Marsden ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Hi, Le 19/09/2012 22:55, Richard Weait a écrit : - Cadastre is not an import. Cadastre is an import. Could you do the same thing if there were no Cadastre to import? No, you couldn't. Cadastre is an import. Cadastre is sometimes an import, sometimes not. Using the same data source, you may upload tons of buildings, or draw a single house thanks to cadastre WMS-like service as a background layer. In the end all buildings, without any difference, will get the Cadastre source tag. For instance : this one is from an import : http://www.openstreetmap.org/browse/way/63994077 this other one was drawn manually : http://www.openstreetmap.org/browse/way/181674272 Both have the same source tag, which is a request from the French Cadastre authority. A reason for a dedicated import account (read above in this topic) is that it helps to make a distinction between data sources in case of incompatibility between the source and a new licence. Let's assume French Cadastre Authority changes its mind and revokes any right to use its data in OSM. Cleaning the database will not consist in reverting changeset of imports, no. The only criteria for removing French Cadastre data will be the value of the source tag. Both buildings above will have to be suppressed. it is not a matter of import here. A lot of contributors mapping in France use Cadastre in such both ways : massive uploads and single edits using the same source. Such a mix make a dedicated account useless in this case. That's why I think that, as you say Richard : - Cadastre is different; and does not apply for a separate account. vincent (first post here, usually on talk-fr) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 19.09.2012 23:45, Vincent de Chateau-Thierry wrote: The only criteria for removing French Cadastre data will be the value of the source tag. That's a bad idea: if someone for what ever reason just decides to remove (or change) the source=cadastre tag of a object (and don't change anything else) you can't identify the object any more. Or to be more precise: you need to use a lot of effort and check all versions of an object (this means: the whole planet) whether it once had the source=cadastre tag. But thats a lot of work to do. Much (!) more easy to identify all the object is if you can take all object created by a special account = just check the changesets. Checking all versions of all objects is the thing we just went through with a lot of pain and effort: creating and using the redaction bot. And we are all aware that this was necessary but not nice and a lot of unporoductive effort. So let's learn from the past and avoid possible issues in the future that can be done easily with very small aditional effort in the presence. Best regards, Michael. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 10:55 PM, Richard Weait rich...@weait.com wrote: Since you[1] are trying to revise guidelines that are found to be acceptable across the community Could you provide evidences about this ? Since the vast majority of the community does not care about import guidelines, I would like to know how you can be sure about this. Pieren ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
Pieren pier...@gmail.com writes: On Wed, Sep 19, 2012 at 10:55 PM, Richard Weait rich...@weait.com wrote: Since you[1] are trying to revise guidelines that are found to be acceptable across the community Could you provide evidences about this ? Since the vast majority of the community does not care about import guidelines, I would like to know how you can be sure about this. I'm mostly a lurker in these discussions, and generally more pro-import than many who participate in import decisions. But I find the 'separate account for import' to be an utterly reasonable (along with the rest of the guidlines), easy to follow rule, and I am boggled by the objections to it. pgpVxIaZdUhH9.pgp Description: PGP signature ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
- Original Message - From: Lucas Nussbaum lu...@lucas-nussbaum.net To: sly (sylvain letuffe) li...@letuffe.org Cc: talk@openstreetmap.org Sent: Wednesday, September 19, 2012 5:59 PM Subject: Re: [OSM-talk] Import guidelines proposal update On 19/09/12 at 16:24 +0200, sly (sylvain letuffe) wrote: Hi, I've read the rather long thread Import guidelines OSMF/DWG governance and ^^ Note that the use of the term guidelines is problematic by itself. Either they are *guidelines*, that is, things that people SHOULD follow, but it's OK (but not recommended) not to follow them. Or they are *rules*, that is, things that people MUST follow. Yes , except for the fact that some of the parts of that part do seem to relate to guidelines, Therefore I suggest : 1) Page Title Import/Guidelines this should be changed to Import/Guidelines and Mandatory Requirements 2) Opening sentences be reworded. The current use of phrases such as there are few hard and fast rules, all this is open to discussion could suggest that the text on the rest of the page are mere suggestions, rather than hard and fast rules. 3) be clear on the page which bits are guidance and which bits are mandatory requirements David If the DWG blocks accounts based on *guidelines*, I think that they should be renamed to *rules*. Lucas ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On 9/19/2012 6:29 PM, Michael Kugelmann wrote: Or to be more precise: you need to use a lot of effort and check all versions of an object (this means: the whole planet) whether it once had the source=cadastre tag. But thats a lot of work to do. Much (!) more easy to identify all the object is if you can take all object created by a special account = just check the changesets. I may be missing something, but the special account has the same problem: any edit that splits an object loses the created-by information unless the history is used. Similarly, any edits change the last-edit owner. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
From: Vincent de Chateau-Thierry [mailto:v...@laposte.net] Subject: Re: [OSM-talk] Import guidelines proposal update Hi, Le 19/09/2012 22:55, Richard Weait a écrit : - Cadastre is not an import. Cadastre is an import. Could you do the same thing if there were no Cadastre to import? No, you couldn't. Cadastre is an import. Cadastre is sometimes an import, sometimes not. Using the same data source, you may upload tons of buildings, or draw a single house thanks to cadastre WMS-like service as a background layer. In the end all buildings, without any difference, will get the Cadastre source tag. For instance : this one is from an import : http://www.openstreetmap.org/browse/way/63994077 this other one was drawn manually : http://www.openstreetmap.org/browse/way/181674272 Both have the same source tag, which is a request from the French Cadastre authority. And the proposed change does nothing to help differentiate between cadastre imports (what the discussion is about) and non-import use of cadastre. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Import guidelines proposal update
On Wed, Sep 19, 2012 at 6:43 PM, Mike N nice...@att.net wrote: On 9/19/2012 6:29 PM, Michael Kugelmann wrote: Or to be more precise: you need to use a lot of effort and check all versions of an object (this means: the whole planet) whether it once had the source=cadastre tag. But thats a lot of work to do. Much (!) more easy to identify all the object is if you can take all object created by a special account = just check the changesets. I may be missing something, but the special account has the same problem: any edit that splits an object loses the created-by information unless the history is used. Similarly, any edits change the last-edit owner. Splitting can a problem, yes. It was decided to ignore it during the ODbL change because it difficult to determine and is a pretty miniscule problem at the end of the day. If a way is split but the nodes aren't touched, the way might still get deleted if all the nodes go away. As for the last-edited getting changed, that's why Michael talked about using the changesets. You can download the OSC of the original upload and have a list of all of the imported objects, regardless of the current state of it. If all changesets of a user are known to be from the same import it makes things simple. If you have to wade through thousands of changesets with varying comments... not so simple. Toby ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
