Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
On Sep 28, 2012 7:27 AM, THEVENON Julien julien_theve...@yahoo.fr wrote: -- Le jeu. 27 sept. 2012 20:18 HAEC, Sarah Hoffmann a écrit : This is the real problem for us. For the sake of completeness: planetwide there are currently 152 million objects. Which means 1/6th of the planet consists of French buildings. Now, there is a real problem. Hi Sara, concerning problem of disk usage by french cadastre data do you have some information?particulary do you know how is it stored in database? to be allowed to use cadastre data we have to add a source key which is long about 40 characters to each way drawn thanks cadastre data due to legal agreement with french office goverment providing cadastre data. do you know is this key is duplicatd for each building in the database or if there is a smart storage? if not it would be interesting to know which part of the size is for the key itself and which part is for the geometry. I think that for buildings composed of one way and 4 nodes the space required by the could be greater than for geometry. if this is the case there is perhaps a way to factorise the source key and dramatically reduce disk usage. I think the biggest cost for long tags that are heavily used is really in the planet file size. A bigger planet takes longer to generate, longer to download, longer to parse. The sheer size of it can be a problem to some potential users. Especially when over 10% of it is just tags from imports that most data consumers couldn't care less about. I think I calculated once that the tiger:upload_uuid tag here in the US is responsible for about 1% of the data in the planet file. Since it is a random string with hundreds of thousands of possible values, it doesn't compress well either. As for database space, it depends on the schema being used. The API database does not store geometry information for ways. It only stores which nodes belong to which ways. And every tag takes one row in the way tags table. There is no consolidation of common tags. The same goes for the planet file XML. So in these cases the tag will take up as much space as is needed for the key+value strings and the geometry only takes a few bytes per node to store the relationship between the way and the node. The default rendering database style (osm2pgsql) explicitly drops the source tag while importing since it is not useful for rendering but of course it still takes CPU cycles to uncompress and parse every tag from the XML/PBF file. One schema where you could actually make a direct comparison is pgsnapshot. It can store listening geometry and it stores all tags in an hstore field. I'm not really sure how the linestring geometry is stored on disk. When queried at a postgres prompt, it returns a string that is 187 characters long for some random 4 node way I picked out. Toby ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
From: THEVENON Julien [mailto:julien_theve...@yahoo.fr] Sent: Thursday, September 27, 2012 10:26 PM Subject: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre to be allowed to use cadastre data we have to add a source key which is long about 40 characters to each way drawn thanks cadastre data due to legal agreement with french office goverment providing cadastre data. My understanding is that the source=* tag is for mappers and cannot be relied upon to provide attribution or for meeting a legal requirement. Nothing in the OSM license prohibits a data consumer from removing tags and redistributing the data - if they can't do this with the cadastre data, is the license compatible? Several programs (e.g. osm2pgsql) strip the source tag when processing data. What is left in many cases is cadastre geometry which is often the same as the geometry the importer downloaded and imported without changes. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
Am 30.09.2012 um 01:21 schrieb Paul Norman penor...@mac.com: My understanding is that the source=* tag is for mappers and cannot be relied upon to provide attribution or for meeting a legal requirement. Nothing in the OSM license prohibits a data consumer from removing tags and redistributing the data - if they can't do this with the cadastre data, is the license compatible? I believe that's not a problem. The following mappers modifying an object can (and probably should) remove this tag, as long as they don't base their edit on cadastre data. Attribution will be kept accessible in the history. Cheers, Martin ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
From: THEVENON Julien [mailto:julien_theve...@yahoo.fr] Sent: Thursday, September 27, 2012 10:14 PM To: penor...@mac.com; cqu...@openstreetmap.fr; si...@poole.ch Cc: talk@openstreetmap.org Subject: Réf.: Re: [OSM-talk] All you've ever wanted to know about the french cadastre Le ven. 28 sept. 2012 02:13 HAEC, Paul Norman a écrit : Obviously buildings are part of it, but is there a list of what else? Hi, I don't think there is a list. the information that you can find are highway references,street names,city boundaries,cemetery boundaries,buildings,house number,hydrographic layer(this one is not really reliable so must be cross check carrefully with other sources),railways. only buildings railways cemetery boundaries and hydrographic shapes are automatically extracted. Other information must be read by contributor in cadastre overlay because automatic solutions are not reliable at the moment How do people know what features they can import without going through additional consultation as a new import? ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
On 28 Sep 2012, at 06:25, THEVENON Julien julien_theve...@yahoo.fr wrote: -- Le jeu. 27 sept. 2012 20:18 HAEC, Sarah Hoffmann a écrit : This is the real problem for us. For the sake of completeness: planetwide there are currently 152 million objects. Which means 1/6th of the planet consists of French buildings. Now, there is a real problem. Hi Sara, concerning problem of disk usage by french cadastre data do you have some information?particulary do you know how is it stored in database? to be allowed to use cadastre data we have to add a source key which is long about 40 characters to each way drawn thanks cadastre data due to legal agreement with french office goverment providing cadastre data. do you know is this key is duplicatd for each building in the database or if there is a smart storage? if not it would be interesting to know which part of the size is for the key itself and which part is for the geometry. I think that for buildings composed of one way and 4 nodes the space required by the could be greater than for geometry. if this is the case there is perhaps a way to factorise the source key and dramatically reduce disk usage. The way to reduce the disk space for stuff imported in the future is to store that source once on the changeset instead. Shaun ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
I know. My thinking is that the source tag would be better placed on the changeset rather than polluting the whole db with source tags and source tags for each and every tag on each object which is starting to happen. You can then use the changeset info to synthesise the source info down to the tag and geometry. Shaun On 28 Sep 2012, at 08:47, Richard Mann richard.mann.westoxf...@gmail.com wrote: They're not allowed to. The licence requires them to put it on the object. On Fri, Sep 28, 2012 at 8:35 AM, Shaun McDonald sh...@shaunmcdonald.me.uk wrote: On 28 Sep 2012, at 06:25, THEVENON Julien julien_theve...@yahoo.fr wrote: -- Le jeu. 27 sept. 2012 20:18 HAEC, Sarah Hoffmann a écrit : This is the real problem for us. For the sake of completeness: planetwide there are currently 152 million objects. Which means 1/6th of the planet consists of French buildings. Now, there is a real problem. Hi Sara, concerning problem of disk usage by french cadastre data do you have some information?particulary do you know how is it stored in database? to be allowed to use cadastre data we have to add a source key which is long about 40 characters to each way drawn thanks cadastre data due to legal agreement with french office goverment providing cadastre data. do you know is this key is duplicatd for each building in the database or if there is a smart storage? if not it would be interesting to know which part of the size is for the key itself and which part is for the geometry. I think that for buildings composed of one way and 4 nodes the space required by the could be greater than for geometry. if this is the case there is perhaps a way to factorise the source key and dramatically reduce disk usage. The way to reduce the disk space for stuff imported in the future is to store that source once on the changeset instead. Shaun ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
Shaun McDonald wrote: My thinking is that the source tag would be better placed on the changeset rather than polluting the whole db with source tags and source tags for each and every tag on each object which is starting to happen. You can then use the changeset info to synthesise the source info down to the tag and geometry. This will show my ignorance, but having tried to look at history on bits WHILE IN POTLATCH, I don't see how to do this? I don't think you can see who created an object? I need to get back to JOSM and dig a bit deeper under the skin. I'd downloaded the shortcut crib sheet only to find it needs updating, but I need to spend a few more hours using it. It *IS* important to identify the sourse of an object and what work has already been done on it, but I'm not sure what facilities are provided to do that? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
I would prefer to keep the source tag with the object. Within a changeset I will often have some roads where source is GPS, have traced some buildings from bing, and added a few pub/shop names where source is survey Phil -- Sent from my Nokia N9 On 28/09/2012 8:55 Shaun McDonald wrote: I know. My thinking is that the source tag would be better placed on the changeset rather than polluting the whole db with source tags and source tags for each and every tag on each object which is starting to happen. You can then use the changeset info to synthesise the source info down to the tag and geometry. Shaun On 28 Sep 2012, at 08:47, Richard Mann richard.mann.westoxf...@gmail.com wrote: They're not allowed to. The licence requires them to put it on the object. On Fri, Sep 28, 2012 at 8:35 AM, Shaun McDonald sh...@shaunmcdonald.me.uk wrote: On 28 Sep 2012, at 06:25, THEVENON Julien julien_theve...@yahoo.fr wrote: -- Le jeu. 27 sept. 2012 20:18 HAEC, Sarah Hoffmann a écrit : This is the real problem for us. For the sake of completeness: planetwide there are currently 152 million objects. Which means 1/6th of the planet consists of French buildings. Now, there is a real problem. Hi Sara, concerning problem of disk usage by french cadastre data do you have some information?particulary do you know how is it stored in database? to be allowed to use cadastre data we have to add a source key which is long about 40 characters to each way drawn thanks cadastre data due to legal agreement with french office goverment providing cadastre data. do you know is this key is duplicatd for each building in the database or if there is a smart storage? if not it would be interesting to know which part of the size is for the key itself and which part is for the geometry. I think that for buildings composed of one way and 4 nodes the space required by the could be greater than for geometry. if this is the case there is perhaps a way to factorise the source key and dramatically reduce disk usage. The way to reduce the disk space for stuff imported in the future is to store that source once on the changeset instead. Shaun ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
Select way or node. Click advanced. Click way/node number. Click more details. Phil -- Sent from my Nokia N9 On 28/09/2012 9:21 Lester Caine wrote: Shaun McDonald wrote: My thinking is that the source tag would be better placed on the changeset rather than polluting the whole db with source tags and source tags for each and every tag on each object which is starting to happen. You can then use the changeset info to synthesise the source info down to the tag and geometry. This will show my ignorance, but having tried to look at history on bits WHILE IN POTLATCH, I don't see how to do this? I don't think you can see who created an object? I need to get back to JOSM and dig a bit deeper under the skin. I'd downloaded the shortcut crib sheet only to find it needs updating, but I need to spend a few more hours using it. It *IS* important to identify the sourse of an object and what work has already been done on it, but I'm not sure what facilities are provided to do that? -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk/wiki/?page=contact EnquirySolve - http://lsces.co.uk/wiki/?page=contact Model Engineers Digital Workshop - http://lsces.co.uk/wiki/?page=contact Rainbow Digital Media - http://lsces.co.uk/wiki/?page=contact ___ talk mailing list talk@openstreetmap.org http://lsces.co.uk/wiki/?page=contact ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
Philip Barnes wrote: Select way or node. Click advanced. Click way/node number. Click more details. You don't even need the fourth step - the dialogue that appears when you click the way/node id is the history. cheers Richard -- View this message in context: http://gis.19327.n5.nabble.com/Ref-Re-All-you-ve-ever-wanted-to-know-about-the-french-cadastre-tp5727997p5728061.html Sent from the General Discussion mailing list archive at Nabble.com. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
Philip Barnes wrote: Select way or node. Click advanced. Click way/node number. Click more details. I think that the question was about changeset tags, in which case there are a couple more steps: View History. Choose the changeset to view information for, and click it. Here's an example: http://www.openstreetmap.org/browse/node/332695404 which has a source tag in this changeset: http://www.openstreetmap.org/browse/changeset/13283634 Is there any easy way (in any editor with any plugin) of getting to this information - preferably a collated list of object / changeset tags? Cheers, Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
Someoneelse wrote: Is there any easy way (in any editor with any plugin) of getting to this information - preferably a collated list of object / changeset tags? I've just done this in P2's history dialogue for 'comment' and 'source': https://github.com/systemed/potlatch2/commit/f827b5368307dfd1a12f717e778ba91b46e242e3 If more changeset tags become relevant then I'll add those too. cheers Richard -- View this message in context: http://gis.19327.n5.nabble.com/Ref-Re-All-you-ve-ever-wanted-to-know-about-the-french-cadastre-tp5727997p5728096.html Sent from the General Discussion mailing list archive at Nabble.com. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
Philip Barnes phil at trigpoint.me.uk writes: I would prefer to keep the source tag with the object. Within a changeset I will often have some roads where source is GPS, have traced some buildings from bing, and added a few pub/shop names where source is survey Changeset info can be obtained only from native OSM services. If someone downloads shapefiles from Geofabrik or Cloudmade or OSM data in GML format from my WFS server the changeset tags are a bit difficult to obtain. If such data contain osm_ids then it is possible to find the history of the objects from OSM services but I do not think it is compulsory to include osm_ids in WFS services or derived ODbL databases. Users can delete or edit the object source tags but perhaps there are still better possibilities that they remain in ODbL chain than the changeset source tags. -Jukka Rahkonen- ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
Richard Fairhurst wrote: Someoneelse wrote: Is there any easy way (in any editor with any plugin) of getting to this information - preferably a collated list of object / changeset tags? I've just done this in P2's history dialogue for 'comment' and 'source': https://github.com/systemed/potlatch2/commit/f827b5368307dfd1a12f717e778ba91b46e242e3 If more changeset tags become relevant then I'll add those too. Excellent - thanks. Cheers, Andy ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
-- Le ven. 28 sept. 2012 02:13 HAEC, Paul Norman a écrit : Obviously buildings are part of it, but is there a list of what else? Hi, I don't think there is a list. the information that you can find are highway references,street names,city boundaries,cemetery boundaries,buildings,house number,hydrographic layer(this one is not really reliable so must be cross check carrefully with other sources),railways. only buildings railways cemetery boundaries and hydrographic shapes are automatically extracted. Other information must be read by contributor in cadastre overlay because automatic solutions are not reliable at the moment Cheers Julien ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
-- Le jeu. 27 sept. 2012 20:18 HAEC, Sarah Hoffmann a écrit : This is the real problem for us. For the sake of completeness: planetwide there are currently 152 million objects. Which means 1/6th of the planet consists of French buildings. Now, there is a real problem. Hi Sara, concerning problem of disk usage by french cadastre data do you have some information?particulary do you know how is it stored in database? to be allowed to use cadastre data we have to add a source key which is long about 40 characters to each way drawn thanks cadastre data due to legal agreement with french office goverment providing cadastre data. do you know is this key is duplicatd for each building in the database or if there is a smart storage? if not it would be interesting to know which part of the size is for the key itself and which part is for the geometry. I think that for buildings composed of one way and 4 nodes the space required by the could be greater than for geometry. if this is the case there is perhaps a way to factorise the source key and dramatically reduce disk usage. Cheers Julien ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
-- Le mer. 26 sept. 2012 21:48 HAEC, Lester Caine a écrit Looking at the source material, there is nothing which can be used to separate the blocks displayed into separate buildings, and since we have no means of identifying different levels of building, adding 'detail' for that seems pointless? All that can be 'accurately' extracted from the source material is that there is a 'block' of buildings? So if you are not actually surveying the buildings and identifying individual buildings, then normal practice is to draw a single box. Frederik's example is the sort of thing that SHOULD have been tidied up before importing. please don't forget that you are talking about something that is considered as a bad import by the french community and that you are currently focusing on one building whereas there thousand correct buildings imported by hardworking mappers. you also seem to consider that there is only cadastre in the life. majority of contributos also use there local knowledge. I personnaly know where buildings are separated in my city and where there is a single one or at least I can go to check directly. I don't know by heart the number of level for each building and I have other priority at the moment(I prefer to concentrate on missing roads by example) but I think it would have no sense to remove correct details provided by cadastre just because I cannot had the details at the moment. a mappers keen of micro mapping or osm2xp will just have to add the missing details if he want. Please don't consider that what you think useless is useless for everyone. few weeks ago we had a request from a guy that would like to use osm data to compute solar energy potential production. For the moment this not possible but it can be done by adding some tags to separated adjacent building to indicate orientation of roof. by keeping only the external shape of building block it will not be possible Hand tracing hundreds of individual elements and not committing them often does not make sense. What I am talking about here is selecting hundreds of vectors from a file without checking them, and having to select each individually would help the checking process. Then perhaps the sort of questionable mapping demonstrated would not happen? Personally I would prefer to see http://www.remote.org/frederik/tmp/funnybuilding.png as a single closed outline box. If the vectors are not providing closed objects then there is something wrong with the data anyway and in my book it should not be allowed to be imported? With a decent editor, one should be able to select the outline of a block and simply import that ... where did you see open building objects ? cheers Julien ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Réf.: Re: All you've ever wanted to know about the french cadastre
THEVENON Julien wrote: Personally I would prefer to seehttp://www.remote.org/frederik/tmp/funnybuilding.png as a single closed outline box. If the vectors are not providing closed objects then there is something wrong with the data anyway and in my book it should not be allowed to be imported? With a decent editor, one should be able to select the outline of a block and simply import that ... where did you see open building objects ? I was referring to the extra line segments within the box ... -- Lester Caine - G8HFL - Contact - http://lsces.co.uk/wiki/?page=contact L.S.Caine Electronic Services - http://lsces.co.uk EnquirySolve - http://enquirysolve.com/ Model Engineers Digital Workshop - http://medw.co.uk Rainbow Digital Media - http://rainbowdigitalmedia.co.uk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
