Since I've now written a tool to do this (and tested it), guess I have to react now :)
On 21-01-16 05:38, Jo wrote: > About that CRAB import. In The Netherlands they already created a tool > to assist with that for their BAG import. I think we should look into > reusing that, adapted to the data we have available. We can't use that tool. If you take a look at the BAG import data, it's not really well done, in Holland most of the addressing is done on nodes, it's not merged into the building, which is what we do here, and it has many advantages to do it on the way. Also, using that tool will actually cause problems at the border, since houses (in Stekene) are aparantly in both BAG and GRB when they are visible from Belgium. I made sure to be able to safeguard both BAG and GRB data for those. I've written a tool to import GRB shapefiles and merge address data that comes with it. I still need to write a blog post on all that research, I'm lacking some time. Basically, we need to import the shape file in JOSM, save it as an osm (xml). and then run it through the (CLI) script to adapt the keys/tag's to osm and to merge basic address information to the node. There are .DBF (yes, that will be dbase ) files included when getting a full export, but that isn't perfect, there are still issues with both GRB addresses and CRAB, but it's a great way to not having to do it manually. We have introduced a set of tags, after some discussion with Sander, I believe is enough to start with. See http://www.openstreetmap.org/way/384159892#map=19/51.20779/4.07462 Stekene has been the testing grounds for the import, since it was quite the empty place. > > At this moment someone is working on converting the GRB data into > something that can be "easily" imported. > > I've put "easily" between quotes, because it will still be a lot of > work in areas where there are already buildings. The reason is that > we should try to keep the history and data of the existing > buildings. With data I mean POI info, heritage information, house > name, 3D, building type, etc. Even when there is no information, it > is polite to keep the names of the previous mappers in the history > of the OSM way. I do not like an approach where all existing data is > deleted. This is to be done with JOSM plugins, "Replace geometry" . that plugin is the whole center of the solution. This way you get to keep the history too. > > I have been replacing old AND data and buildings that I had traced > from Bing imagery with data based on AGIV aerial images and recently > with GRB data. It is labor intensive. Yes, but this source part is automated well. > > The scenario that I follow is > > * move the old house out of the way > * redraw based on GRB > * replace geometry of old building with new (from utilsplugin2). > this keeps the history > * drop source=AND > * add building type based on aerial images and add address (address > from AGIV) using housenumbertool The way to do it is : -preparse the shape file and save as xml (osm) in JOSM - to adapt keys add addressing, run the tool, it will create the correct tags. -use 2 layers, copy/paste from 1 layer to the existing layers (just overlay) -run the validator, fix the warnings first, select crossing buildings, and press CTRL-Shift-G (Replace geometry). Done! - Fix other warnings (there are plenty, but fixable) Building type is automated. > > Sometimes (with a lot of connected houses), I just try to reshape > the old building way, because moving all houses out of the way is > more difficult. > > With an import you do not have to do the drawing, but you still need > to do the replace geometry. The other steps that you have to perform > depends on the import data: is the building type correct ? is the > address data already merged ? > > Since this is so much work to correct, it's a pitty to see that some > power mappers are rushing through the current address data and only > add rough approximations of the buildings. L-shaped buildings become > rectangles, in terraces half of the buildings are ignored. > Correcting this afterwards, might take more time than trying to do > it correctly the first time around. Drawing buildings at the moment is indeed making it harder, but it's a very good check for every building. Here's a text dump of what the tool does: > glenn@slicky:~/Downloads/stekene/Shapefile$ ./grb2osm.php -f > TblAdpAdr46024B500.dbf,TblGbgAdr46024B500.dbf -i stekkie.xml -o stek.osm > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Start > 2016-01-21 08:47:09 [23849]:[2]info - [OsmTool::open_db] - Trying to open > DBase DB TblAdpAdr46024B500.dbf > 2016-01-21 08:47:09 [23849]:[2]info - [OsmTool::open_db] - Opened > TblAdpAdr46024B500.dbf > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Reading records... > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Done. loaded (384) > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Postprocessing > addresses. > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Flattened > multiple records on oid. > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Cleaning up... > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Purged 'nvt' > values . > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Start > 2016-01-21 08:47:09 [23849]:[2]info - [OsmTool::open_db] - Trying to open > DBase DB TblGbgAdr46024B500.dbf > 2016-01-21 08:47:09 [23849]:[2]info - [OsmTool::open_db] - Opened > TblGbgAdr46024B500.dbf > 2016-01-21 08:47:09 [23849]:[3]notic - [OsmTool::open_db] - Reading records... > 2016-01-21 08:47:10 [23849]:[3]notic - [OsmTool::open_db] - Done. loaded > (8263) > 2016-01-21 08:47:10 [23849]:[3]notic - [OsmTool::open_db] - Postprocessing > addresses. > 2016-01-21 08:47:10 [23849]:[3]notic - [OsmTool::open_db] - Flattened > multiple records on oid. > 2016-01-21 08:47:10 [23849]:[3]notic - [OsmTool::open_db] - Cleaning up... > 2016-01-21 08:47:11 [23849]:[3]notic - [OsmTool::open_db] - Purged 'nvt' > values . > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Loading OSM file ... > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Done ... > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Renaming GRB keys for > ways/relations to OSM keys.. > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Done. > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Parsing XML ... > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Done ... > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Analysing ways/relations having > the source:geometry:oidn tag present... > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Deleting ways: > [4491,4497,4500,4503,4507,4512,4514,4517,4520,4526,4529,4532,4533,4538,4542,4543,4548,4554,4559,4560,4561,4563,4567,4571,4573,4574,4575,4577,4582,4583,4586,4587,4589,4591,4597,4598,4606,4620,4622,4625,4626,4627,4628,4629,4638,4640,4643,4644,4649,4658,4660,4663,4669,4671,4672,4673,4675,4680,4681,4683,4689,4690,4694,4705,4716,4717,4722,4724,4728,4739,4755,4760,4762,4763,4776,4778,4788,4793,4795,4797,4803,4811,4813,4814,4819,4820,4822,4823,4829,4835,4840,4850,4854,4862,4868,4870,4873,4879,4882,4886,4891,4904,4905,4909,4912,4918,4920,4921,4926,4928,4930,4932,4937,4938,4944,4951,4958,4961,4971,4973,4979,4981,4985,4988,4991,4995,4997,4998,5011,5012,5014,5018,5019,5021,5024,5026,5027,5037,5038,5039,5040,5043,5053,5061,5065,5066,5067,5072,5075,5076,5082,5083,5091,5111,5122,5139,5156,5158,5161,5162,5166,5169,5173,5174,5180,5186,5187,5189,5192,5198,5201,5206,5211,5220,5222,5223,5226,5240,5241,5243,5246,5251,5253,5257,5260]... > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Done. > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Filtering deleted elements from > XML ... > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Done. > 2016-01-21 08:47:11 [23849]:[3]notic - [] - Generating new XML ... > 2016-01-21 08:47:12 [23849]:[3]notic - [] - Done. > 2016-01-21 08:47:12 [23849]:[3]notic - [] - Writing XML ... > 2016-01-21 08:47:12 [23849]:[3]notic - [] - Done. > 2016-01-21 08:47:12 [23849]:[3]notic - [] - Postprocessing new OSM XML file.. > 2016-01-21 08:47:12 [23849]:[3]notic - [] - Replacing OSM xml header > 2016-01-21 08:47:12 [23849]:[3]notic - [] - Filter namespace extras from OSM > xml > 2016-01-21 08:47:12 [23849]:[3]notic - [] - done writing. > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] matches = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] misses = 773 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > gbg_addressrecords = 10066 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > adp_addressrecords = 500 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > knw_addressrecords = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] merged_huisnr = > 2349 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] merged_busnr = > 1452 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] merged_appartnr > = 8 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] purge_single_nvt > = 16048 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] address_records > = 10566 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_deleted_tags = 3092 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > multi_street_deleted = 44 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > empty_street_deleted = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_exist_addr = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_exist_street = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_exist_flats = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_exist_geometry = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_exist_source = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_exist_housenumber = 0 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > double_street_oids list > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_flattened_huisnr = 389 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_flattened_busnr = 208 > 2016-01-21 08:47:12 [23849]:[1]core - [OsmTool::__destruct] > counter_flattened_appartnr = 2 _______________________________________________ Talk-be mailing list Talk-be@openstreetmap.org https://lists.openstreetmap.org/listinfo/talk-be
