[mkgmap-dev] Weird behavior and problem with splitter r161
Hello Splitter doesn't work work anymore I am downloading the geofabrik european map to build my own benelux map. for this I cut the european PBF with Osmosis to generate a Benelux PBF. I then split it into tiles with the special r161 splitter version that supports PBF as input before running the rest of the processing. This worked and I already generated a usefull map. But since my latest tentative (European data of wednesday and thursday) the splitter stops processing (without any error message) and the generated maps are missing most of their features (ex: roads). The weird behavior I see is that when the splitter starts to process the way occupancy it displays MAP occupancy: 0 and exists immediatly. This should explain why I don't see any roads on my generated maps. This is the output of my splitter: C:\Users\papa\Documents\GPS\OpenStreetMap\be_tilesjava -Xmx2000M -jar ../splitt er-r161/splitter.jar --mapid=5201 --cache=../be_tiles_cache/ --geonames-file =../cities15000.zip --max-nodes=100 ../myBenelux.osm.pbf cache=../be_tiles_cache/ description= geonames-file=../cities15000.zip legacy-mode=false mapid=5201 max-areas=255 max-nodes=100 max-threads=2 (auto) mixed=false no-trim=false overlap=2000 resolution=13 split-file= status-freq=120 write-kml= Elapsed time: 0s Memory: Current 61MB (1MB used, 60MB free) Max 1777MB Time started: Sat Feb 19 08:10:04 CET 2011 Map is being split for resolution 13: - area boundaries are aligned to 0x800 map units - areas are multiples of 0x1000 map units wide and high Processing ../myBenelux.osm.pbf Bounding box 1.801757999 48.908059 7.80029299901 52.214339 Time: Sat Feb 19 08:10:07 CET 2011 Exact map coverage is (48.908042907714844,1.8017578125) to (52.21432685852051,7. 80029296875) Trimmed and rounded map coverage is (48.9111328125,1.8017578125) to (52.25097656 25,7.7783203125) Splitting nodes into areas containing a maximum of 1.000.000 nodes each... Area (49.482421875,1.8896484375) to (51.15234375,4.1748046875) contains 405.624 nodes. DONE! Area (49.04296875,4.1748046875) to (51.15234375,5.4931640625) contains 733.032 n odes. DONE! Area (51.15234375,1.8017578125) to (52.20703125,4.5263671875) contains 620.613 n odes. DONE! Area (51.15234375,4.5263671875) to (52.20703125,5.4931640625) contains 704.554 n odes. DONE! Area (48.9111328125,5.4931640625) to (49.5263671875,7.7783203125) contains 529.6 28 nodes. DONE! Area (49.5263671875,5.4931640625) to (50.4931640625,7.4267578125) contains 658.5 04 nodes. DONE! Area (50.4931640625,5.4931640625) to (51.1083984375,6.2841796875) contains 477.8 42 nodes. DONE! Area (50.4931640625,6.2841796875) to (51.1083984375,7.1630859375) contains 524.1 02 nodes. DONE! Area (51.1083984375,5.4931640625) to (51.3720703125,7.0751953125) contains 491.5 19 nodes. DONE! Area (51.3720703125,5.4931640625) to (52.0751953125,6.9873046875) contains 619.8 61 nodes. DONE! 10 areas: Area 5201 covers (0x233800,0x3e800) to (0x23e800,0x54800) DE-Trier Area 5202 covers (0x245800,0x3e800) to (0x248800,0x50800) DE-Dusseldorf Area 5203 covers (0x22c800,0x3e800) to (0x233800,0x58800) DE-Saarbrucken Area 5204 covers (0x22e000,0x2f800) to (0x246000,0x3e800) BE-Brussels Area 5205 covers (0x23e800,0x3e800) to (0x245800,0x47800) DE-Aachen Area 5206 covers (0x246000,0x33800) to (0x252000,0x3e800) NL-Utrecht Area 5207 covers (0x248800,0x3e800) to (0x250800,0x4f800) DE-Duisburg Area 5208 covers (0x233000,0x15800) to (0x246000,0x2f800) BE-Gent Area 5209 covers (0x23e800,0x47800) to (0x245800,0x51800) DE-Koeln Area 5210 covers (0x246000,0x14800) to (0x252000,0x33800) NL-Rotterdam Writing out split osm files Sat Feb 19 08:10:08 CET 2011 Processing 10 areas in a single pass Starting pass 1 of 1, processing 10 areas (5201 to 5210) Making SparseMultiMap Making SparseMultiMap Processing ../myBenelux.osm.pbf Bounding box 1.801757999 48.908059 7.80029299901 52.214339 Making SparseMultiMap Making SparseMultiMap MAP occupancy: 861501 MAP occupancy: 130753 MAP occupancy: 7746 Making SparseMultiMap MAP occupancy: 1744524 MAP occupancy: 242713 MAP occupancy: 12515 MAP occupancy: 248 MAP occupancy: 2603236 MAP occupancy: 372778 MAP occupancy: 23341 MAP occupancy: 645 MAP occupancy: 3439177 MAP occupancy: 523803 MAP occupancy: 36362 MAP occupancy: 658 MAP occupancy: 4301030 MAP occupancy: 658871 MAP occupancy: 39441 MAP occupancy: 658 MAP occupancy: 5178067 MAP occupancy: 776748 MAP occupancy: 44358 MAP occupancy: 827 coords occupancy MAP occupancy: 5765385 MAP occupancy: 839551 MAP occupancy: 46448 MAP occupancy: 1236 ways occupancy MAP occupancy: 0 Time finished: Sat Feb 19 08:10:47 CET 2011 Total time taken: 43s C:\Users\papa\Documents\GPS\OpenStreetMap\be_tiles - Am I using the latest version of the PBF splitter branch (I also tried the compiled r161-3) ? - Is this a known problem ? - Is something wrong with my parameters ? I was using a windows 7 PC for this. Thanks in advance to
Re: [mkgmap-dev] Weird behavior and problem with splitter r161
Jean-Marc Meessen schrieb am 20.02.2011 14:24: * Am I using the latest version of the PBF splitter branch (I also tried the compiled r161-3) ? * Is this a known problem ? * Is something wrong with my parameters ? I was using a windows 7 PC for this. Thanks in advance to any hint or advise you could give me. This sound like a known (and already solved) problem, where splitter had problems with files 4GB on windof PCs. In the lates version of r-161 this problem was solved, previous versions of r-161 had this problem. Are you sure, you are using the latest versin of r-161? (My splitter.jar has the date 27.01.2011.) Are smaller maps working ok? Gruss Torsten ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Weird behavior and problem with splitter r161
The file I am trying to split is fairly small, Torsten. It is about 30kb large (while the european data is 5gb). The splitter version is from end january. Isn't the extract too small and thus the Osmosis extract went wrong ? I am extracting 50km around the borders of Belgium. Jmm 2011/2/20 Torsten Leistikow de_m...@gmx.de Jean-Marc Meessen schrieb am 20.02.2011 14:24: * Am I using the latest version of the PBF splitter branch (I also tried the compiled r161-3) ? * Is this a known problem ? * Is something wrong with my parameters ? I was using a windows 7 PC for this. Thanks in advance to any hint or advise you could give me. This sound like a known (and already solved) problem, where splitter had problems with files 4GB on windof PCs. In the lates version of r-161 this problem was solved, previous versions of r-161 had this problem. Are you sure, you are using the latest versin of r-161? (My splitter.jar has the date 27.01.2011.) Are smaller maps working ok? Gruss Torsten ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Weird behavior and problem with splitter r161
Jean-Marc Meessen schrieb am 20.02.2011 14:53: The file I am trying to split is fairly small, Torsten. It is about 30kb large (while the european data is 5gb). The splitter version is from end january. Isn't the extract too small and thus the Osmosis extract went wrong ? I am extracting 50km around the borders of Belgium. I haven't read your first mail correctly: I think the Osmosis cutting is your problem. Splitter and Osmosis use the same librarys for the pbf handling, so they both had the same 4GB bug. Is your Osmosis actual? It should also have a date from end of january. Gruss Torsten ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Weird behavior and problem with splitter r161
You have to use an osmosis-build, builded after 28.1.2011. Also it is faster to use splitter with an existing areas.list-file and europe.osm.pbf-file. This will be as fast as splitting your area with osmosis. So you save the time for splitting your myBenelux.osm.pbf Henning ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Splitter issue
Hi OK, I think I have this one solved, or at least worked around. The original code used a character array to buffer up output before I've looked at this and I think that the cause is the same as the other problem with the NPE. However I am not able to reproduce so I cannot be at all certain what is going on. I am not able to see how it goes wrong, although I don't like the approach to batching up write that was added in r160. I am going to merge the branch to trunk anyway, so I will do so without 160 and then add batching in a way that makes sense to me. Hopefully this will fix your problem too. ..Steve ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Weird behavior and problem with splitter r161
Hi Jean-Marc, Im using an areas.list-file for the Benelux, you can use this to split the europe.osm.pbf See areas_bnl.kml and areas_bnl.list on http://mijndev.openstreetmap.nl/~ligfietser/openfietsmap/Scripts/ I've also made a bigger areas.list that divides the bnl in 3 sections: north, middle and south. First step is to split europe.osm.pbf in 3 big osm.gz files with the areas list below. (or maybe merge middle and south to one big osm.gz). Second step is to split the 3 files into smaller tiles with max-nodes=150 or so. I've made this workaround because I couldnt make a working extract with osmosis either (same problems). # List of areas # BNL-north 3110: 2410496,110592 to 2516992,344064 # : 51.723633,2.373047 to 54.008789,7.382813 # BNL-middle 3120: 2361344,110592 to 2410496,307200 # : 50.668945,2.373047 to 51.723633,6.547852 # BNL-south 3130: 2299904,147456 to 2361344,307200 # : 49.350586,3.164063 to 50.668945,6.547852 ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Weird behavior and problem with splitter r161
Hello Henning, Thank you for the tip. Just came back and am trying your suggestion. I think that indeed the problem is coming from Osmosis. I used the latest stable build (previous compilations were made running on Linux but a little low on memory for the rest of the process). I am now running the latest dev build of Osmosis. The produced file is larger then the previous run (10 times larger, what is looking much better). I'll run splitter on that file now. I am optimistic. Jmm 2011/2/20 Henning Scholland o...@aighes.de You have to use an osmosis-build, builded after 28.1.2011. Also it is faster to use splitter with an existing areas.list-file and europe.osm.pbf-file. This will be as fast as splitting your area with osmosis. So you save the time for splitting your myBenelux.osm.pbf Henning ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] POIs for areas
Great patch. Twould be nice to have it commited. Is there interest for a patch file? I could offer one (I'ld just have to take out some of my own patches beforehand). On 19.02.2011 18:30, Torsten Leistikow wrote: Moin, I have taken a look at the add-pois-to-areas function: The function is implemented in MAPMaker.java (makeAreaPOIs), which means, it is active, AFTER the processing of the osm-data. So without a greater change this function can not be modified to be controlled by action rules during the style controlled osm-data processing. So I tried a limited modification, its not perfekt but for me it is an improvement over the actual implementation. This does not allow to create POIs for areas without the creation of a polygon (as a workaround a transparent polygon can be created). But the modification allows to control the POI generation for each polygon type on its own, i.e. via the style file single polygon types can be selected for the POI generation. My modification disables the add-pois-to-areas command line option, instead the POI generation is now triggered by a new command add_poi in the element type definition in the polygon style, e.g.: amenity=bank [0x2f06 level 3 add_poi] Since I am not really familiar with my java environment, I can not provide my modification as a patch, instead you will find the modified files attached to this mail. I have marked my changes with //TTT-start and //TTT-end. Gruss Torsten ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
[mkgmap-dev] Contour lines in Mapsource (was Improved street search in index branch)
On 21/01/2011 18:17, Felix Hartmann wrote: On 21.01.2011 15:12, char...@cferrero.net wrote: Minko (ligfiet...@online.nl) wrote: [snip] Note that I don't put the line mapname: 10010101.img in the line above with description: contourmap because this causes that the contours dont show up in Mapsource. If I leave mapname etc out, it shows the contourlines. No way! Is this why I can't get contour lines to display in MapSource (even though I know they're there because I can see the contour elevation labels)??? I'd given up on ever getting contours to work properly in MapSource. Prize for obscurest workaround of the year so far if this works. ;) Highly off topic: 1. Check: Countourlines have to have exactly the same resolution as normal maps (if normal maps 24,22,20,18 countourline maps have to be 24,22,20 (you can skip the 18) - if normal maps are 24,23,22,21,20 then well, your contourline maps need to be also 24,23,22,21,20 - as soon as you leave out a resolution they won't be displayed anymore) 2. Contourline maps have to have higher mapname. --- both is irrelevant if you use basecamp or on GPS, which always shows contourlines if inside same tdb. Problem is 1, in general contourlines work better if you have 23,21,20 as levels on GPS or for Qlandkarte GT. DEM3 data, is not good enough resolution to justify the 2x space / render requirements of res 23.) Hmmm - tried this and it still doesn't work. My contour style has exactly the same levels as the base style, and the mapnames are higher. All I can see is the contour labels, but not the contours themselves (see http://cferrero.net/maps/img/mkgmapdev_contourissue.png). If I just display the contour maps and not the basemap, it works as expected (see http://cferrero.net/maps/img/mkgmapdev_contourissue2.png) I'm using Mapsource 6.16.3, my base map has mapname 63247001 and the contour tiles are 63247201-63247209, compiled with higher draw priority and --transparent The levels definition in both basemap and contours is levels = 0:24, 1:23, 2:22, 3:20, 4:18, 5:16 -- Charlie ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Contour lines in Mapsource (was Improved street search in index branch)
On 21.02.2011 07:09, Charlie Ferrero wrote: On 21/01/2011 18:17, Felix Hartmann wrote: On 21.01.2011 15:12, char...@cferrero.net wrote: Minko (ligfiet...@online.nl) wrote: [snip] Note that I don't put the line mapname: 10010101.img in the line above with description: contourmap because this causes that the contours dont show up in Mapsource. If I leave mapname etc out, it shows the contourlines. No way! Is this why I can't get contour lines to display in MapSource (even though I know they're there because I can see the contour elevation labels)??? I'd given up on ever getting contours to work properly in MapSource. Prize for obscurest workaround of the year so far if this works. ;) Highly off topic: 1. Check: Countourlines have to have exactly the same resolution as normal maps (if normal maps 24,22,20,18 countourline maps have to be 24,22,20 (you can skip the 18) - if normal maps are 24,23,22,21,20 then well, your contourline maps need to be also 24,23,22,21,20 - as soon as you leave out a resolution they won't be displayed anymore) 2. Contourline maps have to have higher mapname. --- both is irrelevant if you use basecamp or on GPS, which always shows contourlines if inside same tdb. Problem is 1, in general contourlines work better if you have 23,21,20 as levels on GPS or for Qlandkarte GT. DEM3 data, is not good enough resolution to justify the 2x space / render requirements of res 23.) Hmmm - tried this and it still doesn't work. My contour style has exactly the same levels as the base style, and the mapnames are higher. All I can see is the contour labels, but not the contours themselves (see http://cferrero.net/maps/img/mkgmapdev_contourissue.png). If I just display the contour maps and not the basemap, it works as expected (see http://cferrero.net/maps/img/mkgmapdev_contourissue2.png) I'm using Mapsource 6.16.3, my base map has mapname 63247001 and the contour tiles are 63247201-63247209, compiled with higher draw priority and --transparent Where did I write you are supposed to use --transparent. Drop that it causes only problems. The levels definition in both basemap and contours is levels = 0:24, 1:23, 2:22, 3:20, 4:18, 5:16 -- Charlie ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] objects with line and polygon
On Sun, Feb 20, Charlie Ferrero wrote: Use a continue statement in the lines definitions, e.g. barrier=fence [0x?? resolution 23 continue] This enables the landuse to be processed by the polygons style file subsequent to the lines processing. Thanks! It's working fine. Didn't know that this works now cross files, I only found an older discussion where it did not work. You can see a working example of this in this screenshot: http://www.cferrero.net/maps/shot_AD_detail2.html (park with wall around the outside). Between, your link to your style file is broken. Thorsten -- Thorsten Kukuk, Project Manager/Release Manager SLES SUSE LINUX Products GmbH, Maxfeldstr. 5, D-90409 Nuernberg GF: Markus Rex, HRB 16746 (AG Nuernberg) ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
Re: [mkgmap-dev] Contour lines in Mapsource (was Improved street search in index branch)
Hi Charlie, I recently noticed that my method of leaving the mapname: etc out didn't always work as expected :-( What always worked for me, was saving the contourlines files as *.mp instead of img with gpsmapedit. My contourlines are transparent but can have the same draw priority. Process the *.mp tiles together with your osm.gz tiles to make img's. ___ mkgmap-dev mailing list mkgmap-dev@lists.mkgmap.org.uk http://www.mkgmap.org.uk/mailman/listinfo/mkgmap-dev
