Hi Peter and thank you for your help.
I had the same kind of approach and I guess the problem was your 4th point.
I'll give a try at osmconvert with "complex-ways" option as a second step.
But as you mentioned, there will be a problem with features that are
bigger than the margin.
Anyway, this will be a good temporary solution and I will have to move
Jochen's solution as soon as I have a better system with enough RAM.
Thank you again for this detailed description and your sources, I'll
definitely give it a look.
Regards,
Sylvain
On 18/04/2016 13:42, Peter Svensson wrote:
Hi
I've spent a lot of time trying to solve this problem for OSMTileMachine.
OSMTimeMachine is basically a tool that bridges the gap between a full
planet file and a data consumer loads all the data into RAM.
This makes it possible to render a map of a very large region using
Maperitive (which itself cannot handle very large amounts of data, due
to the loads-everything-into-memory-limitation). See example of
results here: www.mtbmap.se <http://www.mtbmap.se> . This map is
rendered using OSMTileMachine. I don't know about your requirements,
but as a working proof of concept, the input do that map is a single
planet.pbf and all of this is rendered on a Windows PC with 8 GB of RAM.
So what I have figured out so far is this:
1. osmconvert is MUCH faster than osmosis for extracting data, at
least for me.
2. For generation of map tiles, complete ways/relations are needed.
This means that at the boundary, everything related to each
way/relation passing through the boundary of the tile image needs to
be complete.
3. osmconvert with the "complex-ways" option is very suited to
generate complete data for visualisation of a geograpically limited
region (square). I use 0.04 degrees of margin when i extract data for
a tile for zoom level 10.
4. it's not possible to use the complex-ways cutmethod at the full
planet due to time and/or RAM/DISK usage.
5. If i extract the data in two steps, it is possible to prepare
complete data for a small box:
6a) First extract a region AROUND the box of interest with cutmethod
"clip". I think this is default for osmconvert. I use 2 degrees of
extra margin around my box of interest here. This makes it possible to
produce anything within reasonable abount of time and RAM / disk usage.
6b) Since the output of 6a) is too big for my data consumer (in my
case maperative) i need to shrink it again and cut away even more
data. This time i clip at the exact box of interest + 0.04 degrees.
(This is what i need to have enough overlap for place-labels etc that
can stick into neighboring map tiles). If i use the complex-ways
method i will not loose any data inside my map file and the results
looks good.
The drawback is that features that are larger than 2 degrees will be
incomplete. This is mainly a problem for very large lakes and of
course the coastline.
Some source code, if you have an interest:
https://github.com/psvenz/OSMTileMachine/blob/master/osmTileMachine/src/osmTileMachine/SplitAndRenderStrategy.java
Hope this helps.
with regards,
zvenzzon
On Mon, Apr 18, 2016 at 10:10 AM, Sylvain Melin <s.me...@alsim.com
<mailto:s.me...@alsim.com>> wrote:
Hi everyone.
I'm working on a 3D earth rendering engine and I want to use osm
data for forest, cities, roads, etc...
Of course it involves a lot preprocessing to filter and convert
vector data into OpenGL friendly format.
My plan is to :
- exploit a planet sized pbf file
- cut it into 1° tiles using osmosis
- filter and extract the data from these tiles as shapefiles using
libosmium
- rasterize these shapefiles with gdal
My problem occurs when trying to cut the big PBF file into tiles.
I can't find an efficient way to prevent the elements from being
clipped on the edges of the tiles.
I've tried to use completeWays=yes and completeRelations=yes but
the first tile was still empty after a few hours when it took 6
minutes to process it without these options.
The only way I can see to limit the damage is to set a wider
bounding box than necessary but there will always be a polygon
bigger than my margin that will be clipped, and the size of my
tiles is also greatly impacted.
Do you have any suggestion about this ?
Thanks.
_______________________________________________
osmosis-dev mailing list
osmosis-dev@openstreetmap.org <mailto:osmosis-dev@openstreetmap.org>
https://lists.openstreetmap.org/listinfo/osmosis-dev
_______________________________________________
osmosis-dev mailing list
osmosis-dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/osmosis-dev
