Hello, I need some help. I am making a project using Raspberry and GPS, so I want to read coordinates and show dots in a maps in live, just like osmand does. I found very interesting this post, but I can't find where to download TIF files for my country. I live in Cuba. Could you help me?
El viernes, 4 de marzo de 2016, 3:32:15 (UTC-8), Andrew Davie escribió: > > Thank you to the developers of OSMAND, and to the community who have > supported it. > > I would like to contribute a "HOW TO" - because this took me quite some > time to figure out and perfect. > > Essentially, I'm going to try to demonstrate how to go from georeferenced > TIFF source files (many) to an OSMAND compatible file which has all of > those source files as a displayable zoomable map. And, although there are > some commercial software packages out there already that do this, they are > not free - and some are far from cheap. I'm going to show you how to do it > completely free. I'll try, too, to make these instructions > platform-agnostic - that is, should work for Windows, OSX, or Linux - > although I'm using OSX myself and this is untested on those other > platforms. YMMV. > > Step 1 - Maps > As source material, I'm assuming you have a bunch of TIF format files > which you want to get displaying in your OSMAND app. Now I'm not going to > cover how to georeference those TIF files (that is, specify the exact place > in the world they represent). That's another subject, and basically > although I did check that out a while back, I'm starting with georeferenced > files already. You can download these pretty cheap - for example, the file > covering my capital city can be had for about US$1.50 so hey, why not. I > am sure you can also find tons of georeferenced map files on the interweb, > too. > > But, let's say we have a directory of files... as I do. 50 will be a nice > round number - it doesn't matter how many; the point being there are lots. > Now, the files I am using have been scanned from physical maps - that is, > paper printouts, scanned, and then georeferenced. It doesn't really matter > - but the point being there are areas in the files which are genuine map > stuff, and there are areas which are NOT map stuff (around the edges where > there's no map). These areas are usually represented in the TIF files as > transparent areas - and that's important for what's to come. > > So, if you want the "mangle all the maps together" to work properly in > this how-to, you will want your source files to have transparent areas > where there's "nothing". > > So, first step is to convert each individual TIF file into a hierarchy of > sub-images, with each level of the heirarchy being higher and higer > magnification. This is all to do with how the data is stored so that OSMAND > can access it efficiently - esentially each level is 2x the dimensions on > each axis of the lower level. Each 'tile' is just a square 256x256 pixel > slice from the original image, but the scaling on the tile is different > from level to level. These scaled image tiles are stored in a directory > hierarchy, with the top level representing the zoom/scale/level, whatever > you want to call it. 0 1 2 3 4.... etc. Now I'm really only interested in > levels 7 or so to about 15 or so - that's from a scale of hundreds of > kilometers down to tens of metres. > > But we don't really need to know much about all of that - it's all done > automatically for you by our first tool - 'gdal2tiles.py' > You will need to download and install a package called GDAL - get that at > http://www.gdal.org > There are direct binary downloads for this at > https://trac.osgeo.org/gdal/wiki/DownloadingGdalBinaries and tht's what I > used. > > We're really only interested in a couple of the GDAL tools - the first > being gdal2tiles.py and the second being 'gdaltranslate'. > When I did the install, the .py file was hidden > in /Library/Frameworks/GDAL.framework/Programs/gdal2tiles.py so you may > need to hunt for it after installation. For me, gdal_translate was in the > path and so just worked. Once you have checked that those are findable, > then you can cut/paste the following and save it as a script (call it > something like 'tile.sh' if you're on OSX or Linux). For windows users, > you will have to figure out how to iterate over files, so that's a bit of > homework for you. Sorry, I don't do windows. OSX/Linux, make sure it's > executable (from a terminal window, type 'chmod +x tile.sh). Windows > should be OK as is. > > Here's the code... > > #!/bin/bash > FILES="*.tif" > for f in $FILES > do > echo "Processing ${f%.*}" > gdal_translate -of vrt -expand rgba $f ${f%.*}.vrt > python /Library/Frameworks/GDAL.framework/Programs/gdal2tiles.py -z 7-14 > ${f%.*}.vrt > done > > OK, so just to explain; it iterates over every '.tif' file in the current > directory, and for each it calls first gdal_translate, and then > gdal2tiles.py > The gdal_translate is just converting the file format form tif to some > format that is recognised by the second part. > > It's the gdal2tiles.py that does all the hard work. It takes one source > image and splits up it into smaller and smaller tiles, creating a directory > for each level of detail. We are specifying those levels (zooms) by that > "-z 7-14" at the end there. You can change it to whatever range you want, > but over 14 and you start gettting tens of thousands of images (literally) > and the resultant database will be huge. So, stick with this range for > now; seems about right for general use. > > This will take a fair while - about a minute per source image, say. So, > just wait patiently for it to finish. Once it's done, you will see a new > directory for each of the source TIF files. So, if your tif was called > '8513.tif' then you will see a directory called '8513' and inside that you > will see subdirectories 7 and 8 and 9 and 10 ... etc... up to 14. Inside > each of THOSE you will see more directories with odd numbering, and inside > THOSE you will see individual image files with png extension, also with > strange numbering. As an aside, the numbering is actually an encoding of > the position of each of the tiles. No need to really worry/understand about > this, but that's what's happening. Encoded directory structure and > filename --> location of png in the real world. > > Right, so if all has gone OK we now have all of our TIF files processed > and we've created a bunch of directories with the sliced PNG tiles inside > each. We now need to combine those directories and PNG files into a single > heirarchy, representing the merged maps. There are a couple of issues here > - firstly, any image tile name combined with its parent directory name > uniquely represents where that tile is, and what scale it is. If we have > two original TIF files which are adjacent, or even overlap, then it's > possible we will have duplicate PNG files. For the same physcial location. > So, the PNG files need to be merged, in that case - that is where the files > parent name is the same, and the filename is the same. This merging > process involves copying the non-transparent pixels from each PNG into a > destination PNG (again, of the same name as the source PNG files). > Fortunately, Python to the rescue here - I wrote a short python program to > do just that - that is, traverse ALL the directories, and find ALL of the > images and merge all the duplicates and recreate the tree structure as a > new 'merged' tree. > > Here's the python program. Copy this code into, say, 'tree.py'... > > import os > from PIL import Image > import shutil > > > def merge(destination,source): > > print 'merging ',destination,'and',source > srcd = Image.open(source) > src = srcd.convert('RGBA') > dst = Image.open(destination).convert('RGBA') > dst.paste(src,(0,0),src) > dst.save(destination) > > > rootdir = '/Users/boo/Downloads/TasTOPO_100K' # TODO: change this to your > source directory > dest = '/Users/boo/Desktop/pymerge' # TODO: change this to your > destination directory > > > # Now merge everything... > > directories = next(os.walk(rootdir))[1] > directories = sorted(directories) > > if os.path.exists(dest): > shutil.rmtree(dest) > > for dir1 in directories: > for root, dirs, files in os.walk(rootdir+'/'+dir1): > for f in files: > if 'png' in f: > dix = root[len(rootdir+'/'+dir1)+1:] > fkey = root+'/' + f > ddir = dest + '/' + dix > if not os.path.isfile(ddir+'/'+f): > try: > if not os.path.exists(ddir): > os.makedirs(ddir) > except: > break > shutil.copyfile(root + '/' + f, ddir + '/' + f) > else: > merge(ddir+'/'+f,root+'/'+f) > > > OK, this is a bit of a quick'n'dirty - you see the comment where it says > "change this to your ...". Well, you gotta do that. > Obviously this could be a stand-alone python program with the paths as an > input, but hey - there's some homework for you. > > As it stands, it's good enough to do the job. > > What it does is grab a list of directories, and then 'walks' down the > heirarchy of these to find the PNG files. Each PNG file is either new (not > yet in the new tree), or it already exists. If it already exists then it > must be pixel-merged with the existing PNG file - and that's where "PIL" > magic comes into action. That little subroutine 'merge' is doing just that > with just a single line of code. Amazing. > > Anyway, this will take a while to run. If you want to go into the > destination directory as it's running and watch one of the images you can > actually see it being progressively upgraded as more and more images are > merged into it. That is, if you have multiple source TIFs. Spellibinding. > Might take a half hour or so to run through all the tens of thousands of > images, so go get a cuppa. > > BUT, once that's done - now you're sitting on data in the right format for > the next tool. > > Before I get to that, though - I need to mention something that's going to > be problematic for those in the Southern Hemisphere - there is something I > don't quite understand with the earlier tool 'gdal_translate' or possibly a > bug - it does not seem to understand the meaning of 'southern' and so all > my maps were treated as if they were in the northern hemisphere. That is, > all of the tiles were shifted from south to north but otherwise in the > correct latitude. So, shifted north and vertically stacked the wrong way. > Immedate thoughts "uh oh!" but it turns out that that file naming > convention I talked about earlier is now advantageous to us, because it's > quite easy to write a quick python program that goes through and renames > all the tiles so that they are correct names for where they SHOULD be (that > is, shift from the wrong northern hemisphere numbering to the correct > southern hemisphere numbering). > > So, here's that little fixup FOR SOUTHERN HEMISPHERE MAPS ONLY!!!. > Northern hemispherians skip this next bit... > OK, first just add this to the top part of the file - it's a new function > to fix the naming. > > import math > > def fixNorthSouthNaming(dir): > > directories = next(os.walk(dir))[1] > directories = sorted(directories) > > for level in directories: > base = math.pow(2.,float(level)-1) > for root,dirs,files in os.walk(dest+'/'+level): > for f in files: > if 'png' in f: > num = float(f.split('.')[0]) > num = base + (base-num) - 1 > print 'level ',level,'rename ',f,' to ',num > os.rename(root+'/'+f,root+'/'+str(int(num))+'.png') > > Lastly, at the very bottom of the file we want to add the call... > > fixNorthSouthNaming(dest) > > > That's it. If you've done things as instructed, then the original > directory walk will first create the merged directory, then this new > addition will go through and rename the tiles to fixup that 'bug' in the > northern/southern hemisphere thing. I'm sure that there is a better way to > do this, but this is the hack I used, and it WORKS, so there's that. > > Right, everyone should be here - both northern and southern folks. We > have a MERGED directory containing the processed tiles generated from > multiple input georeferenced TIF files. Just one thing to add here - those > with eagle eyes might have noticed that I sorted the directories in the > merging code. There's a very specific reason for that. Say we had high > resolution tifs of some parts of the map. Our original maps might be > 1:100000 and we might have one or two areas at 1:25000. We might only want > to display the high resolution maps at zoom levels (say, 12, 13, 14). > Here's where we manually intercede. BEFORE running the merging code, find > the directory with the tiles for each (remember, the directory name is the > same as the TIF file name). Go into it and DELETE the levels you do NOT > want to see. So, I would delete all directories 0,1,2,3,4,5,6,7,8,9,10,11 > and keep 12,13,14. That's the first step - the next step is to RENAME the > directory so that its name comes after all of the other directories in an > alphabetical sense. So, call it 'zzz" or put 'zzz' in front of the name. > Basically that will force that directory to be processed last in the > merge, and the upshot is that any images with pixels will also be processed > last - and so overwrite the low-resolution pixels that may already be in > the merged directory. Trust me, it works. > > Well, here we are ready to actually create our database. We'll do that > with another free tool... MOBAC. That's Mobile Atlas Creator, available > from http://mobac.sourceforge.net/ > Fantastic tool, but horrible UI/design and confusing to learn. We will > use this tool to create a 'sqlite' file that's needed by OSMAND to display > our maps. > So, download and install MOBAC from the sourceforge project page linked > from the previous URL, unzip it and there you go... it's a java program, > but on OSX at least you start it by opening a terminal and runinng > 'start.sh' from the directory where you put it. BUT, don't do that yet!!!! > > MOBAC is designed out of the box to connect to various online map sources, > and grab the data from those. We're NOT doing that. We have custom maps > already waiting in our merged directory and we need to point MOBAC to > those. To do that you have to add a short file to one of the MOBAC > subdirectories, to specify where the data is. I created a file called > 'local.xml' and its contents are... > > <?xml version="1.0" encoding="UTF-8"?> > <localTileFiles> > <name>ZoomTas</name> > <sourceType>DIR_ZOOM_X_Y</sourceType> > <sourceFolder>/Users/boo/Desktop/pymerge</sourceFolder> > <invertYCoordinate>false</invertYCoordinate> <!-- optional --> > <backgroundColor>#000000</backgroundColor> > </localTileFiles> > > See where it says 'sourceFolder'? That's where you need to put the > destination directory you used for the merged stuff we just did. So, fix > that up, and put this file in the MOBAC subdirectory 'mapsources'. You can > also name your source - here i have called mine 'ZoomTas' - call it > whatever you like. Now we're ready to go. Fire up MOBAC with ./start.sh > from its root directory. > > Ugly UI. But lovely program. OK, let's get this happening. > > Click on Atlas/new Atlas. Give it a name, and select "OSMAND SQLite" as > the format. Very important you get this right! If everything has worked, > then we should see the name of the source that we put in the local.xml file > (in my example, 'ZoomTas') appearing in the list of sources in the window > top left of MOBAC. Select it, and everything working OK, our map should > actually load up in the view window on the right hand side (the one with > all the red crosses that made us think nothing was working). If you can't > see anything, try changing that zoom slider to a small number. You can also > click on the map area and scroll around with the arrow keys. But > hopefully, you will be able to find your map and even zoom in and out with > that slider bar. Almost there! > > So if we actually see our map, all we need to do now is package the data > into a database for OSMAND. That involves selecting an area on the map, > selecting the zoom levels we want to include, and adding that selection to > the atlas. So here's where the UI is confusing. Essentially that "Zoom > levels" area with all the tickboxes is only relevant just before you click > on "Add Selection". It tells MOBAC which zoom levels you want to include. > After you've done that, the ticks are meaningless - so don't try and > changing them and go "export" - because export exports what is in the > atlas, not what is shown on the tick-boxes. > > So, go ahead - let's put a square around the entire area you want to > export by selecting it in the map area with your mouse/button and drag. You > will see a reddish square over the area. Now click the zoom levels you > want. Something lick 7-14 will do for now. You can choose less or more. > But if you want a real quick test to see if it's working on your phone, > just select a medium-zoom (perhaps 11 or so). OK, select as mentioned, > click the zoom levels tick-boxes as mentioned, and NOW click "Add > Selection". In the Atlas Content window, where you see the atlas with the > name you gave it, you will now see under that a "Layer" which lists the > zoom levels you ticked. This is pretty much all you HAVE to do. However, > you can if you wish select other parts of your map, and repeat. In other > words, you can add multiple selections. > > Right. almost done... now we generate the map database. Click on 'Create > Atlas'. A window with lots of progress barring and time estimates will > come up, and after a reasonable amount of time you will see it's finished > and the "Open Atlas Folder" button is now clickable. That's where your > file is, baby! > > Now it's time to get that file onto your phone. Naturally you will need > to have enough spare space on the phone to take this file. If you've been > silly and included gazillion levels of a large area, then your file is > going to be big. Smaller areas, or less levels are the way to go. Or, a big > SD card :) In any case, for the whole of my home state, the database file > is about 1GB. That's very usable. We need to connect our phone via USB > connection, and when it comes up look at the directories with a file > browser or somesuch. OSMAND is installed in /osmand - how nice - and under > that is a directory called 'tiles'. THAT directory is where you dump the > .sqlite file that MOBAC just created for you. So, copy it over... and now > start OSMAND. > > You want to tell OSMAND to USE the map you have just created. I typically > put it as an underlay map, but you can choose other options. Not really the > point of this tutorial to show you how to use osmand, but instead just how > to get a map. But in short, do the following.. top left you will see an > icon representing layers of maps. Looks a bit like a chesse sandwich. Click > that. and provided you have a map source already setup, you will see just > under that menu option "Overlay map" and "underlay map". CLick on underlay > map and in the selection of maps that comes up you SHOULD see the map name > of the sqlite file you just created. Select that, and exit back to the > main screen. > > Now, don't be disappointed! It seems that OSMAND needs a fair bit of time > to update/convert that map into an internal format - perhaps indexing it or > something, so that it can access it quicly. So, scroll around a bit, or > click the zoom levels. One thing you will see if it's all OK is a small > blue circle slider bar bottom middle. This is a cross-fade. When you play > with that you can fade between vector maps (I have my offline vectors as my > main map, and as noted my tiled tif map as the underlay). So, slide that > back and forth, and eventually -- once OSMAND has finished converting your > database file.. it should ALL WORK!!! > > Now obviously this first pass of instructions will have a few unclear > areas. I wanted to get this out there as a thank-you to all those who have > written tools such as MOBAC and the GDAL stuff. Fantastic stuff, and I hope > that my own contribution here helps others in turn. > > Cheers > A > > > > > > -- You received this message because you are subscribed to the Google Groups "Osmand" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
