Re: tango and the missing map tiles
On Fri, Jul 24, 2009 at 12:14:22PM +0200, Christophe Badoit wrote: > Le jeudi 23 juillet 2009 22:08:03, Ben Wong a écrit : > > # Modified by Ben Wong to work with BusyBox's wget. ^ > I found this faster (but you need curl) : [cut] > Only one instance of curl is executed, and it uses HTTP1.1 Keep Alive; it's > slow to start, but fast once running. Yes, but then he might as well just install the real wget and use the original script. The whole point of BusyBox is to avoid installing tonnes of stuff. Anyway, it seems that we can rewrite the original script using the real wget such that it doesn't need awk and xargs: $ (cd /usr/local/share/maptiles/tile.openstreetmap.org && \ find -size 0c -print | cut -c3- | \ wget -p -i - -B http://tile.openstreetmap.org/ -x -nH -U Firefox) Downloaded: 6 files, 30K in 0.05s (662 KB/s) Who wants to add support for ETag, If-None-Match: and extended attributes to wget? -- Rask Ingemann Lambertsen Danish law requires addresses in e-mail to be logged and stored for a year ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
Le jeudi 23 juillet 2009 22:08:03, Ben Wong a écrit :
> Oops, that didn't work. That's what I get for trying to cleanup the
> error checking before posting. This version actually works.
>
> #!/bin/sh
>
> # Look for OpenStreetMap tiles that are size zero (TangoGPS has a bug)
> # and download the correct tile. Based on a one-line script by
> # Rask Ingemann Lambertsen , June 2009,
> # Modified by Ben Wong to work with BusyBox's wget.
>
> cd ~/Maps/OSM || exit 1
>
> find -size 0c -print \
>
> | cut -c3- \
> | awk ' { print "http://tile.openstreetmap.org/"; $0 " -O " $0; }' \
> | xargs -n3 wget -U "UpdateTiles"
Hi,
I found this faster (but you need curl) :
find -type f -mtime +7 |
cut -c3- |
awk '{ print "url = http://tile.openstreetmap.org/"; $0 "\noutput = " $0 "\n";
}' |
curl -K -
Only one instance of curl is executed, and it uses HTTP1.1 Keep Alive; it's
slow to start, but fast once running.
Just change the -mtime +7 according to your needs !
--
Tof
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
Oops, that didn't work. That's what I get for trying to cleanup the
error checking before posting. This version actually works.
#!/bin/sh
# Look for OpenStreetMap tiles that are size zero (TangoGPS has a bug)
# and download the correct tile. Based on a one-line script by
# Rask Ingemann Lambertsen , June 2009,
# Modified by Ben Wong to work with BusyBox's wget.
cd ~/Maps/OSM || exit 1
find -size 0c -print \
| cut -c3- \
| awk ' { print "http://tile.openstreetmap.org/"; $0 " -O " $0; }' \
| xargs -n3 wget -U "UpdateTiles"
On Thu, Jul 23, 2009 at 12:46 PM, Ben Wong wrote:
> Finally got around to trying Rask's one-liner. Thanks for posting it,
> it worked great once I modified it to work with BusyBox's version of
> wget. I also had to fake the UserAgent; apparently openstreetmaps.org
> is no longer allowing wget. I've appended my modifications, which
> should work on any system, below.
>
> --Ben
>
> #!/bin/sh
>
> # Look for OpenStreetMap tiles that are size zero (TangoGPS has a bug)
> # and download the correct tile. Based on a one-line script by
> # Rask Ingemann Lambertsen , June 2009,
> # modified by Ben Wong to work with BusyBox's wget.
> cd ~/Maps/OSM || echo "Cannot cd to ~/Maps/OSM" >&2 && exit 1
> find -size 0c -print \
> | cut -c3- \
> | awk ' { print "http://tile.openstreetmap.org/"; $0 " -O " $0; }' \
> | xargs -n3 wget -U "UpdateTiles"
>
>
> On Sat, Jun 27, 2009 at 3:04 AM, Rask Ingemann
> Lambertsen wrote:
>> Running Yaouh! didn't update the about 15 empty map tiles I had. What I
>> did afterwards to update them was:
>>
>> $ (cd ~/Maps/OSM && find -size 0c -print) | cut -c3- | \
>> awk ' { print "http://tile.openstreetmap.org/"; $0; }' | \
>> (cd ~/Maps/OSM && xargs wget --force-directories --no-host-directories -N )
>>
>> --
>> Rask Ingemann Lambertsen
>> Danish law requires addresses in e-mail to be logged and stored for a year
>>
>> ___
>> Openmoko community mailing list
>> community@lists.openmoko.org
>> http://lists.openmoko.org/mailman/listinfo/community
>>
>
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
Finally got around to trying Rask's one-liner. Thanks for posting it,
it worked great once I modified it to work with BusyBox's version of
wget. I also had to fake the UserAgent; apparently openstreetmaps.org
is no longer allowing wget. I've appended my modifications, which
should work on any system, below.
--Ben
#!/bin/sh
# Look for OpenStreetMap tiles that are size zero (TangoGPS has a bug)
# and download the correct tile. Based on a one-line script by
# Rask Ingemann Lambertsen , June 2009,
# modified by Ben Wong to work with BusyBox's wget.
cd ~/Maps/OSM || echo "Cannot cd to ~/Maps/OSM" >&2 && exit 1
find -size 0c -print \
| cut -c3- \
| awk ' { print "http://tile.openstreetmap.org/"; $0 " -O " $0; }' \
| xargs -n3 wget -U "UpdateTiles"
On Sat, Jun 27, 2009 at 3:04 AM, Rask Ingemann
Lambertsen wrote:
> Running Yaouh! didn't update the about 15 empty map tiles I had. What I
> did afterwards to update them was:
>
> $ (cd ~/Maps/OSM && find -size 0c -print) | cut -c3- | \
> awk ' { print "http://tile.openstreetmap.org/"; $0; }' | \
> (cd ~/Maps/OSM && xargs wget --force-directories --no-host-directories -N )
>
> --
> Rask Ingemann Lambertsen
> Danish law requires addresses in e-mail to be logged and stored for a year
>
> ___
> Openmoko community mailing list
> community@lists.openmoko.org
> http://lists.openmoko.org/mailman/listinfo/community
>
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
On Mon, Jun 29, 2009 at 03:23:47PM +0200, Helge Hafting wrote: > Strange - the hash for an empty file will be wrong. Maybe yaouh gives up > when nothing can be read from the file. In that case, consider making a > script that copies some particular tile (or other png image) over any > size=0 file. Yaouh will then notice the wrong hash and fix it. Or just download the correct tiles directly with the one-liner I posted. :-) -- Rask Ingemann Lambertsen Danish law requires addresses in e-mail to be logged and stored for a year ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
Rask Ingemann Lambertsen wrote: > On Thu, Jun 25, 2009 at 12:08:54PM +1200, Robin Paulson wrote: >> 2009/6/25 Tim Abell : >>> I've found that if tango doesn't have a working net connection available >>> when you pan around it will generate empty files for the map images and >>> then never attempt to re-download them leaving nasty gaps in your coverage. > [snip} >>> Feel free to stick this on the wiki or something. I don't know if this >>> needs a bug report. > >IMHO it does. > >> will yaouh perform a similar task? i assume even though they are >> blank, they will have a date and time, thus when yaouh is run, it will >> see they are old/the hash is wrong, and replace them? > >No, Yaouh! (version 0.5.1) doesn't update empty map tiles. > Strange - the hash for an empty file will be wrong. Maybe yaouh gives up when nothing can be read from the file. In that case, consider making a script that copies some particular tile (or other png image) over any size=0 file. Yaouh will then notice the wrong hash and fix it. Helge Hafting ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
On Thu, Jun 25, 2009 at 10:28:11AM +0200, Carlo Minucci wrote:
> Rask Ingemann Lambertsen ha scritto:
> >
> >No, Yaouh! (version 0.5.1) doesn't update empty map tiles
>
> what???
> why not?
No idea, I haven't looked at the source code or straced a run to see why.
> are you sure?
Running Yaouh! didn't update the about 15 empty map tiles I had. What I
did afterwards to update them was:
$ (cd ~/Maps/OSM && find -size 0c -print) | cut -c3- | \
awk ' { print "http://tile.openstreetmap.org/"; $0; }' | \
(cd ~/Maps/OSM && xargs wget --force-directories --no-host-directories -N )
--
Rask Ingemann Lambertsen
Danish law requires addresses in e-mail to be logged and stored for a year
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
Rask Ingemann Lambertsen ha scritto: > >> will yaouh perform a similar task? i assume even though they are >> blank, they will have a date and time, thus when yaouh is run, it will >> see they are old/the hash is wrong, and replace them? >> > >No, Yaouh! (version 0.5.1) doesn't update empty map tiles what??? why not? are you sure? ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
On Thu, 25 Jun 2009 10:17:59 +0200, Rask Ingemann Lambertsen wrote: > On Thu, Jun 25, 2009 at 12:08:54PM +1200, Robin Paulson wrote: >> 2009/6/25 Tim Abell : >> > I've found that if tango doesn't have a working net connection >> > available >> > when you pan around it will generate empty files for the map images and >> > then never attempt to re-download them leaving nasty gaps in your >> > coverage. > [snip} >> > >> > Feel free to stick this on the wiki or something. I don't know if this >> > needs a bug report. > >IMHO it does. > >> will yaouh perform a similar task? i assume even though they are >> blank, they will have a date and time, thus when yaouh is run, it will >> see they are old/the hash is wrong, and replace them? > >No, Yaouh! (version 0.5.1) doesn't update empty map tiles. Some time ago, I posted a Python script (needs python-netclient) that performs a similar task to what Yaouh! did. Including updating the empty files. Just search for "Yaouh! clone" or "fill_gaps" in the archives. In the following weeks, I should have enough time to clean it up and add the functionality some people requested - namely, updating also tiles with lower zoom levels if one with higher was updated, and symlinking empty tiles to one file. -- Cheers, rhn ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
On Thu, Jun 25, 2009 at 12:08:54PM +1200, Robin Paulson wrote: > 2009/6/25 Tim Abell : > > I've found that if tango doesn't have a working net connection available > > when you pan around it will generate empty files for the map images and > > then never attempt to re-download them leaving nasty gaps in your coverage. [snip} > > > > Feel free to stick this on the wiki or something. I don't know if this > > needs a bug report. IMHO it does. > will yaouh perform a similar task? i assume even though they are > blank, they will have a date and time, thus when yaouh is run, it will > see they are old/the hash is wrong, and replace them? No, Yaouh! (version 0.5.1) doesn't update empty map tiles. -- Rask Ingemann Lambertsen Danish law requires addresses in e-mail to be logged and stored for a year ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
On Thu, 25 Jun 2009 00:30:42 +0100
Tim Abell wrote:
> Hi all,
>
> I've found that if tango doesn't have a working net connection
> available when you pan around it will generate empty files for the
> map images and then never attempt to re-download them leaving nasty
> gaps in your coverage.
>
> Here's a quick n dirty script that removes all the empty files so
> that tango will fetch them next time.
>
>
> r...@om-gta02:~# cat cleanmapcache.sh
>
> #!/bin/sh -v
> find Maps/OSM/ -size 0 -type f
> find Maps/OSM/ -size 0 -type f -exec rm {} \;
>
>
> The first line displays the empty tile list, the second deletes them.
How about this: (works if you have coreutils)
alias clean-map="find /path/Maps/OSM/ -size 0 -type f -exec rm -v {} \;"
--
Joseph Booker
signature.asc
Description: PGP signature
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Re: tango and the missing map tiles
2009/6/25 Tim Abell :
> I've found that if tango doesn't have a working net connection available
> when you pan around it will generate empty files for the map images and
> then never attempt to re-download them leaving nasty gaps in your coverage.
>
> Here's a quick n dirty script that removes all the empty files so that
> tango will fetch them next time.
>
>
> r...@om-gta02:~# cat cleanmapcache.sh
>
> #!/bin/sh -v
> find Maps/OSM/ -size 0 -type f
> find Maps/OSM/ -size 0 -type f -exec rm {} \;
>
>
> The first line displays the empty tile list, the second deletes them.
>
> Feel free to stick this on the wiki or something. I don't know if this
> needs a bug report.
will yaouh perform a similar task? i assume even though they are
blank, they will have a date and time, thus when yaouh is run, it will
see they are old/the hash is wrong, and replace them?
i might be wrong of course
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
tango and the missing map tiles
Hi all,
I've found that if tango doesn't have a working net connection available
when you pan around it will generate empty files for the map images and
then never attempt to re-download them leaving nasty gaps in your coverage.
Here's a quick n dirty script that removes all the empty files so that
tango will fetch them next time.
r...@om-gta02:~# cat cleanmapcache.sh
#!/bin/sh -v
find Maps/OSM/ -size 0 -type f
find Maps/OSM/ -size 0 -type f -exec rm {} \;
The first line displays the empty tile list, the second deletes them.
Feel free to stick this on the wiki or something. I don't know if this
needs a bug report.
Om2009
Tim Abell
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
