Re: [OSM-dev] searching for good document of binary map formats

[Scott Crosby]

On Fri, Aug 27, 2010 at 2:35 AM, feverzsj  wrote:
>
> hi, list:
>  I've started to develop a new binary map format for display, geocoding and
> possibly routing on mobile device. After checking out some active routing
> projects, I found most of them have no or poor document for their binary
> format.

I have a binary format thats going into osmosis fairly soon. It is
primarily intended for use as an interchange format, not for
geocoding, routing, etc, but it might be adapted into a display
format. The main perk is that it can contain the entire planet,
including all tags and metadata, in about 6gb. It is also very fast to
read and write. For details on the design, please see my earlier email
threads, in particular,
http://www.mail-archive.com/dev@openstreetmap.org/msg11392.html I have
made only one substantual change since then. I have made DenseNodes
able to store tags, and have thus removed Node's from the design
entirely. (This is not written up anywhere except the protobuf docs.)

>  Navit actually has a page about their binary format, but almost just an
> outline. For Gosmore, there was once a post about its format (about 2 yeas
> ago?), which was also not so detailed.
>  Several years ago, I've developed a binary map using zoom-level based sub
> r*-tree of map tiles. It's very fast even for low end mobile device, but
> missing feature like geocoding and routing.

Could you describe your design and the filesizes?

>  So, it will be grateful, if someone can offer me some good documents.
>

Scott

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] generate_xml.py not generating osm.xml file?

[Lennard]

On 27-8-2010 15:58, Ákos Maróy wrote:

Hi,

Maybe I'm misunderstanding the generic purpose of the generate_xml.py
file from the mapnik rendering toolset, but for me, it does not seem to
touch my osm.xml file in any way.


That's correct.


when I execute generate_xml.py with appropriate database setttings,
osm.xml is not changed in any way. if I remove osm.xml, it is not
generated by the gall to generate_xml.py in any way.


What does happen when you run generate_xml.py is that three files are 
created (or updated) in the inc directory. They are: 
datasource-settings.xml.inc, fontset-settings.xml.inc, settings.xml.inc


Some guides still give a syntax like:

generate_xml.py my_osm.xml 

But that's needed only in rare cases where you actually want it to 
create a monolithic copy of the entire mapnik style. my_osm.xml would 
then be composed of osm.xml and all include files from the inc directory.


Normally, you use generate_xml.py to set the contents of the three files 
I mentioned earlier, and then point your map generating script or app to 
osm.xml. Through XML parsing, osm.xml will dynamically include the 
contents of the files in the inc directory.


Synching the stylesheet to OSM SVN is then only a matter of running svn 
up. You only need generate_xml.py again if you change any of the 
settings, or if a new setting was introduced (in the .template files in 
the inc directory).



am I misunderstanding that generate_xml.py would actually create osm.xml
based on the contents of the PostGIS database it connects to, and based
on the templates in the inc directory?


osm.xml does not contain geographic data. It contains the stylesheet 
that tells mapnik how to render geo data that you load into the postgis 
db with osm2pgsql.


--
Lennard


___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] generate_xml.py not generating osm.xml file?

[Stephan Knauss]

Hi,

Sajjad Anwar wrote:
The following link contains pointers to setting up Mapnik and basic 
rules file modifications 
http://sajjad.in/mapping/setting-up-mapnik-for-rendering-tiles-in-your-language/


you can find the trick to render bilingual maps like 
http://thaimap.osm-tools.org/ or http://greece.osm-tools.org/ on my 
download space:


http://downloads.osm-tools.org/bilingual/

I did set up a view to return the matching names from the DB. This is 
also the place you want to edit in case you like a different fallback rule.


Stephan

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] size of apidb

[Grant Slater]

On 27 August 2010 19:50, Marco Lechner - FOSSGIS e.V.
 wrote:
> Munin reports about 1.2 TB fot the whole Api-DB.
> If I calculate the size of the full-history-dump (unpacked about 338 GB)
> and add the size of the planet-file (unpacked about 155 GB) for the
> current_tables I get a sum of less the 500 GB. Let's add some GB for
> Indexes and things like email-adresses and passwords of the users.
> How can the ApiDB be about twice the size? Of course it's not o.k to
> compare a file with a relational database, but comparing the DB with a
> space-wasting XML-file?
>
> I'm confused and asking for enlightenment.
>

You are forgetting about the GPS/GPX points table.

The compressed raw GPX files total 193GB.

Regards
 Grant

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] size of apidb

[Stefan de Konink]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

Op 27-08-10 20:50, Marco Lechner - FOSSGIS e.V. schreef:
> I'm confused and asking for enlightenment.

Not all data might as well aligned as you might think ;) A lot of
deletes and updates in a table can make it pretty sparse.


Stefan
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.16 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEAREKAAYFAkx4HYYACgkQYH1+F2Rqwn0UOgCeJddCAKIc72Ii2zMg4ac4T6uE
C9wAoIr5uDZFkc1gkN1Tc6RoxXYWmA5k
=B0jC
-END PGP SIGNATURE-

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] size of apidb

[Marco Lechner - FOSSGIS e.V.]

Am 27.08.2010 09:25, schrieb Tom Hughes:
> On 27/08/10 07:53, Marco Lechner - FOSSGIS e.V. wrote:
>
>> What is or"where can i get the recent size of the OSM-apidb?
>
> http://munin.openstreetmap.org/openstreetmap/smaug.openstreetmap/postgres_size_openstreetmap.html
>
>
> Tom
>
Hm, I'm a little bit wondering.

Munin reports about 1.2 TB fot the whole Api-DB.
If I calculate the size of the full-history-dump (unpacked about 338 GB)
and add the size of the planet-file (unpacked about 155 GB) for the
current_tables I get a sum of less the 500 GB. Let's add some GB for
Indexes and things like email-adresses and passwords of the users.
How can the ApiDB be about twice the size? Of course it's not o.k to
compare a file with a relational database, but comparing the DB with a
space-wasting XML-file?

I'm confused and asking for enlightenment.

Marco

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Wishlist API/db: bot-flag WAS new contributor terms ID list

[Andy Allan]

On Fri, Aug 27, 2010 at 4:51 PM, andrzej zaborowski  wrote:
> On 27 August 2010 16:19, M∡rtin Koppenhoefer  wrote:
>> 2010/8/27 andrzej zaborowski :
>  I've ...also used bot=yes on many manual edits,
 IMHO you shouldn't do this.
>>> Why?
>>
>> because thereby you are flagging them as automated edits which they aren't.
>
> Perhaps fixbot is highly automated but other than that the "bots" that
> run on OSM are hardly ever fully automated, in my experience there's
> always some (or a lot of) manual work to do before upload.  So the
> question is how a bot edit is defined, is it an edit that doesn't
> bring in new data, just amends what is already in OSM?  Or (what I
> actually wanted to propose, but expressed it wrongly) let's use the
> flag as meaning "uninteresting" or "likely not the edit you're looking
> for".  This is the same function that "this is a minor edit" on
> wikipedia has.

Then mark your changes as minor edits. And then everyone can ignore
the mark, since I see people (especially on the OSM wiki) trying to
sneak through changes as "minor" when they are anything but.

Really, the problem is not whether or not changes are done by bots or
not. The issue is that you want to see what's changed in the
particular area - by a human, by a bot, by a human pretending to be a
bot, or whatever. That problem is solved by OWL, not by flagging
changesets in any way.

Cheers,
Andy

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Wishlist API/db: bot-flag WAS new contributor terms ID list

[andrzej zaborowski]

On 27 August 2010 16:19, M∡rtin Koppenhoefer  wrote:
> 2010/8/27 andrzej zaborowski :
  I've ...also used bot=yes on many manual edits,
>>> IMHO you shouldn't do this.
>> Why?
>
> because thereby you are flagging them as automated edits which they aren't.

Perhaps fixbot is highly automated but other than that the "bots" that
run on OSM are hardly ever fully automated, in my experience there's
always some (or a lot of) manual work to do before upload.  So the
question is how a bot edit is defined, is it an edit that doesn't
bring in new data, just amends what is already in OSM?  Or (what I
actually wanted to propose, but expressed it wrongly) let's use the
flag as meaning "uninteresting" or "likely not the edit you're looking
for".  This is the same function that "this is a minor edit" on
wikipedia has.

Cheers

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] generate_xml.py not generating osm.xml file?

[Sajjad Anwar]

Dear Akos

>
> I did read the README file, but this seems to be a catch-22 here. if it
> uses osm.xml itself to generate osm.xml - how do you initially create
> the first osm.xml, from the style sheets / templates?
>
> The osm.xml file is a bit confusing in this scenario. You can see that the
osm.xml file that came along with Mapnik is not the dataset, rather, it
contains certain rules which support the rendering of tiles. You are
actually loading the map data into the pgsql db and asking *
generate_image.py *or*generate_tiles.py *to generate the image or set of
tiles as the case may be.

If you want to make some modifications in the styles and fonts for the
rendering, you could tweak this rule file.
Also note that, Mapnik does not intend you to give the data in xml format,
but it reads the data only from the pgsql database.

The following link contains pointers to setting up Mapnik and basic rules
file modifications
http://sajjad.in/mapping/setting-up-mapnik-for-rendering-tiles-in-your-language/


Hope
this helps

Cheers
-- 
Sajjad Anwar | http://geohackers.in | Blog: http://sajjad.in | Twitter:
geohacker
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] generate_xml.py not generating osm.xml file?

[Sajjad Anwar]

Taking the freedom to post the email that skipped the list


From: Ákos Maróy 
Date: Fri, Aug 27, 2010 at 7:43 PM
Subject: Re: [OSM-dev] generate_xml.py not generating osm.xml file?
To: Sajjad Anwar 


Sajjad,

> I think you have misunderstood the purpose of generate_xml.py
> generate_xml.py as far as I understand is to prepare the osm dataset (by
> default osm.xml) ready for rendering. It fetches the style sheets,
> symbol information etc and modifies the dataset to create the new osm.xml
>
> The README coming with mapnik also explains its use.

thanks for the answer.

I did read the README file, but this seems to be a catch-22 here. if it
uses osm.xml itself to generate osm.xml - how do you initially create
the first osm.xml, from the style sheets / templates?


Akos

-- 
Sajjad Anwar | http://geohackers.in | Blog: http://sajjad.in | Twitter:
geohacker
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] generate_xml.py not generating osm.xml file?

[Sajjad Anwar]

Dear Akos,

I think you have misunderstood the purpose of generate_xml.py
generate_xml.py as far as I understand is to prepare the osm dataset (by
default osm.xml) ready for rendering. It fetches the style sheets, symbol
information etc and modifies the dataset to create the new osm.xml

The README coming with mapnik also explains its use.

Regards.

-- 
Sajjad Anwar | http://geohackers.in | Blog: http://sajjad.in | Twitter:
geohacker
___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Wishlist API/db: bot-flag WAS new contributor terms ID list

[M∡rtin Koppenhoefer]

2010/8/27 andrzej zaborowski :
>>>  I've ...also used bot=yes on many manual edits,
>> IMHO you shouldn't do this.
> Why?


because thereby you are flagging them as automated edits which they aren't.

Cheers,
Martin

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

[OSM-dev] generate_xml.py not generating osm.xml file?

[Ákos Maróy]

Hi,

Maybe I'm misunderstanding the generic purpose of the generate_xml.py
file from the mapnik rendering toolset, but for me, it does not seem to
touch my osm.xml file in any way.

when I execute generate_xml.py with appropriate database setttings,
osm.xml is not changed in any way. if I remove osm.xml, it is not
generated by the gall to generate_xml.py in any way.

am I misunderstanding that generate_xml.py would actually create osm.xml
based on the contents of the PostGIS database it connects to, and based
on the templates in the inc directory?


Akos

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Wishlist API/db: bot-flag WAS new contributor terms ID list

[andrzej zaborowski]

On 27 August 2010 12:16, M∡rtin Koppenhoefer  wrote:
> 2010/8/27 andrzej zaborowski :
>> While we're at it I want to point out that anyone can tag their edits
>> with bot=yes if their changes are likely to be uninteresting to a
>> casual mapper looking for something in the history tab.  I've ...also used 
>> bot=yes on many manual edits,
>
> IMHO you shouldn't do this.

Why?

Cheers

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] Wishlist API/db: bot-flag WAS new contributor terms ID list

[M∡rtin Koppenhoefer]

2010/8/27 andrzej zaborowski :
> While we're at it I want to point out that anyone can tag their edits
> with bot=yes if their changes are likely to be uninteresting to a
> casual mapper looking for something in the history tab.  I've ...also used 
> bot=yes on many manual edits,


IMHO you shouldn't do this.

cheers,
Martin

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

[OSM-dev] searching for good document of binary map formats

[feverzsj]

hi, list:
  I've started to develop a new binary map format for display, geocoding and
possibly routing on mobile device. After checking out some active routing
projects, I found most of them have no or poor document for their binary
format.
  Navit actually has a page about their binary format, but almost just an
outline. For Gosmore, there was once a post about its format (about 2 yeas
ago?), which was also not so detailed.
  Several years ago, I've developed a binary map using zoom-level based sub
r*-tree of map tiles. It's very fast even for low end mobile device, but
missing feature like geocoding and routing.
  So, it will be grateful, if someone can offer me some good documents.


  --Regards, zsj
-- 
View this message in context: 
http://gis.638310.n2.nabble.com/searching-for-good-document-of-binary-map-formats-tp5468479p5468479.html
Sent from the Developer Discussion mailing list archive at Nabble.com.

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev

Re: [OSM-dev] size of apidb

[Tom Hughes]

On 27/08/10 07:53, Marco Lechner - FOSSGIS e.V. wrote:


What is or"where can i get the recent size of the OSM-apidb?


http://munin.openstreetmap.org/openstreetmap/smaug.openstreetmap/postgres_size_openstreetmap.html

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
http://lists.openstreetmap.org/listinfo/dev