Re: [OSM-dev-fr] Osmose - patch - code INSEE dans les relations admin_level=7
Le 14/07/2011 07:44, Vincent de Chateau-Thierry a écrit : = L'analyse actuelle deviendrait celle (stricte) du niveau Communes. Il y a au moins deux autres familles de ref:INSEE en cours de propagation : celle des sous-prefectures (admin_level=7) et celle des EPCIs (boundary=local_authority). C'est évoqué ici : http://wiki.openstreetmap.org/wiki/WikiProject_France/Limites_administratives/Tracer_les_limites_administratives/Bascule_des_admin_level%3D7_en_arrondissements_d%C3%A9partementaux Le tag ref:INSEE n'est pas approprié pour identifier autres choses que des communes. Le sous-entendu commun code INSEE (des communes) ne peut s'appliquer ni aux arrondissements ni aux EPCI. Côté arrondissements, l'INSEE propose dans ce fichier : http://insee.fr/fr/methodes/nomenclatures/cog/telechargement/2011/txt/arrond2011.txt la liste des arrondissements. Le code qui nous intéresse est à recomposer en concaténant les colonnes 2 (DEP) et 3 (AR). Pour les arrondissements la codification (DEP+AR) à 3/4 chiffres ne doit pas être contenu dans un tag ref:INSEE. Le simple tag ref serait plus approprié ici. De nombreuses codifications/nomenclatures sont gérées par l'INSEE, on ne peut pas taguer tous les codes par ref:INSEE. Au lieu de : *admin_level* = 7 *boundary* = administrative *type* = boundary *ref:INSEE* = 784 *name* = Versailles Je ne propose pas quelque chose comme : *admin_level* = 7 *boundary* = administrative *type* = boundary _*ref* = 784 *ref:type* = arrondissement *ref:operator* = INSEE _*name* = Versailles mais tout simplement : *admin_level* = 7 *boundary* = administrative *type* = boundary _*ref*_ = 784 *name* = Versailles Côté EPCIs, l'INSEE encore a un gros fichier téléchargeable ici : http://insee.fr/fr/methodes/default.asp?page=zonages/intercommunalite.htm qui donne la liste des EPCIs et leur code. Le code utilisé comme référence dans ce fichier est un numéro SIREN (sans E). Il convient d'être tagué ref:SIREN. Soit pour la communauté d'agglomérationVersailles Grand Parc : _ref:SIREN_ = 247800584 L'INSEE a pour mission de gérer le répertoire SIRENE (avec un E) : L'INSEE attribue un numéro SIREN à tous les établissements publiques quelque soit leur mission et plus généralement à toutes les entreprises de France. Aujourd'hui, dans OSM des communes et des EPCI sont tagués ref:SIREN. Demain, dans OSM mon épicerie sera tagué ref:SIREN avec un numéro à 9 chiffres ou ref:SIRET avec un numéro à 14 chiffres. Mais mon épicerie ne sera jamais tagué ref:INSEE. phillippekerla ___ dev-fr mailing list dev-fr@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev-fr
Re: [OSM-dev-fr] Osmose et last-update.py
Le 13 juillet 2011, Frédéric Rodrigo a écrit : J'en avais proposé un il y a quelques mois pour ma part, sur cette liste. Il était passé un peu inaperçu apparemment, et ne sachant trop qui s'occupait d'Osmose, je n'étais pas allé plus loin. Ce patch s'applique à Name_Toponymie.py et permet de régler le problème des (nombreux) noms de lieu bretons contenant « c'h », dont l'apostrophe est détectée comme une coupure de mot, ce qui déclenche une erreur de toponymie due à la majuscule supposément manquante au « h ». En gros il y a plein de faux positifs. Désolé de ne pas encore l'avoir traité... ton patch dans ma liste de trucs à faire :/ Ça m'a l'air bien compliqué ton patch :) Je le mets de côté pour j'aurais un peu plus de temps. En attendant, est-ce que tu pourras donner un exemple d'URL où l'analyse plante ? Il y a un SVN mais bien au chaud sur le serveur d'osmose... C'est plutôt un git en fait. Enfin deux, un pour le backend, et l'autre pour le frontend. Je ne peux pas dans l'état les rendre public parce qu'ils contiennent des mots de passes. Je verrais ce que je peux faire en août. Merci, Jocelyn ___ dev-fr mailing list dev-fr@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev-fr
Re: [OSM-dev-fr] [OSM-talk-fr] Requêtes SQL spatiale
J'ai commencer à apprendre en regardant/comprenant les requêtes qui passent sur la liste ML (-DEV), comme par exemple : http://www.mail-archive.com/dev-fr@openstreetmap.org/msg00074.html http://www.mail-archive.com/dev-fr@openstreetmap.org/msg1.html Il y a d'autres requêtes très intéressantes qui sont passées, il faut les rechercher. (pour faire ce que tu demande) Le 13 juillet 2011 23:46, Vincent de Chateau-Thierry v...@laposte.net a écrit : Bonjour, Le 14/07/2011 08:13, Frédéric Benninger a écrit : Petite question, j'ai importé des données osm avec osm2pgsql dans ma db gis. gis=# \d Liste des relations Schéma |Nom | Type | Propriétaire ++**---+-- public | geography_columns | vue | mapmaker public | geometry_columns | table | mapmaker public | planet_osm_line| table | mapmaker public | planet_osm_nodes | table | mapmaker public | planet_osm_point | table | mapmaker public | planet_osm_polygon | table | mapmaker public | planet_osm_rels| table | mapmaker public | planet_osm_roads | table | mapmaker public | planet_osm_ways| table | mapmaker public | spatial_ref_sys| table | mapmaker Maintenant j'aimerais savoir comment faire pour lancer quelques requêtes s'appuyant sur le moteur spatial. Par exemple: Compter le nombre de café dans ma commune. Me donner la boite au lettre la plus proche, etc qqun a-t-il un site avec des exemples pour bien démarrer? Je suis un peu perdu avec l'extension spatiale et la dénormalisation de la base. Je n'ai pas de tutorial sous la main mais côté doc, tes requêtes vont s'appuyer sur les opérateurs listés ici : http://www.postgis.org/**documentation/manual-1.5/** reference.html#Spatial_**Relationships_Measurementshttp://www.postgis.org/documentation/manual-1.5/reference.html#Spatial_Relationships_Measurements. Tu as des exemples pour chaque opérateur. N'hésite pas à poursuivre la discussion sur la liste dev-fr (en cc ici), où les syntaxes SQL et autres sont les bienvenues. vincent __**_ dev-fr mailing list dev-fr@openstreetmap.org http://lists.openstreetmap.**org/listinfo/dev-frhttp://lists.openstreetmap.org/listinfo/dev-fr ___ dev-fr mailing list dev-fr@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev-fr
[OSM-dev] Merging Changes into Planet
Hi, If I download the latest set of hourly changes and merge them into planet.osm.pbf using osmosis, this always takes about two hours to complete: osmosis --read-xml-change changes.osc --read-bin planet-latest.osm.pbf --apply-change --write-bin planet-new.osm.pbf This is on an Ubuntu-based quad core server with 8GB of RAM so I think the limitation is disk speed. Does this merge time seem right/reasonable? Are there any approaches or tricks I am missing that can speed it up? thanks, Andy -- Andy PGP Key ID: 0xDC1B5864 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Merging Changes into Planet
2011/7/14 Andrew Ayre a...@britishideas.com: If I download the latest set of hourly changes and merge them into planet.osm.pbf using osmosis, this always takes about two hours to complete: Try to merge change files first, and then apply them as one change file. -- Darafei Komяpa Praliaskouski OSM BY Team xmpp:m...@komzpa.net mailto:m...@komzpa.net ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Merging Changes into Planet
On Thu, Jul 14, 2011 at 09:04:03AM +0100, Andrew Ayre wrote: If I download the latest set of hourly changes and merge them into planet.osm.pbf using osmosis, this always takes about two hours to complete: osmosis --read-xml-change changes.osc --read-bin planet-latest.osm.pbf --apply-change --write-bin planet-new.osm.pbf This is on an Ubuntu-based quad core server with 8GB of RAM so I think the limitation is disk speed. Its almost certainly CPU. Try dd if=planet-latest.osm.pbf of=copy.osm.pbf bs=1M and you'll see how fast the files can be copied. Thats the time needed for the disk. Everything else is CPU. Does this merge time seem right/reasonable? Are there any approaches or tricks I am missing that can speed it up? Try larger buffers: $OSMOSIS --read-xml-change $OSCFILE --read-bin $BASEDIR/var/current-planet.osm.pbf --buffer bufferCapacity=12000 --apply-change --buffer bufferCapacity=12000 --write-pbf $NEWPLANET This takes about an hour every day on a 800MHz machine. Jochen -- Jochen Topf joc...@remote.org http://www.remote.org/jochen/ +49-721-388298 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Merging Changes into Planet
On 7/14/2011 9:26 AM, Jochen Topf wrote: Its almost certainly CPU. Try dd if=planet-latest.osm.pbf of=copy.osm.pbf bs=1M and you'll see how fast the files can be copied. Thats the time needed for the disk. Everything else is CPU. Does this merge time seem right/reasonable? Are there any approaches or tricks I am missing that can speed it up? Try larger buffers: $OSMOSIS --read-xml-change $OSCFILE --read-bin $BASEDIR/var/current-planet.osm.pbf --buffer bufferCapacity=12000 --apply-change --buffer bufferCapacity=12000 --write-pbf $NEWPLANET This takes about an hour every day on a 800MHz machine. Jochen Thanks everyone for the suggestions. I am using osmosis to generate a single changes.osc file already. The server is in a server farm so I don't have control over the disks. The dd command took 6 minutes 40 seconds so that proves Jochen right. With my 766MB change file the original merge command that I posted takes 106 minutes. By using Jochen's merge command with buffers and the same change file it now takes 58 minutes! I knew about the possibility of buffers but I never tried them because [1] appeared quite negative regarding them. Maybe it is out of date? Thanks, Andy [1] http://wiki.openstreetmap.org/wiki/Osmosis/Benchmarking -- Andy PGP Key ID: 0xDC1B5864 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Merging Changes into Planet
On Thu, 14 Jul 2011 12:38:02 +0100, Andrew Ayre wrote: With my 766MB change file the original merge command that I posted takes 106 minutes. By using Jochen's merge command with buffers and the same change file it now takes 58 minutes! I knew about the possibility of buffers but I never tried them because [1] appeared quite negative regarding them. Maybe it is out of date? Benchmarking complex processes like this usually generate results that are very much tied to the configuration you benchmark with. As it says in the conclusion That is not to say it won't help in different applications or with different setups. It was tested with a very old version of osmosis (0.24 while 0.38 was already out) and maybe you are using a different type or make of processor. [1] http://wiki.openstreetmap.org/wiki/Osmosis/Benchmarking I'm sure adding your observations will be a valuable addition. Regards, Maarten ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Partial outage this weekend
Devs, The dev server (errol) will not be available this weekend due to power maintenance work at hosts. Further details and list of servers involved here: http://wiki.openstreetmap.org/wiki/Power_Maintenance_Q3_2011 www.osm.org and API are NOT affected. / Grant Part of Sysadmin Team. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Merging Changes into Planet
Hi Andy, there is another program which I would expect to be significantly faster. However, it has two disadvantages: 1. It's new, it still may have bugs. 2. It lacks in the ability to write PBF and uses .osm or the efficient but less common .o5m format instead. Nevertheless, it could be worth a try. There is an application example in the Wiki: http://wiki.openstreetmap.org/wiki/Daily_update_an_OSM_XML_file Markus Original-Nachricht Datum: Thu, 14 Jul 2011 12:38:02 +0100 Von: Andrew Ayre a...@britishideas.com An: dev dev@openstreetmap.org Betreff: Re: [OSM-dev] Merging Changes into Planet On 7/14/2011 9:26 AM, Jochen Topf wrote: Its almost certainly CPU. Try dd if=planet-latest.osm.pbf of=copy.osm.pbf bs=1M and you'll see how fast the files can be copied. Thats the time needed for the disk. Everything else is CPU. Does this merge time seem right/reasonable? Are there any approaches or tricks I am missing that can speed it up? Try larger buffers: $OSMOSIS --read-xml-change $OSCFILE --read-bin $BASEDIR/var/current-planet.osm.pbf --buffer bufferCapacity=12000 --apply-change --buffer bufferCapacity=12000 --write-pbf $NEWPLANET This takes about an hour every day on a 800MHz machine. Jochen Thanks everyone for the suggestions. I am using osmosis to generate a single changes.osc file already. The server is in a server farm so I don't have control over the disks. The dd command took 6 minutes 40 seconds so that proves Jochen right. With my 766MB change file the original merge command that I posted takes 106 minutes. By using Jochen's merge command with buffers and the same change file it now takes 58 minutes! I knew about the possibility of buffers but I never tried them because [1] appeared quite negative regarding them. Maybe it is out of date? Thanks, Andy [1] http://wiki.openstreetmap.org/wiki/Osmosis/Benchmarking -- Andy PGP Key ID: 0xDC1B5864 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Where to define the directory of generating tiles?
Hello Everyone, I was generating tiles for OSM Tile Server, by default tiles are being generated into /home/username/bin/mapnik From which file I can change the path to generate them in some other directory? Thank You. -- Parveen Arora www.parveenarora.in E-Mail: m...@parveenarora.in ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Merging Changes into Planet
Hi Markus, Thanks for the suggestion - I will take a look at it! Andy On 7/14/2011 3:41 PM, mar...@gmx.eu wrote: Hi Andy, there is another program which I would expect to be significantly faster. However, it has two disadvantages: 1. It's new, it still may have bugs. 2. It lacks in the ability to write PBF and uses .osm or the efficient but less common .o5m format instead. Nevertheless, it could be worth a try. There is an application example in the Wiki: http://wiki.openstreetmap.org/wiki/Daily_update_an_OSM_XML_file Markus ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Where to define the directory of generating tiles?
Hi Parveen, You do not say which program you are using to generate your tiles, so it is difficult to answer. I think you are using generate_tiles.py? In which case there is a command line parameter to specify the output directory. If you are thinking of a different program, please say which one. Regards Graham from my phone On 15 Jul 2011 00:00, Parveen Arora m...@parveenarora.in wrote: Hello Everyone, I was generating tiles for OSM Tile Server, by default tiles are being generated into /home/username/bin/mapnik From which file I can change the path to generate them in some other directory? Thank You. -- Parveen Arora www.parveenarora.in E-Mail: m...@parveenarora.in ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Where to define the directory of generating tiles?
On Fri, Jul 15, 2011 at 10:14 AM, Graham Jones grahamjones...@gmail.com wrote: Hi Parveen, You do not say which program you are using to generate your tiles, so it is difficult to answer. I think you are using generate_tiles.py? Yes, I am using generate_tiles.py In which case there is a command line parameter to specify the output directory. Can we define Inside the generate_tiles.py that where to generate tiles.? -- Parveen Arora www.parveenarora.in E-Mail: m...@parveenarora.in ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
