[OSM-dev-fr] [info] liste pour ceux qui jouent avec des serveurs de tuiles
Des fois que ça permette de regrouper nos combines, j'ai vu que cette liste venait d'être créée : http://lists.openstreetmap.org/listinfo/tile-serving ps: ça fait peut-être un peu doublons avec dev, mais en même temps c'est un peu plus spécifique -- sly (sylvain letuffe) ___ dev-fr mailing list dev-fr@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev-fr
[OSM-dev] Seeking Project Ideas for GSoC
Hi All, The application deadline for GSoC is in a few days (March 29th). We could use a few more project ideas to round things out: http://wiki.openstreetmap.org/wiki/Google_Summer_of_Code/2013/Project_Ideas Do you have any? Please add them to the wiki. Ones you are interested in mentoring would be specifically great, but not required. -Kate ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
Le 26/03/2013 17:16, Kai Krueger a écrit : There have been some major refactorings a couple of days ago in mod_tile / renderd to support new storage backends. That is when the error you reported got introduced. So if you take a snapshot from prior to March 23rd it should be more stable. However, Fedora seems to have upgraded to Apache 2.4, and until a commit 2 days ago, mod_tile had build issues as the apache 2.4 and 2.2 apis are not compatible. I am also hopping to expand http://ci.openstreetmap.org/ to provide automatic (build) testing on a variety of different platforms, including Fedora 18, so that errors with incompatibilities between systems can be spotted faster. So far there are no releases or stable branches of mod_tile / renderd or osm2pgsql. However, as things mature and more and more people rely on them, it is time to have a more proper release process for these software. Kai Hello Kai, I stopped using renderd because of the crashes I got (it can be a thread race condition bug since the bug disappears when running renderd with valgrind). I fully agree about the need of a real release process: it took me many days to figure how to have Nominatim and Tirex running, mainly because the information is split in different wiki pages written from 2010 to early 2013 (I ignored ealier info) and none of them reports the release numbers used when wiki entries were written. One also have to search in mailing list to fix issues that show up one after the other. It was a painful road, nearly each step of the installation brings a new step that does not work at first, it may be necessary to backtrack to some earlier step, change of version/package, retry, etc. For instance F18 provides postgreql 9.2 but postgis 1.5: one has to get postgis 2 but also to apply legacy.sql, bring stuff from mapnik source code, get files (like coastlines) from here and there, figure how to configure postgresql (that I never used before) etc. I'm actually trying to configure a secondary system to check that I didn't forget to note something, to be able to reinstall a system if this is needed in the future: I have currently a list of about *60* different things to do one after the other and I still have to add Tirex setup (and pray that the resulting system is a working one). I have to save copies of the versions of the different projects to be sure of what I use since I can't rely on stable versions of different components. And if someone is using that list in a couple of weeks, some parts will have to be updated/rechecked etc. so the installation is messy at best. Bernard ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
For the record, methods described in http://switch2osm.org works rather well, thanks to Kai's packages. Maybe packaging work could be shared with https://launchpad.net/~ubuntugis too. I somewhat would expect better results in building an osm tool-chain on a debian-based system (Ubuntu), maybe F18 wasn't the good choice at the beginning. Yves ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
Le 27/03/2013 10:39, yvecai a écrit : For the record, methods described in http://switch2osm.org works rather well, thanks to Kai's packages. Maybe packaging work could be shared with https://launchpad.net/~ubuntugis too. I somewhat would expect better results in building an osm tool-chain on a debian-based system (Ubuntu), maybe F18 wasn't the good choice at the beginning. Yves We have everything else on F-something, I want to avoid maintaining different distros, what I may have spared using Ubuntu for the install I lose it in the long term by having to support another distro. My time was mostly spent because of the lack of stable releases and documentation related to package versions, quirks here and there (for instance when making Tirex there is no check of what perl packages are already available, you discover the list of missing packages when Tirex runs, or how do you know that you'll need to setup a definition for a 'default' map?) , broken SVN trunk of mod_tile, cryptic error messages, configuration magic (when applying postgresql/system recommended values for a 32GB system I saw it swapping, the values do not consider the number of running cores in the system), some broken URL (get that file here and you get a 404), etc., and of course, never having used OSM related tools before. Using Ubuntu may have save time on postgres/postgis issues, probably not much else. Bernard ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
On Wed, Mar 27, 2013 at 2:23 PM, Bernard Fouché bernard.fou...@kuantic.com wrote: My time was mostly spent because of the lack of stable releases and documentation related to package versions, quirks here and there (for instance when making Tirex there is no check of what perl packages are already available, you discover the list of missing packages when Tirex runs, or how do you know that you'll need to setup a definition for a 'default' map?) , broken SVN trunk of mod_tile, cryptic error messages, configuration magic OSM data tools are mostly developed by volunteers on their free time. Feel free to improve the wiki documentation if you find issues or incompleteness. Creating an account for wiki edition takes less time than writing your last 5 messages. Do something positive for the project instead of complaining. Thus, the next one installing a tile server on F18 will be grateful for your help. Pieren ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
Le 27/03/2013 15:09, Pieren a écrit : On Wed, Mar 27, 2013 at 2:23 PM, Bernard Fouché bernard.fou...@kuantic.com wrote: My time was mostly spent because of the lack of stable releases and documentation related to package versions, quirks here and there (for instance when making Tirex there is no check of what perl packages are already available, you discover the list of missing packages when Tirex runs, or how do you know that you'll need to setup a definition for a 'default' map?) , broken SVN trunk of mod_tile, cryptic error messages, configuration magic OSM data tools are mostly developed by volunteers on their free time. Feel free to improve the wiki documentation if you find issues or incompleteness. Creating an account for wiki edition takes less time than writing your last 5 messages. Do something positive for the project instead of complaining. Thus, the next one installing a tile server on F18 will be grateful for your help. Pieren I apologize if the feed back I gave seems to be a complaint, it is not, I'm very happy to have an alternative to Google Maps and similar systems. I can update a wiki page, but what wiki page? When googling openstreetmap fedora install there are different pages that show up and I had to look at all of these. What should I write about obtaining mod_tile or osm2pgsql? What revisions a user is supposed to get to have a working system? I can list the revision I have but in a few days, or even already, it may be better to get a later version: I'm the last to be able to declare that a particular version is better than another. At the moment I can write the complete set of things I had to consider to have a working system, that would give a general view of the set of problems to handle, but it will be far from a do this and it will 100% work recipe, mostly because of the lack of stable versions for the most important components of the tool chain and the lack of a set of stable versions known to work correctly together. I don't complain, I supplicate stable versions :-) Bernard ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
Bernard Fouché bernard.fou...@kuantic.com wrote: My time was mostly spent because of the lack of stable releases and documentation related to package versions, quirks here and there (for instance when making Tirex there is no check of what perl packages are already available Hm, this is at least kind of a distro problem. If debian packages are build the required packages are installed automatically via dependencies. Sven -- Der wichtigste Aspekt, den Sie vor der Entscheidung für ein Open Source-Betriebssystem bedenken sollten, ist, dass Sie kein Windows-Betriebssystem erhalten. (von http://www.dell.de/ubuntu) /me is giggls@ircnet, http://sven.gegg.us/ on the Web ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
On 27 March 2013 14:09, Pieren pier...@gmail.com wrote: OSM data tools are mostly developed by volunteers on their free time. And I'm sure we all strive to make high-quality tools. If someone encounters bugs, like crashes and broken documentation, then even just being told by email is valuable, or logging tickets, or mentioning it on IRC. There's no need to round on them for not having fixed all the problems single-handedly. Of course, some people have unreasonable expectations, and we all tire of their demands. But things like being able to compile the software is a perfectly reasonable expectation. Feel free to improve the wiki documentation if you find issues or incompleteness. Creating an account for wiki edition takes less time than writing your last 5 messages. Do something positive for the project instead of complaining. Thus, the next one installing a tile server on F18 will be grateful for your help. And this is one of the reasons our documentation is such a mess. With the best will in the world, someone who doesn't understand the components and has just managed to get them working - for the very first time - isn't in the best position to write clear documentation. The osm2pgsql page on the wiki, like many others, is degrading over the years as people add to it and nobody edits it. Such pages are not converging towards great documentation, they start off as useful documentation and slowly degenerate, while accumulating increasing amounts of cruft, inconsistencies and even inline-patches that you are encouraged to apply. Sheesh. And that doesn't even count the increasing numbers of other pages you can stumble across which duplicate one another and confuse anyone who reads them. So think twice before telling a new user that they should start editing wiki pages. Cheers, Andy ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
On 03/27/2013 03:12 AM, Bernard Fouché wrote: Le 26/03/2013 17:16, Kai Krueger a écrit : There have been some major refactorings a couple of days ago in mod_tile / renderd to support new storage backends. That is when the error you reported got introduced. So if you take a snapshot from prior to March 23rd it should be more stable. However, Fedora seems to have upgraded to Apache 2.4, and until a commit 2 days ago, mod_tile had build issues as the apache 2.4 and 2.2 apis are not compatible. I am also hopping to expand http://ci.openstreetmap.org/ to provide automatic (build) testing on a variety of different platforms, including Fedora 18, so that errors with incompatibilities between systems can be spotted faster. So far there are no releases or stable branches of mod_tile / renderd or osm2pgsql. However, as things mature and more and more people rely on them, it is time to have a more proper release process for these software. Kai Hello Kai, I stopped using renderd because of the crashes I got (it can be a thread race condition bug since the bug disappears when running renderd with valgrind). I think that crash was a buffer overflow and should also now be fixed, thanks to Jon spotting the error and the help of some people on the irc channel #osm-dev. This bug was also introduced just 4 days days ago. Normally mod_tile and renderd should be pretty stable with not many bigger changes. So normally taking the latest SVN snapshot should work fairly reliable. Unfortunately you seem to have hit just the two or three days where I committed some bigger changes to help mod_tile scale up to larger installations that caused some instability. I fully agree about the need of a real release process: it took me many days to figure how to have Nominatim and Tirex running, mainly because the information is split in different wiki pages written from 2010 to early 2013 (I ignored ealier info) and none of them reports the release numbers used when wiki entries were written. Yes, the information is split into too many different wiki pages, which makes keeping them updated pretty much an impossibility. I think that is at least partly the reason why Richard started the webpage switch2osm.org to have a single source of good documentation of how to set things up. Of cause as any documentation (in OSM) there is also room for improvement and expansion but I think that is probably the most up to date and clear descriptions of how to set up the tile rendering infrastructure. It could probably do with an equivalent description for nominatim. One also have to search in mailing list to fix issues that show up one after the other. It was a painful road, nearly each step of the installation brings a new step that does not work at first, it may be necessary to backtrack to some earlier step, change of version/package, retry, etc. Well, it is a system with many components. I always try to make the installation process as easy as possible and improve the error messages to more easily track down setup issues, but the fact that every single linux distribution has its own set of versions of key components, has different packaging systems, puts files into different locations and therefore unique set of issues doesn't directly make that easier. And that is without even touching other unix derivatives like freeBSD, Solaris or Mac OSX, let alone Windows. As mentioned I'd like to try and automate more of the testing on different systems, but so far that infrastructure doesn't exist in OSM. So all we have to go on at the moment are bug reports by people If you have any suggestions on how to improve the situation further, I'd be interested to here them. For instance F18 provides postgreql 9.2 but postgis 1.5: one has to get postgis 2 but also to apply legacy.sql, bring stuff from mapnik source code, get files (like coastlines) from here and there, figure how to configure postgresql (that I never used before) etc. Well, for Ubuntu I have created the PPA packages as linked to on switch2osm.org. Those try and take care of all the necessary steps as much as possible. To the extent even that they make many default decision on things and do other magic setup steps which violate the official packaging guidelines. However, as mentioned above, it is unfortunately difficult to create similar packages for all the different systems, as there are simply not enough developers around to do that. However, there are definitely things that can and should be improved, and figuring out a better release process is clearly one of them. Kai I'm actually trying to configure a secondary system to check that I didn't forget to note something, to be able to reinstall a system if this is needed in the future: I have currently a list of about *60* different things to do one after the other and I still have to add Tirex setup (and pray that the resulting system is a working one). I have
[OSM-dev] PPA vs 64bit IDs
On 3/27/2013 2:03 PM, Kai Krueger wrote: Well, for Ubuntu I have created the PPA packages as linked to on switch2osm.org. Those try and take care of all the necessary steps as much as possible. To the extent even that they make many default decision on things and do other magic setup steps which violate the official packaging guidelines. However, as mentioned above, it is unfortunately difficult to create similar packages for all the different systems, as there are simply not enough developers around to do that. I'm just about to rebuild my tile server and was wondering if the PPA packages fully support the 64bit IDs? Lynn (D) ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
On Wed, Mar 27, 2013 at 6:33 PM, Andy Allan gravityst...@gmail.com wrote: So think twice before telling a new user that they should start editing wiki pages. Instead of 'new user', I would say 'the end users, those from whom the documentation is intended'. Since the devs do not take a high priority to keep the wiki up-to-date and consistent (sigh), anybody else taking the time to search, read the docs with new eyes, follow the process, find and fix issues for himself, etc is able to improve the doc for the next ones. I said improve the doc, not write another 'how-to setup my own tile server'. Pieren ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
Andy Allan wrote On 27 March 2013 14:09, Pieren lt; pieren3@ gt; wrote: OSM data tools are mostly developed by volunteers on their free time. And I'm sure we all strive to make high-quality tools. If someone encounters bugs, like crashes and broken documentation, then even just being told by email is valuable, or logging tickets, or mentioning it on IRC. Indeed, although we as developers obviously try and catch as many bugs and variants as possible before committing things, in the end we do rely on people reporting bugs so that they can be fixed. And reporting bugs often does work. For example I just had a very pleasant experience with that today. A couple of weeks (or months ago), I was trying to get renderd working with the latest snapshot of mapnik. It failed due to a projection bug in (the unstable branch of) mapnik. However, instead of reporting it, I just used mapnik 2.1 instead. Now yesterday some one else on IRC ran into the same issue again and wasted more time with it. After finally reporting it, the bug was fixed in less than 24 hours and no one else needs to run into the issue any more. And in the process, I needed to know nothing about the internals of Mapnik or be able to debug it, so it is something that anyone can do. Granted, the bug reports in mod_tile / renderd have not always been dealt with that quickly. Partly I think it is due to the fact that there are too many channels to report bugs through. E.g. the osm track ticketing system, the github osm repository issue tracking, the various forums, help.osm.org the mailing list, irc, the wiki and other random pages were people mention / complain about problems. I am hoping to improve the situation, yes by creating yet another communication channel... ;-) There is a new mailing list tile-serving ( http://lists.openstreetmap.org/listinfo/tile-serving ) where hopefully all those bug reports from different sources can get collated and make sure they reach all the people who can potentially fix them. Hopefully that will improve the response time to bug reports and thus make overall better and more reliable software. Andy Allan wrote There's no need to round on them for not having fixed all the problems single-handedly. Of course, some people have unreasonable expectations, and we all tire of their demands. But things like being able to compile the software is a perfectly reasonable expectation. Indeed, it would be helpful if users better realise the limitations of developers and their time to spend on hoby projects, but at least as useful would be if developers don't always respond to requests with the Oh it is open source, go fix it your self attitude, as fixing it your self is clearly not a reasonable response for the majority of people who don't have the necessary specific skills. Kai -- View this message in context: http://gis.19327.n5.nabble.com/mod-tile-stable-version-tp5754723p5754943.html Sent from the Developer Discussion mailing list archive at Nabble.com. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
Le 27/03/2013 19:13, Pieren a écrit : On Wed, Mar 27, 2013 at 6:33 PM, Andy Allan gravityst...@gmail.com wrote: So think twice before telling a new user that they should start editing wiki pages. Instead of 'new user', I would say 'the end users, those from whom the documentation is intended'. Since the devs do not take a high priority to keep the wiki up-to-date and consistent (sigh), anybody else taking the time to search, read the docs with new eyes, follow the process, find and fix issues for himself, etc is able to improve the doc for the next ones. I said improve the doc, not write another 'how-to setup my own tile server'. Pieren ___ No problem with Pieren's message, I know how painful it is when you work all day on something and a newcomer complains because of a free lunch not included yet. Anyway I'm close to finish an ansible playbook to setup a VM just installed with the core F18 packages, to have a working tile server + nominatim. I choose ansible because it is extremely easy to setup and requires no server in the system being targeted, but sshd and to have setup ~/.ssh/authorized_keys. It seems a reasonable approach to list exhaustively every step that must be done. Of course this playbook needs to be checked by people knowing the different OSM components, I probably have done very stupid things and I also use ansible for a few days and didn't take time to make the playbook a very nice one (for instance I had problems understanding symlinks handling so I just remove them and 'ln -fs' them afterwards). So much to do and not much time available so I have to follow an aggressive approach about everything these days :-) Bu there is the problem of copying files (.tgz/conf/.sh/etc.) that I built during the process. For instance for Tirex I applied such a patch: Index: backend-mapnik/Makefile === --- backend-mapnik/Makefile (revision 29404) +++ backend-mapnik/Makefile (working copy) @@ -1,7 +1,7 @@ INSTALLOPTS=-g root -o root CFLAGS += -Wall -Wextra -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 CXXFLAGS = $(CFLAGS) `mapnik-config --cflags` -LDFLAGS= `mapnik-config --libs --ldflags --dep-libs` +LDFLAGS= `mapnik-config --libs --ldflags --dep-libs| sed -e s/-l.agg.//` backend-mapnik: renderd.o metatilehandler.o networklistener.o networkmessage.o networkrequest.o networkresponse.o debuggable.o requesthandler.o $(CXX) -o $@ $^ $(LDFLAGS) Index: backend-mapnik/metatilehandler.cc === --- backend-mapnik/metatilehandler.cc (revision 29404) +++ backend-mapnik/metatilehandler.cc (working copy) @@ -23,7 +23,7 @@ #include mapnik/datasource_cache.hpp #include mapnik/font_engine_freetype.hpp #include mapnik/agg_renderer.hpp -#include mapnik/expression.hpp +//#include mapnik/expression.hpp #include mapnik/image_util.hpp #include mapnik/load_map.hpp #include mapnik/box2d.hpp So for Tirex I made up a .tgz of the revision I got including this patch, and have ansible to copy it to the target and recompile it locally. I also copy mapnik fonts, there still a few 'wget svn..' (with no revision number ;-)), etc: this is purely an approach to reach a working system and to avoid digging into the details to make it a perfect one. Bernard ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] announcing a new mailinglist tile-serving
Hello everyone, I would like to announce a new OSM mailinglist called tile-serving [1] to discuss and aid the development of the OSM tile serving infrastructure. Its purpose is modelled after the rails-dev, geocoding or osrm mailinglists that already exist. One of its primary purposes is to collate all of the various bug reporting sources (like trac and github) and ensure that bug reports get sufficient exposure to the developers to try and resolve the reported issues in a timely manor. At the moment it feels like bug reports don't get the necessary attention. It is also a place to discuss the development and feature requests as well as documentation of those components, although the wider issues might still be discussed on dev. In light of the recent emails, perhaps one of the first things one could talk about is to try and set up a release process for mod_tile / renderd / tirex / osm2pgsl. So it would be great if all the developers of these components could sign up to the new mailing list. Thanks, Kai [1] http://lists.openstreetmap.org/listinfo/tile-serving ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] mod_tile stable version ?
For instance for Tirex I applied such a patch: Index: backend-mapnik/Makefile === --- backend-mapnik/Makefile (revision 29404) +++ backend-mapnik/Makefile (working copy) @@ -1,7 +1,7 @@ INSTALLOPTS=-g root -o root CFLAGS += -Wall -Wextra -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 CXXFLAGS = $(CFLAGS) `mapnik-config --cflags` -LDFLAGS= `mapnik-config --libs --ldflags --dep-libs` +LDFLAGS= `mapnik-config --libs --ldflags --dep-libs| sed -e s/-l.agg.//` ^^ this looks either like a mapnik-config bug or a packaging bug. Working around in tirex is fine, but personally I would want to understand why -lagg is in the config flags in the first place before fixing this way. As far as I can tell there is not a mapnik version I see that reports -lagg like this. Can you provide more details about how you installed mapnik and the exact version of mapnik you see this with to a support ticket at: https://github.com/mapnik/mapnik/issues Also, if you are using mapnik from some fedora package, then this may be a bug in fedora packaging. Please provide details. Index: backend-mapnik/metatilehandler.cc === --- backend-mapnik/metatilehandler.cc (revision 29404) +++ backend-mapnik/metatilehandler.cc (working copy) @@ -23,7 +23,7 @@ #include mapnik/datasource_cache.hpp #include mapnik/font_engine_freetype.hpp #include mapnik/agg_renderer.hpp -#include mapnik/expression.hpp +//#include mapnik/expression.hpp Thanks for reporting, this is now fixed: https://trac.openstreetmap.org/changeset/29407/subversion Note that I am not a tirex user or developer and I did not test this but as a Mapnik developer I know the header is both 1) not needed and 2) was only added in Mapnik 2.x, so including it makes for harder compatibility with multiple versions of Mapnik. So, nice catch. So for Tirex I made up a .tgz of the revision I got including this patch, and have ansible to copy it to the target and recompile it locally. I also copy mapnik fonts, You should not need to copy mapnik fonts around. Sounds like a configuration problem or something messed up with the install. Dane ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
