Re: [OSM-dev] C++ implementation of the API - Testing

2008-06-04 Thread Alex Wilson
Hello all, We're having some success with the C++/apache implementation of the API - an implementation of the map query up and running which returns appropriate ways, nodes relations and their associated tags and members. It also seems that the Cherokee/MonetDB version is at a similar stage. So

Re: [OSM-dev] C++ implementation of the API

2008-05-30 Thread Alex Wilson
I'm taking the printf method below - except streaming out using the apache module interface rather than stdout. I'm also streaming as much stuff from the db as possible (nodes, ways, . However I believe for the map query there's a trade-off between local process memory usage and minimising db

Re: [OSM-dev] C++ implementation of the API

2008-05-30 Thread Alex Wilson
Ah. good point - a sensible third way. Apologies for missing it, I'm a relative SQL novice. However, if you fetch the tags along with the ways using a join, given that there are potentially many tags per way - couldn't you end up with considerable duplication of the way data in the query result? I

Re: [OSM-dev] C++ implementation of the API

2008-05-28 Thread Frederik Ramm
Hi, The problem here is the remedial action taken by the user/editor: they're going to try to work out the conflicts by downloading the latest version of the region. If this comes off a replication server that's a couple of minutes behind, You could make it so that the intial map query is

Re: [OSM-dev] C++ implementation of the API

2008-05-28 Thread Dave Stubbs
On Wed, May 28, 2008 at 10:07 AM, Frederik Ramm [EMAIL PROTECTED] wrote: Hi, The problem here is the remedial action taken by the user/editor: they're going to try to work out the conflicts by downloading the latest version of the region. If this comes off a replication server that's a

Re: [OSM-dev] C++ implementation of the API

2008-05-26 Thread Tom Hughes
2008/5/26 Stefan de Konink [EMAIL PROTECTED]: How should one benchmark his solution? Just put the data in and fetch? Well that is certainly a good start - compare the time taken to fetch an area with the time taken by the current code on the same machine. It is also worth comparing the maximum

Re: [OSM-dev] C++ implementation of the API

2008-05-26 Thread Stefan de Konink
On Mon, 26 May 2008, Tom Hughes wrote: 2008/5/26 Stefan de Konink [EMAIL PROTECTED]: How should one benchmark his solution? Just put the data in and fetch? Well that is certainly a good start - compare the time taken to fetch an area with the time taken by the current code on the same

Re: [OSM-dev] C++ implementation of the API

2008-05-26 Thread Joachim Zobel
Am Montag, den 26.05.2008, 15:07 +0100 schrieb Tom Hughes: The reason is that we have to allow about 600Mb or so for each call and that quickly mounts up as you try and add extra simultaneous accesses. If _that_ amount of memory is needed this probably means the XML is build in memory. This

Re: [OSM-dev] C++ implementation of the API

2008-05-26 Thread Frederik Ramm
Hi, If _that_ amount of memory is needed this probably means the XML is build in memory. This could be done the SAX way instead. Looking at the code, which is publicly available, would quickly have told you that neither a DOM tree nor a SAX approach is used on output. It's plain old DIY XML.

Re: [OSM-dev] C++ implementation of the API

2008-05-26 Thread Tom Hughes
In message [EMAIL PROTECTED] Frederik Ramm [EMAIL PROTECTED] wrote: Do we need an Apache server? Why not writing (in c/c++) a small daemon listening on another port (e.g. 8080) and having a pool of threads translating from GET requests to mysql querries and from mysql results to

Re: [OSM-dev] C++ implementation of the API

2008-05-26 Thread Tom Hughes
In message [EMAIL PROTECTED] Ludwig [EMAIL PROTECTED] wrote: My concern is that even if a reimplementation of the API in some other form will not fundamentally solve the scalability problem if all data requests will have to be served by a single machine. What makes you think they

Re: [OSM-dev] C++ implementation of the API

2008-05-26 Thread Tom Hughes
In message [EMAIL PROTECTED] Joachim Zobel [EMAIL PROTECTED] wrote: Am Donnerstag, den 22.05.2008, 00:06 +0100 schrieb Tom Hughes: Writing Apache modules in C is hard, and I don't think using mod_cpp will make it much easier. Doing Apache modules in Perl (mod_perl has API

Re: [OSM-dev] C++ implementation of the API

2008-05-26 Thread Frederik Ramm
Hi, Um... no. Looking at the code would reveal that we do indeed create a DOM tree and then flatten it out. Darn, there goes my credibility, should've looked at the code myself. Sorry Joachim! Bye Frederik -- Frederik Ramm ## eMail [EMAIL PROTECTED] ## N49°00'09 E008°23'33

Re: [OSM-dev] C++ implementation of the API

2008-05-25 Thread Dave Stubbs
On Sat, May 24, 2008 at 8:48 PM, Joachim Zobel [EMAIL PROTECTED] wrote: Am Donnerstag, den 22.05.2008, 00:06 +0100 schrieb Tom Hughes: Writing Apache modules in C is hard, and I don't think using mod_cpp will make it much easier. Doing Apache modules in Perl (mod_perl has API access

Re: [OSM-dev] C++ implementation of the API

2008-05-25 Thread Dave Stubbs
2008/5/25 Ludwig [EMAIL PROTECTED]: Would it not be better to think about new services that could be offered by OSM rather than putting a lot of effort into fixing something that is not broken? Suggestions would be WMS, WFS, i18n, custom layer support, or some sort of support where the OSM

Re: [OSM-dev] C++ implementation of the API

2008-05-25 Thread Frederik Ramm
Hi, Would it not be better to think about new services that could be offered by OSM rather than putting a lot of effort into fixing something that is not broken? While I agree that one should not over-engineer and try to be prepared for everything, if ONE thing is sure then that is a steadily

Re: [OSM-dev] C++ implementation of the API

2008-05-23 Thread Frederik Ramm
Hi, Writing Apache modules in C is hard I don't think so. and I don't think using mod_cpp will make it much easier. Doing Apache modules in Perl (mod_perl has API access including filters) is a lot easier. I don't know what the English would us in this situation but you might be familiar

Re: [OSM-dev] C++ implementation of the API

2008-05-23 Thread Raphael Studer
I'm a frequent user of Perl and it has lots of merits, but I wouldn't want to base a high-performance module on it that reads data from Mysql and shifts it over to Apache. I haven't done the numbers on that but my gut feeling is that you'll be copying the data around a few times more in that

Re: [OSM-dev] C++ implementation of the API

2008-05-23 Thread Frederik Ramm
Hi, Do we need an Apache server? Why not writing (in c/c++) a small daemon listening on another port (e.g. 8080) and having a pool of threads translating from GET requests to mysql querries and from mysql results to xml? Famous last words: Let's just write a small HTTP daemon... ;-) Sure

Re: [OSM-dev] C++ implementation of the API

2008-05-23 Thread Stefan de Konink
Frederik Ramm schreef: Hi, Do we need an Apache server? Why not writing (in c/c++) a small daemon listening on another port (e.g. 8080) and having a pool of threads translating from GET requests to mysql querries and from mysql results to xml? Famous last words: Let's just write a small

Re: [OSM-dev] C++ implementation of the API

2008-05-23 Thread Raphael Studer
On Fri, May 23, 2008 at 8:00 PM, Frederik Ramm [EMAIL PROTECTED] wrote: Hi, Do we need an Apache server? Why not writing (in c/c++) a small daemon listening on another port (e.g. 8080) and having a pool of threads translating from GET requests to mysql querries and from mysql results to xml?

Re: [OSM-dev] C++ implementation of the API

2008-05-22 Thread Alex Wilson
All good answers. Thanks guys. My aim is to keep the memory footprint as low as possible and stream the data out, so mod_apache it is. Cheers, Alex ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev

[OSM-dev] C++ implementation of the API

2008-05-21 Thread Alex Wilson
Hi, Further to earlier discussion on this list, I've been looking into writing a C++ version of the OSM API server. When discussing this before, it was suggested that an Apache module would be the best way to plug such an API into the OSM server. Just out of interest: what are the advantages of a

Re: [OSM-dev] C++ implementation of the API

2008-05-21 Thread Tom Hughes
In message [EMAIL PROTECTED] Alex Wilson [EMAIL PROTECTED] wrote: Further to earlier discussion on this list, I've been looking into writing a C++ version of the OSM API server. When discussing this before, it was suggested that an Apache module would be the best way to plug such an

Re: [OSM-dev] C++ implementation of the API

2008-05-21 Thread Joachim Zobel
Am Mittwoch, den 21.05.2008, 11:14 +0100 schrieb Alex Wilson: Further to earlier discussion on this list, I've been looking into writing a C++ version of the OSM API server. When discussing this before, it was suggested that an Apache module would be the best way to plug such an API into the

Re: [OSM-dev] C++ implementation of the API

2008-05-21 Thread Stefan de Konink
On Wed, 21 May 2008, Joachim Zobel wrote: Writing Apache modules in C is hard, and I don't think using mod_cpp will make it much easier. Doing Apache modules in Perl (mod_perl has API access including filters) is a lot easier. For modules using the C API read Nick Kews book (ISBN

Re: [OSM-dev] C++ implementation of the API

2008-05-21 Thread Tom Hughes
In message [EMAIL PROTECTED] Joachim Zobel [EMAIL PROTECTED] wrote: Am Mittwoch, den 21.05.2008, 11:14 +0100 schrieb Alex Wilson: Further to earlier discussion on this list, I've been looking into writing a C++ version of the OSM API server. When discussing this before, it was