Re: [OSM-dev] Garmin GPX madness
If you had 30 traces down a road and no dop information and you derived a centre line, using only the lat lons of the 30 different traces, how much less precise would the result be than taking 30 traces with dop info and figuring out a centre line? Stefan and Oliver - if you really believe that the community have been brainwashed by Garmin, what's your plan of action to reverse the brainwashing and liberate our minds? On Sun, Sep 21, 2008 at 10:54 PM, Stefan de Konink [EMAIL PROTECTED] wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Richard Fairhurst schreef: When people learn to draw decent curves with polylines, _then_ we'll start worrying about GPS accuracy. Since we are already in serious preparations to go and support NURBS... polylines even will get irrelevant too, less nodes, more accuracy and better data representation :) Oh I love those mapping parties that end up in devtalk ;) Stefan -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkjWwp4ACgkQYH1+F2Rqwn0JcgCglEHr59MvWqJNEMfVW8hjkSOo egwAn0GPRg/ExStV4ndQTx3EDxKrVm9k =31HK -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev -- Nick Black http://www.blacksworld.net ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
On Mon, 22 Sep 2008 11:36:30 +0100 Nick Black [EMAIL PROTECTED] wrote: If you had 30 traces down a road and no dop information and you derived a centre line, using only the lat lons of the 30 different traces, how much less precise would the result be than taking 30 traces with dop info and figuring out a centre line? [...] Um, that would depend on the dop info, i.e., does it make some tracks significantly more precise than the others? It is impossible to make a general prediction. In general, given a 1D value x, measured with RMS error, s, from n independent, non-correlated observations; the effect of averaging all n values is that the RMS error of the mean value, x, is s/sqrt(n), i.e., the precision to which the 1D position is measured improves as the square root of the number of measurements. Regards, Gora ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
On Mon, 22 Sep 2008, Nick Black wrote: If you had 30 traces down a road and no dop information and you derived a centre line, using only the lat lons of the 30 different traces, how much less precise would the result be than taking 30 traces with dop info and figuring out a centre line? The error would be the maximum variance of all tracks, since a track without DOP would not be a track but an area with the maximum error (worst case). Now you can claim if the amount of input is increased by a factor that would compensate the statistical error would outweight the work for it, you are right. But taking an Amaryllo that claims DGPS grade tracks at many locations, I would really prefer that... Stefan and Oliver - if you really believe that the community have been brainwashed by Garmin, what's your plan of action to reverse the brainwashing and liberate our minds? I see many mappers in NL being 'so happy' with Garmin because OSM maps run on it. Happy-happy-joy-joy! As long Garmin is not actively committed to help OSM in the compiling of maps or publishing open specifications, or even open up the firmware, why do you even care for these devices because they have a screen? Wake up! If someone would buy a device for tracking take some el-cheapo NMEA receiver that does export the error-margin, and hook up your phone or GPS, the user will get the same fancy screen... with more possibilities of navigation and mapping. Stefan ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
On 22/09/2008 11:51, Stefan de Konink wrote: Stefan and Oliver - if you really believe that the community have been brainwashed by Garmin, what's your plan of action to reverse the brainwashing and liberate our minds? I see many mappers in NL being 'so happy' with Garmin because OSM maps run on it. Happy-happy-joy-joy! As long Garmin is not actively committed to help OSM in the compiling of maps or publishing open specifications, or even open up the firmware, why do you even care for these devices because they have a screen? Wake up! If someone would buy a device for tracking take some el-cheapo NMEA receiver that does export the error-margin, and hook up your phone or GPS, the user will get the same fancy screen... with more possibilities of navigation and mapping. You're being really unfair attacking us as you have been. We're not trying to promote Garmin, but lots of people do have them. This is the first discussion I can remember on these lists about this issue. There's no flashing lights on the website that says the info from Garmin receivers is lacking. You've only mentioned vaguely algorithms exist to make use of accuracy info, but that's a fat lot of good if they aren't built in to our editors. Database objects don't have references back to the tracks (or other sources) from which they were derived, so you can't tell whether one's additional trace is better than the one from which what's already there is derived. So at the moment, the additional info is irrelevant to ordinary mortals and info that some devices (and not just Garmin I imagine) may have problems in the future is not advertised. I think this is a complete smokescreen at present. David ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
On Mon, Sep 22, 2008 at 12:10:40PM +0100, David Earl wrote: This is the first discussion I can remember on these lists about this issue. There's no flashing lights on the website that says the info from Garmin receivers is lacking. You've only mentioned vaguely algorithms exist to make use of accuracy info, but that's a fat lot of good if they aren't built in to our editors. Database objects don't have references back to the tracks (or other sources) from which they were derived, so you can't tell whether one's additional trace is better than the one from which what's already there is derived. FWIW, merkaartor already shows the speed at a certian point of a track by varying the thickness of the track. This way the user can discard tracks which have an inconsistent speed reading (which nearly always means the position is jerky due to different satellites coming in/going out of view). It would be trivial to use precision information. The color of the track represents the slope. I am sure Chris who is nowadays responsible for most of these nifty additions to merkaartor can tell you more. cu bart ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] [OSM-talk] first pictograms of the 2020 iconset, place_of_workship
On Mon, Sep 22, 2008 at 01:00:22PM +0200, sergio sevillano wrote: first pictograms of the 2020 iconset, place_of_workship they are public domain svg intended for 20x20px size http://wiki.openstreetmap.org/index.php/User:Sergionaranja/Es:Map_Icons_Design_Standards#2020_iconset:_plantillas_y_ejemplos Its good to have a pastafarian place of workship :D -- Celso González (PerroVerd) http://mitago.net ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
On Mon, Sep 22, 2008 at 12:51 PM, Stefan de Konink [EMAIL PROTECTED]wrote: If someone would buy a device for tracking take some el-cheapo NMEA receiver that does export the error-margin, and hook up your phone or GPS, the user will get the same fancy screen... with more possibilities of navigation and mapping. True, and I'm thinking constantly about switching to that solution, but my Garmin unit still has some good sides (when hiking): - it's robust - it's waterproof - it uses AA batteries and can run pretty long on a pair of them + you can have extras just in case - it's a unit dedicated to a single job, which is sometimes better than a multipurpose one Best regards, Igor ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
Igor Brejc wrote: True, and I'm thinking constantly about switching to that solution, but my Garmin unit still has some good sides (when hiking): - it's robust Dropped phones PDAs definitely don't bounce, a GPS 60 does (resulting in nothing more than a minor scuff) - it's waterproof But unfortunately they don't yet seem to offer a model fitted with a little windscreen wiper, which would have been very handy last week. - it uses AA batteries and can run pretty long on a pair of them + you can have extras just in case This is extremely important so you don't end up with a unit which is useless due to lack of battery availability. (Rechargeable batts don't last forever). - it's a unit dedicated to a single job, which is sometimes better than a multipurpose one As demonstrated at the leeds mapping party when one of the guys had his bluetooth GPS disconnect from his phone, resulting in a fair amount of missing track. Jon ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Richard Fairhurst schreef: I'm trying to inject a little levity (you might even have noticed my this needs to be rvted comment on the wiki edit history) into a bunch of idiot points made by po-faced fundamentalists You could also have added a column supports DOP? Don't feed the trolls. Stefan -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkjXqroACgkQYH1+F2Rqwn2rNQCgkCWFeCmcuA/gGl3j0Yx/W6bO mFwAn3GhRGtfw5F/i/6PHwLY3NTTbUEI =ToCI -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
Stefan de Konink wrote: You could also have added a column supports DOP? Don't feed the trolls. Excellent idea - and you actually know what DOP is, which is more than I do, so I'll leave it to you. :) cheers Richard ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
2008/9/22 Richard Fairhurst [EMAIL PROTECTED] [redirected to list] Oliver Eichler wrote: Do you really think your childish behaviour fits OSM? Oh right, and saying Because Garmin users are brain washed and And they won't stop to cheerleader for Garmin until the day they drop dead is utterly, utterly rational and adult. And the shit recorded by Garmin is a technical description and not an emotive insult at all. No, I do not think that those comments were appropriate, but at least they are being kept on the list, as opposed to making modifications to a wiki. Keeping a discussion/argument on a mailing list (or possibly the discussions section of the wiki) is the correct course, if we all chose to have arguments in the wiki, and randomly making inane changes to tables, would result in an unmanagable mess, irrespective of whether or not you added a comment along the lines of this needs to be rvted. I think the original poster of those comments has a point, if what he states is correct. It is a pity that he chose to say it in the manner that he did, which could be considered argumentative. I'm trying to inject a little levity (you might even have noticed my this needs to be rvted comment on the wiki edit history) into a bunch of idiot points made by po-faced fundamentalists - while gently making a serious point, that people are able to make up their own minds with the assistance of a helpful comparison table which many of us have contributed to. If you don't like levity I suggest you go and play on Wikipedia, where disappearing up one's arse appears to be par for the course. Based on the tone of your emails both on list and off list, I sincerely doubt that levity has any thing to do with it. If you don't like what some one says, as opposed to getting emotional about it, tackle the points one by one, and if you can't agree, then just agree to disagree. This is the last I'll be saying on the topic. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Richard Fairhurst schreef: Stefan de Konink wrote: You could also have added a column supports DOP? Don't feed the trolls. Excellent idea - and you actually know what DOP is, which is more than I do, so I'll leave it to you. :) I'm not very into wiki tables :) Can they be reordered? Stefan -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkjXrGwACgkQYH1+F2Rqwn1U8gCfVrtiLzHSBZ854eh5msyPcphZ jwkAn320ehE41xvTYwcABbgi+HhbDO6W =QMYC -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Richard Fairhurst schreef: Stefan de Konink wrote: I'm not very into wiki tables :) Can they be reordered? Heh, you're now making me wonder what actually _do_ I know about? (and I'm sure I'm not the first to wonder that). I'm no wikitable export either, but yes, they can be reordered. If you click the little '' logo at the top of each column, it sorts by that field. And all you need to do to add the new column is to type over my Warm glow of satisfaction with your Supports DOP. :) Actually I mean if I want to insert a column. Or to move a column from right to the middle. Stefan -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEAREKAAYFAkjXryUACgkQYH1+F2Rqwn3wEQCdEZ5bZ6Gr99kr3lBVaAKT94Im fAsAniaA6yzwJrSx2h0eNE8TJl2S35yr =A6cy -END PGP SIGNATURE- ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
Douglas Furlong wrote: Based on the tone of your emails both on list and off list, I sincerely doubt that levity has any thing to do with it. You can doubt all you like - you must be new round here, as the phrase goes, or you'd know I tend to take the piss... which may be why people keep accusing me of the heinous crime of being Fake SteveC (pah, if I were FSC I'd post more regularly, hint, hint). But, yes, let's kill this one. Richard ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
Richard, Oh right, and saying Because Garmin users are brain washed and And they won't stop to cheerleader for Garmin until the day they drop dead is utterly, utterly rational and adult. And the shit recorded by Garmin is a technical description and not an emotive insult at all. you definetly missed to cite the last line of the original posting SCNR trolling ;) I'm trying to inject a little levity (you might even have noticed my this needs to be rvted comment on the wiki edit history) into a bunch of idiot points made by po-faced fundamentalists - while gently making a serious point, that people are able to make up their own minds with the assistance of a helpful comparison table which many of us have contributed to. And how does that add to a valid discussion on how propperly recorded GPS data would allow real track post processing? Not just some subjective interpolation by the eye with reduced track information. Oliver ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
Oliver Eichler wrote: Sent: 21 September 2008 6:26 PM To: dev@openstreetmap.org Subject: Re: [OSM-dev] Garmin GPX madness OK, I understand now. But I still don't know what I'd do with the information if I had it. The satellites are where they are when I'm out surveying, there's nothing I can do about the positional accuracy. It's usually pretty obvious when a track has gone wildly off (usually due to woodland cover or urban canyons), and I'm still generally going to use the track otherwise. If you have all information and points recorded with a fixed sample rate you can apply algorithms to minimize the error. But with the shit recorded by Garmin you can't do anything. It's spoiled data you have to take as is. Most users try to overcome this shortcoming by simply moving the points to the location they think it should be. But that is spoofing data the worst way. Having spent time using both standard NMEA out devices and Garmin devices I have a number of observations. These observations tell me that nothing that any of the devices produces is shit. You might shit, I might shit but GPS receivers really aren't clever enough to shit. I find that devices that stream NMEA sentences which include HDOP data (I'm ignoring VDOP on the basis that OSM has never sought to use GPS for height information) require post processing before upload to OSM because they regularly contain data that is wholly misleading, either because the horizontal accuracy was very poor (high HDOP) or because it contains bad data (partial sentences etc). Arguably out of the box a basic device only outputting NMEA data is not a great OSM tool because of these limiters. Agreed, if you spend the time to analyse/filter the raw data then you get a more reliable result than with no analysis. A Garmin device is not designed to be something that you analyse. The analysis is done by the device itself before data is presented to the user (on screen or as data output). Obviously if you want to make the analysis decisions yourself then you will go and buy a device that punts out NMEA and do it yourself, otherwise you pick the best tool for the job you are doing and go mapping. So, a Garmin device may not be doing the analysis in the way you would prefer, but at least its doing it. Having tried both approaches over the last three years of my activity with OSM I can safely say that both approaches work. Both have their limitations (NMEA needs filtering, Garmin data will include some misleading data under trees etc.) but overall neither is perfect but both are more than good enough for OSM. The newer high sensitivity receivers (in all devices) have made the question of accuracy rather a moot point. Regardless of device we can get streets and objects placed where we need them on the map, even under tree cover and even in urban canyons. Surely that's all we are concerned about here? OSM is not meant to be the product that sets objects at 2.5cm accuracy. 10m is good enough to place objects relative to each other and it is relative placement surely that is important in our map, not positional accuracy. Cheers Andy Oliver ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.169 / Virus Database: 270.7.0/1683 - Release Date: 21/09/2008 10:10 AM ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
On Monday, 22. September 2008 17:42:42 Richard Fairhurst wrote: Oliver Eichler wrote: you definetly missed to cite the last line of the original posting SCNR trolling ;) Hey, if I'm being slated for daring to take the piss, you can be too. ;) LOL ok :) Oliver ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
So anyway, for all us losers who use garmins which output sub-optimal GPX files... can we make potlatch, or something, automatically not zoom all the way out and to the start of the track, but zoom in to the end of the track? That strikes me as better than asking everyone to sell their GPS units or take up GPX editing as a hobby. On 22 Sep 2008, at 07:58, Richard Fairhurst wrote: Douglas Furlong wrote: Based on the tone of your emails both on list and off list, I sincerely doubt that levity has any thing to do with it. You can doubt all you like - you must be new round here, as the phrase goes, or you'd know I tend to take the piss... which may be why people keep accusing me of the heinous crime of being Fake SteveC (pah, if I were FSC I'd post more regularly, hint, hint). But, yes, let's kill this one. Richard ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev Best Steve ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Reimplementation of the GPX importer
Hi, After conversations with TomH and others, I have finally gotten around to rewriting the GPX importer in C. Attached is r14 of my code, and I believe it to be *ALMOST* feature-complete. In essence, it functions equivalently to the ruby importer, however it consumes significantly less system resources. TomH suggested to me on IRC that the ruby importer consumed the equivalent of 30% CPU continuously on the box on which it runs. My tests, although tainted by the fact that I ran them against an effectively blank database, show that the C reimplementation takes approximately 0.4% of that, so approximately 0.12% of a CPU on average through the month. The particular gain however is that an import which took almost 2min30 with the ruby, took less than a second with my C code. Naturally, the C version is not configured by the ruby environment which does introduce issues wrt. keeping it consistent with the rest of the system, however it is configured by environment variables set in settings.sh prior to it being run. As such, it would be possible to write a ruby daemon wrapper which configured the environment and then ran my code, were such desirable. (I'm *really* not a ruby hacker, so someone else would have to do that). My code does not replicate the delete all traces which are set to visible=0 functionality as I am unsure of when this behaviour would be required. The images are generated using libgd rather than imagemagick. As such, they are *marginally* different, but functionally equivalent. To build this, unpack the tarball, cd into the src/ subdirectory and type 'make' You will need the MySQL client library headers, and libgd's headers, along with the Expat library headers for the GPX parsing. To run it, from the directory containing src, templates etc, run: ./settings.sh src/gpx-import This assumes a localhost database, /tmp/osm/traces/* and /tmp/osm/images/* Trivial inspection of settings.sh show how to reconfigure that. I'd really appreciate any comments anyone might have, and if at all possible, a functionality review from one of the OSM admins. Modulo some kind of daemon management (gpx-import runs against its console by default), and the question about invisible traces, I believe it to be ready for deployment. Regards, Daniel Silverstone. (Kinnison on the wiki) -- Daniel Silverstone http://www.digital-scurf.org/ PGP mail accepted and encouraged.Key Id: 2BC8 4016 2068 7895 gpx-import-r14.tar.gz Description: application/compressed-tar ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Reimplementation of the GPX importer
On Mon, 2008-09-22 at 19:51 +0100, Daniel Silverstone wrote: My tests, although tainted by the fact that I ran them against an effectively blank database, show that the C reimplementation takes approximately 0.4% of that, so approximately 0.12% of a CPU on average through the month. I realise this isn't exactly obvious, so here are the raw numbers which might be easier to understand: [The unit 'CS' is CPU Second, not centisecond] The ruby takes ca. 1CS to start up (hot cache) and consumes ca. 139CS to process a 21186 point GPX and insert it into a blank database. At the end it has 45MB resident size. This size does not seem to increase over time or over repeated reimports which all take approximately the same time. The GPX import daemon (Compiled -O0, although -O2 only makes a tiny improvement as most of the time is in the database and libraries, not my code) takes an immeasurably small amount of time to start up, and consumes ca. 0.55CS to import the same 21186 point GPX and insert it into a blank database. At the end it has 2.2MB resident size, this size does not seem to increase over time, and valgrind reports it to be leak-free. Reimports take approximately the same amount of time. Wallclock, the ruby takes about 2:20s to complete, the C daemon about 2s. I'd be interested to know its performance on a non-empty DB. Thanks, Daniel. -- Daniel Silverstone http://www.digital-scurf.org/ PGP mail accepted and encouraged.Key Id: 2BC8 4016 2068 7895 ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Reimplementation of the GPX importer
On 22 Sep 2008, at 19:51, Daniel Silverstone wrote: My code does not replicate the delete all traces which are set to visible=0 functionality as I am unsure of when this behaviour would be required. This is used when people delete their traces. As it is a time consuming operation, it is done as a batch. The files are hidden on the site immediately, without actually being deleted from the db until the daemon gets them. Shaun ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
SteveC wrote: So anyway, for all us losers who use garmins which output sub- optimal GPX files... can we make potlatch, or something, automatically not zoom all the way out and to the start of the track, but zoom in to the end of the track? Yeah, no problem. I like the idea of the fifth point along or something. You just missed an update :| but I'll put it in the next one - until then you can munge the URL if you like (lat=somethinglon=somethinggpx=12345 - I sometimes just copy the gpx= bit and paste it onto the end of a URL created by clicking the Edit tab). cheers Richard ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Garmin GPX madness
On Mon, Sep 22, 2008 at 2:20 PM, Jon Stockill [EMAIL PROTECTED] wrote: - it's a unit dedicated to a single job, which is sometimes better than a multipurpose one As demonstrated at the leeds mapping party when one of the guys had his bluetooth GPS disconnect from his phone, resulting in a fair amount of missing track. Just get a bluetooth GPS with logger, then you can log the datapoints and just turn on the PDA to get a map of the area. Having tried it on a Nokia 770 the other day I can honestly say it works very well to reference the map. ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Reimplementation of the GPX importer
On Mon, 2008-09-22 at 23:46 +0100, Jon Burgess wrote: On Mon, 2008-09-22 at 19:51 +0100, Daniel Silverstone wrote: I'd really appreciate any comments anyone might have Hope you don't mind lots of comments :-) Overall it looks great and a huge performance win over the old code. Sorry I forgot to say that earlier, I was getting carried away reviewing the code. Jon ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Reimplementation of the GPX importer
On Mon, 2008-09-22 at 23:46 +0100, Jon Burgess wrote: I'd really appreciate any comments anyone might have Hope you don't mind lots of comments :-) The more the better. main.c: If the GPX_SLEEP_TIME environment variable is not set then getenv() returns NULL and causes a segv in atoi(). Well caught, If you have an error talking to the db (i.e. job-error is set on return from db_find_work()) then cstart will be uninitialised leading to bogus timing information being printed. Moved initialisation of cstart to top of main loop. db.c: #define FN(V) - Should not be required, free(NULL) is fine. My brain was clearly frazzled when I wrote that code. Cleaned up. mysql.c: - May be useful to print mysql_error(handle) when mysql_real_connect() fails. Mmm, would be handy, done so. filename.c: - Would be good to validate getenv(base) != NULL or report error. Good idea -- WARN and default to '.' if can't find it. gpx.c: - The old parser used to accept gz, bzip2, tar, zip etc, is this still supported? I couldn't find this in gpx.rb -- is it some magical behaviour of the ruby libxml binding?! In gpx_handle_end_element() - It looks like you've made ctx-got_ele mandatory. I'm not sure the old code enforced this. I *thought* it was mandatory, but a reread of the ruby leaves me confused. I shall make it optional. - Looks like you are missing a check of lat/long against +-90/180 limits D'oh, fixed. In gpx_parse_coord() - It would be worth checking to see if anyone has uploaded any files which use ',' as the decimal separator as is used in some locales. I've no idea if this is strictly a legal in a GPX file. My understanding of the XML schema is that it's not valid, but who knows for sure. Is there an easy way to grep this? I have no access to the OSM server filesystems. image.c: gpx_parse_coord() - May want to emit an error if fopen(outfilename, wb) fails Nod, done. mercator.c: mercator_sheet_y() - Bad things happen if you try to project 90 degrees of latitude into mercator. Clamping the input range to +-85.0511 degrees would fit the limits of the normal slippy map. Okay, clamps engaged :-) interpolate.c: interpolate() - missing fclose(inputfile) ? Argh, I am so angry I missed that, Thank you. That review was excellent and caught many good points. If someone in the know can clarify how .gz .bz2 etc were all handled before, then I'll work out what I can do to support it. Overall it looks great and a huge performance win over the old code. Sorry I forgot to say that earlier, I was getting carried away reviewing the code. As I said, the review was excellent and raised many good points. I am just embarrassed that I missed so much. One last bug: interpolate.c: In function ‘interpolate’: interpolate.c:101: warning: format ‘%s’ expects type ‘char *’, but argument 3 has type ‘struct FILE *’ ERROR(Unable to open input file %s (errno=%s), inputfile, strerror(errno)); == should be inputpath not inputfile. Fixed. I got this warning after updating log.h with: extern void _gpxlog(const char *level, const char *fmt, ...) __attribute__ ((format (printf, 2, 3))); Added to log.h, thanks for that. Attached is r18 of the codebase. I hope I've handled your points well enough. Let's see if we can get the answers to the above, and then move on to some real-world testing. If I can get hold of a tarball of a bunch of the GPXes without having to script up something to stress the webserver, that'd be ace. D. -- Daniel Silverstone http://www.digital-scurf.org/ PGP mail accepted and encouraged.Key Id: 2BC8 4016 2068 7895 gpx-import-r18.tar.gz Description: application/compressed-tar ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Reimplementation of the GPX importer
Daniel Silverstone wrote: On Mon, 2008-09-22 at 23:46 +0100, Jon Burgess wrote: gpx.c: - The old parser used to accept gz, bzip2, tar, zip etc, is this still supported? I couldn't find this in gpx.rb -- is it some magical behaviour of the ruby libxml binding?! The xml_file method in the trace model takes care of doing any unpacking and returning an uncompressed file which the daemon then processes. Tom -- Tom Hughes ([EMAIL PROTECTED]) http://www.compton.nu/ ___ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
