Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 2010-10-16 10:01 AM, Richard Weait wrote: Also IIRC you can use gpsbabel to set the timestamps in your file to one second gaps beginning at unix epoch, so that would give your vehicles outrageous speeds during 1970. Also you can choose the privacy settings for your traces to prevent sharing the timestamps. I've got the go-ahead to start uploading. Now I'm just trying to finish the conversion script, and I'm not all that well-versed in gpsbabel. If anyone's got a moment, could they look at http://help.openstreetmap.org/questions/1260/how-to-create-fake-timestamps-with-gpsbabel ? Thanks! - Sam. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On Thu, Oct 21, 2010 at 9:27 PM, Sam Wilson s...@archives.org.au wrote: On 2010-10-16 10:01 AM, Richard Weait wrote: Also IIRC you can use gpsbabel to set the timestamps in your file to one second gaps beginning at unix epoch, so that would give your vehicles outrageous speeds during 1970. Also you can choose the privacy settings for your traces to prevent sharing the timestamps. I've got the go-ahead to start uploading. Now I'm just trying to finish the conversion script, and I'm not all that well-versed in gpsbabel. If anyone's got a moment, could they look at http://help.openstreetmap.org/questions/1260/how-to-create-fake-timestamps-with-gpsbabel try this? http://help.openstreetmap.org/questions/1260/how-to-create-fake-timestamps-with-gpsbabel Hat tip, Jonathan Bennett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 2010-10-13 3:50 PM, Sam Wilson wrote: Hi, I work for a company in Western Australia that has a dozen or so vehicles traveling all over regional WA and all equipped with GPS trackers, taking waypoints at 30s intervals. I am going to be allowed to upload the data to OSM. I'm just wondering what issues I'm facing... 1. We use OSM maps for navigation, and so management are quite able to see the value that could arise from our giving this data to OSM, because it would make the maps better. But there is a bit of a gap between lots-of-GPS-traces and lots-of-well-tagged-roads! What can I do to bridge this gap? (I mean, trace over the GPS trace, I know, but that's going to leave a lot of 'highway=road' tags; is that okay?) 2. What form of permission do I need to get from management, and where does it get saved? Is there a pro forma letter somewhere? What licence do we release the data under? 3. Is a single daily trace suitable? I mean, all vehicles' traces put together in one file and uploaded (under, I guess, a separate account -- but still owned by me)? Or is there some automatic way of doing this? But really: is this worthwhile? Will my company benefit? Will OSM benefit? Thanks for any help, Sam. Okay, I've uploaded the first few traces. I posted a diary entry about it all: http://www.openstreetmap.org/user/Sam%20Wilson/diary/12085 ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On Sat, Oct 16, 2010 at 2:59 PM, Sam Wilson s...@archives.org.au wrote: We've got, it looks like, the last three months or so of tracks from about twenty vehicles, and I can break the data up into whatever chunks I want. I've been working with one track file per seven days, which seems to be about a 6-8MB gpx file. Would this be okay to upload? Is there any issue with it containing only one track (i.e. there are some connections between points that are hundreds of kilometers apart). Best to split it at points like that. Otherwise you end up with these spurious lines that don't represent roads - just random connections. So if you can have multiple tracks within the one gps, with each track being a real contiguous sequence of points, that would be best. As for the drivers: they know the vehicles movements are being logged, and they know that I'm working on making their maps better; there wouldn't be any problem with getting their permission, but I'm not sure it's necessary -- there's no way to link any part of the logging to any particular vehicle, or driver. Hmm... there will be a concentration of points near drivers' homes... but then again, also near every place they stop as well. Is this something to worry about? That's a discussion you need to have with them, or with other people in the company. Remember, once you release the information, it's pretty hard to get the genie back in the bottle. Steve ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On Fri, Oct 15, 2010 at 8:59 PM, Sam Wilson s...@archives.org.au wrote: Hmm... there will be a concentration of points near drivers' homes... but then again, also near every place they stop as well. Is this something to worry about? If you felt protective about it, you could make a list of polygons surrounding each driver's house and extending a short ways away, and use gpsbabel to drop all points within those polygons. I do that for my own gps traces I upload. - Alan ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
Or, if there are clear working hours, ditch anything before 9am or after 6pm, or whatever. Steve On Sun, Oct 17, 2010 at 6:00 AM, Alan Millar amillar...@gmail.com wrote: On Fri, Oct 15, 2010 at 8:59 PM, Sam Wilson s...@archives.org.au wrote: Hmm... there will be a concentration of points near drivers' homes... but then again, also near every place they stop as well. Is this something to worry about? If you felt protective about it, you could make a list of polygons surrounding each driver's house and extending a short ways away, and use gpsbabel to drop all points within those polygons. I do that for my own gps traces I upload. - Alan ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
But really: is this worthwhile? Will my company benefit? Will OSM benefit? Hi Sam, it definitely sounds worthwhile to me, and your company and OSM will benefit. One thing you didn't mention: how many gpx tracks do you have so far? Or is this just for going forward? If you can increase the frequency of trackpoints, do so, but for that kind of road, I suspect even 30s frequency will be pretty useful. A point every 1km for those kind of highways is definitely of value. I'd also double check that there aren't privacy concerns - gpx traces contain time information - is your company happy releasing that? Would it compromise them? Do the drivers get a say? When it's all up, post on the talk-au list. I'd be happy to spend some time tracing out the gpx'es. Steve ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On Fri, Oct 15, 2010 at 9:51 PM, Steve Bennett stevag...@gmail.com wrote: But really: is this worthwhile? Will my company benefit? Will OSM benefit? I'd also double check that there aren't privacy concerns - gpx traces contain time information - is your company happy releasing that? Would it compromise them? Do the drivers get a say? Good point. If privacy or business practices are a concern, you might munge the timestamps a bit. IIRC, OSM requires timestamps and will ignore files without them. Also IIRC you can use gpsbabel to set the timestamps in your file to one second gaps beginning at unix epoch, so that would give your vehicles outrageous speeds during 1970. Also you can choose the privacy settings for your traces to prevent sharing the timestamps. http://wiki.openstreetmap.org/wiki/Visibility_of_GPS_traces ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 16/10/10 10:01 AM, Richard Weait wrote: I'd also double check that there aren't privacy concerns - gpx traces contain time information - is your company happy releasing that? Would it compromise them? Do the drivers get a say? Good point. If privacy or business practices are a concern, you might munge the timestamps a bit. IIRC, OSM requires timestamps and will ignore files without them. Also IIRC you can use gpsbabel to set the timestamps in your file to one second gaps beginning at unix epoch, so that would give your vehicles outrageous speeds during 1970. Also you can choose the privacy settings for your traces to prevent sharing the timestamps. http://wiki.openstreetmap.org/wiki/Visibility_of_GPS_traces Yeah, I'd thought about this. I've been playing with gpsbabel, and figuring out the whole workflow, including timestamp munging. I was going to upload as 'private' tracks; are there any drawbacks to doing so? We've got, it looks like, the last three months or so of tracks from about twenty vehicles, and I can break the data up into whatever chunks I want. I've been working with one track file per seven days, which seems to be about a 6-8MB gpx file. Would this be okay to upload? Is there any issue with it containing only one track (i.e. there are some connections between points that are hundreds of kilometers apart). As for the drivers: they know the vehicles movements are being logged, and they know that I'm working on making their maps better; there wouldn't be any problem with getting their permission, but I'm not sure it's necessary -- there's no way to link any part of the logging to any particular vehicle, or driver. Hmm... there will be a concentration of points near drivers' homes... but then again, also near every place they stop as well. Is this something to worry about? I should have an official sign-off on the uploading sometime this week. - Sam. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 13/10/10 4:39 PM, Elizabeth Dodd wrote: Would you consider uploading the traces to an independent point, specifying the licence of the traces and giving permission for them to be traced into OSM / other maps (you specify what) and perhaps this could then be downloaded as a separate layer into editors? Yes, I could do that, absolutely. Do you think this is a better option? It would make the traces less apparent to lots of people (i.e. they wouldn't be included when someone clicks the Also get GPS traces checkbox in Merkaartor for example), but I guess I could advertise their existence to Australian mappers etc. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 13/10/2010 08:50, Sam Wilson wrote: 1. We use OSM maps for navigation, and so management are quite able to see the value that could arise from our giving this data to OSM, because it would make the maps better. But there is a bit of a gap between lots-of-GPS-traces and lots-of-well-tagged-roads! What can I do to bridge this gap? (I mean, trace over the GPS trace, I know, but that's going to leave a lot of 'highway=road' tags; is that okay?) That's fine -- it's much better than nothing 2. What form of permission do I need to get from management, and where does it get saved? Is there a pro forma letter somewhere? What licence do we release the data under? You'd be releasing the data under CC-BY-SA and ODbL. If you are acting on behalf of the copyright owner, then you don't need to add any further documentation -- simply uploading the data is enough to show you agree to the licences. 3. Is a single daily trace suitable? I mean, all vehicles' traces put together in one file and uploaded (under, I guess, a separate account -- but still owned by me)? Or is there some automatic way of doing this? A separate account for the traces may well be a good idea. As for aggregating the traces, I don't see any benefit to OSM of doing that, especially if it involves you doing more work. If it's the effort of uploading multiple traces that worries you, there's an Alpha-quality Java utility that I'm happy to help you to use: http://www.chainring.co.uk/Tracey.jar But really: is this worthwhile? Will my company benefit? Will OSM benefit? Yes. More source data is always good, especially if it's based on real movements. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On Wed, 13 Oct 2010 16:53:59 +0800 Sam Wilson s...@archives.org.au wrote: On 13/10/10 4:39 PM, Elizabeth Dodd wrote: Would you consider uploading the traces to an independent point, specifying the licence of the traces and giving permission for them to be traced into OSM / other maps (you specify what) and perhaps this could then be downloaded as a separate layer into editors? Yes, I could do that, absolutely. Do you think this is a better option? It would make the traces less apparent to lots of people (i.e. they wouldn't be included when someone clicks the Also get GPS traces checkbox in Merkaartor for example), but I guess I could advertise their existence to Australian mappers etc. There was a Russian transport mob who managed to completely overload the track upload system trying to put up gps traces to the main database. Separate hosting would keep that from happening - WA is on the same huge scale as Russia. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
Elizabeth Dodd wrote: There was a Russian transport mob who managed to completely overload the track upload system trying to put up gps traces to the main database. Separate hosting would keep that from happening - WA is on the same huge scale as Russia. Different issue. The issue with the Russian transport mob is that they uploaded the tracks all in one go with no delay between them. Simply putting sleep 60 in your upload script between each track fixes this. Sam: this is great. Go for it. cheers Richard -- View this message in context: http://gis.638310.n2.nabble.com/Ongoing-bulk-uploads-of-GPS-traces-tp5629920p5630878.html Sent from the General Discussion mailing list archive at Nabble.com. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
2010/10/13 Sam Wilson s...@archives.org.au: Hi, I work for a company in Western Australia that has a dozen or so vehicles traveling all over regional WA and all equipped with GPS trackers, taking waypoints at 30s intervals. I am going to be allowed to upload the data to OSM. I'm just wondering what issues I'm facing... I feel that 30s per trackpoint is not quite much. For a truck travelling at 80km per hour this would be one trackpoint every 667 metres. Not really enough to get even the idea where a curve is, but OK if the streets are perfectly straight and there is nothing around ;-) Cheers, Martin ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On Wed, Oct 13, 2010 at 4:04 PM, M∡rtin Koppenhoefer dieterdre...@gmail.com wrote: 2010/10/13 Sam Wilson s...@archives.org.au: Hi, I work for a company in Western Australia that has a dozen or so vehicles traveling all over regional WA and all equipped with GPS trackers, taking waypoints at 30s intervals. I am going to be allowed to upload the data to OSM. I'm just wondering what issues I'm facing... I feel that 30s per trackpoint is not quite much. For a truck travelling at 80km per hour this would be one trackpoint every 667 metres. Not really enough to get even the idea where a curve is, but OK if the streets are perfectly straight and there is nothing around ;-) And perfectly fine if there are multiple journeys along the same route. This data is definitely of value. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On Wed, Oct 13, 2010 at 3:50 AM, Sam Wilson s...@archives.org.au wrote: Hi, I work for a company in Western Australia that has a dozen or so vehicles traveling all over regional WA and all equipped with GPS trackers, taking waypoints at 30s intervals. I am going to be allowed to upload the data to OSM. I'm just wondering what issues I'm facing... Thank you for taking the initiative arrange this with your company. If you get the opportunity to change your data acquisition to one track point per second you'll find the traces much nicer for creating junctions, ramps, exits, fuel stations etc. Are you really taking waypoints every 30 second or track points? I've always worked with tracks / track points. 1. We use OSM maps for navigation, and so management are quite able to see the value that could arise from our giving this data to OSM, because it would make the maps better. But there is a bit of a gap between lots-of-GPS-traces and lots-of-well-tagged-roads! What can I do to bridge this gap? (I mean, trace over the GPS trace, I know, but that's going to leave a lot of 'highway=road' tags; is that okay?) This is typical for mapping new areas. You'll make improvements with each pass. Highway=road first, perhaps junctions as you add crossing traces next, a new amenity=fuel when you stop at a new place... Improved road classifications as driver observations and memory allow. 3. Is a single daily trace suitable? I mean, all vehicles' traces put together in one file and uploaded (under, I guess, a separate account -- but still owned by me)? Or is there some automatic way of doing this? Individual traces seems a better approach to me. I've used combined traces but recall that some tools did not handle the split between traces very well. One left a connection between the end one one trace and the start of the next. Another ignored traces after the first trace. I don't recall which tools these were; it was some time ago. But really: is this worthwhile? Will my company benefit? Will OSM benefit? Is this worthwhile? It seems to be exactly what OSM is for. Additional contributors, participating in OpenStreetMap by telling us more about their part of the World. Perfect. Tell some of your trusted colleagues how much fun it is too. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On Wed, 13 Oct 2010 17:04:20 +0200 M∡rtin Koppenhoefer dieterdre...@gmail.com wrote: OK if the streets are perfectly straight and there is nothing around ;-) Perfect for rural and remote Western Australia then. (This is the part of the world that owns the longest railway straight in existence) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 13 October 2010 20:35, Elizabeth Dodd ed...@billiau.net wrote: On Wed, 13 Oct 2010 17:04:20 +0200 M∡rtin Koppenhoefer dieterdre...@gmail.com wrote: OK if the streets are perfectly straight and there is nothing around ;-) Perfect for rural and remote Western Australia then. (This is the part of the world that owns the longest railway straight in existence) I agree, plus if the traces are numerous, the coverage could end up being quite good. In addition, they might be good enough to provide some initial sketch of roads. Emilie Laffray ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 2010-10-13 5:12 PM, Jonathan Bennett wrote: You'd be releasing the data under CC-BY-SA and ODbL. If you are acting on behalf of the copyright owner, then you don't need to add any further documentation -- simply uploading the data is enough to show you agree to the licences. Great! I might just get a letter from management as well, stating that they're permitting me to do this; never know what might happen in the future. A separate account for the traces may well be a good idea. As for aggregating the traces, I don't see any benefit to OSM of doing that, especially if it involves you doing more work. If it's the effort of uploading multiple traces that worries you, there's an Alpha-quality Java utility that I'm happy to help you to use: http://www.chainring.co.uk/Tracey.jar I've been looking into the format that the traces will come in, and it looks like a weekly NMEA dump is going to be the least work for me -- a single file, with all vehicles, that I can gpsbabel into GPX and upload. But I'm open to suggestions on that - Sam. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 2010-10-14 3:35 AM, Elizabeth Dodd wrote: On Wed, 13 Oct 2010 17:04:20 +0200 M∡rtin Koppenhoeferdieterdre...@gmail.com wrote: OK if the streets are perfectly straight and there is nothing around ;-) Perfect for rural and remote Western Australia then. (This is the part of the world that owns the longest railway straight in existence) Yep, perfectly straight and nothing around: that's the WA wheatbelt exactly! ;-) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Ongoing bulk uploads of GPS traces?
On 2010-10-14 1:35 AM, Richard Weait wrote: Thank you for taking the initiative arrange this with your company. If you get the opportunity to change your data acquisition to one track point per second you'll find the traces much nicer for creating junctions, ramps, exits, fuel stations etc. I'll look into the options, but I seem to remember hearing that we were already on the highest frequency possible (with, I guess, our pricing plan). Are you really taking waypoints every 30 second or track points? I've always worked with tracks / track points. Ah, well, yes: I'm probably just displaying my ignorance. ;-) What's the difference between a waypoint and a track point? This is typical for mapping new areas. You'll make improvements with each pass. Highway=road first, perhaps junctions as you add crossing traces next, a new amenity=fuel when you stop at a new place... Improved road classifications as driver observations and memory allow. I should make one point clear: I'm not actually surveying these places. I sit in an office in Perth, and don't really know the country through which the vehicles travel. I'll just be making the best of the GPS tracks and Yahoo imagery. Obviously it'd be better to have people who actually *know* the country editing the map, but I don't think that's going to happen any time soon! And thank you everyone for your support! I'm excited about being able to contribute to the map like this. Perhaps I should organise an *armchair* mapping party to help trace the roads; anyone here in WA?! - Sam. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
