Re: [OSM-dev] [OSM-talk] Ideas for OSM enhancements

2008-01-01 Thread Brett Henderson
Karl Newman wrote: Okay, that's no big deal. It's actually astounding to me that you put were able to the entire planet into a single img file... (obviously in pre-TIGER days). Anyway, I'm planning on writing a tiling task for Osmosis (probably not part of the Osmosis codebase, more likely a

Re: [OSM-dev] [OSM-talk] Osmosis and large bounding polygons

2008-01-08 Thread Brett Henderson
Nice work on the script! Frederik Ramm wrote: A better way to do this would be using the algorithms I nicked from GPSBabel and implemented in the simplify way code for JOSM but I'm not in a mood to perlify them atm. Would it be simpler to add the polygon simplification into the polygon

Re: [OSM-dev] Osmosis: Subtiling the planet with nice ways proposal

2008-01-14 Thread Brett Henderson
Lambertus wrote: Hi Brett, Lets see if I can put this down nice and short: I've had a little chat on the IRC today about splitting the planet file into subtiles for my Garmin endeavors. I complained about missing ways where they cross tile boundaries because of how the bboxes are

Re: [OSM-dev] Namefinder - what to do

2008-01-22 Thread Brett Henderson
Jon Burgess wrote: An update to the planet dump code will also be required. The code should be a clone of what we do already for the ways tags. Please remind me to take a look when the changes occur. Ditto for osmosis (including daily and hourly dumps).

Re: [OSM-dev] UTF-8 errors in our DB, or elsewhere?

2008-02-11 Thread Brett Henderson
Christopher Schmidt wrote: On Tue, Feb 12, 2008 at 03:11:05AM +0100, Frederik Ramm wrote: And about the UTF-8 bugs in the database: are they real? For example: http://www.openstreetmap.org/api/0.5/way/8138279 doesn't seem proper UTF-8 to me but maybe I'm wrong. It is. You're

Re: [OSM-dev] {To All} Re: OSM Data Question from FireGirl

2008-02-12 Thread Brett Henderson
Fire Girl wrote: Meep Mepp, information overload! LOL! I see so many paths and almost stressed out thinking about them all! Haha. I like the pre-staged data Jeremy described, and that looks useful, and think Jason's concepts are very good. I was thinking of setting up my own Database to

Re: [OSM-dev] Osmosis broken

2008-03-02 Thread Brett Henderson
Just happened to check my email, I won't be back for another week or so. To fix the problem, do the following steps (I can't access anything from internet cafes myself). Login as bretth on dev Change to the ~/app/osmosis directory Modify

Re: [OSM-dev] Osmosis patch for filtering ways

2008-03-26 Thread Brett Henderson
Hi Christoph, Thanks for the patch. I haven't had a chance to look at it yet but I will get to it eventually. Brett Christoph Sommer wrote: Dear all, I recently needed to filter OSM data to only include motorways and trunk roads. Finding no easy way to achieve that in a batch run, I

Re: [OSM-dev] Odd data in daily diffs (potlatch related?)

2008-03-31 Thread Brett Henderson
Osmosis is using the inbuilt Java SAX parser which directly reads data from an InputStream, there is no simple place where you can check data and sanitise it before processing it. It may be possible to write my own FilterInputStream that sits in between the underlying data stream and the SAX

[OSM-dev] PostGIS Based Bounding Box Extraction

2008-03-31 Thread Brett Henderson
Hi All, I've finished coding new bounding box extraction in osmosis. It seems to work correctly although I admit I haven't tested it thoroughly yet. If anybody wishes to take a look or better yet improve the PostGIS usage I'll welcome all feedback. BACKGROUND I've had a number of abortive

Re: [OSM-dev] Odd data in daily diffs (potlatch related?)

2008-03-31 Thread Brett Henderson
Martijn van Oosterhout wrote: On Mon, Mar 31, 2008 at 12:26 PM, Brett Henderson [EMAIL PROTECTED] wrote: Anyway, yell if anybody violently disagrees and thinks there is a bug that needs fixing in osmosis. While I agree in general, osmoss does currently make the assumption

Re: [OSM-dev] Osmosis patch for filtering ways

2008-04-03 Thread Brett Henderson
Cool, thanks for that. Patch is applied. Christoph Sommer wrote: Hi Brett, Brett Henderson wrote: The UsedNodeFilter uses a HashSetLong to track the node ids. Could you use a BitSetIdTracker or even better use the IdTracker interface[...] Sure - here's an updated version

Re: [OSM-dev] Osmosis patch for filtering ways

2008-04-03 Thread Brett Henderson
wrote: Hi Brett, Brett Henderson wrote: The UsedNodeFilter uses a HashSetLong to track the node ids. Could you use a BitSetIdTracker or even better use the IdTracker interface[...] Sure - here's an updated version of the patch: http://www.deltadevelopment.de/users/christoph

Re: [OSM-dev] all things to all people

2008-04-23 Thread Brett Henderson
On Wed, Apr 23, 2008 at 7:54 AM, Frederik Ramm [EMAIL PROTECTED] wrote: Hi, Since the minute-diff was introduced OSMXAPI has been running, on average, about 6 minutes behind the main database. Sigh, this project is too large already for one person to keep the overview. Minute diffs

Re: [OSM-dev] dev server /home diskusage

2008-04-23 Thread Brett Henderson
Turns out I had an old 1.2GB diff file in there, no idea how it got there in the first place. Gone now. I've done a few other minor cleanups so I'm down around 300MB now. On Wed, Apr 23, 2008 at 7:06 PM, Sebastian Spaeth [EMAIL PROTECTED] wrote: Our /home partition on dev is 83% full and I

Re: [OSM-dev] Osmosis Bug Report: Node TileIDs Incorrect

2008-04-27 Thread Brett Henderson
answer for the example given by cschmidt below. For lat=51.4781325 and lon=-0.1474929 I now get the correct answer of 2062265654 but I'm not sure if this is a fluke or whether this is now the correct implementation. Christopher Schmidt wrote: On Wed, Apr 23, 2008 at 09:07:50AM +1000, Brett Henderson

Re: [OSM-dev] Osmosis Bug Report: Node TileIDs Incorrect

2008-04-28 Thread Brett Henderson
Well I've fixed the constant and it seems to have fixed the problem ... I would never have found that. I've only tested it for the one node example provided by crschmidt so it hasn't had exhaustive testing :-) Thanks all for the assistance. It's checked into svn now.

Re: [OSM-dev] 0.6 api - same story

2008-05-05 Thread Brett Henderson
That's fine, really I don't have any issues with the way things have been done, I'm all for people jumping in and doing something. I'm sure things will appear in due course. I'm just piping up so that things aren't forgotten and to hint that I might need some lead time if I'm to change over

Re: [OSM-dev] 0.6

2008-05-07 Thread Brett Henderson
Sebastian Spaeth wrote: Brett Henderson wrote: Being fairly intimate with the issues involved in trying to split files into bboxes and polygons I'd love to see a polygon concept. Currently it is almost impossible to split osm files into tiles suitable for devices such as a garmin

Re: [OSM-dev] 0.6

2008-05-08 Thread Brett Henderson
Martijn van Oosterhout wrote: On Wed, May 7, 2008 at 10:44 AM, Brett Henderson [EMAIL PROTECTED] wrote: Actually my problem was much simpler. I was just looking for a way of taking a big bounding box (ie. a box around Australia) and splitting it into smaller tiles (let's say 10km

Re: [OSM-dev] OSM Developer List

2008-05-13 Thread Brett Henderson
bretth = 'Brett Henderson' Joerg Ostertag (OSM Munich/Germany) wrote: Hi, currently I'm trying to complete the debian changelog. For this I'd like to use the realnames of alll developers. So I started a conversion table from the svn-login-names to the realnames. If someone can complete

Re: [OSM-dev] New Replication Functionality

2008-05-26 Thread Brett Henderson
I stuff3d up the 0.27 release. 0.28 is up now and fixes a major runtime problem preventing it from ever launching. Brett Henderson wrote: Hi All, I've just released osmosis 0.27 which adds some additional support for downloading and applying changesets from the planet server. As you may

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

2008-06-09 Thread Brett Henderson
Christopher Schmidt wrote: On Mon, Jun 09, 2008 at 01:25:20PM -0700, Mikel Maron wrote: Are you familiar with setting up Mapnik and TileCache, and other components of the OSM stack? As a comment: with the recent changes in Mapnik to mmap shapefiles, TileCache + OSM + Mapnik is

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

2008-06-10 Thread Brett Henderson
I hope I'm close to getting mod_tile running but when attempting to render this URL: http://localhost/osm_tiles2/0/0/0.png I get output like this from renderd. Rendering daemon Got incoming connection, fd 5, number 1 Render fd(5) z(0), x(0), y(0) DONE TILE 0 0-0 0-0 Problem reading sub tiles

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

2008-06-10 Thread Brett Henderson
I've just found something, when attempting to render this URL. http://localhost/osm_tiles2/6/57/24.png I get errors in the apache error_log stating that a child exited with a Segmentation fault. That sounds ominous unless there's just some error handling going awry. Brett Henderson wrote: Oh

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

2008-06-10 Thread Brett Henderson
Oh, one other thing I tried was changing the owner of the /var/www/html/direct directory to be owned by the tilegen user (the renderd user). Not sure if that is necessary or not but it didn't make any difference either. Brett Henderson wrote: I hope I'm close to getting mod_tile running

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

2008-06-10 Thread Brett Henderson
(0) --- Process 13081 detached Jon Burgess wrote: 2008/6/10 Brett Henderson [EMAIL PROTECTED]: I've just found something, when attempting to render this URL. http://localhost/osm_tiles2/6/57/24.png I get errors in the apache error_log stating that a child exited with a Segmentation fault

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

2008-06-10 Thread Brett Henderson
Found it. My fault, had an incorrect path in render_config.h. I had /home/www... instead of /var/www... If only I'd thought to use strace sooner ... Thanks for the help Jon. Brett Henderson wrote: Apache fails after renderd errors. The apache crash isn't ideal but it sounds like

Re: [OSM-dev] diff mechanism is broken

2008-06-21 Thread Brett Henderson
If the daily or hourly files are 110 bytes it just means that no changes were made in that period. In other words they appear to be working correctly and no data should have been missed. The daily diff is more of a hack so I can't guarantee it's behaved properly with the db server down. I

Re: [OSM-dev] Hwo to get a diff (and apply it?)

2008-06-21 Thread Brett Henderson
If you do wish to automate osmosis diff application, you can use this script, adapt it to your bounding box and trigger it from cron: http://www.bretth.com/myanmar/replicate_osm_file.sh Installation instructions for the script are here. It will need to be adapted for your area because the

[OSM-dev] New Daily Diff Files

2008-06-24 Thread Brett Henderson
Hi All, Apologies for cross-posting to two lists but discussions have been split across both lists. Please reply to the dev list. There are now some new daily diff files available on the planet server that address some issues with the existing daily diff files. These new files are produced

Re: [OSM-dev] simple tool for finding minimal set of osmosis diff files

2008-07-01 Thread Brett Henderson
Hi Timo, There is already a way to do this but I'll leave it up to you to decide whether it suits your purposes. Osmosis has a task named --read-change-interval which will download all changes from the last time it ran (it maintains a timestamp file), and merge them into a single changeset.

Re: [OSM-dev] Osmosis diffs: delete shows the wrong user?

2008-07-04 Thread Brett Henderson
Hakan Tandogan wrote: Hi, I'm watching the areas I'm interested in by downloading daily excerpts from osmxapi and building a diff via osmosis. For the delete actions, the diffs show the wrong user. In one example node, the user djanda deleted a node, but the diffs show the last user that

[OSM-dev] New Osmosis Release (v0.29)

2008-07-04 Thread Brett Henderson
Hi All, I've just released osmosis 0.29. It has some new features which might interest people: * Plugin support. It is now possible to register a plugin which allows new tasks to be used within osmosis without requiring a full osmosis recompile. * Lazy string parsing. For processing xml

[OSM-dev] 0.6 API Status

2008-07-08 Thread Brett Henderson
Hi All, Just wondering where the API 0.6 changes are up to. I haven't been paying much attention and might have missed something but haven't noticed any discussion recently. I've checked out the 0.6 wiki page and it seems fairly quiet. Is it still soldiering on quietly in the background or

Re: [OSM-dev] Area support

2008-07-09 Thread Brett Henderson
Karl Newman wrote: On Wed, Jul 9, 2008 at 4:00 AM, Frederik Ramm [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi, No need for ids or ordering. area id=1234 tag k=landuse v=industrial / outer nd ref=1/ nd ref=2/

Re: [OSM-dev] Area support

2008-07-09 Thread Brett Henderson
Richard Fairhurst wrote: Frederik Ramm wrote: Don't know. But this time it would be supported by JOSM so everyone would use it ;-) This isn't talk-de, people here use other editors too you know. : I'm interested in areas because it makes osm file processing much simpler and allows

Re: [OSM-dev] Area support

2008-07-09 Thread Brett Henderson
Stefan Keller wrote: Agreed. An area (or polygon) simply is a basic geometry type; it's a fact in practice and computational geometry. The actual XML encoding of ways never really was according to best practices and will not scale. This is because even for ways we have to read in all

[OSM-dev] New Trac Component for Osmosis

2008-07-24 Thread Brett Henderson
Hi All, Currently I'm managing Osmosis TODO items, feature requests and bug reports using a combination of emails and text files. It would probably make more sense to manage this in Trac. How do I go about getting a new osmosis component added to Trac? Somehow I've avoided this part of the

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

2008-07-25 Thread Brett Henderson
Hmm dunno. It certainly looks like you're doing the right thing ... there's no reason why this shouldn't work. Osmosis doesn't care much about the size of the file. Are you sure you're running osmosis against the correct segment_earth.osm file? I notice you had both a compressed (bz2 file) and

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

2008-07-25 Thread Brett Henderson
if you wish to improve performance. It will take up some more disk space but will run *much* faster. On Fri, Jul 25, 2008 at 4:30 PM, Brett Henderson [EMAIL PROTECTED] wrote: Hmm dunno. It certainly looks like you're doing the right thing ... there's no reason why this shouldn't work

Re: [OSM-dev] New Trac Component for Osmosis

2008-07-25 Thread Brett Henderson
No worries, Trac ticket created ;-) I've added a please in the description, hope that gets me over the line :-) Seriously though, no rush. I've made do for the past 18 months, another few days won't hurt. On Fri, Jul 25, 2008 at 4:49 PM, Tom Hughes [EMAIL PROTECTED] wrote: Brett Henderson

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

2008-07-25 Thread Brett Henderson
- From: Brett Henderson To: Fire Girl Subject: Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why? Date: Fri, 25 Jul 2008 16:37:53 +1000 I just noticed you said the 27MB file was truncated. Do you mean it didn't get closed with an /osm tag at the end? Or do you

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

2008-07-30 Thread Brett Henderson
On Wed, Jul 30, 2008 at 1:24 PM, Brett Henderson [EMAIL PROTECTED] wrote: 2008/7/26 Karl Newman [EMAIL PROTECTED] Well, I ran it for a while until it failed because of lack of heap space (I had -Xmx512M for the Java command but it wasn't enough for the ListIdTracker). But regardless

[OSM-dev] Daily bzip changesets about to be discontinued

2008-08-07 Thread Brett Henderson
Hi All, I'd like to disable the daily bz2 changeset files once and for all. I intend to do this over the next day or two. Yell if you want me to keep them going for a bit longer. http://planet.openstreetmap.org/daily/ Note that when you switch to the gz files, you should apply the same day

Re: [OSM-dev] Chopped of ways. New flag for OSM XML?

2008-08-13 Thread Brett Henderson
Jochen Topf wrote: Hi! When creating excerpts of OSM data with Osmosis, Osmosis will (depending on flags) chop of ways, i.e. the output will contain ways with only some of the nodes in the way. But looking at the XML there is no way to tell that this has happened. The way still has the same

Re: [OSM-dev] Chopped of ways. New flag for OSM XML?

2008-08-13 Thread Brett Henderson
Karl Newman wrote: Even still, it's a valid concern and there are other operations (i.e., cutting into tiles or tag transforms) that can manipulate the data into a form that probably shouldn't be uploaded. Karl Yep, agree. If it wasn't the default behaviour it would just be far less

Re: [OSM-dev] Chopped of ways. New flag for OSM XML?

2008-08-14 Thread Brett Henderson
Jochen Topf wrote: But people *want* to use the data they get from some kind of extraction process to base decisions upon. And they should be able to. Not all the data is invalidated by an extraction, only some of it. It makes sense to mark those parts invalid that are, so that an automated

Re: [OSM-dev] Chopped of ways. Change Default or remove ID

2008-08-14 Thread Brett Henderson
Wolschon Softwaredesign und Beratung wrote: On Thu, 14 Aug 2008 10:31:04 +0200, Jochen Topf [EMAIL PROTECTED] wrote: I propose to not modify ways by default and to remove the way-id of ways that are modified during extraction. What should a tool care if the way references node-ids that it

[OSM-dev] [Fwd: Re: Chopped of ways. New flag for OSM XML?]

2008-08-15 Thread Brett Henderson
Forwarding to list ... Original Message Subject:Re: [OSM-dev] Chopped of ways. New flag for OSM XML? Date: Fri, 15 Aug 2008 06:21:14 +0200 From: Rogier Wolff [EMAIL PROTECTED] Organisation: BitWizard.nl To: Brett Henderson [EMAIL PROTECTED] References

Re: [OSM-dev] Daily Diff parsing - tile expiring

2008-08-20 Thread Brett Henderson
I have something in the works which could help with this although I'm intending to release it along with version 0.6 updates of osmosis so won't be ready for a little while. Osmosis provides a postgresql schema (called pgsql) which contains complete osm data without history. It is already

Re: [OSM-dev] UTF8 problem with last night's daily .osc

2008-08-30 Thread Brett Henderson
Frederik, I just saw your emails but I'm about to head out and can't look at it now, I'll try to take a look tomorrow but it sounds like you've already diagnosed the problem. If you can organise for the problematic way to be deleted or fixed in the database I'll re-create all changesets since

Re: [OSM-dev] Update osm2pgsql patch mode

2008-08-31 Thread Brett Henderson
On Mon, Sep 1, 2008 at 6:50 AM, Martijn van Oosterhout [EMAIL PROTECTED]wrote: Umm, yeah. There's that. The way I solved it was with the patch below, which is a gross hack but it works. Basically it turns every create into a modify so it deletes any conflicting rows before inserting. It may

Re: [OSM-dev] Update osm2pgsql patch mode

2008-08-31 Thread Brett Henderson
On Mon, Sep 1, 2008 at 8:57 AM, Karl Newman [EMAIL PROTECTED] wrote: I'm curious why the planet dump doesn't cut off at midnight, so you wouldn't get these duplicates. Is it because it's way more expensive to compare timestamps? I doubt it, because the minutely diff generation can run

Re: [OSM-dev] Object IDs not unique across object type?

2008-08-31 Thread Brett Henderson
They're not unique, this is normal. Although it is a remarkable coincidence that you have two with the same id near the same location ... On Mon, Sep 1, 2008 at 10:02 AM, Chris Browet [EMAIL PROTECTED] wrote: I always thought that ID's were unique, even across object types. However, I have an

Re: [OSM-dev] Update osm2pgsql patch mode

2008-08-31 Thread Brett Henderson
On Mon, Sep 1, 2008 at 9:57 AM, Robert (Jamie) Munro [EMAIL PROTECTED]wrote: In theory, you only need to do this once on a planet file. After that, everything should be consistent and remain consistent, and it's probably worth putting the checks back in, because a subsequent error would imply

Re: [OSM-dev] Update osm2pgsql patch mode

2008-09-01 Thread Brett Henderson
Matt Amos wrote: Martijn van Oosterhout wrote: In any case, the patch applying is fast enough as it is once its going so maybe I'll make the modify-instead-of-add the default for the time being until a better implementation comes along. yep. works for me here :-) daily diff with

Re: [OSM-dev] Update osm2pgsql patch mode

2008-09-01 Thread Brett Henderson
Brett Henderson wrote: Matt Amos wrote: Martijn van Oosterhout wrote: In any case, the patch applying is fast enough as it is once its going so maybe I'll make the modify-instead-of-add the default for the time being until a better implementation comes along. yep. works for me here

Re: [OSM-dev] Osmosis to filter ways on exclusion keys?

2008-09-07 Thread Brett Henderson
On Sun, Sep 7, 2008 at 11:09 PM, [EMAIL PROTECTED] wrote: Osmosis currently supports inclusion filtering on ways and nodes based on keys and values (e.g. the --way-key-value option). For the OpenStreetMap routing service I would like to exclude ways and nodes that are irellevant to routing

Re: [OSM-dev] osmosis pgsql schema / way.bbox removal/option

2008-09-10 Thread Brett Henderson
On Tue, Sep 9, 2008 at 4:43 PM, Florian Lohoff [EMAIL PROTECTED] wrote: On Tue, Sep 09, 2008 at 07:32:34AM +1000, Brett Henderson wrote: As Jochen mentioned, the --write-pgsql-dump task and subsequent import using COPY statements should be faster. I've looked at getting osmosis to stream

Re: [OSM-dev] [Fwd: Re: Extracting any size data from planet dump]

2008-09-17 Thread Brett Henderson
Yatzek wrote: Thanks. OSMosis is very useful. I just do not understand why it does not accept : /osmosis --read-xml file=inFile.osm --bounding-polygon file=country2pts.txt --write-xml file=outFile.osm/ while it accepts command: /osmosis --read-xml inFile.osm --bounding-polygon

Re: [OSM-dev] Osmosis and non latin characters, please add a force option

2008-09-17 Thread Brett Henderson
Joachim Zobel wrote: Hi Brett. Could you please make the parameters useUnicode=truecharacterEncoding=UTF-8 switchable with a --force-utf8 option. Not everybody can or likes to change the database charsets. This shouldn't be too hard to add. Before I do it though, have you tried this to

Re: [OSM-dev] Osmosis: Bounding polygon does not support change data as input?

2008-09-18 Thread Brett Henderson
Andreas Kalsch wrote: This is a clear fact, of course. But Osmosis could retrive the nodes from the database instead and try to find it out, like you described below. All is not lost however. The simple answer is just to import the diff for the entire world. It's approximately 10MB of

Re: [OSM-dev] Osmosis: Bounding polygon does not support change data as input?

2008-09-18 Thread Brett Henderson
On Fri, Sep 19, 2008 at 2:18 AM, Andreas Kalsch [EMAIL PROTECTED]wrote: Thank you for your help. The fourth point means that in my planned application I am not going to use OSM database but just a subset of it - so I need kind of trigger which updates my application data from my local OSM

Re: [OSM-dev] 0.6 Replication

2008-09-24 Thread Brett Henderson
Andy Allan wrote: On Wed, Sep 24, 2008 at 12:30 PM, Brett Henderson [EMAIL PROTECTED] wrote: Perhaps I completely misunderstood you. I was worried that you might do something like this: 1. way 12345 is currently at version 1. 2. Via potlatch you edit way 12345 adding a new tag

Re: [OSM-dev] Osmosis cannot connect to DB

2008-09-29 Thread Brett Henderson
On Mon, Sep 29, 2008 at 11:51 AM, Andreas Kalsch [EMAIL PROTECTED]wrote: Hi, I haven't found a solution by search to this problem. Osmosis cannot connect to my database. I can connect on my system with the same user/password. The same input worked in the past - and I haven't changed the

Re: [OSM-dev] way 27483626 UTF-8 truncation

2008-10-03 Thread Brett Henderson
Florian Lohoff wrote: On Fri, Oct 03, 2008 at 01:36:31PM +0100, Matt Amos wrote: Subject: [OSM-dev] way 27483626 UTF-8 truncation i just noticed that the hourly change file 2008100310-2008100311.osc.gz has an invalid UTF-8 string in the note tag for way 27483626 (

Re: [OSM-dev] way 27483626 UTF-8 truncation

2008-10-03 Thread Brett Henderson
Florian Lohoff wrote: On Fri, Oct 03, 2008 at 01:36:31PM +0100, Matt Amos wrote: Subject: [OSM-dev] way 27483626 UTF-8 truncation i just noticed that the hourly change file 2008100310-2008100311.osc.gz has an invalid UTF-8 string in the note tag for way 27483626 (

Re: [OSM-dev] way 27483626 UTF-8 truncation

2008-10-03 Thread Brett Henderson
Brett Henderson wrote: Florian Lohoff wrote: On Fri, Oct 03, 2008 at 01:36:31PM +0100, Matt Amos wrote: Subject: [OSM-dev] way 27483626 UTF-8 truncation i just noticed that the hourly change file 2008100310-2008100311.osc.gz has an invalid UTF-8 string in the note tag for way 27483626

Re: [OSM-dev] way 27483626 UTF-8 truncation

2008-10-05 Thread Brett Henderson
On Sun, Oct 5, 2008 at 2:12 AM, Matt Amos [EMAIL PROTECTED] wrote: On Sat, Oct 4, 2008 at 9:36 AM, Florian Lohoff [EMAIL PROTECTED] wrote: To get the ROMA database in sync again i replaced the notes by broken-utf8 - As notes typically get not rendered thats not a problem for me though.

Re: [OSM-dev] way 7062297, is this new?

2008-10-08 Thread Brett Henderson
On Thu, Oct 9, 2008 at 1:44 PM, Ian Dees [EMAIL PROTECTED] wrote: On Wed, Oct 8, 2008 at 9:35 PM, Stefan de Konink [EMAIL PROTECTED] wrote: No way! The database[1] uses indexing under the hood automatically. So every created_by k or JOSM v is automatically indexed. This gives a significant

Re: [OSM-dev] Database Schema

2008-10-08 Thread Brett Henderson
Hi Sagar, I hope to get time to test this at home tonight but hopefully others have better answers in the meantime. These links might also help in the meantime: http://wiki.openstreetmap.org/index.php/The_Rails_Port http://wiki.openstreetmap.org/index.php/Rails_on_Fedora Cheers, Brett On Thu,

Re: [OSM-dev] Database Schema

2008-10-09 Thread Brett Henderson
MilesTogoe wrote: ruby and rails have gone thru some changes the past few months - you most likely need a newer gems package - try downloading the latest gems source and compiling that - then from the newer gems install the rails packages I currently have rubygems 1.2.0. I see the latest

Re: [OSM-dev] Database Schema

2008-10-09 Thread Brett Henderson
Brett Henderson wrote: I haven't had much luck either. Does anybody have any suggestions on where I've gone wrong below. I suspect my errors have something to do with some missing externals. I better fix that before making any more noise and wasting anybody's time

Re: [OSM-dev] Database Schema

2008-10-09 Thread Brett Henderson
Brett Henderson wrote: Hi Sagar, I hope to get time to test this at home tonight but hopefully others have better answers in the meantime. These links might also help in the meantime: http://wiki.openstreetmap.org/index.php/The_Rails_Port http://wiki.openstreetmap.org/index.php

Re: [OSM-dev] Database Schema

2008-10-09 Thread Brett Henderson
Thanks for all the help. I now have a working rake installation and a database at version 15. I've installed the following gems. gem install -v=2.0.2 rails gem install libxml-ruby (produced a number of what appeared to be errors, hopefully not an issue) gem install -v=0.9.93

Re: [OSM-dev] osm time format

2008-10-13 Thread Brett Henderson
On Tue, Oct 14, 2008 at 3:42 AM, Shaun McDonald [EMAIL PROTECTED] wrote: The time will be UK time. The +0100 is the difference from UTC. There is a simple rails setting to make the server use UTC in the bottom of config/environment.rb Shaun On 13 Oct 2008, at 16:33, Joachim Zobel wrote:

Re: [OSM-dev] 0.6 API hack-a-thon Date

2008-10-14 Thread Brett Henderson
Shaun McDonald wrote: Hi Devs, With the rapid progress on the 0.6 API, I'd like to set the date of the 0.6 API hack-a-thon in London to complete the transition to the 0.6 API. I would like to propose the weekend of 9th November 2008, in the CloudMade offices in London. Is this data

[OSM-dev] Complete History Changesets

2008-10-14 Thread Brett Henderson
Hi All, The topic of getting access to bulk history data has come up a few times now so I'm wondering if people see a need for this. Osmosis already provides the ability to produce a changeset of a specific time interval. This is currently being used to produce minute, hourly and daily

Re: [OSM-dev] Faster loading with scabies

2008-10-15 Thread Brett Henderson
Hi Joachim, Nice work, any info on how to improve load speed is much appreciated. On Thu, Oct 16, 2008 at 6:15 AM, Joachim Zobel [EMAIL PROTECTED]wrote: Hi. I have written a utility named scabies to load data from osm files into a mysql database. See http://www.heute-morgen.de/scabies/

Re: [OSM-dev] 0.6 API hack-a-thon Date

2008-10-15 Thread Brett Henderson
On Wed, Oct 15, 2008 at 12:47 AM, Shaun McDonald [EMAIL PROTECTED]wrote: On 14 Oct 2008, at 10:18, Brett Henderson wrote: Shaun McDonald wrote: [..] Ideally we need to have all the main editors and osm tools ready for the 0.6 API transition by the time the 0.6 API goes live in November

Re: [OSM-dev] 0.6 API hack-a-thon Date

2008-10-16 Thread Brett Henderson
Shaun McDonald wrote: Am I right in thinking that when you import the planet [extract] into mysql using osmosis it will populate the history tables for you, and with each changeset, will add to the history, updating the current tables? I've not yet had a need to use osmosis. That's right,

Re: [OSM-dev] Faster loading with scabies

2008-10-17 Thread Brett Henderson
Sascha Silbe wrote: On Thu, Oct 16, 2008 at 09:21:54AM +1100, Brett Henderson wrote: The biggest problem I found wasn't the actual processing of INSERT statements, it was MySQL scaling non-linearly with the number of rows. MyISAM tables are very fast to import regardless of number of rows

Re: [OSM-dev] Faster loading with scabies

2008-10-17 Thread Brett Henderson
Joachim Zobel wrote: Am Donnerstag, den 16.10.2008, 09:21 +1100 schrieb Brett Henderson: Do you have any numbers to show what sort of performance increase can be expected. I can load europe.osm into the current_* tables in less than 12 hours (on my laptop - overpowered CPU, 5400

[OSM-dev] Changeset And Replication

2008-10-21 Thread Brett Henderson
Hi All, I'm in the process of updating Osmosis to work with API 0.6, or more specifically to work with the new MySQL schema. The biggest change is the introduction of changesets. I'm interested in people's thoughts on how this should be done. Option 1 My initial plan is not to look

Re: [OSM-dev] Changeset And Replication

2008-10-21 Thread Brett Henderson
On Wed, Oct 22, 2008 at 2:50 AM, 80n [EMAIL PROTECTED] wrote: On Tue, Oct 21, 2008 at 12:39 PM, Brett Henderson [EMAIL PROTECTED]wrote: Hi All, Changesets are not atomic transactions, so I don't see any point in trying to identify and work with closed changesets. There's no rollback

[OSM-dev] Broken UTF-8 In Changesets

2008-10-23 Thread Brett Henderson
Hi All, It looks like there's some broken UTF-8 in changesets again. Presumably another tag has been truncated half way through a UTF-8 sequence when writing to the database. I'm heading away for the weekend and won't have time to look into this before I leave. I'll be back on Monday and will

Re: [OSM-dev] Broken UTF-8 In Changesets

2008-10-26 Thread Brett Henderson
Florian Lohoff wrote: On Fri, Oct 24, 2008 at 12:49:47PM +1100, Brett Henderson wrote: It looks like there's some broken UTF-8 in changesets again. Presumably another tag has been truncated half way through a UTF-8 sequence when writing to the database. I'm heading away

Re: [OSM-dev] HEADS UP osmosis pgsql schema users Was: psql osmosis simple shema / smallint out of range

2008-11-02 Thread Brett Henderson
Florian Lohoff wrote: On Fri, Oct 31, 2008 at 11:17:16AM +0100, Florian Lohoff wrote: Hi, i just discovered that osmosis was not able to apply the hourly osc file starting 2008-10-29T20:00:00Z - It failed with: 2008-10-31 11:09:52 CET ERROR: smallint out of range 2008-10-31 11:09:52

Re: [OSM-dev] HEADS UP osmosis pgsql schema users Was: psql osmosis simple shema / smallint out of range

2008-11-03 Thread Brett Henderson
Jochen Topf wrote: On Sun, Nov 02, 2008 at 08:40:45PM +1100, Brett Henderson wrote: Florian Lohoff wrote: On Fri, Oct 31, 2008 at 11:17:16AM +0100, Florian Lohoff wrote: Hi, i just discovered that osmosis was not able to apply the hourly osc file starting 2008-10-29T20

Re: [OSM-dev] Problem installing Osmosis on OSX

2008-11-06 Thread Brett Henderson
That's very strange. I compile it with jdk1.6 and it appears you're running 1.6 so I don't understand what is happening. The UnsupportedClassVersionError usually only occurs if you try to run osmosis with an old version of java. Perhaps try getting the java command line out of the osmosis

Re: [OSM-dev] Hibernate Spatial

2008-11-07 Thread Brett Henderson
Jo_ wrote: Hi, I use Hibernate Spatial PostGIS. With the osmosis sql schema I created a postgis db and filled it via osmosis. After that I've created some foreign keys, made a new collumn for way geometry and used hibernate tools to create the mapping files. The problem is, that there's

Re: [OSM-dev] Osmosis on Australia extract

2008-11-08 Thread Brett Henderson
Richard Chirgwin wrote: Hi, I'm trying to load the Australian OSM extract found here: http://www.osmaustralia.org/osmausextract.php Into a MySQL database using Osmosis. The failure (short version here, long below signature) is: Unable to parse xml file /home/richard/Desktop/australia.osm.

Re: [OSM-dev] XML Schema

2008-11-08 Thread Brett Henderson
Frederik Ramm wrote: Does http://wiki.openstreetmap.org/index.php/OSM_Protocol_Version_0.5/DTD not work? You're right about the change files though, I don't know if there's anything for that. Then again, while it is very easy to validate your application's output against a DTD, it is

[OSM-dev] Osmosis is 0.6 ready

2008-11-10 Thread Brett Henderson
Hi All, Osmosis is now API 0.6 ready. Preferably use the version in svn, although the latest interim version is currently linked to from the API 0.6 wiki page http://wiki.openstreetmap.org/index.php/OSM_Protocol_Version_0.6#Osmosis It supports 0.5 and 0.6 at the same time with 0.5 and 0.6

Re: [OSM-dev] Osmosis is 0.6 ready

2008-11-11 Thread Brett Henderson
As of now it also supports ordered relations so make sure you update svn or download the latest version 0.29.4. Brett Henderson wrote: Hi All, Osmosis is now API 0.6 ready. Preferably use the version in svn, although the latest interim version is currently linked to from the API 0.6 wiki

[OSM-dev] Changeset Files Now Include User Id Attribute

2008-11-18 Thread Brett Henderson
As of 10:55 UTC, the daily/hourly/minute changeset files now include a uid attribute on all entities. This brings osmosis into alignment with the planet format which has included uid details for some time now. This will aid the transition to the 0.6 API which will also expose the uid

Re: [OSM-dev] Visualising change (.osc) files

2008-11-25 Thread Brett Henderson
On Wed, Nov 26, 2008 at 6:26 AM, S Knox [EMAIL PROTECTED] wrote: Hi List, I am already on the talk list, but this is my first post here. Hope it makes some semblance of sense. Having seen Osmdiff in action, I thought it would be useful to have a mechanism to see daily changes to the

Re: [OSM-dev] Data corruption :) II

2008-11-26 Thread Brett Henderson
Simon Ward wrote: On Wed, Nov 26, 2008 at 02:26:27PM +0100, Frederik Ramm wrote: I have never used that distinction for anything. It is implicit in the object id anyway - any node that is less than the maximum id has existed at some point in time because we have auto increments ;-) and

Re: [OSM-dev] Data corruption :) II

2008-11-27 Thread Brett Henderson
Matt Amos wrote: rails relies on mysql's auto_increment or postgres' sequence functionality to ensure no ids are used twice. the question is one of practicality: if we take deleted elements out of the current tables then we get an advantage (foreign key constraints ensure that no way/relation

  1   2   3   4   5   6   7   >