Re: [OSM-dev] New Replication Functionality

I stuff3d up the 0.27 release. 0.28 is up now and fixes a major runtime problem preventing it from ever launching. Brett Henderson wrote: > Hi All, > > I've just released osmosis 0.27 which adds some additional support for > downloading and applying changesets from the plan

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

Christopher Schmidt wrote: > On Mon, Jun 09, 2008 at 01:25:20PM -0700, Mikel Maron wrote: > >> Are you familiar with setting up Mapnik and TileCache, and other components >> of the OSM stack? >> > > As a comment: with the recent changes in Mapnik to mmap shapefiles, > TileCache + OSM + Map

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

Jon Burgess wrote: > It should not be too hard to get mod_tile to work. I developed it > originally on Fedora 8. The packages that you are likely to need > installed are as follows: > > - Mapnik (obviously) > - Apache packages: httpd, httpd-devel > - APR packages: apr, apr-devel, apr-util, apr-util

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

I hope I'm close to getting mod_tile running but when attempting to render this URL: http://localhost/osm_tiles2/0/0/0.png I get output like this from renderd. Rendering daemon Got incoming connection, fd 5, number 1 Render fd(5) z(0), x(0), y(0) DONE TILE 0 0-0 0-0 Problem reading sub tiles for

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

I've just found something, when attempting to render this URL. http://localhost/osm_tiles2/6/57/24.png I get errors in the apache error_log stating that a child exited with a Segmentation fault. That sounds ominous unless there's just some error handling going awry. Brett Henderson w

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

Oh, one other thing I tried was changing the owner of the /var/www/html/direct directory to be owned by the tilegen user (the renderd user). Not sure if that is necessary or not but it didn't make any difference either. Brett Henderson wrote: > I hope I'm close to getting mod_til

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

= 0 rt_sigaction(SIGSEGV, {SIG_DFL}, {SIG_DFL}, 8) = 0 kill(13081, SIGSEGV) = 0 sigreturn() = ? (mask now []) --- SIGSEGV (Segmentation fault) @ 0 (0) --- Process 13081 detached Jon Burgess wrote: > 2008/6/10 Brett Henderson <[EMAIL

Re: [OSM-dev] Volunteers needed now. OSM for Myanmar disaster relief.

Found it. My fault, had an incorrect path in render_config.h. I had /home/www... instead of /var/www... If only I'd thought to use strace sooner ... Thanks for the help Jon. Brett Henderson wrote: > Apache fails after renderd errors. The apache crash isn't ideal but > it

Re: [OSM-dev] diff mechanism is broken

If the daily or hourly files are 110 bytes it just means that no changes were made in that period. In other words they appear to be working correctly and no data should have been missed. The daily diff is more of a hack so I can't guarantee it's behaved properly with the db server down. I hop

Re: [OSM-dev] Hwo to get a diff (and apply it?)

If you do wish to automate osmosis diff application, you can use this script, adapt it to your bounding box and trigger it from cron: http://www.bretth.com/myanmar/replicate_osm_file.sh Installation instructions for the script are here. It will need to be adapted for your area because the scrip

[OSM-dev] New Daily Diff Files

Hi All, Apologies for cross-posting to two lists but discussions have been split across both lists. Please reply to the dev list. There are now some new daily diff files available on the planet server that address some issues with the existing daily diff files. These new files are produced u

Re: [OSM-dev] simple tool for finding minimal set of osmosis diff files

Hi Timo, There is already a way to do this but I'll leave it up to you to decide whether it suits your purposes. Osmosis has a task named --read-change-interval which will download all changes from the last time it ran (it maintains a timestamp file), and merge them into a single changeset. T

Re: [OSM-dev] simple tool for finding minimal set of osmosis diff files

Timo Juhani Lindfors wrote: > Aha, due to java issues I was using the java5 compatible version which > does not have that option: > > $ grep -ri read-change-interval osmosis-0.24.1-java5 > $ > Hmm, I haven't updated it for ages. I should create a new java5 version one of these days ... > >>

Re: [OSM-dev] Osmosis diffs: "delete" shows the wrong user?

Hakan Tandogan wrote: > Hi, > > > I'm watching the areas I'm interested in by downloading daily excerpts > from osmxapi and building a diff via osmosis. > > For the delete actions, the diffs show the wrong user. In one example > node, the user "djanda" deleted a node, but the diffs show the last us

[OSM-dev] New Osmosis Release (v0.29)

Hi All, I've just released osmosis 0.29. It has some new features which might interest people: * Plugin support. It is now possible to register a plugin which allows new tasks to be used within osmosis without requiring a full osmosis recompile. * Lazy string parsing. For processing xml files

[OSM-dev] 0.6 API Status

Hi All, Just wondering where the API 0.6 changes are up to. I haven't been paying much attention and might have missed something but haven't noticed any discussion recently. I've checked out the 0.6 wiki page and it seems fairly quiet. Is it still soldiering on quietly in the background or

Re: [OSM-dev] osm2pgsql can parse diffs now (for testing anyway)

Andy Allan wrote: > On Wed, Jul 9, 2008 at 4:43 PM, Martijn van Oosterhout > <[EMAIL PROTECTED]> wrote: > >> Usage is simple enough. first load your planet dump with --slim mode >> (if you have the memory, give it a big cache): >> osm2pgsql --slim my-planet-extract.osm.gz >> After that you can u

Re: [OSM-dev] Area support

Karl Newman wrote: > On Wed, Jul 9, 2008 at 4:00 AM, Frederik Ramm <[EMAIL PROTECTED] > > wrote: > > Hi, > > > No need for ids or ordering. > > > > > > > > > > > > > > > > > > >

Re: [OSM-dev] Area support

Richard Fairhurst wrote: > Frederik Ramm wrote: > > >> Don't know. But this time it would be supported by JOSM so everyone >> would use it ;-) >> > > This isn't talk-de, people here use other editors too you know. : I'm interested in areas because it makes osm file processing much simpler

Re: [OSM-dev] Area support

Stefan Keller wrote: > Agreed. An area (or polygon) simply is a basic geometry type; it's a > fact in practice and computational geometry. > > The actual XML encoding of ways never really was according to best > practices and will not scale. This is because even for ways we have to > read in

Re: [OSM-dev] Osmosis bug?

2008/7/16 Karl Newman <[EMAIL PROTECTED]>: > > At some point (somewhere around 0.24 or 0.25 if I recall correctly), the > pipe processing order changed to more of a stack-based system. So, you might > try to write out your xml files (--wx) after each bounding box (--bb) call > instead of deferring

[OSM-dev] New Trac Component for Osmosis

Hi All, Currently I'm managing Osmosis TODO items, feature requests and bug reports using a combination of emails and text files. It would probably make more sense to manage this in Trac. How do I go about getting a new "osmosis" component added to Trac? Somehow I've avoided this part of the pr

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

Hmm dunno. It certainly looks like you're doing the right thing ... there's no reason why this shouldn't work. Osmosis doesn't care much about the size of the file. Are you sure you're running osmosis against the correct segment_earth.osm file? I notice you had both a compressed (bz2 file) and

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

ip if you wish to improve performance. It will take up some more disk space but will run *much* faster. On Fri, Jul 25, 2008 at 4:30 PM, Brett Henderson <[EMAIL PROTECTED]> wrote: > Hmm dunno. It certainly looks like you're doing the right thing ... > there's no rea

Re: [OSM-dev] New Trac Component for Osmosis

No worries, Trac ticket created ;-) I've added a please in the description, hope that gets me over the line :-) Seriously though, no rush. I've made do for the past 18 months, another few days won't hurt. On Fri, Jul 25, 2008 at 4:49 PM, Tom Hughes <[EMAIL PROTECTED]> wrot

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

> since my syntax seems to work against the planet.osm file, is maybe such > extraction is not feasable i hope someone can confirm my analysis if > psble. > > > > > - Original Message - > From: "Brett Henderson" > To: "Fire Girl" > Subj

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

2008/7/26 Karl Newman <[EMAIL PROTECTED]> > > Well, I ran it for a while until it failed because of lack of heap space (I > had -Xmx512M for the Java command but it wasn't enough for the > ListIdTracker). But regardless, it already generated a 18.6 GB uncompressed > OSM file (and that's only nodes

Re: [OSM-dev] Osmosis woes, something not right - can someone pls. see why?

On Wed, Jul 30, 2008 at 1:24 PM, Brett Henderson <[EMAIL PROTECTED]> wrote: > 2008/7/26 Karl Newman <[EMAIL PROTECTED]> > >> >> Well, I ran it for a while until it failed because of lack of heap space >> (I had -Xmx512M for the Java command but it wasn

Re: [OSM-dev] Osmosis and non-latin characters

Can you please show the output of the status command? Check the four characterset fields at the bottom. All should be set to utf8. Any non-utf8 values might cause the problem. Osmosis should just work if the database is configured correctly. mysql> status -- mysql Ver 14.12 Dist

[OSM-dev] Daily bzip changesets about to be discontinued

Hi All, I'd like to disable the daily bz2 changeset files once and for all. I intend to do this over the next day or two. Yell if you want me to keep them going for a bit longer. http://planet.openstreetmap.org/daily/ Note that when you switch to the gz files, you should apply the same day a

Re: [OSM-dev] Chopped of ways. New flag for OSM XML?

Jochen Topf wrote: > Hi! > > When creating excerpts of OSM data with Osmosis, Osmosis will (depending > on flags) chop of ways, i.e. the output will contain ways with only some > of the nodes in the way. But looking at the XML there is no way to tell > that this has happened. The way still has the

Re: [OSM-dev] Chopped of ways. New flag for OSM XML?

Karl Newman wrote: > Even still, it's a valid concern and there are other operations (i.e., > cutting into tiles or tag transforms) that can manipulate the data > into a form that probably shouldn't be uploaded. > > Karl Yep, agree. If it wasn't the default behaviour it would just be far less l

Re: [OSM-dev] Chopped of ways. New flag for OSM XML?

Jochen Topf wrote: > But people *want* to use the data they get from some kind of extraction > process to base decisions upon. And they should be able to. Not all the > data is invalidated by an extraction, only some of it. It makes sense > to mark those parts invalid that are, so that an automated

Re: [OSM-dev] Chopped of ways. Change Default or remove ID

Wolschon Softwaredesign und Beratung wrote: > On Thu, 14 Aug 2008 10:31:04 +0200, Jochen Topf <[EMAIL PROTECTED]> wrote: > >>> I propose to not modify ways by default and to remove the way-id of >>> ways that are modified during extraction. >>> What should a tool care if the way references node-

Re: [OSM-dev] Chopped of ways. New flag for OSM XML?

Karl Newman wrote: > On Thu, Aug 14, 2008 at 12:47 AM, Brett Henderson <[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> wrote: > > Jochen Topf wrote: > > But people *want* to use the data they get from some kind of > extraction >

[OSM-dev] [Fwd: Re: Chopped of ways. New flag for OSM XML?]

Forwarding to list ... Original Message Subject:Re: [OSM-dev] Chopped of ways. New flag for OSM XML? Date: Fri, 15 Aug 2008 06:21:14 +0200 From: Rogier Wolff <[EMAIL PROTECTED]> Organisation: BitWizard.nl To: Brett Henderson <[EMAIL PROTECTED]>

Re: [OSM-dev] [Fwd: Re: Chopped of ways. New flag for OSM XML?]

Rogier Wolff wrote: > > So, then, the recommendation should be that when extracting data, this > same procedure is followed, and if not, those ways should be marked > incomplete in a way that as much as possible doesn't interfere with > current tools. Making a node-id negative, which in other tools

Re: [OSM-dev] [Fwd: Re: Chopped of ways. New flag for OSM XML?]

Rogier Wolff wrote: > On Fri, Aug 15, 2008 at 07:59:07AM -0700, Karl Newman wrote: > >>> Making some parameter negative to mean "this object is incomplete" >>> sounds like a hack to me. It's something that you might do in an >>> internal database, or in an in-memory datastructure, but it should

Re: [OSM-dev] Daily Diff parsing - tile expiring

I have something in the works which could help with this although I'm intending to release it along with version 0.6 updates of osmosis so won't be ready for a little while. Osmosis provides a postgresql schema (called pgsql) which contains complete osm data without history. It is already avai

Re: [OSM-dev] Daily Diff parsing - tile expiring

This sounds much simpler :-) Frederik Ramm wrote: > Hi, > > >> I think that adding the Lat/Lon tag to node references in the ways is not an >> option since it would require a huge amount of queries on the database when >> creating the diffs. The positive side of this would that it would only

Re: [OSM-dev] UTF8 problem with last night's daily .osc

Frederik, I just saw your emails but I'm about to head out and can't look at it now, I'll try to take a look tomorrow but it sounds like you've already diagnosed the problem. If you can organise for the problematic way to be deleted or fixed in the database I'll re-create all changesets since t

Re: [OSM-dev] UTF8 problem with last night's daily .osc

Grant Slater wrote: > Karl Newman wrote: > >> If I recall correctly, the database column is not actually set for >> UTF-8 (but is double-encoded to return actual UTF-8 to the client...). >> Wouldn't it be a better long-term fix to change the database to UTF-8 >> (or whatever), then presumably

Re: [OSM-dev] Update osm2pgsql patch mode

On Mon, Sep 1, 2008 at 6:50 AM, Martijn van Oosterhout <[EMAIL PROTECTED]>wrote: > Umm, yeah. There's that. The way I solved it was with the patch below, > which is a gross hack but it works. Basically it turns every create > into a modify so it deletes any conflicting rows before inserting. It >

Re: [OSM-dev] Update osm2pgsql patch mode

On Mon, Sep 1, 2008 at 8:57 AM, Karl Newman <[EMAIL PROTECTED]> wrote: > > I'm curious why the planet dump doesn't cut off at midnight, so you > wouldn't get these duplicates. Is it because it's way more expensive to > compare timestamps? I doubt it, because the minutely diff generation can run >

Re: [OSM-dev] Object IDs not unique across object type?

They're not unique, this is normal. Although it is a remarkable coincidence that you have two with the same id near the same location ... On Mon, Sep 1, 2008 at 10:02 AM, Chris Browet <[EMAIL PROTECTED]> wrote: > I always thought that ID's were unique, even across object types. > However, I have

Re: [OSM-dev] Update osm2pgsql patch mode

On Mon, Sep 1, 2008 at 9:57 AM, Robert (Jamie) Munro <[EMAIL PROTECTED]>wrote: > > In theory, you only need to do this once on a planet file. After that, > everything should be consistent and remain consistent, and it's probably > worth putting the checks back in, because a subsequent error would

Re: [OSM-dev] Update osm2pgsql patch mode

Matt Amos wrote: > Martijn van Oosterhout wrote: > >> In any case, the patch applying is fast enough as it is once its going >> so maybe I'll make the modify-instead-of-add the default for the time >> being until a better implementation comes along. >> > > yep. works for me here :-) > > dai

Re: [OSM-dev] Update osm2pgsql patch mode

Brett Henderson wrote: > Matt Amos wrote: >> Martijn van Oosterhout wrote: >> >>> In any case, the patch applying is fast enough as it is once its going >>> so maybe I'll make the modify-instead-of-add the default for the time >>>

Re: [OSM-dev] Osmosis to filter ways on exclusion keys?

On Sun, Sep 7, 2008 at 11:09 PM, <[EMAIL PROTECTED]> wrote: > > Osmosis currently supports inclusion filtering on ways and nodes based on > keys and values (e.g. the --way-key-value option). For the OpenStreetMap > routing service I would like to exclude ways and nodes that are irellevant > to rou

Re: [OSM-dev] osmosis pgsql schema / way.bbox removal/option

On Tue, Sep 9, 2008 at 4:01 AM, Florian Lohoff <[EMAIL PROTECTED]> wrote: > On Mon, Sep 08, 2008 at 08:05:56AM -0500, Ian Dees wrote: > > Subject: Re: [OSM-dev] osmosis pgsql schema / way.bbox removal/option > > On Mon, Sep 8, 2008 at 7:29 AM, Jochen Topf <[EMAIL PROTECTED]> wrote: > > > > > > > >

Re: [OSM-dev] osmosis pgsql schema / way.bbox removal/option

On Tue, Sep 9, 2008 at 4:43 PM, Florian Lohoff <[EMAIL PROTECTED]> wrote: > On Tue, Sep 09, 2008 at 07:32:34AM +1000, Brett Henderson wrote: > > As Jochen mentioned, the --write-pgsql-dump task and subsequent import > using > > COPY statements should be faster. I've

Re: [OSM-dev] [Fwd: Re: Extracting any size data from planet dump]

Yatzek wrote: > Thanks. OSMosis is very useful. I just do not understand why it does > not accept : > > /osmosis --read-xml file="inFile.osm" --bounding-polygon > file="country2pts.txt" --write-xml file="outFile.osm"/ > > while it accepts command: > > /osmosis --read-xml inFile.osm --bounding-pol

Re: [OSM-dev] Osmosis and non latin characters, please add a force option

Joachim Zobel wrote: > Hi Brett. > > Could you please make the parameters > useUnicode=true&characterEncoding=UTF-8 > switchable with a --force-utf8 option. Not everybody can or likes to > change the database charsets. > This shouldn't be too hard to add. Before I do it though, have you tried

Re: [OSM-dev] Osmosis: Bounding polygon does not support change data as input?

Andreas Kalsch wrote: > I have set up a OSM database with Germany data. Now I want to update my > data daily. I have not found diff files for Germany so I want to use the > global diff files, which are not too big. > To re-import German data every day would be too expensive - it already > took s

Re: [OSM-dev] Osmosis: Bounding polygon does not support change data as input?

Andreas Kalsch wrote: > This is a clear fact, of course. But Osmosis could retrive the nodes > from the database instead and try to find it out, like you described > below. >> >> All is not lost however. >> The simple answer is just to import the diff for the entire world. >> It's approximately

Re: [OSM-dev] Osmosis: Bounding polygon does not support change data as input?

On Fri, Sep 19, 2008 at 2:18 AM, Andreas Kalsch <[EMAIL PROTECTED]>wrote: > Thank you for your help. > The fourth point means that in my planned application I am not going to use > OSM database but just a subset of it - so I need kind of trigger which > updates my application data from my local OS

[OSM-dev] 0.6 Replication (was: Number of Edits)

Andy Allan wrote: > On Wed, Sep 24, 2008 at 11:26 AM, Andy Allan <[EMAIL PROTECTED]> wrote: > >> On Wed, Sep 24, 2008 at 9:27 AM, Richard Fairhurst <[EMAIL PROTECTED]> wrote: >> >>> Brett Henderson wrote: >>> >>> >>>&

Re: [OSM-dev] 0.6 Replication

Andy Allan wrote: > On Wed, Sep 24, 2008 at 12:30 PM, Brett Henderson <[EMAIL PROTECTED]> wrote: > > >> Perhaps I completely misunderstood you. I was worried that you might do >> something like this: >> 1. way 12345 is currently at version 1. >> 2. Via p

Re: [OSM-dev] Osmosis cannot connect to DB

On Mon, Sep 29, 2008 at 11:51 AM, Andreas Kalsch <[EMAIL PROTECTED]>wrote: > Hi, > > I haven't found a solution by search to this problem. Osmosis cannot > connect to my database. I can connect on my system with the same > user/password. The same input worked in the past - and I haven't changed >

Re: [OSM-dev] way 27483626 UTF-8 truncation

Florian Lohoff wrote: > On Fri, Oct 03, 2008 at 01:36:31PM +0100, Matt Amos wrote: > >> Subject: [OSM-dev] way 27483626 UTF-8 truncation >> >> i just noticed that the hourly change file >> 2008100310-2008100311.osc.gz has an invalid UTF-8 string in the note >> tag for way 27483626 ( >> http://ww

Re: [OSM-dev] way 27483626 UTF-8 truncation

Florian Lohoff wrote: > On Fri, Oct 03, 2008 at 01:36:31PM +0100, Matt Amos wrote: > >> Subject: [OSM-dev] way 27483626 UTF-8 truncation >> >> i just noticed that the hourly change file >> 2008100310-2008100311.osc.gz has an invalid UTF-8 string in the note >> tag for way 27483626 ( >> http://ww

Re: [OSM-dev] way 27483626 UTF-8 truncation

Brett Henderson wrote: > Florian Lohoff wrote: >> On Fri, Oct 03, 2008 at 01:36:31PM +0100, Matt Amos wrote: >> >>> Subject: [OSM-dev] way 27483626 UTF-8 truncation >>> >>> i just noticed that the hourly change file >>> 2008100310-2008100311.osc

Re: [OSM-dev] way 27483626 UTF-8 truncation

Florian Lohoff wrote: > On Sat, Oct 04, 2008 at 03:24:12PM +1000, Brett Henderson wrote: > >>> Another 2 change files contain utf-8 bugs and osmosis refuses to process >>> them: >>> >>> 200810031022-200810031023.osc >>> 200810031023-20081003102

Re: [OSM-dev] way 27483626 UTF-8 truncation

On Sun, Oct 5, 2008 at 2:12 AM, Matt Amos <[EMAIL PROTECTED]> wrote: > On Sat, Oct 4, 2008 at 9:36 AM, Florian Lohoff <[EMAIL PROTECTED]> wrote: > > To get the ROMA database in sync again i replaced the notes by > > "broken-utf8" - As notes typically get not rendered thats not a problem > > for me

Re: [OSM-dev] way 7062297, is this new?

On Thu, Oct 9, 2008 at 1:44 PM, Ian Dees <[EMAIL PROTECTED]> wrote: > On Wed, Oct 8, 2008 at 9:35 PM, Stefan de Konink <[EMAIL PROTECTED]> wrote: > >> >> >> No way! The database[1] uses indexing under the hood automatically. So >> every created_by k or JOSM v is automatically indexed. This gives a

Re: [OSM-dev] Database Schema

Hi Sagar, I hope to get time to test this at home tonight but hopefully others have better answers in the meantime. These links might also help in the meantime: http://wiki.openstreetmap.org/index.php/The_Rails_Port http://wiki.openstreetmap.org/index.php/Rails_on_Fedora Cheers, Brett On Thu, O

Re: [OSM-dev] Database Schema

MilesTogoe wrote: > ruby and rails have gone thru some changes the past few months - you > most likely need a newer gems package - try downloading the latest > gems source and compiling that - then from the newer gems install the > rails packages I currently have rubygems 1.2.0. I see the late

Re: [OSM-dev] Database Schema

Brett Henderson wrote: > > I haven't had much luck either. Does anybody have any suggestions on > where I've gone wrong below. > I suspect my errors have something to do with some missing externals. I better fix that before making any more noise and

Re: [OSM-dev] Database Schema

Brett Henderson wrote: > Hi Sagar, > > I hope to get time to test this at home tonight but hopefully others > have better answers in the meantime. > > These links might also help in the meantime: > http://wiki.openstreetmap.org/index.php/The_Rails_Port > http://wiki.ope

Re: [OSM-dev] Database Schema

Thanks for all the help. I now have a working rake installation and a database at version 15. I've installed the following gems. gem install -v=2.0.2 rails gem install libxml-ruby (produced a number of what appeared to be errors, hopefully not an issue) gem install -v=0.9.93 composite_primary_ke

Re: [OSM-dev] osm time format

On Tue, Oct 14, 2008 at 3:42 AM, Shaun McDonald <[EMAIL PROTECTED]> wrote: > The time will be UK time. > The +0100 is the difference from UTC. > > There is a simple rails setting to make the server use UTC in the bottom of > config/environment.rb > > Shaun > On 13 Oct 2008, at 16:33, Joachim Zobel

Re: [OSM-dev] 0.6 API hack-a-thon Date

Shaun McDonald wrote: > Hi Devs, > > With the rapid progress on the 0.6 API, I'd like to set the date of > the 0.6 API hack-a-thon in London to complete the transition to the > 0.6 API. > > I would like to propose the weekend of 9th November 2008, in the > CloudMade offices in London. Is this da

[OSM-dev] Complete History Changesets

Hi All, The topic of getting access to bulk history data has come up a few times now so I'm wondering if people see a need for this. Osmosis already provides the ability to produce a changeset of a specific time interval. This is currently being used to produce minute, hourly and daily change

Re: [OSM-dev] Faster loading with scabies

Hi Joachim, Nice work, any info on how to improve load speed is much appreciated. On Thu, Oct 16, 2008 at 6:15 AM, Joachim Zobel <[EMAIL PROTECTED]>wrote: > Hi. > > I have written a utility named scabies to load data from osm files into > a mysql database. See > > http://www.heute-morgen.de/scab

Re: [OSM-dev] 0.6 API hack-a-thon Date

On Wed, Oct 15, 2008 at 12:47 AM, Shaun McDonald <[EMAIL PROTECTED]>wrote: > > On 14 Oct 2008, at 10:18, Brett Henderson wrote: > > Shaun McDonald wrote: >> >>> [..] >>> >>> Ideally we need to have all the main editors and osm tools ready for

Re: [OSM-dev] 0.6 API hack-a-thon Date

Shaun McDonald wrote: > > Am I right in thinking that when you import the planet [extract] into > mysql using osmosis it will populate the history tables for you, and > with each changeset, will add to the history, updating the current > tables? I've not yet had a need to use osmosis. That's rig

Re: [OSM-dev] Faster loading with scabies

Sascha Silbe wrote: > On Thu, Oct 16, 2008 at 09:21:54AM +1100, Brett Henderson wrote: > >> The biggest problem I found wasn't the actual processing of INSERT >> statements, it was MySQL scaling non-linearly with the number of rows. >> MyISAM tables are very fast to i

Re: [OSM-dev] Faster loading with scabies

Joachim Zobel wrote: > Am Donnerstag, den 16.10.2008, 09:21 +1100 schrieb Brett Henderson: > >> Do you have any numbers to show what sort of performance increase can >> be expected. >> > > I can load europe.osm into the current_* tables in less than 12 hours

[OSM-dev] Changeset And Replication

Hi All, I'm in the process of updating Osmosis to work with API 0.6, or more specifically to work with the new MySQL schema. The biggest change is the introduction of changesets. I'm interested in people's thoughts on how this should be done. Option 1 My initial plan is not to look

Re: [OSM-dev] Changeset And Replication

On Wed, Oct 22, 2008 at 2:50 AM, 80n <[EMAIL PROTECTED]> wrote: > On Tue, Oct 21, 2008 at 12:39 PM, Brett Henderson <[EMAIL PROTECTED]>wrote: > >> Hi All, >> >> >> > > Changesets are not atomic transactions, so I don't see any point in

[OSM-dev] Broken UTF-8 In Changesets

Hi All, It looks like there's some broken UTF-8 in changesets again. Presumably another tag has been truncated half way through a UTF-8 sequence when writing to the database. I'm heading away for the weekend and won't have time to look into this before I leave. I'll be back on Monday and will t

Re: [OSM-dev] Broken UTF-8 In Changesets

Florian Lohoff wrote: > On Fri, Oct 24, 2008 at 12:49:47PM +1100, Brett Henderson wrote: > >>It looks like there's some broken UTF-8 in changesets again. Presumably >>another tag has been truncated half way through a UTF-8 sequence when >>writing

[osmosis-dev] Osmosis 0.41 Released

Hi All, I've just released Osmosis 0.41. http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-0.41.tgz http://bretth.dev.openstreetmap.org/osmosis-build/osmosis-0.41.zip http://dev.openstreetmap.de:23457/hudson/job/osmosis-release/6/ >From changes.txt: - All entities can now pass metadat

[OSM-dev] Broken Daily Replication Diffs

Hi All, Please note that there have been some problems with the daily replication diffs in the last couple of days. http://planet.openstreetmap.org/redaction-period/day-replicate/ BACKGROUND All files after 148 were incorrectly created. The data in them was more or less valid data, but they were

Re: [OSM-dev] Osmosis exception

Hi Stéphane, On 31 August 2012 23:11, Stéphane Henriod wrote: > Hello > > it's the first time I post on this list, so I hope it's the right place to > ask this question! > > I want to use Osmosis to keep a PostGIS database up-to-date (for use with > Mapnik), as explained here > http://wiki.opens

Re: [OSM-dev] Osmosis exception

On 2 September 2012 04:26, Stéphane Henriod wrote: > Hi Brett > > Thanks for your answer! > > Yes I have compiled the source with the command ant build. > What command did you run to build the software, and which directory did you run the command from? Did the ant command complete without error

Re: [OSM-dev] Broken Daily Replication Diffs

On 4 September 2012 15:48, Stéphane Henriod wrote: > Hi Brett > > Is this problem only for the daily diffs, or also for the minutes and > hours? I am trying to get the minutely diffs working and get some strange > error messages "*Unable to read the changeset file on the server*" > It was only d

Re: [OSM-dev] XML planets (was Re: New proposed directory layout for planet.openstreetmap.org)

On 7 September 2012 17:28, Roland Olbricht wrote: > The planet file is necessary for the first startup. Afterwards, it can > work forever solely on minute diffs. And new instances can be cloned from > the exisiting instance over a public interface, without a planet file. > > This initial startup

[OSM-dev] Fwd: Streaming Replication

Oops, I sent my previous email to the wrong list ... -- Forwarded message -- From: Brett Henderson Date: 13 October 2012 15:43 Subject: Streaming Replication To: osmosis-dev Hi All, For those of you who currently use the minute diffs to keep a local database up to date, you

Re: [OSM-dev] Fwd: Streaming Replication

On 14 October 2012 09:46, sly (sylvain letuffe) wrote: > > Hi All, > > Hi Brett, > > > To solve this, a new streaming replication mechanism has been developed. > > Under the covers the same database queries are utilised, but the process > > performing the queries runs continously and polls the da

Re: [OSM-dev] Streaming Replication

One extra bit of info that might be of interest. There are a number of different URLs supported by the server. They are described here: http://wiki.openstreetmap.org/wiki/Osmosis/Detailed_Usage#--send-replication-data_.28--srd.29 To summarise: - You can download just state information, or st

Re: [OSM-dev] Fwd: Streaming Replication

On 14 October 2012 13:59, Brett Henderson wrote: > On 14 October 2012 09:46, sly (sylvain letuffe) wrote:note > : I have a strange issue with all objects's timestamp, they seam > to be > >> shifted by -1 hour >> > > Really? I might have to look into this this

Re: [OSM-dev] Fwd: Streaming Replication

On 15 October 2012 23:08, Matt Amos wrote: > On Sun, 2012-10-14 at 22:39 +1100, Brett Henderson wrote: > > The timestamp columns in the database are set to "timestamp without > > time zone" which presumably means the timezones of dates aren't > > automatical

Re: [OSM-dev] Osmosis bug with --write-pgsql task

Hi Mike, On 16 October 2012 05:57, Michal Migurski wrote: > Hello, > > I've encountered an error with Osmosis and writing to the snapshot schema > that might be a bug. I am attempting to add a new extract to an existing > database, and seeing an error where a new index is trying to be created. A

Re: [OSM-dev] Timestamp in PBF files

On 21 November 2012 19:43, Frederik Ramm wrote: > Hi, > > To be self-contained, it should be sufficient to include the "baseURL" > from configuration.txt, no? > > So maybe: > > > optional string writingprogram = 16; > optional string source = 17; > optional sint64 timestamp = 18; > opt

Re: [OSM-dev] Timestamp in PBF files

Hi Markus, On 24 November 2012 00:04, wrote: > Hi Brett, > > > *If* this information is intended to be used as an input into replication > > processes then the sequence number is essential. Osmosis writes a > > timestamp in the state.txt file, but it only for identifying the right > > sequence

Re: [OSM-dev] Timestamp in PBF files

On 12 December 2012 01:53, Frederik Ramm wrote: > Hi, > > It would be great if someone were to add support to Osmosis which is > likely to be a bit tricky as you have to shove replication information > through the pipeline, but if all else fails I might have a go at it during > the holidays.

Re: [OSM-dev] Osmosis crash (what am I doing wrong)

Hi Serge, On 14 December 2012 15:11, Serge Wroclawski wrote: > > What have I done wrong? > The following line in the stack trace indicates that you've run out of disk space during processing. Caused by: java.io.IOException: No space left on device The --used-node task stores all data to tempor

Re: [OSM-dev] osmosis 32/64 ID failure

On 11 February 2013 07:33, Jochen Topf wrote: > On Sun, Feb 10, 2013 at 09:31:16AM -0700, Kai Krueger wrote: > > On 02/09/2013 04:13 PM, Lynn W. Deffenbaugh (Mr) wrote: > > >I'm running Osmosis Version 0.34 to pull and apply minutely > > >updates to my database for use by mod_tile/renderd. These

Re: [OSM-dev] osmosis 32/64 ID failure

On 13 February 2013 06:21, NopMap wrote: > > Hi! > > > Brett Henderson wrote > > I believe it was fixed in Osmosis 0.38. But I'd recommend using 0.41 > (ie. > > the latest) if possible. > > Unfortunately, it is only partially fixed. I tried Osmosis 0.

<    1   2   3   4   5   6   >