Re: [OSM-talk] [osmosis-dev] Osmosis replication fails
On 18 December 2011 10:12, Martijn van Exel m...@rtijn.org wrote: On Sat, Dec 17, 2011 at 1:49 PM, Paul Norman penor...@mac.com wrote: When switching between minutely and hourly you have to change the sequenceNumber in state.txt Thanks -- how do you figure out the new sequence number though? You need to pick a sequence number by examining the timestamp property in the state files. Pick a timestamp in the hour replication state files that is earlier than the last timestamp received from the minute replication. It's a bit tedious. I seem to remember somebody creating a web-based too to assist with this but I don't have a link handy. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] [osmosis-dev] Osmosis replication fails
On 13 December 2011 06:31, Martijn van Exel m...@rtijn.org wrote: Hi all, I have a replication task set up, initially with a longer interval because my planet file is a few weeks old. I have had it running in a cron job with a two hour interval but I get a lot of errors similar to this one: SEVERE: Thread for task 1-rri failed org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to parse xml file /tmp/change8301792328184763034.tmp. publicId=(null), systemId=(null), lineNumber=7384, columnNumber=3. Caused by: org.xml.sax.SAXParseException; lineNumber: 746; columnNumber: 3; The element type osmChange must be terminated by the matching end-tag /osmChange. It sounds like the change files are incomplete. Perhaps some of the downloads are failing. Is your network connection usually reliable? Out of 50 executions, this error appeared 23 times. I have my replication interval set to one day, could that be the problem? Processing (when it succeeds) takes about 90 minutes. I have the cron job set to execute every two hours. How is your replication configured? Specifically which replication files are you using (ie. minute, hour or day)? It may be worth switching to files with a longer interval if you are patching a file to reduce the number of downloads required. Minute replication files would typically be more suitable to patching a database where small files can be applied quickly. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Osmosis exception
Do any of those paths have spaces in them? That might cause the issue. Also, are multiple drive letters involved? The osmosis.bat launch script appears to have an issue where it requires Osmosis to be on the same drive as the working directory. Brett On Sun, Nov 7, 2010 at 11:46 PM, Carsten Nielsen list_re...@toensberg.dkwrote: When extracting a OSM dataset for denmark from the geofabrik europe file, I use a line like this on my Windows 7 system call %OSMTOOLS%\Osmosis\osmosis-0.37\bin\osmosis.bat --read-bin %DATADIR%\europe.osm.pbf --bounding-polygon-0.6 file=%DATADIR%\CTN OSM DK mm.poly.txt --write-xml-0.6 file=%DATADIR%\denmark_mm.osm But I get en error like this Exception in thread main java.lang.NoClassDefFoundError: org/codehaus/classworlds/Launcher Caused by: java.lang.ClassNotFoundException: org.codehaus.classworlds.Launcher at java.net.URLClassLoader$1.run(Unknown Source) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) at sun.misc.Launcher$AppClassLoader.loadClass(Unknown Source) at java.lang.ClassLoader.loadClass(Unknown Source) Could not find the main class: org.codehaus.classworlds.Launcher. Program will exit. Any clues to what I might do wrong ? Carsten ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] osmosis question
On Mon, May 24, 2010 at 4:45 PM, Gary68 g...@gary68.de wrote: command: ~/osmosis/osmosis-0.30/bin/osmosis --read-xml-0.6 file=osmdata/hessen.osm --way-key-value keyValueList=highway.motorway --write-xml-0.6 file=bab.osm error: May 24, 2010 8:43:02 AM com.bretth.osmosis.core.Osmosis main SEVERE: Execution aborted. com.bretth.osmosis.core.OsmosisRuntimeException: Task 2-way-key-value does not support data provided by default pipe stored at level 1 in the default pipe stack. Hi Gary, The error is indicating that the way key value task can't accept the input provided by the previous task. That could be because you're trying to feed a 0.6 task into a 0.5 task. However as Frederik has pointed out, I'd recommend upgrading to the latest Osmosis. 0.30 was created during the transition from 0.5 to 0.6 and there may have been some incorrectly mapped tasks at that time. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] full history of a way?
MP wrote: There are no full history dumps currently - having such dump would enable this type of query quite easily. I think the daily diffs could be used for help if the changes are fresh (less than 14 days). Are there any nearby plans for full history dumps? This shouldn't probably be very hard, instead of having only one node/way/relation in the result you will have many with same ID and different timestamp as the history progresses. You'll have just to mark deleted objects somehow (those that have history, but they are not in current data) Basically, once the initial dump will be done, you can just concatenate any changes :) There are plans for full history dumps. In fact osmosis (which produces the existing diffs) can already produce full history diffs but they're not enabled at the moment. The current minute changesets have a problem where they occasionally miss data. Once we get this ironed out I'll take another look at full history diffs. Some sample data is available here: http://planet.openstreetmap.org/history/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Setting comment on existing changeset
Frederik Ramm wrote: Hi, Brett Henderson wrote: Something to keep in mind is that I plan to include changesets in changesets one of these days. Um, let's reword that :-) I plan to include changeset information in the osmosis deltas including their tags. If comments can be updated on old changesets then there will be a limitation that these updates will not be replicated. It shouldn't necessary change the outcome of this discussion, but it does have implications that need to be considered. Would we need a changeset history table then ;-) Hehe, yep that would solve the problem. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Setting comment on existing changeset
Frederik Ramm wrote: Hi, Ed Avis wrote: IMHO it would make sense for the comment on a changeset (and only that) to be settable even after the changeset is closed. Someone who would want to see all changesets that have a certain word in the description would then have to scan all changesets all the time because someone might change his description, rather than simply checking all those that were closed during the last 24 hours... Something to keep in mind is that I plan to include changesets in changesets one of these days. Um, let's reword that :-) I plan to include changeset information in the osmosis deltas including their tags. If comments can be updated on old changesets then there will be a limitation that these updates will not be replicated. It shouldn't necessary change the outcome of this discussion, but it does have implications that need to be considered. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] more changeset question
Andy Allan wrote: 2009/4/22 Iván Sánchez Ortega i...@sanchezortega.es: El Miércoles, 22 de Abril de 2009, maning sambale escribió: Thanks! seems reasonable for my regular editing session. What about bulk imports? There is one script to bulk-upload stuff in SVN (apps/utils/import/bulk_upload_06). It does split big files into smaller chunks. Expect it to be buggy and untested. However, for really large datasets, I'm partial to running osmosis directly on the DB server. It better create appropriately sized changesets in any case! It does actually. It's currently set to 50,000 the same as the api. But it doesn't update all changeset attributes properly such as bounding box values, so it's not quite ready to be a bulk importer just yet. Patches welcome of course. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] We're back
Shaun McDonald wrote: On 21 Apr 2009, at 12:27, Steve Hill wrote: On Tue, 21 Apr 2009, Richard Fairhurst wrote: ...with API 0.6, Postgres and the new server. But everyone's uploading at once, so don't expect to do much serious editing for the time being. :) The deltas in http://planet.openstreetmap.org/minute aren't being updated at the moment - what's the plan for these (I notice that the test06 directory has more up to date deltas, but these have stopped now too)? The minute diffs are currently down until Brett updates Osmois on dev. I'm on the train. Give me an hour or so ... ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
[OSM-talk] Minute Diffs and 0.6 Breakages
Hi All, I'm cross-posting to dev and talk. Please reply to dev only to avoid unnecessary noise on the mailing lists. The osmosis minute diffs were broken after 0.6 went live. I'm hoping the problems have been fixed now and have re-generated all impacted changesets. If you are consuming diffs please note the following: * All diffs are now in 0.6 format. * Minute diffs from 200904210807-200904210808.osc.gz should be re-applied in order to ensure incorrect data is overwritten. * Hourly and daily diffs are now also being produced in 0.6 format, and don't require any re-application. If you're interested in gory details, this was caused by two bugs in osmosis: 1. An incorrect query ORDER BY clause. This prevented tags from being matched to their owning node/way/relations. 2. Incorrect entity version selection. If an entity was modified twice in a time interval, the first one was being selected rather than the last. If you notice any issues with the current diffs please reply with details of the problem. Please provide an example entity id, entity type, and entity version when describing any issues. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Possible UTF-8 encoding errors in changesets
Matt Amos wrote: On Sat, Apr 4, 2009 at 8:01 PM, Florian Lohoff f...@rfc822.org wrote: Will this problem be fixed with API 0.6? Its quite annoying to fix this again and again manually. yep. the problem with bad UTF-8 in the database will be solved, both in the server code and in the database tables. It's probably too late for most people but the changeset files have now been fixed. As someone who has spent many hours dealing with the current database utf-8 issues I'll be ecstatic when 0.6 goes live :-) Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] [osmosis-dev] Osmosis running forever with completeWays=yes?
Frederik Ramm wrote: Hi, (f'up set to osmosis-dev) Karl Newman wrote: Anyway, the tee can choke things up with all the temporary files. It would be nice to be able to share the stored node and ways files between tee tasks, but I haven't created that infrastructure yet. It would be even better to have an extended --bp task that somehow takes a list of disjoint polygons and uses some kind of point location algorithm to determine which node belongs to which polygon. The rationale being of course that with the classic --bp/--tee approach, each node is duplicated n times and tested against each of the polygons which is a waste of time, especially with a large input file and many polygons (e.g. split up the US into counties or so). Just to be clear, the --tee task doesn't duplicate nodes. It simply passes nodes to multiple downstream tasks. The problem with the --bp task is that when it is used downstream of a --tee task each instance persists the node information. This exact problem is why I originally created the customdb tasks which aimed to create a single random access dataset (with appropriate indexing) which could be built once then queried many times for many bounding boxes. When that provided dismal performance I created the pgsql tasks instead. There is a --dataset-bounding-box task which can be used to read a bbox from a database. osmosis --read-pgsql --data-bounding-box left=xx . --write-xml myextract.osm I've been distracted by other things recently so haven't spent much time on the bounding box implementation for a while. I've been meaning to load up a full pgsql db to see how it performs for tile cutting. Does the task and stream model that osmosis uses theoretically support tasks where the number of output streams they create is not fixed, but dependent on their parameters? So that e.g. a bp file=a.poly file=b.poly (or bp files=a.poly,b.poly) creates two entity streams and so on? Hmm, perhaps this is a better way to do it. I hadn't thought of keeping a single copy of nodes with references to the polygons they reside in. If somebody can come up with a faster implementation than pgsql I'll be ecstatic. I've wasted a lot of time on this one. Note that the pgsql (and customdb) implementations solve the problem of ways crossing bounding boxes without having nodes in them which might be difficult to solve using an alternative solution. Yes, osmosis can support variable numbers of output streams so long as they are known at startup time. This is pretty much what the --tee task does. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Osmosis running forever with completeWays=yes?
Karl Newman wrote: On Sun, Feb 1, 2009 at 1:55 PM, Frederik Ramm frede...@remote.org mailto:frede...@remote.org wrote: Hi, (f'up set to osmosis-dev) Karl Newman wrote: Anyway, the tee can choke things up with all the temporary files. It would be nice to be able to share the stored node and ways files between tee tasks, but I haven't created that infrastructure yet. It would be even better to have an extended --bp task that somehow takes a list of disjoint polygons and uses some kind of point location algorithm to determine which node belongs to which polygon. The rationale being of course that with the classic --bp/--tee approach, each node is duplicated n times and tested against each of the polygons which is a waste of time, especially with a large input file and many polygons (e.g. split up the US into counties or so). Does the task and stream model that osmosis uses theoretically support tasks where the number of output streams they create is not fixed, but dependent on their parameters? So that e.g. a bp file=a.poly file=b.poly (or bp files=a.poly,b.poly) creates two entity streams and so on? Bye Frederik What you're asking is possible. The number of input and output pipes has to be known at invocation because the pipes are connected before any tasks are run, but if it's a parameter passed to the task, then the task can report to the pipeline manager how many output pipes it has. The tricky part might be connecting the downstream tasks. It might be confusing because of the stack-based pipeline ordering. If you want to see how this works, check out the SinkMultiSource interface which defines a task with a single input and multiple outputs. It is implemented by the EntityTee class which is the --tee task. It is integrated into the pipeline by the SinkMultiSourceManager class. The SinkMultiSource interface defines a method called getSourceCount which allow tasks to tell the manager how many pipe outputs they have. It is called by SinkMultiSourceManager during pipeline startup. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] 0.6 move and downtime
SteveC wrote: Dear all We had a review call today on the technical side of OSM and one of the things that came up was the transition to API 0.6 http://wiki.openstreetmap.org/wiki/OSM_Protocol_Version_0.6 API 0.6 is the latest and greatest in OSM APIs and brings scrummy goodness like changesets. You can read more at the wiki page. A lot of work has gone in to it from a number of people. The downside is that there needs to be some database downtime while the change happens. This means a period of time without login, editing and so on. The slippy map will of course still work. We have agreed a date of 21/22 March. During this time, and possibly for a little while after, you won't be able to log in or edit while things are upgraded. Technical questions on this should be thrown at Matt Amos and TomH. So, apologies, but hopefully this is enough advance warning for you to avoid activities around that time. From there OSM will be stronger, better, faster, longer and higher than before. On a related note (note sure about the best way to announce this), there will be an interruption in the the osmosis daily/hourly/minute changesets during this time. I'll start them again when the db comes online again after the migration and they'll be in 0.6 format. It will be necessary to download a new planet and resync again. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Proposed Relations
On Thu, Nov 6, 2008 at 11:10 AM, Frederik Ramm [EMAIL PROTECTED] wrote: Hi, Karl Newman wrote: 2. Make sure the big three editors support ordering. Nobody will notice, nobody has to change what he does. Also, try and get Osmosis and the various mirror services (ROMA, XAPI) to support ordered relations. I can confirm that Osmosis already maintains the order of relations. That's good but in order to allow Osmosis-based database thingies like ROMA (at least I believe it uses Osmosis?), one would probably have to amend the database input/output code to populate/use a sequence number field in the relation_members table! Yep, the main change from an osmosis point of view would be supporting database schema changes. The pipeline structures stores members as a java.util.List so will preserve the order once it is loaded. Currently osmosis supports 3 schemas, the API MySQL schema, a PostGIS schema I refer to as pgsql (used as the basis of ROMA), and a custom file-based db which isn't used by anybody to the best of my knowledge which I might delete anyway to make life simpler. But don't jump ahead and do something, my idea might still be shot down and/or shown to be flawed or useless in some way, I haven't discussed it with the others working on 0.6 yet. No problem, I do lazy very well :-) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] osm2pgsql planet: frustrations, cutoffs, and idempotence
Others have already commented on most of your points but I'll add my thoughts in case there's some gaps. Michal Migurski wrote: Hi, I've been trying to keep up to date with the dumps and diffs from http://planet.openstreetmap.org/ , and I'm running into a number of bugs related to cutoff dates. In keeping my Bay Area tiles (http://mike.teczno.com/notes/cascadenik-openstreetmap.html ) up to date, I've been grabbing complete planet.osm dumps about once per month, and filling in the intervening time with daily diffs. I've noticed some misalignments between the data in the dumps and the osm2pgsql importer that leads to unavoidable holes in the data. It seems that they could be fixed in either osm2pgsql, the planet files, or both. The final event in each weekly planet dump does not fall on an even day boundary. In the case of the most recent Oct. 22nd planet.osm, it was necessary to experiment with hourly diffs from that day to find that the boundary was approx. 2:00pm. Hourlies up to and including 2008102213-2008102214.osc.gz failed, hourlies after that succeeded. I could go more granular here, checking the minute diffs as well for a more precise breakpoint, but it seems odd that the planet dump does not break cleanly on a midnight boundary so that it's possible to pick up the differences moving forward. Yep, as others have commented there are two tables types in the osm database; current tables, and history tables. The planet dumper just reads current tables which is the fastest approach. Unfortunately the current tables change constantly during the planet generation process resulting in inconsistencies. It is possible to produce a consistent snapshot reading history tables and osmosis has the ability to do just that but it is significantly slower. It is also possible to produce a consistent snapshot by taking an inconsistent planet and applying changesets from a point in time prior to the planet dump beginning through to a point after completion, this effectively produces the same result at much reduced load on the main database. osm2pgsql itself notifies the user of inconsistencies by failing. I can see that effort has been put into making it more resilient (e.g. http://trac.openstreetmap.org/changeset/10464) . Does osm2pgsql have something like a `--force` switch? I haven't been able to find one. In looking at the diff files, it seems that it should be possible to ignore possible conflicts by simply overwriting whatever's in the DB with whatever's in the .osc file. Yes, that's true. I can't comment on osm2pgsql but when osmosis processes changeset files it does exactly that. Finally, the boundaries between the hourlies and dailies seem misaligned. This shouldn't be the case. After running the remaining hourlies for the 22nd, I attempted to pick up on the 23rd with a daily. The final hourly I used was 2008102223-2008102300.osc.gz. It's my expectation that I should be able to immediately follow that with 20081023-20081024.osc.gz, but this led to duplicate key violation suggesting that there's an overlap between the two files. Continuing with hourlies *works*, but is tedious and I suspect slower than the dailies. You should have been able to do what you've suggested. If you are finding problems, please provide me with some example data which is misaligned between the two types of changesets. I've gone to a fair bit of trouble to ensure that timestamp management is correct. For example, all changesets and file names are using UTC even though the database itself is using BST. If I've made a mistake somewhere I'd like to know about it. Given that daily, hourly and minute changesets are using *identical* code, I find it hard to believe they're inconsistent with each other. My sense from reading other people's experiences has been that it's a common pattern to rely solely on the weekly planet dumps, incurring the substantial overhead of parsing and importing the full 5GB dump once every week, and then re-rendering the complete set of tiles. For a long time weekly planet dumps were the only bulk data available. Osmosis changesets have been on the scene for some time now though and are gradually being utilised by more and more clients. As the planet grows, this will become more critical. Who knows, if the kinks gradually get ironed out of the osm2pgsql program we may even begin to see the main mapnik tile generator move to using changesets. My hope has been to proceed in a more incremental fashion, since this makes it possible to track what specific tiles need to be re-rendered on a near-constant schedule, based on actual content or activity, vs. simple cache expiration. Right now I'm doing this daily, I'd like to do it as often as hourly. Yep, that was one of my original aims. I can see a few possible solutions. The cutoff times for
Re: [OSM-talk] osm2pgsql planet: frustrations, cutoffs, and idempotence
Jochen Topf wrote: If the planet dump plus the diff from the same day is what everybody wants anyway, why not do this on the server side and hold the planet back after the first diff is available, run this over the planet and then publish that as the planet? It would add delay to the planet creation process. I don't know how much of an issue that would be. How many people still download the full planet on a regular basis? I would hope that people would begin to use changesets even if they only require a complete xml file. For bandwidth reasons alone the gains are well worthwhile, plus you can get far more regular updates than weekly. The script below automates keeping a snapshot file in sync: http://svn.openstreetmap.org/applications/utils/osmosis/script/contrib/replicate_osm_file.sh ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] osm2pgsql planet: frustrations, cutoffs, and idempotence
On Tue, Oct 28, 2008 at 7:39 AM, Michal Migurski [EMAIL PROTECTED] wrote: Finally, the boundaries between the hourlies and dailies seem misaligned. This shouldn't be the case. After running the remaining hourlies for the 22nd, I attempted to pick up on the 23rd with a daily. The final hourly I used was 2008102223-2008102300.osc.gz. It's my expectation that I should be able to immediately follow that with 20081023-20081024.osc.gz, but this led to duplicate key violation suggesting that there's an overlap between the two files. Continuing with hourlies *works*, but is tedious and I suspect slower than the dailies. You should have been able to do what you've suggested. If you are finding problems, please provide me with some example data which is misaligned between the two types of changesets. Try the two files mentioned above - that's where I saw this behavior, they're quite recent. 2008102223-2008102300.osc.gz 20081023-20081024.osc.gz I need you to provide some specific examples of broken data. If you can say that way 27123456 is created in both of the above files even though they are for different time periods then I can take a look at why this may have occurred. Just saying that there is misalignment between those two files doesn't help me at all. Presumably you ran into a specific problem and received a specific error message, this is the kind of information I need. I only do this project in my spare time and can't go looking for problems that I'm not sure even exist, I have enough known problems to look into already :-) My sense from reading other people's experiences has been that it's a common pattern to rely solely on the weekly planet dumps, incurring the substantial overhead of parsing and importing the full 5GB dump once every week, and then re-rendering the complete set of tiles. For a long time weekly planet dumps were the only bulk data available. Osmosis changesets have been on the scene for some time now though and are gradually being utilised by more and more clients. As the planet grows, this will become more critical. Who knows, if the kinks gradually get ironed out of the osm2pgsql program we may even begin to see the main mapnik tile generator move to using changesets. I would love to rely on these exclusively, it's much more efficient. But, I was seeing a fair bit of information fall through the cracks so that's why I'm re-synching to planet every four weeks. Again, please provide some specific examples. If data is being missed I'd like to know about it. Osmosis provides some tools that may be useful here. You can download a planet, apply changesets for a week, then compare against the next planet and see what the differences are. Obviously both planets would need appropriate changesets applied to make them consistent before performing a comparison to eliminate noise. I probably should do some of these comparisons myself, but again just haven't found time yet and nobody else has complained about missing data. The minute changesets run 5 minutes behind the API so could potentially miss data if a lock is held for several minutes. The daily and hourly changesets run at least 20 minutes behind API (forget off the top of my head) and should be extremely unlikely to miss data. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Searching for recently added names not working
On Thu, Aug 21, 2008 at 8:19 PM, David Earl [EMAIL PROTECTED]wrote: ... The problem was that when I switched it to using the daily planet gz instead of bz2 files, I hadn't realized that the name of the XML tag for additions also changed from add ... to create The log files all look reasonable: it was showing a large number of ways etc being processed, but of course it was only counting modified ones, not new ones, skipping the entries enclosed by create completely. So I only discovered this when I searched for some streets I added this week and couldn't find some but did find others (those I had later modified) and investigated why. ... Sorry about this. The rules changed under my feet! Are you sure? I didn't remember changing anything in this space so checked svn. My original osmosis code import to svn in September 2007 had the xml tag named as create. The same code is used for creating the xml within the bz2 and gz files so there shouldn't be any difference between the two. The difference between the bz2 and gz processes was in the extract automation, not the database and xml code. I wouldn't bring it up except that it sounds like you might be missing more data than just that which changed in the past couple of weeks. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/listinfo/talk
Re: [OSM-talk] Milestone
On Tue, Jul 29, 2008 at 6:08 PM, Andy Robinson (blackadder-lists) [EMAIL PROTECTED] wrote: With a readymade image containing an OSM rendering toolchain - Mapnik, osm2pgsql and TileCache - plus some instructions on the wiki, we'd have a roll your own cartography kit. A WYSIWYG stylesheet editor would be the icing on the cake but not necessary at the start. I am way out of my depth here technically, but wouldn't that be so, so cool? +1, very cool and something I'd try my hand at. So how might we kick something off, even if it's a bit too challenging for some of us, me included? I'm not sure how relevant this is, but I've made some steps in this direction here: http://wiki.openstreetmap.org/index.php/OnDemandTileServer It's only a few weeks old but it already needs some improvements: 1. Ideally should use fedora 9 rather than fedora 8, I didn't have time to download 9 at the time (I have now but haven't started from scratch yet). 2. Mod_tile has seen a number of improvements recently, font paths have changed among other things. 3. osm2pgsql is introducing the ability to apply changesets directly. Not sure if this supports a small region. The vmware image runs well in 512MB of RAM. It may not scale well to a full planet but I suspect most people won't be interested in a full planet anyway, if they do it should be a matter of tweaking vmware and postgres memory settings. I'm a complete noob to mapnik so haven't looked at stylesheet customisations yet. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] multiple bounding-polygons in one file?
Frederik Ramm wrote: Hi, Olga Sacharow wrote: Hello! I just started working with osmosis and the first problem I have is that I can not find an option how to extract several polygons out of the planet.osm file into one single file. I need one osm file with the openstreetmap data from several adjoining European countries together. Can please somebody tell me if that is possible? Option 1: Create ONE polygon that encompasses the full area you need. If you have existing polygon files, use poly2osm, then modify them in JOSM, then use osm2poly. Yep, that is the best approach. The polygon file can contain any number of polygon sections. If you have individual polygon files, it should be relatively straightforward to merge them together using a simple text editor. Option 2, which will be slower, and is completely untested by me but should work in theory: Cut out individual polygons then merge them like so: osmosis --rx input.osm --tee 2 --bp file=1.poly --bp file=2.poly --m --wx output.osm where 1.poly and 2.poly are your polygons. If you have more than 2 polygons, you'll have to increase the number behind --tee, and you will have to specify the --m option one time less often than the number of files, e.g. osmosis --rx input.osm --tee 4 --bp ... --bp ... --bp ... --bp ... --m --m --m --wx output.osm But check with BrettH if unsure. Yep, that will also work. The command line will get fairly complicated, it will be easiest if you use explicit pipe names when connecting tasks together. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
[OSM-talk] New Daily Diff Files
Hi All, Apologies for cross-posting to two lists but discussions have been split across both lists. Please reply to the dev list. There are now some new daily diff files available on the planet server that address some issues with the existing daily diff files. These new files are produced using a more reliable mechanism that is already used by the hourly and minute diffs. They have a different naming standard so there's no conflict. http://planet.openstreetmap.org/daily/ The timestamp.txt file will tell you what the latest file is available to be downloaded rather than relying on 404 server responses if new files aren't available. The osmosis --read-change-interval task will work with the new files but not the old. The task will merge all available files into a single change stream that can then be written to a file using --write-xml-change (for subsequent import to a db) or passed to another task such as --apply-change for merging into existing xml files. The new files are gzip compressed due to performance issues with bzip2 compression in java. This means they're bigger but we're still only talking approximately 10MB per day. There is one big GOTCHA. The new files use UTC timing, the old files use BST timing. This means that the contents of the files are different. If you transition to the new file format you should re-apply the new gzip file corresponding to the most recently applied bzip2 file in order to capture the missing hour. If you don't do this I *think* you'll miss an hours worth of data from 11pm to midnight. If the last file you imported was daily-20080622-20080623.osc.bz2, you should start on the new files from 20080622-20080623.osc.gz. Let me know if you see any problems. Cheers, Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Create extra Planet files for syncing
I never got around to checking out what happened to the diffs during that period, apologies. Are you sure that there's a gap in data? It isn't a huge amount of effort to re-generate a daily file, that can be generated at any time. I'm not sure how simple it is for you to utilise, but the hourly (and minute) diffs will provide a continuous stream of diffs even during outages. It would be possible to apply those as a workaround. Another option is for me to update the daily diff generator to use the same code as the hourly and minute diffs. The two issues with this approach are that: 1. I will have to switch to gzip format due to java bzip2 performance issues. 2. It will lock the MySQL history tables for a longer period. Frederik Ramm wrote: Hi, The coastline checker won't see anything after the 19th because the daily diff for that day is missing. Either the diff will appear or it will resync on the next planet dump. That's a problem I am having (for the Geofabrik Excerpts) as well - once a daily diff has failed, I'm stuck for the rest of the week. Actually this time I simply applied the new diffs anyway, ignoring possible consistency problems, but it's not the best thing. Would it be possible to issue extra planet dumps after events that disrupt the diff chain, like we had last week? So that those who rely on diffs for a current version of the data have a clean start to work on, without having to wait for the next regular planet? What does OSMXAPI do in cases like these? Bye Frederik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Create extra Planet files for syncing
Frederik Ramm wrote: Sure, I can do that. I was assuming they would somehow be incomplete as well but if they are ok then I can just switch to using these. The hourly minute are definitely more reliable than daily. I won't be surprised to see missing data in daily files. If you see *any* problems with the hourly or minute files I'd like to know about it. The daily diffs are created using a shell script that doesn't fail gracefully and doesn't have the ability to generate multiple files if the db has been down for a long period. The hourly and minute diffs are created using the osmosis-mysql-extract application (included in the osmosis distribution) which aborts if the db is down (spamming me with cron failures every minute ...) and catches up again when the db comes up again generating as many files as required to become current again. The minute diffs encounter intermittent failures on a regular basis due to the db or connectivity being lost for brief periods but always recover again. It would take literally 5 minutes to setup daily diffs with the new mechanism. The gzip format is something people would presumably get used to fairly quickly. But the downside is that the compressed files are created directly whereas the daily script extracts to an uncompressed file to minimise db locking time then compresses separately. If we eventually switch to all InnoDB tables where locking isn't such an issue I'll definitely cut it over. Bye Frederik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Create extra Planet files for syncing
David Earl wrote: Would it really be that much slower: yes it is more work, but OTOH, it is fewer disk writes? The daily diffs are approximately 6MB of data to write but take a couple of minutes to produce. The disk overhead is negligible. But this also means that the compression overhead is probably also negligible and wouldn't add much to the overall time. When I first set this up we were dealing with TIGER imports and much larger daily files, the volumes are much smaller at the moment. I'll set up a daily diff using the hourly/minute mechanism and see if any problems occur. I rely on these for the Namefinder updates, and I've always been worried that they may not form a continuous sequence, especially if something goes wrong, the consequence of which is to repair it I'd have to do a full database import which takes a week or so to run. Understand, my aim is for this to be a reliable means of keeping in sync. It would be a simple matter to switch to gzip, so long as I know when it is to change. I'll set up the new one in parallel for a little while. I noticed that the day after the empty file, the file was larger than usual. Did it in fact catch the diffs since the previous file, or just in the previous day? I've just had a quick look, I'm fairly sure data has been missed. The larger file is presumably because people had queued uploads. From the Namefinder POV, if I miss a file, I catch up with it later (but it did break when you changed the convention to span two days a while back after a failure, but I fixed that). But if there is a gap in the sequence that's very hard to repair because I'd already have applied later updates. David I have just re-generated the file from the 19th-20th. http://planet.openstreetmap.org/daily/daily-20080619-20080620.osc.bz2 If you wish to re-import, just import the files in sequence again. Assuming your import scripts won't break in some unexpected way, this will ensure that all updates are applied in the correct order. Note that osmosis now has a task called --read-change-interval which can download all hourly or minute diffs since the last invocation, merge them into a single changeset, and send to subsequent tasks in the osmosis pipeline. The consuming task can be an xml change file writer or a database writing task if you have one. It tracks the latest downloaded timestamp in a timestamp file and will generate empty changesets if no updates are available yet and will abort completely if the planet server can't be reached. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Mapnik Tilecache Memory Error (Myanmar Cyclone Relief)
I suspect I didn't with tilecache (not sure if it's possible), but I do now with mod_tile. Martijn van Oosterhout wrote: A bit late and maybe a stupid question but: do you have metatiling turned on. Rendering is going to really suck without it. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Mapnik Tilecache Memory Error (Myanmar Cyclone Relief)
OJ W wrote: It's only outline code now (i.e. no rendering rules), but pyrender shares those objectives (render on demand, get small amounts of data as required) http://svn.openstreetmap.org/applications/rendering/pyrender/ Thanks for the info. Is this something that could be configured to generate tiles for an OpenLayers enabled web site? I have mod_tile up and running now so will go with that approach for now. One thing that might trigger a switch to another rendering mechanism is resource consumption, although mod_tile doesn't seem too heavy. Brett ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Mapnik Tilecache Memory Error (Myanmar Cyclone Relief)
Hi All,
Just to give some background on what I'm trying to do here. I'm helping
a team in Myanmar setup a software package called Sahana which is an
open source disaster response package. It will be used to help in the
response to the recent Cyclone.
http://www.sahana.lk/
It has a map component based on OpenLayers allowing it to display maps
of affected areas. They wish to use OSM data to provide this because it
provides them with the best chance of producing decent maps of the area.
A constraint they face is extremely poor and intermittent network
connectivity. I can limit the bandwidth required by using osmosis to
download incremental OSM data updates into a local OSM file and
regularly re-import it into a mapnik postgis database.
They also wish to setup users as OSM editors to directly edit OSM data
in the region, at this point we will attempt to use JOSM's offline
editing. In future we might look into a full replicated OSM instance
with a replication mechanism back to the main OSM database but this is
complicated and lower priority right now.
The Sahana guys have already been able to get Sahana configured to
retrieve tiles from OSM tile servers so the client side is working. The
next step is to render these tiles locally without requiring an Internet
connection. I don't want to pre-render tiles because this constrains
the turnaround time on making new data available for display, on-demand
rendering is far preferable because I'm expecting the load to be
relatively low and disk space will be significantly reduced this way.
I am trying to use tilecache to generate mapnik tiles but have run into
the issue described in my previous email. It is probably something
screwy with the way I've configured things but I've been unable to
diagnose the problem.
The tilecache and mapnik software is all installed under a
/home/tilecache directory tree. I want to keep everything in one place
if possible to make it easier to transfer between machines. All
directories and files are owned by root (will change this to a more
limited user at some point) except the cache directory which is owned by
apache.
* /home/tilecache/app contains the tilecache application
* /home/tilecache/mapnik contains the mapnik scripts from the osm svn
* /home/tilecache/cache contains the tilecache cache files
* /home/tilecache/world_boundaries contains the mapnik world boundary data
My tilecache.cfg file contains the following entry (I assume I should
re-enable tms_type but have been playing with different settings):
[osm]
type=Mapnik
mapfile=/home/tilecache/mapnik/osm.xml
spherical_mercator=true
#tms_type=google
I have an apache configuration section that looks like this:
Alias /tilecache /home/tilecache/app
Directory /home/tilecache/app
AddHandler python-program .py
PythonHandler TileCache.Service
PythonOption TileCacheConfig /home/tilecache/app/tilecache.cfg
PythonPath ['/home/tilecache/app'] + sys.path
PythonDebug On
/Directory
I create an OpenLayers layer with a code snippet like this:
layer = new OpenLayers.Layer.TMS( OSM,
tilecache.py/, {layername: 'osm', type: 'png'} );
You can test the site here:
http://www.bretth.com/tilecache/
Some tiles are working, some are failing. Sometimes retrying fixes
broken tiles, sometimes not.
I am running out of ideas. If anybody has any suggestions or can help
in any way please let me know.
Regards,
Brett
Brett Henderson wrote:
Hi All,
I'm trying to setup a server rendering mapnik tiles on demand using
tilecache.
It seems to be working intermittently but most requests result in the
following response being returned to the browser when requesting new
tiles.
An error occurred: failed mapping file: Cannot allocate memory
File /home/tilecache/app/TileCache/Service.py, line 221, in
modPythonHandler
host )
File /home/tilecache/app/TileCache/Service.py, line 205, in
dispatchRequest
return self.renderTile(tile, params.has_key('FORCE'))
File /home/tilecache/app/TileCache/Service.py, line 138, in renderTile
data = layer.render(tile)
File /home/tilecache/app/TileCache/Layer.py, line 411, in render
return self.renderTile(tile)
File /home/tilecache/app/TileCache/Layers/Mapnik.py, line 38, in
renderTile
mapnik.load_map(m,self.mapfile)
If I refresh the tile it eventually gets generated but sometimes takes
several tries. The apache access_log shows the request as a 500. The
apache error_log doesn't contain anything. I have the PythonDebug
directive set to On.
Any ideas what is causing this?
Regards,
Brett
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] OsmChange Files/Datasets.
The create/modify/delete action is applied at the node/way/relation level. There is no way of applying changes at a tag level. When osmosis receives a modified entity in the --apply-change task, it replaces the existing entity with the new one. So in other words if you wish to add tags you need to include all existing tags in your modified entry. [EMAIL PROTECTED] wrote: I've started looking at change sets to upload blocks of data (population figures in this case) and am a little confused. From the work I have done so far I can produce a change set such as --- $ cat population.osc ?xml version='1.0' standalone='no'? osmChange version='0.5' generator='nasty_script_hack' modify version='0.5' generator='nasty_script_hack' node id='51971466' lat='49.485667' lon='-113.9502919' user='Mungewell' osmxapi:users='Mungewell' timestamp='2008-03-27T23:52:01Z' tag k='name' v='Pincher Creek'/ tag k=place v=town/ tag k=population v=3625/ tag k=population_date v=16-may-06/ tag k='is_in' v='Alberta'/ /node /modify /osmChange --- However when I use Osmosis to apply this change set (to a local '.osm' file) it wipes out any of the already existing tags for this node, rather than just modifying/adding those tags which I specify. Is there some other way to apply the change set, or do I need to modify my script to include all the of the original tags for each node in the change file? Cheers, Simon. ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
[OSM-talk] Mapnik Tilecache Memory Error
Hi All,
I'm trying to setup a server rendering mapnik tiles on demand using tilecache.
It seems to be working intermittently but most requests result in the following
response being returned to the browser when requesting new tiles.
An error occurred: failed mapping file: Cannot allocate memory
File /home/tilecache/app/TileCache/Service.py, line 221, in modPythonHandler
host )
File /home/tilecache/app/TileCache/Service.py, line 205, in dispatchRequest
return self.renderTile(tile, params.has_key('FORCE'))
File /home/tilecache/app/TileCache/Service.py, line 138, in renderTile
data = layer.render(tile)
File /home/tilecache/app/TileCache/Layer.py, line 411, in render
return self.renderTile(tile)
File /home/tilecache/app/TileCache/Layers/Mapnik.py, line 38, in renderTile
mapnik.load_map(m,self.mapfile)
If I refresh the tile it eventually gets generated but sometimes takes several
tries. The apache access_log shows the request as a 500. The apache error_log
doesn't contain anything. I have the PythonDebug directive set to On.
Any ideas what is causing this?
Regards,
Brett
___
talk mailing list
talk@openstreetmap.org
http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmosis/Mkgmap: Missing ways at tile borders
Hi Christian, This does sound strange, the completeWays functionality should avoid the problem you're seeing. Can you provide me the id of the way that is causing you problems? I'd like to replicate this bug if possible. If it turns out to be a limitation of the current design (ie. not easily fixable) then there's a backup plan. I've finished writing a new bounding box implementation which might fix your problems. It uses a PostgreSQL database with PostGIS extensions to perform true spatial queries. It is *supposed* to avoid these types of problems but I haven't tested it well yet. This sounds like a good test case. It might take me a few days to get back to you on this though, I'm away for the next few days. Brett Christian Linder wrote: I am using a Garmin 60CSx handheld. As you suggested, I looked at the tiles in JOSM. The problem is the same: In the original osm file, the way is contiguous: --- I call osmosis with something like bzcat ~/Desktop/germany.osm.bz2 | java -Xmx512M -jar ~/osm/osmosis/osmosis.jar --rx file=/dev/stdin enableDateParsing=no --tee 2 --bb left=8 right=9 bottom=50.9 top=51 completeWays=yes completeRelations=yes --bb left=8 right=9 bottom=51 top=51.1 completeWays=yes completeRelations=yes --wx e008n50-e009n51.osm --wx e008n51-e009n52.osm This gives me two tiles, but right at the border between the two tiles (at 51 degree north), there is one segment missing as one tile contains only data north of the border, the other contains only data south of the border: --- --- I reproduced it with different tiles several times, and when I download the german states from Geofabrik http://download.geofabrik.de/osm/europe/germany/ and try to concatenate them to one map for the whole of germany, it is the same problem. So I am kind of confident it is not me doing something wrong. Any suggestions? 2008/4/2, Karl Newman [EMAIL PROTECTED] mailto:[EMAIL PROTECTED]: On Wed, Apr 2, 2008 at 6:24 AM, Christian Linder [EMAIL PROTECTED] mailto:[EMAIL PROTECTED] wrote: Hi everybody, I am trying to split osm data into 1x1 degree tiles with OSMOSIS, then I want to use MKGMAP to create maps for GARMIN devices, assembled from these tiles. Although I use the flags completeWays=yes and completeRelations=yes in OSMOSIS, there are always some segments missing right at the border between tiles if I look at the map on my GARMIN device. Before I investigate further, does anyone happen to know about this issue, and wether it is a problem of OSMOSIS, MKGMAP or the GARMIN device? Best regards Chrischan I wrote the completeWays and completeRelations addition for Osmosis. I'm not aware of any problem with it. Try loading the resulting tile into JOSM to see what's going on. Also, which Garmin device are you using? I'm not aware of any issues with the handheld variety, but the Nuvi, etc. work just a bit differently and it's possible that's where the problem lies. Karl ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] OSMXapi error 501
Christopher Schmidt wrote: Minutely, so far as I can tell based on the way that the server behaves. Regards, It's every minute unless recent issues have changed something. The minute diffs have been somewhat unadvertised but are now available at the top level of the planet server: http://planet.openstreetmap.org/minute/ ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Local map making - truncating ways on boundary?
By default, osmosis will truncate at the last node inside the boundary which isn't always ideal. The completeWays argument will include all way nodes outside the boundary which again isn't perfect. The new PostGIS based bounding box functionality will never truncate ways but may leave out nodes outside the box depending on whether completeWays is specified. Introducing truncation with phantom nodes is very difficult to do in a memory efficient manner. If an in-memory implementation was sufficient it would potentially be much simpler but doesn't exist yet. The new osmosis PostGIS schema would be a good starting point for this. A single task implementation may need to be used (ie. not one bbox task instance per tile) so that it can maintain lists of phantom nodes created at boundaries shared between multiple tiles. I think it was Karl who was looking at creating a tiling task. Not sure whether he's still looking at this. I'd like to play with this but realistically I'm not going to find the time any time soon. Frederik Ramm wrote: Hi, I'm trying to put to together a procedure for making an overview map of a local area. I can use XAPI to get appropriate data, but some ways cross the specified boundary (bbox or bpoly) and continue 'outside the box'. Is there a method for truncating ways at the boundary? None that I know of, but Osmosis will at least truncate ways at the first node outside of the specified box/polygon (or at the last node inside...? not sure ATM). I know that one of the Osmosis guys, probably Karl Newman, wanted to implement something like proper truncation with phantom nodes or so in order to prepare data for Garmin imports. I'd even consider using a method to truncate the SVG output from osmarender, however with the number of layers produced that's not terribly easy either. You know that if you specify a proper bounds element in the Osmarender rules file, Osmarender will create SVG clipping for the specified area? Bye Frederik ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Local map making - truncating ways on boundary?
I hope the job loss isn't too much of a downer, best of luck finding something better. As for tiling, I hadn't considered polygons. They sound nasty. I'd been thinking of something far simpler. For ways I was thinking of splitting them at tile boundaries, adding synthetic nodes as required, and creating new way ids where one way becomes multiple ways. Polygons change all that. Initial thoughts are just detect closed ways and split accordingly to make closed polygons inside each tile. It sounds like that may not be appropriate though. Splitting polygon types according to tags and rules adds a huge level of complexity and will constantly require updates as new tags and polygon types are defined. I also get the impression from your wiki page that this will be very Garmin specific, and perhaps that's the only way to go. I'm going to have to back away slowly from this one and pretend I didn't see anything ;-) Karl Newman wrote: Yep, I'm still planning to do it, but I've just lost my job and surprisingly have even LESS time to work on OSM stuff. It's all going to be tied in with my new proposed rules file (so I can deal differently with the variety of ways--routable, polygons, and simple polylines), so if anyone wants to take a look at it and comment about the structure, etc., it's on my Wiki page here: http://wiki.openstreetmap.org/index.php/User:SiliconFiend/Ruleset I need to do some proper OOAD for my addition, too, and I want to play with the UML editor for Eclipse. But, along with my job I lost my great development laptop, so I'm trying to get back up and running on my older, slow laptop with limited hard drive space. Karl ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] User umehlig and some really nasty edits
Would a version number be a cleaner solution? I'm always uneasy about using dates for this purpose. 80n wrote: If the server were to provide the original timestamp as an additional attribute, and reject if it didn't match on upload, then problems like this could be prevented. It would also be a proper solution to update conflicts. 80n ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
Re: [OSM-talk] Osmosis UTF-8 problem (again)
Frederik Ramm wrote: Hi, Any idea what the user name should be? I find it hard to believe that user=jos??¯® (from the API) is correct. Well on 05 December I did have a problem with the planet diff, quoting from old E-Mail: latest daily planet diff has an UTF-8 problem on line 58267: node id=25254929 timestamp=2007-12-04T17:26:52Z user=josé ... Seems like the user names don't get encoded properly. Username looks conspicuously similar ;) I remember that email, I was hoping the problem would magically disappear ;-) Checking the history of that node from the API again gives user=jos逴巊 »H´ (hopefully this is coming through okay, it includes a bunch of Chinese-like characters). I'll check it out in more detail soon. It does look like it should be user=josé but given that the API is also returning interesting data it sounds like there's a deeper problem somewhere. Either way, osmosis shouldn't be emitting invalid UTF-8, but fixing it may not be easy. It might have something to do with characters that can't be represented with 16-bit characters. If it does turn out to be a problem elsewhere I can try to put a hack in place to at least emit valid UTF-8, but it will require me doing some more reading of unicode standards which I'm not excited about :-) ___ talk mailing list talk@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/talk
