On Tue, Dec 9, 2008 at 4:16 PM, Brett Henderson <[EMAIL PROTECTED]> wrote:
> Andrew Ayre wrote: > >> Karl Newman wrote: >> >>> On Tue, Dec 9, 2008 at 2:40 PM, Andrew Ayre <[EMAIL PROTECTED]<mailto: >>> [EMAIL PROTECTED]>> wrote: >>> >>> Brett Henderson wrote: >>> >>> Karl Newman wrote: >>> >>> So, sounds like the most recent changes are not the issue. >>> Do you know if SRTM2OSM creates entities with negative ids? >>> If not, you may run into collisions with the other data file >>> that you're trying to merge. >>> >>> Karl >>> >>> Andrew, it would be great if you could put your files online >>> somewhere. I'll take a look this evening if Karl doesn't beat >>> me to it. >>> >>> My biggest concern at this point is that you don't appear to be >>> getting a decent error message. Perhaps I'm losing an exception >>> stack trace somewhere. It makes it hard to diagnose the problem. >>> >>> Brett >>> >>> >>> Karl - I'm not sure if SRTM2OSM produces entities with negative IDs. >>> I'll try to check into that. But I think other people are doing the >>> same thing I'm trying to do, making me wonder if I am doing >>> something wrong here. >>> >>> Brett - I've uploaded the files to: >>> >>> http://files.britishideas.com/index.php?dir=osmosis/ >>> >>> Note that I am running this on a set of 26 pairs of tiles. I get >>> this errors on six of the pairs. The others seem to be OK, although >>> I haven't rendered them yet to check. >>> >>> Thanks for the help! Andy >>> >>> >>> Looking at your source data (generated by srtm2osm), there are a couple >>> potential issues. The first is that the ids are not negative, although they >>> start at 1E9, so you probably won't run into a collision (I don't think the >>> new entity ids are anywhere near that yet). The bigger problem is that it >>> intermingles nodes and ways, and generally Osmosis works better if all the >>> nodes come first, then all the ways, then all the relations. For a merge, >>> they have to be sorted. That's easy enough to do, though--just add a --sort >>> task to your command line (for both input files). i.e., osmosis --rx >>> osmfile.osm --sort --rx srtmfile.osm --sort --merge --wx outfile.osm >>> >>> In reality, maybe the problem is entirely with your osm data and not at >>> all with the SRTM. It's possible some of your osm data files have items out >>> of order, which is causing a problem with the merge. >>> >> >> Thanks for looking into this. I added --sort after each input file name >> and it still gives the same error. The OSM map file was generated by >> osmosis, which I used to split planet.osm. >> >> java -Xmx1560m -jar /home/nav/scripts/osmosis.jar --read-xml >> file="/home/nav/temp/tile.63255095.osm" --sort --read-xml >> file="/home/nav/temp/eletile.63255095.osm" --sort --merge --write-xml >> file="/home/nav/temp/mergedtile.63255095.osm" >> >> I had a quick play and think I've found the problem. > > Firstly, when you receive an error in osmosis, look further down the stack > trace. The real reason is almost always in there somewhere. Osmosis is > multi-threaded so it is difficult to have the real reason show up at the > start of the error message. In your case the first bit of error was from > the input xml readers detecting that the merge task had aborted. The merge > task output was further down the error messages. > > In this case there were two problems: > 1. The data was unsorted. > 2. Some data doesn't appear to have dates attached (I haven't found the > offending entry yet). > > This command line works: > osmosis --rx tile.63255095.osm.gz enableDateParsing=false --sort --rx > eletile.63255095.osm.gz enableDateParsing=false --sort --merge --wx > out.osm.gz > > It sorts all data before the merge, and ignores all dates in the input data > (replacing them with current system time). This is a bit of a kludge, > ideally the data itself should be fixed to include the correct dates but > hopefully it points you in the right direction. > > Hope that helps. > Brett > > The SRTM data has no timestamps, which is causing the problem. So you only need to set enableDateParsing=false for that file. It might be good to show a warning if an entity has no timestamp attribute in the XML. I propose adding the following lines in BaseElementProcessor.java: + private static final Logger log = Logger.getLogger(BaseElementProcessor.class.getName()); ... protected TimestampContainer createTimestampContainer(String data) { if (enableDateParsing) { + if (data == null || data == "") { + log.warning("Element missing timestamp attribute."); + } return new UnparsedTimestampContainer(timestampFormat, data); } else { return dummyTimestampContainer; } } That encapsulates it in the lowest possible level, but the downside is that it doesn't report anything about the problematic element--Ideally it would show the id and whether it's a node, way or relation. That could conceivably be solved by passing that information to the createTimestampContainer method, but that seems sloppy. Karl
_______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
