On Tue, Dec 9, 2008 at 4:41 PM, Karl Newman <[EMAIL PROTECTED]> wrote:
> On Tue, Dec 9, 2008 at 4:16 PM, Brett Henderson <[EMAIL PROTECTED]> wrote: > >> Andrew Ayre wrote: >> >>> Karl Newman wrote: >>> >>>> On Tue, Dec 9, 2008 at 2:40 PM, Andrew Ayre <[EMAIL PROTECTED]<mailto: >>>> [EMAIL PROTECTED]>> wrote: >>>> >>>> Brett Henderson wrote: >>>> >>>> Karl Newman wrote: >>>> >>>> So, sounds like the most recent changes are not the issue. >>>> Do you know if SRTM2OSM creates entities with negative ids? >>>> If not, you may run into collisions with the other data file >>>> that you're trying to merge. >>>> >>>> Karl >>>> >>>> Andrew, it would be great if you could put your files online >>>> somewhere. I'll take a look this evening if Karl doesn't beat >>>> me to it. >>>> >>>> My biggest concern at this point is that you don't appear to be >>>> getting a decent error message. Perhaps I'm losing an exception >>>> stack trace somewhere. It makes it hard to diagnose the problem. >>>> >>>> Brett >>>> >>>> >>>> Karl - I'm not sure if SRTM2OSM produces entities with negative IDs. >>>> I'll try to check into that. But I think other people are doing the >>>> same thing I'm trying to do, making me wonder if I am doing >>>> something wrong here. >>>> >>>> Brett - I've uploaded the files to: >>>> >>>> http://files.britishideas.com/index.php?dir=osmosis/ >>>> >>>> Note that I am running this on a set of 26 pairs of tiles. I get >>>> this errors on six of the pairs. The others seem to be OK, although >>>> I haven't rendered them yet to check. >>>> >>>> Thanks for the help! Andy >>>> >>>> >>>> Looking at your source data (generated by srtm2osm), there are a couple >>>> potential issues. The first is that the ids are not negative, although they >>>> start at 1E9, so you probably won't run into a collision (I don't think the >>>> new entity ids are anywhere near that yet). The bigger problem is that it >>>> intermingles nodes and ways, and generally Osmosis works better if all the >>>> nodes come first, then all the ways, then all the relations. For a merge, >>>> they have to be sorted. That's easy enough to do, though--just add a --sort >>>> task to your command line (for both input files). i.e., osmosis --rx >>>> osmfile.osm --sort --rx srtmfile.osm --sort --merge --wx outfile.osm >>>> >>>> In reality, maybe the problem is entirely with your osm data and not at >>>> all with the SRTM. It's possible some of your osm data files have items out >>>> of order, which is causing a problem with the merge. >>>> >>> >>> Thanks for looking into this. I added --sort after each input file name >>> and it still gives the same error. The OSM map file was generated by >>> osmosis, which I used to split planet.osm. >>> >>> java -Xmx1560m -jar /home/nav/scripts/osmosis.jar --read-xml >>> file="/home/nav/temp/tile.63255095.osm" --sort --read-xml >>> file="/home/nav/temp/eletile.63255095.osm" --sort --merge --write-xml >>> file="/home/nav/temp/mergedtile.63255095.osm" >>> >>> I had a quick play and think I've found the problem. >> >> Firstly, when you receive an error in osmosis, look further down the stack >> trace. The real reason is almost always in there somewhere. Osmosis is >> multi-threaded so it is difficult to have the real reason show up at the >> start of the error message. In your case the first bit of error was from >> the input xml readers detecting that the merge task had aborted. The merge >> task output was further down the error messages. >> >> In this case there were two problems: >> 1. The data was unsorted. >> 2. Some data doesn't appear to have dates attached (I haven't found the >> offending entry yet). >> >> This command line works: >> osmosis --rx tile.63255095.osm.gz enableDateParsing=false --sort --rx >> eletile.63255095.osm.gz enableDateParsing=false --sort --merge --wx >> out.osm.gz >> >> It sorts all data before the merge, and ignores all dates in the input >> data (replacing them with current system time). This is a bit of a kludge, >> ideally the data itself should be fixed to include the correct dates but >> hopefully it points you in the right direction. >> >> Hope that helps. >> Brett >> >> > The SRTM data has no timestamps, which is causing the problem. So you only > need to set enableDateParsing=false for that file. It might be good to show > a warning if an entity has no timestamp attribute in the XML. I propose > adding the following lines in BaseElementProcessor.java: > > + private static final Logger log = > Logger.getLogger(BaseElementProcessor.class.getName()); > > ... > > protected TimestampContainer createTimestampContainer(String data) { > if (enableDateParsing) { > + if (data == null || data == "") { > + log.warning("Element missing timestamp attribute."); > + } > return new UnparsedTimestampContainer(timestampFormat, data); > } else { > return dummyTimestampContainer; > } > } > > That encapsulates it in the lowest possible level, but the downside is that > it doesn't report anything about the problematic element--Ideally it would > show the id and whether it's a node, way or relation. That could conceivably > be solved by passing that information to the createTimestampContainer > method, but that seems sloppy. > > Karl > Oops, that should be "".equals(data) or data.length() == 0 Karl
_______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
