Hi Aaron, Did you get any further with this? The error is unlikely to be caused by Osmosis. The following line in the stack trace appears to indicate a network problem: Caused by: java.net.SocketException: Connection reset
I've tried the same command on the planet server itself (downloading from the local Apache server) and it appears to have downloaded all files without an issue. I started from replication file 3216 which I believe is the one you started from. It's actually still running because it takes a while to sort the data after the download but it's completed the download portion of the execution okay. I'm also running the command locally at home. It's taking a long time to run due to the time taken to download a large number of hourly files. It hasn't reached the same point you encountered errors yet, but so far so good. I'll let you know if I see the same problem. There are a few things you can try. - I never set the maxInterval to 0. It is worth setting to a smaller time period so that if encounter problems you don't have to start from the start. You've only downloaded about 30 files which is just over a day when encountering the error though so this isn't a huge number. But the more files you download, the greater the chances of tripping over a network issue. I'd suggest setting it to one day (ie. 86400) and running it several times until you catch up. If you're importing into a database then there's no reason to try to get it all in one hit. Patching a planet file is more time consuming, but if you're doing that you probably should be using daily diffs anyway. - Have you tried using wget or curl to download the file? It would be worth verifying that you can download it outside of Osmosis. Hope that helps. Brett On Wed, Apr 7, 2010 at 3:43 AM, Aaron Schaefer <[email protected]> wrote: > So I'm trying to set up downloads of the hour-replicate diffs and > osmosis seems to be having issues reading certain files from the > server. I did a full planet download and dump into my database from > the 03-Apr-2010 03:38 planet-latest.osm.bz2. > > I created a working directory for the diffs and ran 'osmosis --rri > workingDirectory=.' to initialize the necessary files and then > downloaded the Fri Apr 02 23:02:09 UTC 2010 state.txt file (a few > hours earlier than my previous full import). I updated > configuration.txt to have the proper baseUrl and set the maxInterval = > 0. Now when I try to get the diffs, this is what happens: > > -------- > $ osmosis --rri workingDirectory=. --wxc diffs.osc.gz > Apr 6, 2010 12:21:07 PM org.openstreetmap.osmosis.core.Osmosis run > INFO: Osmosis Version 0.34 > Apr 6, 2010 12:21:08 PM org.openstreetmap.osmosis.core.Osmosis run > INFO: Preparing pipeline. > Apr 6, 2010 12:21:08 PM org.openstreetmap.osmosis.core.Osmosis run > INFO: Launching pipeline execution. > Apr 6, 2010 12:21:08 PM org.openstreetmap.osmosis.core.Osmosis run > INFO: Pipeline executing, waiting for completion. > Apr 6, 2010 12:25:26 PM > org.openstreetmap.osmosis.core.pipeline.common.ActiveTaskManager > waitForCompletion > SEVERE: Thread for task 1-rri failed > org.openstreetmap.osmosis.core.OsmosisRuntimeException: Unable to read > the changeset file 000/003/246.osc.gz from t > he server. > at > org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.downloadReplicationFile(BaseReplicat > ionDownloader.java:128) > at > org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.download(BaseReplicationDownloader.j > ava:236) > at > org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.runImpl(BaseReplicationDownloader.ja > va:275) > at > org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.run(BaseReplicationDownloader.java:3 > 45) > at java.lang.Thread.run(Unknown Source) > Caused by: java.net.SocketException: Connection reset > at java.net.SocketInputStream.read(Unknown Source) > at java.io.BufferedInputStream.read1(Unknown Source) > at java.io.BufferedInputStream.read(Unknown Source) > at sun.net.www.MeteredStream.read(Unknown Source) > at java.io.FilterInputStream.read(Unknown Source) > at > sun.net.www.protocol.http.HttpURLConnection$HttpInputStream.read(Unknown > Source) > at java.io.BufferedInputStream.read1(Unknown Source) > at java.io.BufferedInputStream.read(Unknown Source) > at java.io.FilterInputStream.read(Unknown Source) > at > org.openstreetmap.osmosis.core.merge.v0_6.BaseReplicationDownloader.downloadReplicationFile(BaseReplicationDownloader.java:114) > ... 4 more > Apr 6, 2010 12:25:26 PM org.openstreetmap.osmosis.core.Osmosis main > SEVERE: Execution aborted. > org.openstreetmap.osmosis.core.OsmosisRuntimeException: One or more > tasks failed. > at > org.openstreetmap.osmosis.core.pipeline.common.Pipeline.waitForCompletion(Pipeline.java:146) > at org.openstreetmap.osmosis.core.Osmosis.run(Osmosis.java:85) > at org.openstreetmap.osmosis.core.Osmosis.main(Osmosis.java:30) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) > at java.lang.reflect.Method.invoke(Unknown Source) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launchStandard(Launcher.java:329) > at > org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:239) > at > org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:409) > at > org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:352) > at org.codehaus.classworlds.Launcher.main(Launcher.java:31) > -------- > > It always fails on the 000/003/246.osc.gz file, and I've verified that > it does exist on the server. I tried using both > planet.openstreetmap.org and the heanet mirror with the same results. > I had actually tried this process previously with an older full planet > dump, and it was giving me an error on the 000/003/014.osc.gz file > (which was also present on the server)...I figured maybe it was just a > fluke and that particular file had been corrupted, but now I'm seeing > it error out on the more recent diff as well. > > Is anyone else seeing these errors when trying to generate a diff file? > > -- > Aaron Schaefer > > _______________________________________________ > dev mailing list > [email protected] > http://lists.openstreetmap.org/listinfo/dev >
_______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
