Re: [osmosis-dev] Reading file over http
On Mon, 20 Jun 2016 at 02:49 Frederik Ramm wrote: > Hi, > > On 06/19/2016 04:16 PM, Stephen Knox wrote: > > This might be a bit of a dumb question, but is it possible already (or > > would it be possible with minor modification) to read a remote osm xml > > file over http and stream it, so that the file does not have to be fully > > downloaded? > > wget -O- http://somewhere/file.osm.bz2 | bzcat | osmosis --read-xml - > --do-something-else > > Sadly the more efficient .osm.pbf files cannot be consumed in streaming > fashion due to Osmosis attempting to seek in the file. > Actually, I think it can. Try this: wget -O - http://somewhere/file.pbf | osmosis --read-pbf-fast /dev/stdin --do-something-else I possibly should make the --read-pbf-fast task the default pbf reading task. I don't think it has any downsides. ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Reading file over http
Hi, On 06/19/2016 09:09 PM, Stephen Knox wrote: > I should have said, I am using Osmosis as a Java library and don't have > access to the command line as I am using Platform as a service, so I guess > the same applies so long as I can find a decent Bzip converter to stream to > Osmosis? Osmosis includes bzip2 support so it could decode the .bz2 itself but since Java bzip2 is much slower than the standalone commandline program, the version with bzip2 in the pipe is usually recommended. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
[osmosis-dev] Reading file over http
On 06/19/2016 04:49 PM, Frederik Ramm wrote: > wget -O- http://somewhere/file.osm.bz2 | bzcat | osmosis --read-xml - > --do-something-else > Sadly the more efficient .osm.pbf files cannot be consumed in streaming > fashion due to Osmosis attempting to seek in the file. Thanks On 06/19/2016 04:16 PM, Stephen Knox wrote: >>* This might be a bit of a dumb question, but is it possible already (or *>>* would it be possible with minor modification) to read a remote osm xml *>>* file over http and stream it, so that the file does not have to be fully *>>* downloaded? * I should have said, I am using Osmosis as a Java library and don't have access to the command line as I am using Platform as a service, so I guess the same applies so long as I can find a decent Bzip converter to stream to Osmosis? Stephen ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
Re: [osmosis-dev] Reading file over http
Hi, On 06/19/2016 04:16 PM, Stephen Knox wrote: > This might be a bit of a dumb question, but is it possible already (or > would it be possible with minor modification) to read a remote osm xml > file over http and stream it, so that the file does not have to be fully > downloaded? wget -O- http://somewhere/file.osm.bz2 | bzcat | osmosis --read-xml - --do-something-else Sadly the more efficient .osm.pbf files cannot be consumed in streaming fashion due to Osmosis attempting to seek in the file. Bye Frederik -- Frederik Ramm ## eMail frede...@remote.org ## N49°00'09" E008°23'33" ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
[osmosis-dev] Reading file over http
This might be a bit of a dumb question, but is it possible already (or would it be possible with minor modification) to read a remote osm xml file over http and stream it, so that the file does not have to be fully downloaded? Or would other software be better suited to this task? Thanks Stephen ___ osmosis-dev mailing list osmosis-dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/osmosis-dev
