Re: [OSM-dev] Python XML Parser on Windows
Hi! PyOsmium https://osmcode.org/pyosmium/ should work on Windows. Jochen On Tue, Aug 06, 2019 at 09:20:39PM -0700, Spencer Gardner wrote: > Date: Tue, 6 Aug 2019 21:20:39 -0700 > From: Spencer Gardner > To: dev@openstreetmap.org > Subject: [OSM-dev] Python XML Parser on Windows > > I'm sort of resurrecting an old thread I started last year. I find > Martijn's Overpass Python library to be very useful for small queries but I > frequently run into usage limits. I'd like to query an XML download > directly to save bandwidth and reduce consumption of Overpass resources, > but I can't find any suitable Python libraries that are WIndows-ready. Is > there something I'm missing? Has anyone else found a solution that works? > > By way of example, Geoff Boeing's excellent osmnx package appears to use a > home-grown OSM parser due to lack of a cross-platform option. > https://github.com/gboeing/osmnx/blob/eebbca0ed1d59a6dfe07c90493d54c0beab45145/osmnx/utils.py#L1181 > > Thanks for any help > ___ > dev mailing list > dev@openstreetmap.org > https://lists.openstreetmap.org/listinfo/dev -- Jochen Topf joc...@remote.org https://www.jochentopf.com/ +49-351-31778688 ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Python XML Parser on Windows
Hi Spencer, I'd definitely recommend lxml: https://lxml.de/. One of the benefits of lxml is that it is cross platform. You can find Windows binaries for lxml and many other Python libraries here: https://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml Regards, Frank On 07-08-2019 06:20, Spencer Gardner wrote: I'm sort of resurrecting an old thread I started last year. I find Martijn's Overpass Python library to be very useful for small queries but I frequently run into usage limits. I'd like to query an XML download directly to save bandwidth and reduce consumption of Overpass resources, but I can't find any suitable Python libraries that are WIndows-ready. Is there something I'm missing? Has anyone else found a solution that works? By way of example, Geoff Boeing's excellent osmnx package appears to use a home-grown OSM parser due to lack of a cross-platform option. https://github.com/gboeing/osmnx/blob/eebbca0ed1d59a6dfe07c90493d54c0beab45145/osmnx/utils.py#L1181 Thanks for any help ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Python XML Parser on Windows
I'm sort of resurrecting an old thread I started last year. I find Martijn's Overpass Python library to be very useful for small queries but I frequently run into usage limits. I'd like to query an XML download directly to save bandwidth and reduce consumption of Overpass resources, but I can't find any suitable Python libraries that are WIndows-ready. Is there something I'm missing? Has anyone else found a solution that works? By way of example, Geoff Boeing's excellent osmnx package appears to use a home-grown OSM parser due to lack of a cross-platform option. https://github.com/gboeing/osmnx/blob/eebbca0ed1d59a6dfe07c90493d54c0beab45145/osmnx/utils.py#L1181 Thanks for any help ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Export: (limit is 50000).
On 06/08/2019 16:32, marc marc wrote: josm allow to download data from overpass, which allows a larger limit. Overpass's greater capacity is one reason why I was querying OSM.org's limitations. Using OP to send data to JOSM for mass editing is one of the occasions I do use it to edit. However it's usually to filter the data first. DaveF ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Export: (limit is 50000).
Le 06.08.19 à 17:11, Dave F via dev a écrit : >> - all that export does is redirect >> to the same API that JOSM etc use. > Yes, I've noticed the limitations of JOSM josm allow to download data from overpass, which allows a larger limit. osm-fr had also an api proxy that redirect some request to overpass to reduce api load Unfortunately too few mapper was using it and it is currently offline ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Export: (limit is 50000).
On 06/08/2019 15:06, Tom Hughes wrote: On 06/08/2019 14:45, Dave F via dev wrote: Why are you mentioning 'rendering'? Because I thought we were talking about map exports? I am. "Main Page > Export" makes *no* mention of rendering. I assume you meant the raw data export? That is the standard API limit & I'm enquiring if that limit can be updated. - all that export does is redirect to the same API that JOSM etc use. Yes, I've noticed the limitations of JOSM. It's one of the reasons I don't use it as my general editor. That really isn't something that I would expect anybody much to use - if you want bulk data then sue the planet. The API is for editors. I don't think anyone would expect to "bulk" download, but surely a town or even a suburb should be acceptable? Again we should really never have added that to the share tab Again, the 'share tab' is something I never mentioned Why are you reluctant to read what's in front of you? as it just encourages people to think that is the correct way to reuse OSM data. It's up to them how the reuse the data. DaveF ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Export: (limit is 50000).
On 06/08/2019 14:45, Dave F via dev wrote: Why are you mentioning 'rendering'? Because I thought we were talking about map exports? I assume you meant the raw data export? That is the standard API limit - all that export does is redirect to the same API that JOSM etc use. That really isn't something that I would expect anybody much to use - if you want bulk data then sue the planet. The API is for editors. Again we should really never have added that to the share tab as it just encourages people to think that is the correct way to reuse OSM data. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Export: (limit is 50000).
Why are you mentioning 'rendering'? On 06/08/2019 14:36, Tom Hughes wrote: On 06/08/2019 14:24, Dave F via dev wrote: Is this based on server & connection speeds etc? If so has it been reviewed recently? With the increase in general computing power is there a reason this limit could not be increased? Rendering custom maps is very expensive and is not really a core part of our mission - we should probably never have added that feature really. Every time that call is used you are going direct to one of the render servers whose main job is to render tiles for the map and you're making it do a custom select from the database just for and then render that data. The node limit is basically about limiting the amount of data that has to be sucked out the database and manipulated to produce the result. While compute power may have increased somewhat so have the number of users hitting that endpoint... Tom ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Export: (limit is 50000).
On 06/08/2019 14:24, Dave F via dev wrote: Is this based on server & connection speeds etc? If so has it been reviewed recently? With the increase in general computing power is there a reason this limit could not be increased? Rendering custom maps is very expensive and is not really a core part of our mission - we should probably never have added that feature really. Every time that call is used you are going direct to one of the render servers whose main job is to render tiles for the map and you're making it do a custom select from the database just for and then render that data. The node limit is basically about limiting the amount of data that has to be sucked out the database and manipulated to produce the result. While compute power may have increased somewhat so have the number of users hitting that endpoint... Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Export: (limit is 50000).
Hi Main Page > Export*. I'm often getting the following message: "You requested too many nodes (limit is 5). Either request a smaller area, or use planet.osm" To get within this limit the downloadable area has been greatly reduced as more data is added to OSM, especially in urban areas. Is this based on server & connection speeds etc? If so has it been reviewed recently? With the increase in general computing power is there a reason this limit could not be increased? * I'm aware of alternatives for retrieving OSM data, but I'm inquiring specifically about this, on the main page where the vast majority have their initial interaction with OSM. Error messages just deter potential contributors. Cheers DaveF ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Auto adding attribution to image creation.
On 06/08/2019 13:47, Tom Hughes wrote: On 06/08/2019 13:29, Dave F via dev wrote: On 06/08/2019 12:26, Tom Hughes wrote: On 06/08/2019 12:04, Dave F via dev wrote: A user has asked why the image creation tool under 'Share' couldn't automatically add an attribution. It would certainly save me a lot of email sending asking users to add it. If you mean the option I'm surprised your confused. I thought I was perfectly clear. The "share" tool offers a number of options and I just wanted to be clear which one we were talking about. I mentioned 'image' & I'm posting to the Development forum not Newbie. that gives you an image based on the default rendering then the answer is that all that option does is to to a custom render using mapnik, and that render has no way to add an attribution. Would it be much coding effort? Yes. I mean you can certainly do it, but it means writing a whole bunch of code to work out what font size to use then render some text and overlay it on the mapnik generated image. There must be calculations performed to create the clipped image especially considering the Scale: option provided. Inserting an image of the attribution into the corner of the map image at a defined percentage of its size can't add that much additional overheads, There are plenty of 'image stamper' programs on the market which do similar. The script is here: https://github.com/openstreetmap/chef/blob/master/cookbooks/tile/templates/default/export.erb Patches are welcome. Make sure they work for all the output formats ;-) Yeah, that's not really the answer, is it? My OSM time is spent creating data for the OSM database. I then spend additional time updating & maintaining that data. It a shame the creators of the main website are reluctant to do the same for their contributions. It's looking a bit tired & 'Win95ish'. Cheers DaveF ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Auto adding attribution to image creation.
On 06/08/2019 13:29, Dave F via dev wrote: On 06/08/2019 12:26, Tom Hughes wrote: On 06/08/2019 12:04, Dave F via dev wrote: A user has asked why the image creation tool under 'Share' couldn't automatically add an attribution. It would certainly save me a lot of email sending asking users to add it. If you mean the option I'm surprised your confused. I thought I was perfectly clear. The "share" tool offers a number of options and I just wanted to be clear which one we were talking about. that gives you an image based on the default rendering then the answer is that all that option does is to to a custom render using mapnik, and that render has no way to add an attribution. Would it be much coding effort? Yes. I mean you can certainly do it, but it means writing a whole bunch of code to work out what font size to use then render some text and overlay it on the mapnik generated image. There must be calculations performed to create the clipped image especially considering the Scale: option provided. Inserting an image of the attribution into the corner of the map image at a defined percentage of its size can't add that much additional overheads, There are plenty of 'image stamper' programs on the market which do similar. The script is here: https://github.com/openstreetmap/chef/blob/master/cookbooks/tile/templates/default/export.erb Patches are welcome. Make sure they work for all the output formats ;-) Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Auto adding attribution to image creation.
On 06/08/2019 12:26, Tom Hughes wrote: On 06/08/2019 12:04, Dave F via dev wrote: A user has asked why the image creation tool under 'Share' couldn't automatically add an attribution. It would certainly save me a lot of email sending asking users to add it. If you mean the option I'm surprised your confused. I thought I was perfectly clear. that gives you an image based on the default rendering then the answer is that all that option does is to to a custom render using mapnik, and that render has no way to add an attribution. Would it be much coding effort? Yes. I mean you can certainly do it, but it means writing a whole bunch of code to work out what font size to use then render some text and overlay it on the mapnik generated image. There must be calculations performed to create the clipped image especially considering the Scale: option provided. Inserting an image of the attribution into the corner of the map image at a defined percentage of its size can't add that much additional overheads, There are plenty of 'image stamper' programs on the market which do similar. Then you need somebody to answer all the complaints when that text gets dropped over the top of something important. Sorry, but that's a ridiculous argument. How many complaints are received about the main page map attribution? DaveF ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] Auto adding attribution to image creation.
On 06/08/2019 12:04, Dave F via dev wrote: A user has asked why the image creation tool under 'Share' couldn't automatically add an attribution. It would certainly save me a lot of email sending asking users to add it. If you mean the option that gives you an image based on the default rendering then the answer is that all that option does is to to a custom render using mapnik, and that render has no way to add an attribution. Would it be much coding effort? Yes. I mean you can certainly do it, but it means writing a whole bunch of code to work out what font size to use then render some text and overlay it on the mapnik generated image. Then you need somebody to answer all the complaints when that text gets dropped over the top of something important. Tom -- Tom Hughes (t...@compton.nu) http://compton.nu/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
[OSM-dev] Auto adding attribution to image creation.
Hi A user has asked why the image creation tool under 'Share' couldn't automatically add an attribution. It would certainly save me a lot of email sending asking users to add it. Would it be much coding effort? Cheers DaveF ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
