[OSM-dev] Python XML Parser on Windows

2019-08-06 Thread Spencer Gardner
I'm sort of resurrecting an old thread I started last year. I find
Martijn's Overpass Python library to be very useful for small queries but I
frequently run into usage limits. I'd like to query an XML download
directly to save bandwidth and reduce consumption of Overpass resources,
but I can't find any suitable Python libraries that are WIndows-ready. Is
there something I'm missing? Has anyone else found a solution that works?

By way of example, Geoff Boeing's excellent osmnx package appears to use a
home-grown OSM parser due to lack of a cross-platform option.
https://github.com/gboeing/osmnx/blob/eebbca0ed1d59a6dfe07c90493d54c0beab45145/osmnx/utils.py#L1181

Thanks for any help
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Export: (limit is 50000).

2019-08-06 Thread Dave F via dev



On 06/08/2019 16:32, marc marc wrote:


josm allow to download data from overpass,
which allows a larger limit.


Overpass's greater capacity is one reason why I was querying OSM.org's 
limitations.


Using OP to send data to JOSM for mass editing is one of the occasions I 
do use it to edit. However it's usually to filter the data first.


DaveF





___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Export: (limit is 50000).

2019-08-06 Thread marc marc
Le 06.08.19 à 17:11, Dave F via dev a écrit :
>> - all that export does is redirect
>> to the same API that JOSM etc use.
> Yes, I've noticed the limitations of JOSM

josm allow to download data from overpass,
which allows a larger limit.

osm-fr had also an api proxy that redirect
some request to overpass to reduce api load
Unfortunately too few mapper was using it and it is currently offline
___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Export: (limit is 50000).

2019-08-06 Thread Dave F via dev



On 06/08/2019 15:06, Tom Hughes wrote:

On 06/08/2019 14:45, Dave F via dev wrote:


Why are you mentioning 'rendering'?


Because I thought we were talking about map exports?


I am.
"Main Page > Export" makes *no* mention of rendering.



I assume you meant the raw data export? That is the
standard API limit 


& I'm enquiring if that limit can be updated.


- all that export does is redirect
to the same API that JOSM etc use.


Yes, I've noticed the limitations of JOSM. It's one of the reasons I 
don't use it as my general editor.




That really isn't something that I would expect
anybody much to use - if you want bulk data then
sue the planet. The API is for editors.


I don't think anyone would expect to "bulk" download, but surely a town 
or even a suburb should be acceptable?




Again we should really never have added that to
the share tab 


Again,  the 'share tab' is something I never mentioned Why are you 
reluctant to read what's in front of you?



as it just encourages people to think
that is the correct way to reuse OSM data.


It's up to them how the reuse the data.

DaveF


___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Export: (limit is 50000).

2019-08-06 Thread Tom Hughes

On 06/08/2019 14:45, Dave F via dev wrote:


Why are you mentioning 'rendering'?


Because I thought we were talking about map exports?

I assume you meant the raw data export? That is the
standard API limit - all that export does is redirect
to the same API that JOSM etc use.

That really isn't something that I would expect
anybody much to use - if you want bulk data then
sue the planet. The API is for editors.

Again we should really never have added that to
the share tab as it just encourages people to think
that is the correct way to reuse OSM data.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Export: (limit is 50000).

2019-08-06 Thread Tom Hughes

On 06/08/2019 14:24, Dave F via dev wrote:

Is this based on server & connection speeds etc? If so has it been 
reviewed recently? With the increase in general computing power is there 
a reason this limit could not be increased?


Rendering custom maps is very expensive and is not really
a core part of our mission - we should probably never have
added that feature really.

Every time that call is used you are going direct to one
of the render servers whose main job is to render tiles for
the map and you're making it do a custom select from the
database just for and then render that data.

The node limit is basically about limiting the amount of
data that has to be sucked out the database and manipulated
to produce the result.

While compute power may have increased somewhat so have the
number of users hitting that endpoint...

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


[OSM-dev] Export: (limit is 50000).

2019-08-06 Thread Dave F via dev

Hi

Main Page > Export*. I'm often getting the following message:

"You requested too many nodes (limit is 5). Either request a smaller 
area, or use planet.osm"


To get within this limit the downloadable area has been greatly reduced 
as more data is added to OSM, especially in urban areas.


Is this based on server & connection speeds etc? If so has it been 
reviewed recently? With the increase in general computing power is there 
a reason this limit could not be increased?


* I'm aware of alternatives for retrieving OSM data, but I'm inquiring 
specifically about this, on the main page where the vast majority have 
their initial interaction with OSM. Error messages just deter potential 
contributors.


Cheers
DaveF

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Auto adding attribution to image creation.

2019-08-06 Thread Dave F via dev



On 06/08/2019 13:47, Tom Hughes wrote:

On 06/08/2019 13:29, Dave F via dev wrote:



On 06/08/2019 12:26, Tom Hughes wrote:

On 06/08/2019 12:04, Dave F via dev wrote:

A user has asked why the image creation tool under 'Share' couldn't 
automatically add an attribution. It would certainly save me a lot 
of email sending asking users to add it.


If you mean the option 


I'm surprised your confused. I thought I was perfectly clear.


The "share" tool offers a number of options and I just wanted
to be clear which one we were talking about.


I mentioned 'image' & I'm posting to the Development forum not Newbie.




that gives you an image based on the
default rendering then the answer is that all that option does
is to to a custom render using mapnik, and that render has
no way to add an attribution.


Would it be much coding effort?


Yes.

I mean you can certainly do it, but it means writing a whole
bunch of code to work out what font size to use then render
some text and overlay it on the mapnik generated image.


There must be calculations performed to create the clipped image
  especially considering the Scale: option provided.
Inserting an image of the attribution into the corner of the map 
image at a defined percentage of its size can't add that much 
additional overheads,


There are plenty of 'image stamper' programs on the market which do 
similar.


The script is here:

https://github.com/openstreetmap/chef/blob/master/cookbooks/tile/templates/default/export.erb 



Patches are welcome.

Make sure they work for all the output formats ;-)


Yeah, that's not really the answer, is it?

My OSM time is spent creating data for the OSM database. I then spend 
additional time updating & maintaining that data.


It a shame the creators of the main website are reluctant to do the same 
for their contributions. It's looking a bit tired & 'Win95ish'.


Cheers
DaveF

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Auto adding attribution to image creation.

2019-08-06 Thread Tom Hughes

On 06/08/2019 13:29, Dave F via dev wrote:



On 06/08/2019 12:26, Tom Hughes wrote:

On 06/08/2019 12:04, Dave F via dev wrote:

A user has asked why the image creation tool under 'Share' couldn't 
automatically add an attribution. It would certainly save me a lot of 
email sending asking users to add it.


If you mean the option 


I'm surprised your confused. I thought I was perfectly clear.


The "share" tool offers a number of options and I just wanted
to be clear which one we were talking about.


that gives you an image based on the
default rendering then the answer is that all that option does
is to to a custom render using mapnik, and that render has
no way to add an attribution.


Would it be much coding effort?


Yes.

I mean you can certainly do it, but it means writing a whole
bunch of code to work out what font size to use then render
some text and overlay it on the mapnik generated image.


There must be calculations performed to create the clipped image
  especially considering the Scale: option provided.
Inserting an image of the attribution into the corner of the map image 
at a defined percentage of its size can't add that much additional 
overheads,


There are plenty of 'image stamper' programs on the market which do 
similar.


The script is here:

https://github.com/openstreetmap/chef/blob/master/cookbooks/tile/templates/default/export.erb

Patches are welcome.

Make sure they work for all the output formats ;-)

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


Re: [OSM-dev] Auto adding attribution to image creation.

2019-08-06 Thread Tom Hughes

On 06/08/2019 12:04, Dave F via dev wrote:

A user has asked why the image creation tool under 'Share' couldn't 
automatically add an attribution. It would certainly save me a lot of 
email sending asking users to add it.


If you mean the option that gives you an image based on the
default rendering then the answer is that all that option does
is to to a custom render using mapnik, and that render has
no way to add an attribution.


Would it be much coding effort?


Yes.

I mean you can certainly do it, but it means writing a whole
bunch of code to work out what font size to use then render
some text and overlay it on the mapnik generated image.

Then you need somebody to answer all the complaints when that
text gets dropped over the top of something important.

Tom

--
Tom Hughes (t...@compton.nu)
http://compton.nu/

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev


[OSM-dev] Auto adding attribution to image creation.

2019-08-06 Thread Dave F via dev

Hi

A user has asked why the image creation tool under 'Share' couldn't 
automatically add an attribution. It would certainly save me a lot of 
email sending asking users to add it.


Would it be much coding effort?

Cheers
DaveF

___
dev mailing list
dev@openstreetmap.org
https://lists.openstreetmap.org/listinfo/dev