Re: [OSM-talk] Is tile rendering having a crisis?

[Rory McCann]

On 10/08/16 09:33, Komяpa wrote:
> There was
> also a site using OSM tiles for a popular Pokemon Go map causing lots
> of additional traffic, once this was identified the site was blocked.
> 
>  
> What was the site, and was it using OSM properly? (attribution, no
> evident js bugs, etc)
> 
> What was the load? 

It was FastPokeMap.com and it was responsible for 13% of the tile server
usage.

https://github.com/openstreetmap/chef/pull/78




signature.asc
Description: OpenPGP digital signature
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Is tile rendering having a crisis?

[Dave F]

I hope there was more to the situation than just "identified & blocked". 
OSM should be encouraging the use of it's database. Hopefully it was 
explained to the site's developers how to implement it within OSM's 
guidelines.


Dave F.


On 09/08/2016 23:34, Jon Burgess wrote:

On Tue, 2016-08-09 at 14:41 -0700, Ben Discoe wrote:

The tile renderer, "renderd", has been heavily overloaded for quite a
while, like 90% of the time it is dropping, and the dirty queue is
entirely ignored.  However, in the past day, something has happened
so
that it is even more overloaded, even the standard request queue is
not getting handled, and the dropped is out of control:

http://munin.openstreetmap.org/openstreetmap/orm.openstreetmap/render
d_processed.html

Does anyone know what's going on?  This is a large frustration for
me,
and I'd happily donate money to beefing up our render server(s).

There was an update to the rendering style a couple of days ago and
this triggered all of the existing tiles on Orm to be marked as needing
a fresh render the next time someone requests them. This means there
are far more requests hitting the render daemon than normal. It
typically takes about a week for the request queues to return to normal
when this happens. If you look at the "by-year" chart you can see
spikes in the dropped counts where this has happened about a dozen
times in the past year.

The OSM operations team are aware of the increased load on the tile
servers. There was a recent jump in usage when one of the other tile
providers stopped unlimited free access to their service. There was
also a site using OSM tiles for a popular Pokemon Go map causing lots
of additional traffic, once this was identified the site was blocked.

Jon

(member of the OSM operations team)

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk



___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Is tile rendering having a crisis?

[Komяpa]

>
> There was
> also a site using OSM tiles for a popular Pokemon Go map causing lots
> of additional traffic, once this was identified the site was blocked.
>

What was the site, and was it using OSM properly? (attribution, no evident
js bugs, etc)

What was the load?

How exactly Tile CDN should be scaled for this case to be handled not by
banning the site, but embracing their OSM usage?

I'm sure "to get this Pokemon Go site running bring us servers" could be
the way to solve it, making experience better not only for Pokemon Go
players but also for OSM mappers and users.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Is tile rendering having a crisis?

[Paul Johnson]

On Tue, Aug 9, 2016 at 5:34 PM, Jon Burgess  wrote:

> There was
> also a site using OSM tiles for a popular Pokemon Go map causing lots
> of additional traffic, once this was identified the site was blocked.
>

If it was possible to reach that developer, I hope it was made clear that
we're happy they're using OSM, just that they should maybe consider
rendering their own tiles or hit up a tile provider that specializes in
this for them.
___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

Re: [OSM-talk] Is tile rendering having a crisis?

[Jon Burgess]

On Tue, 2016-08-09 at 14:41 -0700, Ben Discoe wrote:
> The tile renderer, "renderd", has been heavily overloaded for quite a
> while, like 90% of the time it is dropping, and the dirty queue is
> entirely ignored.  However, in the past day, something has happened
> so
> that it is even more overloaded, even the standard request queue is
> not getting handled, and the dropped is out of control:
> 
> http://munin.openstreetmap.org/openstreetmap/orm.openstreetmap/render
> d_processed.html
> 
> Does anyone know what's going on?  This is a large frustration for
> me,
> and I'd happily donate money to beefing up our render server(s).

There was an update to the rendering style a couple of days ago and
this triggered all of the existing tiles on Orm to be marked as needing
a fresh render the next time someone requests them. This means there
are far more requests hitting the render daemon than normal. It
typically takes about a week for the request queues to return to normal
when this happens. If you look at the "by-year" chart you can see
spikes in the dropped counts where this has happened about a dozen
times in the past year.

The OSM operations team are aware of the increased load on the tile
servers. There was a recent jump in usage when one of the other tile
providers stopped unlimited free access to their service. There was
also a site using OSM tiles for a popular Pokemon Go map causing lots
of additional traffic, once this was identified the site was blocked.

Jon 

(member of the OSM operations team)

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk

[OSM-talk] Is tile rendering having a crisis?

[Ben Discoe]

The tile renderer, "renderd", has been heavily overloaded for quite a
while, like 90% of the time it is dropping, and the dirty queue is
entirely ignored.  However, in the past day, something has happened so
that it is even more overloaded, even the standard request queue is
not getting handled, and the dropped is out of control:

http://munin.openstreetmap.org/openstreetmap/orm.openstreetmap/renderd_processed.html

Does anyone know what's going on?  This is a large frustration for me,
and I'd happily donate money to beefing up our render server(s).

Thanks,
Ben

___
talk mailing list
talk@openstreetmap.org
https://lists.openstreetmap.org/listinfo/talk