Re: [QGIS-Developer] QGIS3: no referrer on HTTP(S)-requests

2018-04-14 Thread Tobias Wendorff
Hi Richard,

Am Sa, 14.04.2018, 10:29 schrieb Richard Duivenvoorde:
> Are you sure they need a 'referrer'?

According to OSM's munin, they're blocking IPs without a referrer
more often:

https://munin.openstreetmap.org/openstreetmap/tile.openstreetmap/index.html

> Some days ago I asked a question about the 'User-Agent'-header in
> (OSM)-requests [1], with a link to the 'Tile Usage Policy page [2],
> which states:
> "If known, a valid HTTP Referer."

Argh, I remember your post now :)

> And are people in a corporate network not always sending the proxy as
> referrer? Or their actual domain?

I know maaany corporations using IP masquerading (Universities also
do here). But I've never seen them adding anything to the referrer.

Best regards,
Tobias

___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer

Re: [QGIS-Developer] QGIS3: no referrer on HTTP(S)-requests

2018-04-14 Thread Richard Duivenvoorde
Hi Tobias,

Are you sure they need a 'referrer'?
Some days ago I asked a question about the 'User-Agent'-header in
(OSM)-requests [1], with a link to the 'Tile Usage Policy page [2],
which states:
"If known, a valid HTTP Referer."

In our case, I'm wondering what the referrer should be...

And are people in a corporate network not always sending the proxy as
referrer? Or their actual domain?

Mmm, see [3]: you have to add a 'referer' header (no type) :-)

Funny stuff those standards

Regards,

Richard

[0] https://en.wikipedia.org/wiki/HTTP_referer
[1] https://lists.osgeo.org/pipermail/qgis-developer/2018-April/052902.html
[2] https://operations.osmfoundation.org/policies/tiles/
[3] https://en.wikipedia.org/wiki/List_of_HTTP_header_fields

On 14-04-18 00:23, Tobias Wendorff wrote:
> Hi there,
> 
> after having problems with loading XYZ tiles from OSM (many timeouts),
> I've read that IPs without referrer get blocked faster and longer than
> IPs with referrer due to OSM's tile policy.
> 
> QGIS isn't sending out a referrer at all. So for users with a fixed
> IP (like a corporate network), will get blocked way faster and longer.
> On the other hand, community's opinion about this is that affected
> corporations should get a paid or own tile server to not slow down
> the existing ones.
> 
> I've checked the requests of QGIS and there's no referrer in header.
> Would it be possible to add an option to set an referrer in general?
> Not to override OSM's limits, but to give the user more control if
> the target service needs a referrer. libcurl and GDAL support this
> of course.
> 
> Right now, the only way is to setup a local proxy and add an referrer.
> But when accessing an HTTPS service, that is not what you want at all ;)
> 
> Best regards,
> Tobias
> 
> ___
> QGIS-Developer mailing list
> QGIS-Developer@lists.osgeo.org
> List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer
> 

___
QGIS-Developer mailing list
QGIS-Developer@lists.osgeo.org
List info: https://lists.osgeo.org/mailman/listinfo/qgis-developer
Unsubscribe: https://lists.osgeo.org/mailman/listinfo/qgis-developer