Re: [OSM-dev] GDPR implementation on planet.osm.org
Hi Roland,
Am 2018-06-20 um 20:16 schrieb Roland Olbricht:
> On the technical side, things are even worse. The elephant in the room
> is OAuth. OAuth is built on in particular the assumptions that
> - the consumer ("the website") acts stateful
> - sessions are relatively long-lived, i.e. some seconds to some hours
> - the identity provider has the cross-origin assets
> All three are not true for Overpass API which means that I have to work
> around OAuth or significantly mess with it.
>
> For example, implementing to have sessions on Overpass API will require
> to develop a full-fledged security system to deal with the hundres of
> potential modes of attacks on session based systms. Even if that works,
> the median runtime for a request on Overpass API is well below a second,
> and just the roundtrip times for the OAuth threesome communication sum
> up to more. We have not even started to talk about the plethora of error
> messages that need to be formulated, explained, and implemented.
You do not need the full roundtrip for each request. I have implemented
the authentication of the protected part of Geofabrik's download service
(https://osm-internal.download.geofabrik.de/). Its source code can be
found at https://github.com/geofabrik/sendfile_osm_oauth_protector
1. If a user requests a protected resource https://HOST/PATH for the
first time, he will receive the landing page containing a link to
https://HOST/PATH?show_landing_page.
2. If he follows this link, the web application will check if he
attached a cookie to his request. If no cookie was attached, the
application will retrieve a temporary request token from
https://www.openstreetmap.org/oauth/request_token and reply with a
redirect (302 Found) to
https://www.openstreetmap.org/authorize?oauth_token=X_token_secret=Y_callback=https://HOST/PATH?oauth_token_secret_encry=Y_ENCRYTPED
3. The browser will call the URL in the Location header of the response
of item 2. If the user is already logged in into OSM, he will be asked
to grant a permission to the application "Geofabrik Downloads".
Otherwise, he has to log in first.
4. If the user grants the permission and clicks on "Grant permissions",
his browser fill send a HTTP POST request to
https://www.openstreetmap.org/oauth/authorize. The OSM website will
respond with code 302 and pointing him to
https://HOST/PATH?oauth_token_secret_encr=Y_ENCRYPTED_token=X
5. The user calls (HTTP GET)
https://HOST/PATH?oauth_token_secret_encr=Y_ENCRYPTED_token=X. The
web application of the download server recognizes the URL parameters
oauth_token and oauth_token_secret. The web application retrieves a
permanent OAuth access token from the OSM API by calling
https://www.openstreetmap.org/access_token. If that works, it is able to
call https://api.openstreetmap.org/api/0.6/user/details (with the
permanent access token in the HTTP Authentication header). If this
request does not fail, the access token is valid and the web application
has ensured that the client has a valid OSM account. The web application
sets a cookie as described in
https://github.com/geofabrik/sendfile_osm_oauth_protector/blob/master/doc/cookie.md
and responds with the requested resource.
The cookies contains the login status (unencrypted, unsigned), the name
of the key set which was used by the server to encrypt and sign the
cookie and a encrypted and signed part consisting of the access token,
the access token and the expiry date (48 hours).
6. The client sends this cookie with all future requests to the server.
The server decrypts the cookie and checks the signature. If it is ok and
the expiry date has not passed yet, the request is answered immediately
without further OAuth round trips. If the expiry date has passed, the
server responds with a redirect (code 302) as described in item 2.
Our solution does not need any session management on our side. The
session IDs are stored in the cookies. They are encrypted and signed to
prevent clients to manipulate them (or the expiry date).
It would be possible to avoid the round trip every 48 hours if the
server calls https://api.openstreetmap.org/api/0.6/user/details again
using the permanent access token (it's in the cookie). This means, you
could revalidate the validity of the OSM account every 48 hours.
However, this feature bears a security risk. It is implemented in my
tool but we at Geofabrik decided not to use it. If a user accidentally
publishes his cookie on GitHub (e.g. forgotten to remove it from the
invokation of curl), someone else could use it forever (until the access
token is revoked by the user which usually does not happen). Instead, we
require the user to re-enter his OSM account credentials every 48 hours
and require such malicious users to publish their OSM account credentials.
> On top of that, the OAuth idea means that each and every sequence of
> user data access will trigger an event on the central OSM OAuth server.
> This is quite Orwellian. Even if you do not store that
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
On Saturday 23 June 2018, Yves wrote: > > Which is not going to change if everybody scripts its own way :) > I did not followed the discussion around that change, but it would be > a good idea to make changes upstream to come up with an equivalent > style without special code. Yves The question is how long would you be committed to wait for an upstream change to be implemented? Fixes for major bugs like here: https://github.com/mapnik/mapnik/issues/3558 can take a year. Not sure how requests for feature additions fare on average. But i think without an existing implementation being submitted as a pull request the chances are relatively small. It is commendable that Mapbox still does some basic maintainance and bugfixing there but this is hardly more than keeping the lights on. And any feature addition would also need support in Carto which - as Daniel already mentioned - is definitely and officially not maintained any more. It is all open source of course so anyone can pick up development but you need to ask yourself if this would be worthwhile of course. Both the technological and cartographic environment have changed a lot since these programs were originally designed. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
W dniu 23.06.2018 o 11:18, Yves pisze: > | Mapnik and CartoCSS being essentially > | unmaintained, > > Which is not going to change if everybody scripts its own way :) > I did not followed the discussion around that change, but it would be > a good idea to make changes upstream to come up with an equivalent > style without special code. CartoCSS is dead end now - first it has been abandoned by Mapbox, and lately by nebulon42, who made 1.0.0 release shortly before his departure. Mapnik is a different beast. It's still being developed by artemp (founder) and talaj at least and has some big deployments, like Mapy.cz. I don't remember right now if we asked about adding support for our solution, but solving things upstream is what I try to promote. However it always takes some coding to be done and it might be not what maintainers are focusing on. So if there's somebody interested in the issue, please let us know. -- "My method is uncertain/ It's a mess but it's working" [F. Apple] ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
W dniu 23.06.2018 o 09:21, Sven Geggus pisze: > Are you unaware of the fact that there are forks of OpenStreetMap Carto like > German style which are keept in sync by merging them with every upstream > release? Hi, Sven! I was aware that your fork is quite closely following osm-carto, unlike many others, so I did not expect such problem. Moreover this particular approach to surface rendering has been developed for about a year, so I'm surprised you missed the whole discussion. -- "My method is uncertain/ It's a mess but it's working" [F. Apple] ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
| Mapnik and CartoCSS being essentially | unmaintained, Which is not going to change if everybody scripts its own way :) I did not followed the discussion around that change, but it would be a good idea to make changes upstream to come up with an equivalent style without special code. Yves ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
On Saturday 23 June 2018, Sven Geggus wrote: > > I'm doing this for German style for more than 2 years now without > that much of a hassle up till this release. > > This will now kill my ability to keep on doing this at least as > roads.mss is concerned! This is to be expected. You are kind of spoiled because during the last year this is about the only larger technologically more sophisticated change in the style. Discussion of this change can be found in: https://github.com/gravitystorm/openstreetmap-carto/pull/2640 and discussion leading up to this in: https://github.com/gravitystorm/openstreetmap-carto/issues/110 The reason for the technological complexity of this and the resulting mangling of the code is the lack of support to render lines with a 2d pattern in Mapnik. I still hope Lucas will write up a few details about the technical approach that would enable you to more easily re-implement this in a different style. Note there have been voices to choose a more conservative approach to rendering unpaved roads in OSM-Carto - like yet another form of dashing in addition to the various dashing types already used in road rendering. There are very good cartographic reasons not to do that though. All in all this is a good example for OSM-Carto being at a crossroads (and having been for quite some time) between staying avant-garde and pushing the boundaries of cartographic design and technology or being satisfied with shuffling the options offered by the cartographic mainstream within the technological framework used - and which, due to Mapnik and CartoCSS being essentially unmaintained, becomes more narrow and limiting every year. -- Christoph Hormann http://www.imagico.de/ ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
Re: [OSM-dev] OpenStreetMap Carto release v4.12.0
Daniel Koć wrote: > Changes > - Added rendering “surface” tag on roads with a pattern This is not supposed to be a huge change, right? Are you unaware of the fact that there are forks of OpenStreetMap Carto like German style which are keept in sync by merging them with every upstream release? I'm doing this for German style for more than 2 years now without that much of a hassle up till this release. This will now kill my ability to keep on doing this at least as roads.mss is concerned! The commit which causes the hassle is most likely this one: Author: Lukas Sommer Date: Fri May 11 10:57:04 2018 + Render “surface” tag on roads with a pattern (#2640) * code re-ordering - no rendering change * Render unpaved road surface with a pattern WTF. Did you have a good reason which was worth killing your forks ability to stay in sync with upstream? Regards Sven -- "Whenever there is a conflict between human rights and property rights, human rights must prevail." (Abraham Lincoln) /me is giggls@ircnet, http://sven.gegg.us/ on the Web ___ dev mailing list dev@openstreetmap.org https://lists.openstreetmap.org/listinfo/dev
