dcausse created this task. dcausse added projects: Wikidata-Query-Service, serviceops, SRE-swift-storage. Restricted Application added a subscriber: Aklapper.
TASK DESCRIPTION Swift with tmpauth does seem to do some hostname negociation that makes the use of envoy defaults problematic. Contacting swift requires the request header `Host` to be set properly: this can be achieved using `host_rewrite_header` (done in https://gerrit.wikimedia.org/r/c/operations/puppet/+/704960/). But this is not enough as swift will return X-Storage-Url <https://docs.openstack.org/swift/latest/api/object_api_v1_overview.html> that the client must follow to continue: curl -v -H"X-Storage-User: wdqs:flink" -H"X-Storage-Pass: some_secret" https://thanos-swift.discovery.wmnet/auth/v1.0 > GET /auth/v1.0 HTTP/1.1 > Host: thanos-swift.discovery.wmnet > User-Agent: curl/7.64.0 > Accept: */* > X-Storage-User: wdqs:flink > X-Storage-Pass: some_secret < HTTP/1.1 200 OK < x-storage-url: https://thanos-swift.discovery.wmnet/v1/AUTH_wdqs < x-auth-token-expires: 332392 < x-auth-token: AUTH_some_token < content-type: text/html; charset=UTF-8 < x-storage-token: AUTH_some_token < x-trans-id: some_id < x-openstack-request-id: some_id < content-length: 0 < date: Mon, 19 Jul 2021 16:56:14 GMT < x-envoy-upstream-service-time: 9 < server: envoy The solution we used in the past is (for applications owning the http client) defining a set of http routes: - `www.wikidata.org` -> http://localhost:6500 (//mwapi-async//) - `meta.wikimedia.org` -> http://localhost:6500 (//mwapi-async//) - `schema.wikimedia.org` -> https://localhost:6023 (//schema//) The application is configured with e.g. `https://schema.wikimedia.org` and the http client is told to do the routing (host headers kept intact) Main drawback is that it requires patching the swift client. Other strategies might be: - create a dedicated envoy profile that understands this negotiation and do more header rewrites - request headers `localhost` -> `thanos-swift.discovery.wmnet` - response headers `https://thanos-swift.discovery.wmnet/(.*)` -> `http://localhost:6022/$1` Or perhaps making envoy act as a forward proxy (assuming the swift client does support setting-up a forward proxy which is the case for the one needed flink)? TASK DETAIL https://phabricator.wikimedia.org/T286935 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: dcausse Cc: Joe, fgiunchedi, jijiki, dcausse, Aklapper, Lalamarie69, MPhamWMF, LSobanski, wkandek, JMeybohm, CBogen, Chambersjay, Namenlos314, Conradrock, Gq86, Lucas_Werkmeister_WMDE, EBjune, merbst, elukey, Jonas, Xmlizer, jkroll, Wikidata-bugs, Jdouglas, aude, Tobias1984, Manybubbles, Dzahn
_______________________________________________ Wikidata-bugs mailing list -- [email protected] To unsubscribe send an email to [email protected]
