Hi Ryan,
Apologies for the late reply. The problem here is that no v2.0
identity endpoint is returned in the service catalog. As you say,
jclouds does the version discovery and it can only work with the
endpoints returned in the service catalog. An initial request is
performed to authenticate the user to the endpoint configured in the
ContextBuilder, and all subsequent operations are done against the
corresponding endpoint discovered from the service catalog. That's why
the second request to keystone is done to the v3 endpoint.
I'm not an OpenStack expert, and i don't know how OpenStack should be
configured to make the v2 endpoint part of the service catalog
response, but that needs to happen in order to let jclouds discover
the version it needs.
Apart from that (and for completeness), if several endpoints are
returned for the same service, jclouds will pick the first one,
unless:
* A "versionId" field comes in the endpoint object in the json
response (according to this blueprint [1]; I don't know the state of
the art of Openstack and if that field is still returned or not).
* AND you use the "ContextBuilder.apiVersion" method to force the
version you want.
HTH,
I.
On 5 January 2017 at 20:33, Ryan Shoemaker
<ryan.shoema...@enterprisedb.com> wrote:
> Hi,
>
> I'm hoping to get a better understanding of how jclouds discovers and
> decides which identity endpoint to use. Feel free to redirect me to docs if
> this is already written up somewhere.
>
> In my case, we have a community Mitaka installation running and an app that
> uses jclouds v2.0.0. Initially, nothing identity related was working
> because Mitaka wasn't configured to respond to V2.0 Identity API requests -
> only V3, which isn't supported by jclouds yet. I didn't setup/configure
> this installation of OpenStack, so I'm not sure if that's a common issue or
> not - I don't know if the V2.0 Identity API is normally turned off in a
> fresh install or what. It's entirely possible that there's something
> misconfigured in there causing the behavior I describe below. Anyway, once
> the V2.0 Identity endpoint was enabled, I turned on tracing and can see that
> even though I create my context builder explicitly using the V2.0 endpoint
> URL, jclouds switches over and uses whatever comes back in the
> 'serviceCatalog' param in the token response. In my case, that happens to
> point to the V3 Identity endpoint. At that point, jclouds fails to work
> because it is making V2.0 requests on the V3 endpoints. Here's some tracing
> showing that in more detail:
>
>
> FINE: >>
> "{"auth":{"passwordCredentials":{"username":"someuser","password":"somepass"},"tenantName":"sometenant"}}"
> FINE: >> POST http://mitaka-host.com:5000/v2.0/tokens HTTP/1.1
> FINE: >> Accept: application/json
> FINE: >> Content-Type: application/json
> FINE: >> Content-Length: 108
>
> FINE: << HTTP/1.1 200 OK
> FINE: << Vary: X-Auth-Token
> FINE: << Date: Wed, 04 Jan 2017 20:45:54 GMT
> FINE: << Keep-Alive: timeout=5, max=97
> FINE: << Connection: Keep-Alive
> FINE: << x-openstack-request-id: req-390cebba-7cec-472c-88f4-8be6f6a0c236
> FINE: << Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5
> FINE: << Content-Type: application/json
> FINE: << Content-Length: 3620
> FINE: <<
> "{"access":{
> "token":{
> "issued_at":"2017-01-04T20:45:54.000000Z",
> "expires":"2017-01-04T21:45:54Z",
>
> "id":"gAAAAABYbV8C5pTf30IRs7ybxmbflLk539RDldDU6Q7DBHTTXTY79pabgotPSfy41Nt2nZ82isP0RxsfRuxWRlb1fnlWy1E8zlQdM3xZbrkjP27gSlmLcsV298v-CH8R4GY7wnIvfJjO0MO4DMAgFvvMIA12ByG4kqIJHZ92VV_F6Vby33gR574",
> "tenant":{
> "description":"somedescription",
> "enabled":true,
> "id":"6408ffc2d9034498bc764c230a90e591",
> "name":"sometenant"
> },
> "audit_ids":[
> "sEaG03ZdRXSeRMOr9GVCDA"
> ]
> },
> "serviceCatalog":[
> {
> "endpoints":[
> {
>
> "adminURL":"http://mitaka-host.com:8774/v2.1/6408ffc2d9034498bc764c230a90e591";,
> "region":"uk",
>
> "internalURL":"http://mitaka-host.com:8774/v2.1/6408ffc2d9034498bc764c230a90e591";,
> "id":"9d531ab7259e44738c953ec8766b0bc4",
>
> "publicURL":"http://mitaka-host.com:8774/v2.1/6408ffc2d9034498bc764c230a90e591";
> }
> ],
> "endpoints_links":[
>
> ],
> "type":"compute",
> "name":"nova"
> },
> {
> "endpoints":[
> {
> "adminURL":"http://mitaka-host.com:9696";,
> "region":"uk",
> "internalURL":"http://mitaka-host.com:9696";,
> "id":"1312e551abfe4c6d886e1a471f92bddd",
> "publicURL":"http://mitaka-host.com:9696";
> }
> ],
> "endpoints_links":[
>
> ],
> "type":"network",
> "name":"neutron"
> },
> {
> "endpoints":[
> {
>
> "adminURL":"http://mitaka-host.com:8776/v2/6408ffc2d9034498bc764c230a90e591";,
> "region":"uk",
>
> "internalURL":"http://mitaka-host.com:8776/v2/6408ffc2d9034498bc764c230a90e591";,
> "id":"02fb1ff3b41a40ceb4cb28abfa5bd547",
>
> "publicURL":"http://mitaka-host.com:8776/v2/6408ffc2d9034498bc764c230a90e591";
> }
> ],
> "endpoints_links":[
>
> ],
> "type":"volumev2",
> "name":"cinderv2"
> },
> {
> "endpoints":[
> {
> "adminURL":"http://mitaka-host.com:9292";,
> "region":"uk",
> "internalURL":"http://mitaka-host.com:9292";,
> "id":"3681888e8b62421f81e93d4270201520",
> "publicURL":"http://mitaka-host.com:9292";
> }
> ],
> "endpoints_links":[
>
> ],
> "type":"image",
> "name":"glance"
> },
> {
> "endpoints":[
> {
>
> "adminURL":"http://mitaka-host.com:8776/v1/6408ffc2d9034498bc764c230a90e591";,
> "region":"uk",
>
> "internalURL":"http://mitaka-host.com:8776/v1/6408ffc2d9034498bc764c230a90e591";,
> "id":"252797f5a4bb44c0844bff5586a23e16",
>
> "publicURL":"http://mitaka-host.com:8776/v1/6408ffc2d9034498bc764c230a90e591";
> }
> ],
> "endpoints_links":[
>
> ],
> "type":"volume",
> "name":"cinder"
> },
> {
> "endpoints":[
> {
> "adminURL":"http://mitaka-host.com:8080/v1";,
> "region":"uk",
>
> "internalURL":"http://mitaka-host.com:8080/v1/AUTH_6408ffc2d9034498bc764c230a90e591";,
> "id":"32f4fa49d90e4ad58b2b694bc1aab1ed",
>
> "publicURL":"http://mitaka-host.com:8080/v1/AUTH_6408ffc2d9034498bc764c230a90e591";
> }
> ],
> "endpoints_links":[
>
> ],
> "type":"object-store",
> "name":"swift"
> },
> {
> "endpoints":[
> {
> "adminURL":"http://mitaka-host.com:35357/v3";,
> "region":"uk",
> "internalURL":"http://mitaka-host.com:5000/v3";,
> "id":"23ace8e63eab431d81ccf5bfabc1ad53",
> "publicURL":"http://mitaka-host.com:5000/v3";
> }
> ],
> "endpoints_links":[
>
> ],
> "type":"identity",
> "name":"keystone"
> }
> ],
> "user":{
> "username":"someuser",
> "roles_links":[
>
> ],
> "id":"a0da46b26af74a44a7c89cab6d9a6439",
> "roles":[
> {
> "name":"admin"
> }
> ],
> "name":"someuser"
> },
> "metadata":{
> "is_admin":0,
> "roles":[
> "dca7c9a77e294f97a619e2b7da246550"
> ]
> }
> }
> }"
>
>
> FINE: >> GET http://mitaka-host.com:35357/v3/tenants?name=sometenant
> HTTP/1.1
> FINE: >> Accept: application/json
> FINE: >> X-Auth-Token:
> gAAAAABYbV8C5pTf30IRs7ybxmbflLk539RDldDU6Q7DBHTTXTY79pabgotPSfy41Nt2nZ82isP0RxsfRuxWRlb1fnlWy1E8zlQdM3xZbrkjP27gSlmLcsV298v-CH8R4GY7wnIvfJjO0MO4DMAgFvvMIA12ByG4kqIJHZ92VV_F6Vby33gR574
>
> FINE: << HTTP/1.1 404 Not Found
> FINE: << Vary: X-Auth-Token
> FINE: << Date: Wed, 04 Jan 2017 20:45:55 GMT
> FINE: << Keep-Alive: timeout=5, max=100
> FINE: << Connection: Keep-Alive
> FINE: << x-openstack-request-id: req-94bea180-8447-485a-8485-bf3f57e8d225
> FINE: << Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5
> FINE: << Content-Type: application/json
> FINE: << Content-Length: 93
> FINE: << "{"error": {"message": "The resource could not be found.", "code":
> 404, "title": "Not Found"}}"
>
>
> Again, even though I created the jclouds ContextBuilder with the V2.0
> endpoint, it looks like jclouds read the identity admin endpoint from the
> 'serviceCatalog' in the response and then switched to using it, even though
> it is a V3 endpoint. Is this normal expected behavior for jclouds? I'm
> guessing it has more to do with how Mitaka is configured, but I don't see
> how jclouds would be able to talk to newer versions of OpenStack unless they
> disable the V3 Identity endpoint or configure it in such a way that the V2.0
> endpoint is exposed for jclouds to see it. Are there any READMEs on how to
> use jclouds keystone APIs with newer versions of OpenStack?
>
> In general, it seems like OpenStack is moving more towards a model where API
> versioning is supposed to be discovered by clients. For example:
> https://review.openstack.org/#/c/130669/, specifically the patch comment
> that says "if some one is using third party client, it will break. Only
> keystone client handles version discovery. Keystone doesn't even document
> how to handle version discovery, so any third party clients such as jcloud
> will likely fail"
>
> Is there any update on jclouds adding support for the V3 Identity API?
>
> Thanks,
>
> --Ryan
