On Sun 15 Sep 2013 20:50:41 EST, John Arbash Meinel wrote: > I'm trying to sort through the ssl stuff, and I noticed the recent > changes stopped calling "openstack.NewPublicClient". > > This means that PublicStorage is returning an authenticating client > (s.client). >
The code is this area has not changed since May. NewPublicClient is still called when creating a public storage instance based on the configured public-bucket-url for openstack providers. Note though that it is a requirement that public-bucket-url be defined for this to happen; the fallback is to simply use the current user's credentials to create a named container on the user's account. > Which I *think* means that if you had been setting public-bucket to > point to an account which isn't yours, we'll be making requests for > that data with your creds (rather than no creds). > What has changed though, is that the process to find tools for bootstraping juju first tries using simplestreams. simplestreams does not use the public-bucket-url setting which is eventually going away. It looks in order at: 1. tools-url setting 2. control bucket 3. keystone juju-tools endpoint (for openstack) 4. public tools repository at https://juju.canonical.com/tools If none of the above work it falls back to the old way which uses the public bucket. The simplestreams work is still in progress, so documentation has not been communicated, but as of Friday last week, for Canonistack you can use this setting in your env yaml: tools-url : <public-bucket-url>/tools where <public-bucket-url> is the originally configured public-bucket-url setting. The above tools url has metadata and tools for released versions of Juju up to 1.13.3 simplestreams tools lookup almost always just performs http gets against configured urls (no credentials required), except for when it tries looking in the control/private bucket in which case is uses an authenticated storage reader. > Even with the changes to DataStore, you still go via the StorageReader. > This is *only* for the control/private bucket which requires credentials. > I just tested it with Canonistack and public-bucket-url set (and > FORCE-VERSION=1.13.3). > > With those bits, I see it try to connect to my private bucket, and > then fall back to what seems to be public-bucket-url. Both of which > fail. Then it tries the same public URL again and it succeeds. > > So I'm pretty sure that public-bucket-url is just broken now, and > bootstrap is only succeeding because it falls back to the keystone > entry which it then accesses as a Raw URL rather than a Swift bucket. > > Also, with all of the URL probing it now does, bootstrap is *much* > slower. It took about 1m40s for bootstrap to complete. See > http://pad.lv/1219582 where I added some debug logs. public-bucket-url is still used for legacy tools lookup which is invoked if simplestreams fails. So public-bucket-url is not broken, but it is ignored for simplestreams. simplestreams tools lookup does do a bit of file probing via http get, because it needs to first check for signed metadata and then fallback to unsigned metadata, and it does this for each possible URL. But the URL probing is pretty fast (< 1 second for each). It's the control bucket lookup that is slow (see below). The 10s "timeout" is only relevant when trying to load non-existent files from the control/private bucket via swift, which is one of the 3 (or 4 on openstack) places both simplestrams and legacy tools lookup uses. I didn't realise this delay was there. But it is now made worse because each simplestreams tools lookup looks for both signed and unsigned metadata. And it happens 3 times during the bootstrap sequence: 1. look for tools to see if sync is required 1.1 (perform sync) 2. look for tools again after sync to ensure they are there 2.1 (start bootstrap) 3. look for tools during bootstrap We can do some things to optimise this eg let the bootstrap process know where the tools are as a result of step 2, or reduce the swift timeout for tools lookup. Or only look for unsigned metadata in the control bucket. There's a few possibilities. This issue is only in trunk so we'll need to get it sorted before the next release. -- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
