Hey James, Thanks for the detailed information! I think you did find a bug, let me try to explain.
First, it looks like your vpc-ff069a98 meets the minimum connectivity requirements of Juju (i.e. it's deemed suitable to host a controller), so passing --config vpc-id-force=true at bootstrap shouldn't be necessary (if it is, I'd like to know more about how your VPC's subnets and route tables look like, if possible!). The bug I think you've discovered is related to how Juju does automatic distribution of instances across AZ for redundancy. Since EC2 subnets can only be part of one AZ, and when using a non-default VPC the AWS API requires you to pass SubnetID, not just AvailZone to RunInstances used to start an instance. Security groups are VPC-specific as well. The essence of the issue I think is demonstrated here: 2016-08-02 16:57:13 INFO juju.provider.ec2 environ.go:516 selected random subnet "subnet-f30d3fd9" from all matching in zone "us-east-1a": [subnet-9b013ab1 subnet-f30d3fd9 subnet-fb380ad1] I strongly suspect those other 2 subnets: subnet-9b013ab1, subnet-fb380ad1 in AZ us-east-1a are part of a different VPC, but are still picked up "randomly" during StartInstance in the EC2 provider. In other words Juju is not filtering subnets in a given AZ by the user-specified vpc-id when deciding which one to pick. I looked at the code and it looks easy to fix this, fortunately. If I give you a patch to apply on top of the tip of juju/juju master branch, would you be willing to give it a try to see if it works? Thanks for your patience! I'll file a bug about what you've discovered and start working on a fix for beta14! Cheers, Dimiter On 08/02/2016 08:33 PM, James Beedy wrote: > Ok, here we go .... hopefully this will provide a better overhead view. > > `juju bootstrap creativedrive aws --credential creativedrive --config > vpc-id=vpc-ff069a98 --config vpc-id-force='true' --upload-tools --debug > --config logging-config='<root>=TRACE'` <- http://paste.ubuntu.com/21914113/ > > cat machine-0.log <- http://paste.ubuntu.com/21914847/ > > > ### debug add-model > > `juju add-model consul --credential creativedrive --config > vpc-id=vpc-ff069a98 --config vpc-id-force='true' --debug` <- > http://paste.ubuntu.com/21915182/ > > ### juju status on new model > > juju status --format yaml <- http://paste.ubuntu.com/21915512/ > > ### debug add space and subnet > > `juju add-space common-infrastructure --debug` <- > http://paste.ubuntu.com/21915888/ > > #### add a subnet in my vpc to the newly created space > `juju add-subnet subnet-9b013ab1 common-infrastructure us-east-1a > --debug` <- http://paste.ubuntu.com/21916055/ > > ### list spaces and subnets > > `juju subnets && juju spaces` <- http://paste.ubuntu.com/21916258/ > > > ### deploy something to the model > `juju deploy ubuntu --debug` <- http://paste.ubuntu.com/21916540/ > > > ### deploy something to a network space > > `juju deploy ubuntu ubuntu-spaces --constraints > spaces=common-infrastructure --debug` <- http://paste.ubuntu.com/21916804/ > > > ### juju status now shows a success for the machine deployed to the > model w/out a space constraint, and and error for the instance deployed > to the space. > > `juju status --format yaml` <- http://paste.ubuntu.com/21917006/ > > > No matter what, I can't seem to get anything deployed to a "space".... > > So strange ... possibly I have stumbled upon a bug? > > > Thanks again for your insight here. > > > > On Tue, Aug 2, 2016 at 9:51 AM, James Beedy <[email protected] > <mailto:[email protected]>> wrote: > > To my utter dismay, setting the correct config 'vpc-id-force' gave > me the same result.... > > Let me scrub and collect my machine-0.log for you. > > > > > On Tue, Aug 2, 2016 at 9:36 AM, James Beedy <[email protected] > <mailto:[email protected]>> wrote: > > Dimiter, > > Thanks for the insight. > / > / > /Can you please also paste the full logs (scrubbed of secrets) > of `juju > bootstrap ... --debug` (with the vpc-id etc., but please also > include > `--config logging-config='<root>=TRACE'`), and machine-0.log from > /var/log/juju on the bootstrap node, once completed? That will help > figuring out the issue. > / > `juju bootstrap creativedrive aws --credential creativedrive > --config vpc-id=vpc-ff069a98 --config force-vpc-id='true' > --config loggin-config='<root>=TRACE' --upload-tools --debug` <- > http://paste.ubuntu.com/21908548/ > > machine-0.log shows "2016-08-02 16:16:16 TRACE juju.apiserver > request_notifier.go:127 -> [2] machine-0 > > {"request-id":53,"response":{"config":{"access-key":"","agent-version":"2.0-beta13","authorized-keys":"juju-client-key\nssh-rsa > ssh-rsa > > juju-system-key\n","automatically-retry-hooks":true,"default-series":"","development":false,"disable-network-management":false,"firewall-mode":"instance","force-vpc-id":true,"ignore-machine-addresses":false,"logging-config":"\u003croot\u003e=TRACE;unit=DEBUG","name":"controller","proxy-ssh":false,"region":"us-east-1","secret-key":"/E","ssl-hostname-verification":true,"storage-default-block-source":"ebs","test-mode":false,"type":"ec2","uuid":"259be235-a255-416d-8bbf-75e128d05794","vpc-id":"vpc-ff069a98","vpc-id-force":false}}}" > > > Just realizing now, I have been specifying 'vpc-force-id', not > 'vpc-id-force' (grrrr). > > I would expect to see this resolved when I apply the correct > config. I'll report back shortly. > > Thanks for your time! > > /From what I can understand, you're trying to bootstrap on a > non-default, > possibly private VPC (accessed via its internal address over a VPN > connection maybe?), and then add a model with the same VPC and > credentials. > > / > ^ Exactly/ > / > / > If OTOH, the VPC used for add-model is different, the > machines there won't be able to talk to the controller's VPC > unless it > has a public address (cross VPC communication currently relies > on having > that, fancier setups with VPN gateways is not yet supported)./ > > ^ > > The error in status implies 2 separate VPCs are used (or a VPC and > EC2-Classic - i.e. no VPC) for the controller and hosted model. > > Cheers, > Dimiter > > > > > -- Dimiter Naydenov <[email protected]> Juju Core Sapphire team <http://juju.ubuntu.com>
signature.asc
Description: OpenPGP digital signature
-- Juju-dev mailing list [email protected] Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/juju-dev
