[Cloud-init-dev] [Merge] ~b1sandmann/cloud-init:migrate-lp-to-github into cloud-init:master

2021-04-06 Thread Dan Watkins
The proposal to merge ~b1sandmann/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~b1sandmann/cloud-init/+git/cloud-init/+merge/399100
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~b1sandmann/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~b1sandmann/cloud-init:migrate-lp-to-github into cloud-init:master

2021-04-06 Thread Dan Watkins
Thanks!  Marking this as Rejected (which is the expected end state for this MP).
-- 
https://code.launchpad.net/~b1sandmann/cloud-init/+git/cloud-init/+merge/399100
Your team cloud-init Commiters is requested to review the proposed merge of 
~b1sandmann/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~lp-markusschade/cloud-init:migrate-lp-to-github into cloud-init:master

2020-10-27 Thread Dan Watkins
The proposal to merge ~lp-markusschade/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~lp-markusschade/cloud-init/+git/cloud-init/+merge/392777
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~lp-markusschade/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~lp-markusschade/cloud-init:migrate-lp-to-github into cloud-init:master

2020-10-27 Thread Dan Watkins
Thanks!  I've confirmed that this matches in GitHub, so I'm going to mark this 
as Rejected.  This is the expected final state of this MP.
-- 
https://code.launchpad.net/~lp-markusschade/cloud-init/+git/cloud-init/+merge/392777
Your team cloud-init Commiters is requested to review the proposed merge of 
~lp-markusschade/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~vtqanh/cloud-init:migrate-lp-to-github into cloud-init:master

2020-07-15 Thread Dan Watkins
Thanks!  I will now be marking this MP rejected, which is the expected end 
state here.
-- 
https://code.launchpad.net/~vtqanh/cloud-init/+git/cloud-init/+merge/387447
Your team cloud-init Commiters is requested to review the proposed merge of 
~vtqanh/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~chad.smith/cloud-init:feature/ec2-secondary-nics into cloud-init:master

2020-05-21 Thread Dan Watkins
Hi Andrea, this branch was migrated over to GitHub (along with the rest of 
cloud-init's code hosting :) and was proposed here: 
https://github.com/canonical/cloud-init/pull/114

It landed in March and was included in the 20.2 cloud-init release.  It is 
available in Ubuntu 20.04 (Focal Fossa), and will be backported to all stable 
Ubuntu releases (i.e. 16.04 and later) in the next couple of weeks.

Thanks for your interest!
-- 
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/369792
Your team cloud-init Commiters is requested to review the proposed merge of 
~chad.smith/cloud-init:feature/ec2-secondary-nics into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~momousta/cloud-init:momousta-migrate-lp-to-github into cloud-init:master

2020-05-14 Thread Dan Watkins
The proposal to merge ~momousta/cloud-init:momousta-migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~momousta/cloud-init/+git/cloud-init/+merge/383902
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~momousta/cloud-init:momousta-migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~momousta/cloud-init:momousta-migrate-lp-to-github into cloud-init:master

2020-05-14 Thread Dan Watkins
Thanks!  I'm moving this MP to Rejected, which is the expected end state for it.
-- 
https://code.launchpad.net/~momousta/cloud-init/+git/cloud-init/+merge/383902
Your team cloud-init Commiters is requested to review the proposed merge of 
~momousta/cloud-init:momousta-migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~killermoehre/cloud-init:migrate-lp-to-github into cloud-init:master

2020-03-26 Thread Dan Watkins
The proposal to merge ~killermoehre/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~killermoehre/cloud-init/+git/cloud-init/+merge/381247
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~killermoehre/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~killermoehre/cloud-init:migrate-lp-to-github into cloud-init:master

2020-03-26 Thread Dan Watkins
Thanks for this.  Marking this as Rejected, which is the expected end state for 
this MP.
-- 
https://code.launchpad.net/~killermoehre/cloud-init/+git/cloud-init/+merge/381247
Your team cloud-init Commiters is requested to review the proposed merge of 
~killermoehre/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init:master into cloud-init:master

2020-01-08 Thread Dan Watkins
Dan Watkins has proposed merging ~daniel-thewatkins/cloud-init:master into 
cloud-init:master.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init-1/+merge/377307
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init:master into cloud-init:master.
diff --git a/tools/.lp-to-git-user b/tools/.lp-to-git-user
index 4a68759..a943e1d 100644
--- a/tools/.lp-to-git-user
+++ b/tools/.lp-to-git-user
@@ -7,6 +7,7 @@
  "bitfehler": "bitfehler",
  "chad.smith": "blackboxsw",
  "d-info-e": "do3meli",
+ "daniel-thewatkins": "OddBloke",
  "eric-lafontaine1": "elafontaine",
  "fredlefebvre": "fred-lefebvre",
  "goneri": "goneri",
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~otubo/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-08 Thread Dan Watkins
The proposal to merge ~otubo/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~otubo/cloud-init/+git/cloud-init/+merge/377041
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~otubo/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~otubo/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-08 Thread Dan Watkins
Thanks!  (Moving this to Rejected now, which is the expected end state here. :)
-- 
https://code.launchpad.net/~otubo/cloud-init/+git/cloud-init/+merge/377041
Your team cloud-init Commiters is requested to review the proposed merge of 
~otubo/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~afranceschini/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-07 Thread Dan Watkins
Thanks!  (I'm going to mark this as Rejected, which is the expected state here. 
:)
-- 
https://code.launchpad.net/~afranceschini/cloud-init/+git/cloud-init/+merge/377244
Your team cloud-init Commiters is requested to review the proposed merge of 
~afranceschini/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~afranceschini/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-07 Thread Dan Watkins
The proposal to merge ~afranceschini/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~afranceschini/cloud-init/+git/cloud-init/+merge/377244
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~afranceschini/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~madhuri-rai07/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-07 Thread Dan Watkins
The proposal to merge ~madhuri-rai07/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~madhuri-rai07/cloud-init/+git/cloud-init/+merge/377187
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~madhuri-rai07/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~madhuri-rai07/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-07 Thread Dan Watkins
Thanks!  (Moving this to Rejected, which is the expected end state here. :)
-- 
https://code.launchpad.net/~madhuri-rai07/cloud-init/+git/cloud-init/+merge/377187
Your team cloud-init Commiters is requested to review the proposed merge of 
~madhuri-rai07/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~louis/cloud-init:fix_scaleway_datasource_backtrace into cloud-init:master

2020-01-06 Thread Dan Watkins
I'm going to close this out as it's been moved to GitHub: 
https://github.com/canonical/cloud-init/pull/128/
-- 
https://code.launchpad.net/~louis/cloud-init/+git/cloud-init/+merge/377090
Your team cloud-init Commiters is requested to review the proposed merge of 
~louis/cloud-init:fix_scaleway_datasource_backtrace into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~louis/cloud-init:fix_scaleway_datasource_backtrace into cloud-init:master

2020-01-06 Thread Dan Watkins
The proposal to merge ~louis/cloud-init:fix_scaleway_datasource_backtrace into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~louis/cloud-init/+git/cloud-init/+merge/377090
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~louis/cloud-init:fix_scaleway_datasource_backtrace into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~madhuri-rai07/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-06 Thread Dan Watkins
Dan Watkins has proposed merging ~madhuri-rai07/cloud-init:migrate-lp-to-github 
into cloud-init:master.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~madhuri-rai07/cloud-init/+git/cloud-init/+merge/377187
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~madhuri-rai07/cloud-init:migrate-lp-to-github into cloud-init:master.
diff --git a/tools/.lp-to-git-user b/tools/.lp-to-git-user
index c2e0c9a..959598b 100644
--- a/tools/.lp-to-git-user
+++ b/tools/.lp-to-git-user
@@ -12,6 +12,7 @@
  "i.galic": "igalic",
  "larsks": "larsks",
  "legovini": "paride",
+ "madhuri-rai07": "madhuri-rai07",
  "pengpengs": "PengpengSun",
  "powersj": "powersj",
  "raharper": "raharper",
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~askon/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-06 Thread Dan Watkins
The proposal to merge ~askon/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~askon/cloud-init/+git/cloud-init/+merge/377185
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~askon/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~askon/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-06 Thread Dan Watkins
Thanks!  (Marking as Rejected, which is the expected final state here. :)
-- 
https://code.launchpad.net/~askon/cloud-init/+git/cloud-init/+merge/377185
Your team cloud-init Commiters is requested to review the proposed merge of 
~askon/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~louis/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-06 Thread Dan Watkins
Thanks, Louis!  (I'll be marking this as Rejected, which is the expected end 
state. :)
-- 
https://code.launchpad.net/~louis/cloud-init/+git/cloud-init/+merge/377091
Your team cloud-init Commiters is requested to review the proposed merge of 
~louis/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~louis/cloud-init:migrate-lp-to-github into cloud-init:master

2020-01-06 Thread Dan Watkins
The proposal to merge ~louis/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~louis/cloud-init/+git/cloud-init/+merge/377091
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~louis/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~goneri/cloud-init:migrate-lp-to-github into cloud-init:master

2019-12-19 Thread Dan Watkins
The proposal to merge ~goneri/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/377008
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~goneri/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~goneri/cloud-init:migrate-lp-to-github into cloud-init:master

2019-12-19 Thread Dan Watkins
Thanks!  (Rejected is the expected status here. :)
-- 
https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/377008
Your team cloud-init Commiters is requested to review the proposed merge of 
~goneri/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~adobrawy/cloud-init:migrate-lp-to-github into cloud-init:master

2019-12-17 Thread Dan Watkins
(Rejected is the normal state for this to end up in, no more work needed for 
this!)
-- 
https://code.launchpad.net/~adobrawy/cloud-init/+git/cloud-init/+merge/376922
Your team cloud-init Commiters is requested to review the proposed merge of 
~adobrawy/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~adobrawy/cloud-init:migrate-lp-to-github into cloud-init:master

2019-12-17 Thread Dan Watkins
The proposal to merge ~adobrawy/cloud-init:migrate-lp-to-github into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~adobrawy/cloud-init/+git/cloud-init/+merge/376922
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~adobrawy/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~adobrawy/cloud-init:migrate-lp-to-github into cloud-init:master

2019-12-17 Thread Dan Watkins
Confirmed at 
https://github.com/canonical/cloud-init/commit/2267cd31443d2c1032190102dd4b6011076ed182
-- 
https://code.launchpad.net/~adobrawy/cloud-init/+git/cloud-init/+merge/376922
Your team cloud-init Commiters is requested to review the proposed merge of 
~adobrawy/cloud-init:migrate-lp-to-github into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~rjschwei/cloud-init:breakLink into cloud-init:master

2019-10-24 Thread Dan Watkins
One inline question, probably for my own edification.  Thanks!

Diff comments:

> diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
> index e381596..44a2f11 100644
> --- a/cloudinit/net/sysconfig.py
> +++ b/cloudinit/net/sysconfig.py
> @@ -708,6 +708,12 @@ class Renderer(renderer.Renderer):
>  resolv_content = self._render_dns(network_state,
>existing_dns_path=dns_path)
>  if resolv_content:
> +# netconfig checks if the resolv file is a link and if that

Forgive my ignorance about the stack here, Robert, but is there anything we can 
do to configure DNS in netconfig itself so that even if it does assume control 
it won't break the configuration we want?

> +# is true it assumes it is in controil of the file and
> +# will clobber our changes, break the link to ensure the user
> +# configuration sticks
> +if os.path.islink(dns_path):
> +os.unlink(dns_path)
>  util.write_file(dns_path, resolv_content, file_mode)
>  if self.networkmanager_conf_path:
>  nm_conf_path = util.target_path(target,


-- 
https://code.launchpad.net/~rjschwei/cloud-init/+git/cloud-init/+merge/374517
Your team cloud-init Commiters is requested to review the proposed merge of 
~rjschwei/cloud-init:breakLink into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:travis into cloud-init:master

2019-10-21 Thread Dan Watkins
You can see a successful run with this configuration here: 
https://travis-ci.org/OddBloke/cloud-init/builds/600851758
-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/374462
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:travis into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:travis into cloud-init:master

2019-10-21 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:travis into cloud-init:master.

Commit message:
introduce .travis.yml

This captures the CI testing that is currently performed by the Ubuntu
Server Jenkins instance into a Travis configuration, which is part of
the migration of cloud-init code hosting from Launchpad to GitHub.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/374462
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:travis into cloud-init:master.
diff --git a/.travis.yml b/.travis.yml
new file mode 100644
index 000..834a568
--- /dev/null
+++ b/.travis.yml
@@ -0,0 +1,55 @@
+language: python
+dist: bionic
+
+install:
+# Required so `git describe` will definitely find a tag; see
+# https://github.com/travis-ci/travis-ci/issues/7422
+- git fetch --unshallow
+- pip install tox
+
+script:
+- tox
+
+matrix:
+fast_finish: true
+include:
+- python: 3.6
+  env: TOXENV=py3
+- install:
+- git fetch --unshallow
+- sudo apt-get build-dep -y cloud-init
+- sudo apt-get install -y --install-recommends sbuild ubuntu-dev-tools fakeroot tox
+# These are build deps but not pulled in by the build-dep call above
+- sudo apt-get install -y --install-recommends dh-systemd python3-coverage python3-contextlib2
+- pip install .
+- pip install tox
+# bionic has lxd from deb installed, remove it first to ensure
+# pylxd talks only to the lxd from snap
+- sudo apt remove --purge lxd lxd-client
+- sudo rm -Rf /var/lib/lxd
+- sudo snap install lxd
+- sudo lxd init --auto
+- sudo mkdir --mode=1777 -p /var/snap/lxd/common/consoles
+- sudo usermod -a -G lxd $USER
+- sudo sbuild-adduser $USER
+- cp /usr/share/doc/sbuild/examples/example.sbuildrc /home/$USER/.sbuildrc
+  script:
+# Ubuntu LTS: Build
+- ./packages/bddeb -S
+# Use this to get a new shell where we're in the sbuild group
+- sudo -E su $USER -c 'mk-sbuild xenial'
+- sudo -E su $USER -c 'sbuild --nolog --verbose --dist=xenial cloud-init_*.dsc'
+# Ubuntu LTS: Integration
+- sg lxd -c 'tox -e citest -- run --verbose --preserve-data --data-dir results --os-name xenial --test modules/apt_configure_sources_list.yaml --test modules/ntp_servers --test modules/set_password_list --test modules/user_groups --deb cloud-init_*_all.deb'
+- python: 2.7
+  env: TOXENV=py27
+- python: 3.4
+  env: TOXENV=xenial
+  # Travis doesn't support Python 3.4 on bionic, so use xenial
+  dist: xenial
+- python: 3.6
+  env: TOXENV=pycodestyle
+- python: 3.6
+  env: TOXENV=pyflakes
+- python: 3.6
+  env: TOXENV=pylint
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~d-info-e/cloud-init:gitignore-venv into cloud-init:master

2019-10-08 Thread Dan Watkins
The proposal to merge ~d-info-e/cloud-init:gitignore-venv into 
cloud-init:master has been updated.

Description changed to:

add .venv/ to .gitignore

For more details, see:
https://code.launchpad.net/~d-info-e/cloud-init/+git/cloud-init/+merge/373628
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~d-info-e/cloud-init:gitignore-venv into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/xenial into cloud-init:ubuntu/xenial

2019-10-04 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/xenial into 
cloud-init:ubuntu/xenial.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1846535 in cloud-init: "cloud-init 19.2.36 fails with python exception 
"Not all expected physical devices present ..."  during bionic image deployment 
from MAAS"
  https://bugs.launchpad.net/cloud-init/+bug/1846535

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373660
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/xenial into 
cloud-init:ubuntu/xenial.
diff --git a/debian/changelog b/debian/changelog
index 5b03045..ecd454e 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+cloud-init (19.2-36-g059d049c-0ubuntu2~16.04.1) xenial; urgency=medium
+
+  * cherry-pick a7d8d032: get_interfaces: don't exclude bridge and bond
+members (LP: #1846535)
+
+ -- Daniel Watkins   Fri, 04 Oct 2019 12:01:19 -0400
+
 cloud-init (19.2-36-g059d049c-0ubuntu1~16.04.1) xenial; urgency=medium
 
   * New upstream snapshot. (LP: #1844334)
diff --git a/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members b/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members
new file mode 100644
index 000..4415fcb
--- /dev/null
+++ b/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members
@@ -0,0 +1,150 @@
+From a7d8d032a65e807007f1467a9220b7f161625668 Mon Sep 17 00:00:00 2001
+From: Daniel Watkins 
+Date: Fri, 4 Oct 2019 15:34:41 +
+Subject: [PATCH] get_interfaces: don't exclude bridge and bond members
+
+The change that introduced this issue was handling interfaces that are
+bonded in the kernel, in a way that doesn't present as "a bond" to
+userspace in the normal way.  Both members of this "bond" will share a
+MAC address, so we filter one of them out to avoid incorrect MAC address
+collision warnings.
+
+Unfortunately, the matching condition was too broad, so that change also
+affected normal bonds and bridges.  This change specifically excludes
+bonds and bridges from that determination, to address that regression.
+
+LP: #1846535
+---
+ cloudinit/net/__init__.py| 24 ++---
+ cloudinit/net/tests/test_init.py | 59 +---
+ 2 files changed, 73 insertions(+), 10 deletions(-)
+
+--- a/cloudinit/net/__init__.py
 b/cloudinit/net/__init__.py
+@@ -109,8 +109,22 @@ def is_bond(devname):
+ return os.path.exists(sys_dev_path(devname, "bonding"))
+ 
+ 
+-def has_master(devname):
+-return os.path.exists(sys_dev_path(devname, path="master"))
++def get_master(devname):
++"""Return the master path for devname, or None if no master"""
++path = sys_dev_path(devname, path="master")
++if os.path.exists(path):
++return path
++return None
++
++
++def master_is_bridge_or_bond(devname):
++"""Return a bool indicating if devname's master is a bridge or bond"""
++master_path = get_master(devname)
++if master_path is None:
++return False
++bonding_path = os.path.join(master_path, "bonding")
++bridge_path = os.path.join(master_path, "bridge")
++return (os.path.exists(bonding_path) or os.path.exists(bridge_path))
+ 
+ 
+ def is_netfailover(devname, driver=None):
+@@ -158,7 +172,7 @@ def is_netfail_master(devname, driver=No
+ 
+ Return True if all of the above is True.
+ """
+-if has_master(devname):
++if get_master(devname) is not None:
+ return False
+ 
+ if driver is None:
+@@ -215,7 +229,7 @@ def is_netfail_standby(devname, driver=N
+ 
+ Return True if all of the above is True.
+ """
+-if not has_master(devname):
++if get_master(devname) is None:
+ return False
+ 
+ if driver is None:
+@@ -790,7 +804,7 @@ def get_interfaces():
+ continue
+ if is_bond(name):
+ continue
+-if has_master(name):
++if get_master(name) is not None and not master_is_bridge_or_bond(name):
+ continue
+ if is_netfailover(name):
+ continue
+--- a/cloudinit/net/tests/test_init.py
 b/cloudinit/net/tests/test_init.py
+@@ -157,11 +157,40 @@ class TestReadSysNet(CiTestCase):
+ ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))
+ self.assertTrue(net.is_bond('eth0'))
+ 
+-def test_has_master(self):
+-"""has_master is True when /sys/net/devname/master exists."""
+-self.assertFalse(net.has_master('enP1s1'))
+-ensure_file(os.path.join(self.sysdir, 'enP1s1', 'master'))
+-self.assertTrue(net.has_master('enP1s1'))
++def test_

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/disco into cloud-init:ubuntu/disco

2019-10-04 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/disco into 
cloud-init:ubuntu/disco.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1846535 in cloud-init: "cloud-init 19.2.36 fails with python exception 
"Not all expected physical devices present ..."  during bionic image deployment 
from MAAS"
  https://bugs.launchpad.net/cloud-init/+bug/1846535

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373657
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/disco into 
cloud-init:ubuntu/disco.
diff --git a/debian/changelog b/debian/changelog
index e975bc2..befbd26 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+cloud-init (19.2-36-g059d049c-0ubuntu2~19.04.1) disco; urgency=medium
+
+  * cherry-pick a7d8d032: get_interfaces: don't exclude bridge and bond
+members (LP: #1846535)
+
+ -- Daniel Watkins   Fri, 04 Oct 2019 11:46:15 -0400
+
 cloud-init (19.2-36-g059d049c-0ubuntu1~19.04.1) disco; urgency=medium
 
   * New upstream snapshot. (LP: #1844334)
diff --git a/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members b/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members
new file mode 100644
index 000..4415fcb
--- /dev/null
+++ b/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members
@@ -0,0 +1,150 @@
+From a7d8d032a65e807007f1467a9220b7f161625668 Mon Sep 17 00:00:00 2001
+From: Daniel Watkins 
+Date: Fri, 4 Oct 2019 15:34:41 +
+Subject: [PATCH] get_interfaces: don't exclude bridge and bond members
+
+The change that introduced this issue was handling interfaces that are
+bonded in the kernel, in a way that doesn't present as "a bond" to
+userspace in the normal way.  Both members of this "bond" will share a
+MAC address, so we filter one of them out to avoid incorrect MAC address
+collision warnings.
+
+Unfortunately, the matching condition was too broad, so that change also
+affected normal bonds and bridges.  This change specifically excludes
+bonds and bridges from that determination, to address that regression.
+
+LP: #1846535
+---
+ cloudinit/net/__init__.py| 24 ++---
+ cloudinit/net/tests/test_init.py | 59 +---
+ 2 files changed, 73 insertions(+), 10 deletions(-)
+
+--- a/cloudinit/net/__init__.py
 b/cloudinit/net/__init__.py
+@@ -109,8 +109,22 @@ def is_bond(devname):
+ return os.path.exists(sys_dev_path(devname, "bonding"))
+ 
+ 
+-def has_master(devname):
+-return os.path.exists(sys_dev_path(devname, path="master"))
++def get_master(devname):
++"""Return the master path for devname, or None if no master"""
++path = sys_dev_path(devname, path="master")
++if os.path.exists(path):
++return path
++return None
++
++
++def master_is_bridge_or_bond(devname):
++"""Return a bool indicating if devname's master is a bridge or bond"""
++master_path = get_master(devname)
++if master_path is None:
++return False
++bonding_path = os.path.join(master_path, "bonding")
++bridge_path = os.path.join(master_path, "bridge")
++return (os.path.exists(bonding_path) or os.path.exists(bridge_path))
+ 
+ 
+ def is_netfailover(devname, driver=None):
+@@ -158,7 +172,7 @@ def is_netfail_master(devname, driver=No
+ 
+ Return True if all of the above is True.
+ """
+-if has_master(devname):
++if get_master(devname) is not None:
+ return False
+ 
+ if driver is None:
+@@ -215,7 +229,7 @@ def is_netfail_standby(devname, driver=N
+ 
+ Return True if all of the above is True.
+ """
+-if not has_master(devname):
++if get_master(devname) is None:
+ return False
+ 
+ if driver is None:
+@@ -790,7 +804,7 @@ def get_interfaces():
+ continue
+ if is_bond(name):
+ continue
+-if has_master(name):
++if get_master(name) is not None and not master_is_bridge_or_bond(name):
+ continue
+ if is_netfailover(name):
+ continue
+--- a/cloudinit/net/tests/test_init.py
 b/cloudinit/net/tests/test_init.py
+@@ -157,11 +157,40 @@ class TestReadSysNet(CiTestCase):
+ ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))
+ self.assertTrue(net.is_bond('eth0'))
+ 
+-def test_has_master(self):
+-"""has_master is True when /sys/net/devname/master exists."""
+-self.assertFalse(net.has_master('enP1s1'))
+-ensure_file(os.path.join(self.sysdir, 'enP1s1', 'master'))
+-self.assertTrue(net.has_master('enP1s1'))
++def test_get_master

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into cloud-init:ubuntu/devel

2019-10-04 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1846535 in cloud-init: "cloud-init 19.2.36 fails with python exception 
"Not all expected physical devices present ..."  during bionic image deployment 
from MAAS"
  https://bugs.launchpad.net/cloud-init/+bug/1846535

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373656
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.
diff --git a/debian/changelog b/debian/changelog
index 4195d2c..cfe1e0f 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+cloud-init (19.2-36-g059d049c-0ubuntu2) eoan; urgency=medium
+
+  * cherry-pick a7d8d032: get_interfaces: don't exclude bridge and bond
+members (LP: #1846535)
+
+ -- Daniel Watkins   Fri, 04 Oct 2019 11:42:12 -0400
+
 cloud-init (19.2-36-g059d049c-0ubuntu1) eoan; urgency=medium
 
   * New upstream snapshot.
diff --git a/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members b/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members
new file mode 100644
index 000..4415fcb
--- /dev/null
+++ b/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members
@@ -0,0 +1,150 @@
+From a7d8d032a65e807007f1467a9220b7f161625668 Mon Sep 17 00:00:00 2001
+From: Daniel Watkins 
+Date: Fri, 4 Oct 2019 15:34:41 +
+Subject: [PATCH] get_interfaces: don't exclude bridge and bond members
+
+The change that introduced this issue was handling interfaces that are
+bonded in the kernel, in a way that doesn't present as "a bond" to
+userspace in the normal way.  Both members of this "bond" will share a
+MAC address, so we filter one of them out to avoid incorrect MAC address
+collision warnings.
+
+Unfortunately, the matching condition was too broad, so that change also
+affected normal bonds and bridges.  This change specifically excludes
+bonds and bridges from that determination, to address that regression.
+
+LP: #1846535
+---
+ cloudinit/net/__init__.py| 24 ++---
+ cloudinit/net/tests/test_init.py | 59 +---
+ 2 files changed, 73 insertions(+), 10 deletions(-)
+
+--- a/cloudinit/net/__init__.py
 b/cloudinit/net/__init__.py
+@@ -109,8 +109,22 @@ def is_bond(devname):
+ return os.path.exists(sys_dev_path(devname, "bonding"))
+ 
+ 
+-def has_master(devname):
+-return os.path.exists(sys_dev_path(devname, path="master"))
++def get_master(devname):
++"""Return the master path for devname, or None if no master"""
++path = sys_dev_path(devname, path="master")
++if os.path.exists(path):
++return path
++return None
++
++
++def master_is_bridge_or_bond(devname):
++"""Return a bool indicating if devname's master is a bridge or bond"""
++master_path = get_master(devname)
++if master_path is None:
++return False
++bonding_path = os.path.join(master_path, "bonding")
++bridge_path = os.path.join(master_path, "bridge")
++return (os.path.exists(bonding_path) or os.path.exists(bridge_path))
+ 
+ 
+ def is_netfailover(devname, driver=None):
+@@ -158,7 +172,7 @@ def is_netfail_master(devname, driver=No
+ 
+ Return True if all of the above is True.
+ """
+-if has_master(devname):
++if get_master(devname) is not None:
+ return False
+ 
+ if driver is None:
+@@ -215,7 +229,7 @@ def is_netfail_standby(devname, driver=N
+ 
+ Return True if all of the above is True.
+ """
+-if not has_master(devname):
++if get_master(devname) is None:
+ return False
+ 
+ if driver is None:
+@@ -790,7 +804,7 @@ def get_interfaces():
+ continue
+ if is_bond(name):
+ continue
+-if has_master(name):
++if get_master(name) is not None and not master_is_bridge_or_bond(name):
+ continue
+ if is_netfailover(name):
+ continue
+--- a/cloudinit/net/tests/test_init.py
 b/cloudinit/net/tests/test_init.py
+@@ -157,11 +157,40 @@ class TestReadSysNet(CiTestCase):
+ ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))
+ self.assertTrue(net.is_bond('eth0'))
+ 
+-def test_has_master(self):
+-"""has_master is True when /sys/net/devname/master exists."""
+-self.assertFalse(net.has_master('enP1s1'))
+-ensure_file(os.path.join(self.sysdir, 'enP1s1', 'master'))
+-self.assertTrue(net.has_master('enP1s1'))
++def test_get_master(self):
++"

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/bionic into cloud-init:ubuntu/bionic

2019-10-04 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/bionic into 
cloud-init:ubuntu/bionic.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1846535 in cloud-init: "cloud-init 19.2.36 fails with python exception 
"Not all expected physical devices present ..."  during bionic image deployment 
from MAAS"
  https://bugs.launchpad.net/cloud-init/+bug/1846535

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373655
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/bionic into 
cloud-init:ubuntu/bionic.
diff --git a/debian/changelog b/debian/changelog
index 31760d7..10ac80b 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+cloud-init (19.2-36-g059d049c-0ubuntu2~18.04.1) bionic; urgency=medium
+
+  * cherry-pick a7d8d032: get_interfaces: don't exclude bridge and bond
+members (LP: #1846535)
+
+ -- Daniel Watkins   Fri, 04 Oct 2019 11:35:54 -0400
+
 cloud-init (19.2-36-g059d049c-0ubuntu1~18.04.1) bionic; urgency=medium
 
   * New upstream snapshot. (LP: #1844334)
diff --git a/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members b/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members
new file mode 100644
index 000..4415fcb
--- /dev/null
+++ b/debian/patches/cpick-a7d8d032-get_interfaces-don-t-exclude-bridge-and-bond-members
@@ -0,0 +1,150 @@
+From a7d8d032a65e807007f1467a9220b7f161625668 Mon Sep 17 00:00:00 2001
+From: Daniel Watkins 
+Date: Fri, 4 Oct 2019 15:34:41 +
+Subject: [PATCH] get_interfaces: don't exclude bridge and bond members
+
+The change that introduced this issue was handling interfaces that are
+bonded in the kernel, in a way that doesn't present as "a bond" to
+userspace in the normal way.  Both members of this "bond" will share a
+MAC address, so we filter one of them out to avoid incorrect MAC address
+collision warnings.
+
+Unfortunately, the matching condition was too broad, so that change also
+affected normal bonds and bridges.  This change specifically excludes
+bonds and bridges from that determination, to address that regression.
+
+LP: #1846535
+---
+ cloudinit/net/__init__.py| 24 ++---
+ cloudinit/net/tests/test_init.py | 59 +---
+ 2 files changed, 73 insertions(+), 10 deletions(-)
+
+--- a/cloudinit/net/__init__.py
 b/cloudinit/net/__init__.py
+@@ -109,8 +109,22 @@ def is_bond(devname):
+ return os.path.exists(sys_dev_path(devname, "bonding"))
+ 
+ 
+-def has_master(devname):
+-return os.path.exists(sys_dev_path(devname, path="master"))
++def get_master(devname):
++"""Return the master path for devname, or None if no master"""
++path = sys_dev_path(devname, path="master")
++if os.path.exists(path):
++return path
++return None
++
++
++def master_is_bridge_or_bond(devname):
++"""Return a bool indicating if devname's master is a bridge or bond"""
++master_path = get_master(devname)
++if master_path is None:
++return False
++bonding_path = os.path.join(master_path, "bonding")
++bridge_path = os.path.join(master_path, "bridge")
++return (os.path.exists(bonding_path) or os.path.exists(bridge_path))
+ 
+ 
+ def is_netfailover(devname, driver=None):
+@@ -158,7 +172,7 @@ def is_netfail_master(devname, driver=No
+ 
+ Return True if all of the above is True.
+ """
+-if has_master(devname):
++if get_master(devname) is not None:
+ return False
+ 
+ if driver is None:
+@@ -215,7 +229,7 @@ def is_netfail_standby(devname, driver=N
+ 
+ Return True if all of the above is True.
+ """
+-if not has_master(devname):
++if get_master(devname) is None:
+ return False
+ 
+ if driver is None:
+@@ -790,7 +804,7 @@ def get_interfaces():
+ continue
+ if is_bond(name):
+ continue
+-if has_master(name):
++if get_master(name) is not None and not master_is_bridge_or_bond(name):
+ continue
+ if is_netfailover(name):
+ continue
+--- a/cloudinit/net/tests/test_init.py
 b/cloudinit/net/tests/test_init.py
+@@ -157,11 +157,40 @@ class TestReadSysNet(CiTestCase):
+ ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))
+ self.assertTrue(net.is_bond('eth0'))
+ 
+-def test_has_master(self):
+-"""has_master is True when /sys/net/devname/master exists."""
+-self.assertFalse(net.has_master('enP1s1'))
+-ensure_file(os.path.join(self.sysdir, 'enP1s1', 'master'))
+-self.assertTrue(net.has_master('enP1s1'))
++def test_

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lp1846535 into cloud-init:master

2019-10-04 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1846535 into cloud-init:master.

Commit message:
get_interfaces: don't exclude bridge and bond members

The change that introduced this issue was handling interfaces that are
bonded in the kernel, in a way that doesn't present as "a bond" to
userspace in the normal way.  Both members of this "bond" will share a
MAC address, so we filter one of them out to avoid incorrect MAC address
collision warnings.

Unfortunately, the matching condition was too broad, so that change also
affected normal bonds and bridges.  This change specifically excludes
bonds and bridges from that determination, to address that regression.

LP: #1846535

Requested reviews:
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1846535 in cloud-init: "cloud-init 19.2.36 fails with python exception 
"Not all expected physical devices present ..."  during bionic image deployment 
from MAAS"
  https://bugs.launchpad.net/cloud-init/+bug/1846535

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373649
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1846535 into cloud-init:master.
diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
index 5de5c6d..307da78 100644
--- a/cloudinit/net/__init__.py
+++ b/cloudinit/net/__init__.py
@@ -109,8 +109,22 @@ def is_bond(devname):
 return os.path.exists(sys_dev_path(devname, "bonding"))
 
 
-def has_master(devname):
-return os.path.exists(sys_dev_path(devname, path="master"))
+def get_master(devname):
+"""Return the master path for devname, or None if no master"""
+path = sys_dev_path(devname, path="master")
+if os.path.exists(path):
+return path
+return None
+
+
+def master_is_bridge_or_bond(devname):
+"""Return a bool indicating if devname's master is a bridge or bond"""
+master_path = get_master(devname)
+if master_path is None:
+return False
+bonding_path = os.path.join(master_path, "bonding")
+bridge_path = os.path.join(master_path, "bridge")
+return (os.path.exists(bonding_path) or os.path.exists(bridge_path))
 
 
 def is_netfailover(devname, driver=None):
@@ -158,7 +172,7 @@ def is_netfail_master(devname, driver=None):
 
 Return True if all of the above is True.
 """
-if has_master(devname):
+if get_master(devname) is not None:
 return False
 
 if driver is None:
@@ -215,7 +229,7 @@ def is_netfail_standby(devname, driver=None):
 
 Return True if all of the above is True.
 """
-if not has_master(devname):
+if get_master(devname) is None:
 return False
 
 if driver is None:
@@ -790,7 +804,7 @@ def get_interfaces():
 continue
 if is_bond(name):
 continue
-if has_master(name):
+if get_master(name) is not None and not master_is_bridge_or_bond(name):
 continue
 if is_netfailover(name):
 continue
diff --git a/cloudinit/net/tests/test_init.py b/cloudinit/net/tests/test_init.py
index 999db98..6db93e2 100644
--- a/cloudinit/net/tests/test_init.py
+++ b/cloudinit/net/tests/test_init.py
@@ -157,11 +157,40 @@ class TestReadSysNet(CiTestCase):
 ensure_file(os.path.join(self.sysdir, 'eth0', 'bonding'))
 self.assertTrue(net.is_bond('eth0'))
 
-def test_has_master(self):
-"""has_master is True when /sys/net/devname/master exists."""
-self.assertFalse(net.has_master('enP1s1'))
-ensure_file(os.path.join(self.sysdir, 'enP1s1', 'master'))
-self.assertTrue(net.has_master('enP1s1'))
+def test_get_master(self):
+"""get_master returns the path when /sys/net/devname/master exists."""
+self.assertIsNone(net.get_master('enP1s1'))
+master_path = os.path.join(self.sysdir, 'enP1s1', 'master')
+ensure_file(master_path)
+self.assertEqual(master_path, net.get_master('enP1s1'))
+
+def test_master_is_bridge_or_bond(self):
+bridge_mac = 'aa:bb:cc:aa:bb:cc'
+bond_mac = 'cc:bb:aa:cc:bb:aa'
+
+# No master => False
+write_file(os.path.join(self.sysdir, 'eth1', 'address'), bridge_mac)
+write_file(os.path.join(self.sysdir, 'eth2', 'address'), bond_mac)
+
+self.assertFalse(net.master_is_bridge_or_bond('eth1'))
+self.assertFalse(net.master_is_bridge_or_bond('eth2'))
+
+# masters without bridge/bonding => False
+write_file(os.path.join(self.sysdir, 'br0', 'address'), bridge_mac)
+write_file(os.path.join(self.sysdir, 'bond0', 'address'), bond_mac)
+
+os.symlink('../br

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:net into cloud-init:master

2019-09-27 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:net into cloud-init:master.

Commit message:
debian/ubuntu: add missing word to netplan/ENI header

Specifically, add in "reboot" to make it clear what people should expect
when modifying the file.

This also renames the variable to indicate it is used for netplan and
ENI, not just ENI.

LP: #1845669


Requested reviews:
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1845669 in cloud-init: "The meaning of "Changes to it will not persist 
across an instance." in 50-cloud-init.yaml is unclear"
  https://bugs.launchpad.net/cloud-init/+bug/1845669

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373329
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:net into cloud-init:master.
diff --git a/cloudinit/distros/debian.py b/cloudinit/distros/debian.py
index 0ad93ff..cf082c7 100644
--- a/cloudinit/distros/debian.py
+++ b/cloudinit/distros/debian.py
@@ -29,9 +29,10 @@ APT_GET_WRAPPER = {
 'enabled': 'auto',
 }
 
-ENI_HEADER = """# This file is generated from information provided by
-# the datasource.  Changes to it will not persist across an instance.
-# To disable cloud-init's network configuration capabilities, write a file
+NETWORK_FILE_HEADER = """\
+# This file is generated from information provided by the datasource.  Changes
+# to it will not persist across an instance reboot.  To disable cloud-init's
+# network configuration capabilities, write a file
 # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
 # network: {config: disabled}
 """
@@ -48,9 +49,9 @@ class Distro(distros.Distro):
 }
 renderer_configs = {
 "eni": {"eni_path": network_conf_fn["eni"],
-"eni_header": ENI_HEADER},
+"eni_header": NETWORK_FILE_HEADER},
 "netplan": {"netplan_path": network_conf_fn["netplan"],
-"netplan_header": ENI_HEADER,
+"netplan_header": NETWORK_FILE_HEADER,
 "postcmds": True}
 }
 
diff --git a/cloudinit/distros/ubuntu.py b/cloudinit/distros/ubuntu.py
index e5fcbc5..23be3bd 100644
--- a/cloudinit/distros/ubuntu.py
+++ b/cloudinit/distros/ubuntu.py
@@ -30,9 +30,9 @@ class Distro(debian.Distro):
 }
 self.renderer_configs = {
 "eni": {"eni_path": self.network_conf_fn["eni"],
-"eni_header": debian.ENI_HEADER},
+"eni_header": debian.NETWORK_FILE_HEADER},
 "netplan": {"netplan_path": self.network_conf_fn["netplan"],
-"netplan_header": debian.ENI_HEADER,
+"netplan_header": debian.NETWORK_FILE_HEADER,
 "postcmds": True}
 }
 
diff --git a/tests/unittests/test_distros/test_netconfig.py b/tests/unittests/test_distros/test_netconfig.py
index 07b5c0a..6720995 100644
--- a/tests/unittests/test_distros/test_netconfig.py
+++ b/tests/unittests/test_distros/test_netconfig.py
@@ -91,9 +91,9 @@ V1_NET_CFG = {'config': [{'name': 'eth0',
   'version': 1}
 
 V1_NET_CFG_OUTPUT = """\
-# This file is generated from information provided by
-# the datasource.  Changes to it will not persist across an instance.
-# To disable cloud-init's network configuration capabilities, write a file
+# This file is generated from information provided by the datasource.  Changes
+# to it will not persist across an instance reboot.  To disable cloud-init's
+# network configuration capabilities, write a file
 # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
 # network: {config: disabled}
 auto lo
@@ -125,9 +125,9 @@ V1_NET_CFG_IPV6 = {'config': [{'name': 'eth0',
 
 
 V1_TO_V2_NET_CFG_OUTPUT = """\
-# This file is generated from information provided by
-# the datasource.  Changes to it will not persist across an instance.
-# To disable cloud-init's network configuration capabilities, write a file
+# This file is generated from information provided by the datasource.  Changes
+# to it will not persist across an instance reboot.  To disable cloud-init's
+# network configuration capabilities, write a file
 # /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
 # network: {config: disabled}
 network:
@@ -154,9 +154,9 @@ V2_NET_CFG = {
 
 
 V2_TO_V2_NET_CFG_OUTPUT = """\
-# This file is generated from information provided by
-# the datasource.  Changes to it will not persist across an instance.
-# To disable cloud-init's network configuration capabilities

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:whitespace into cloud-init:master

2019-09-25 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:whitespace into cloud-init:master.

Commit message:
analyze/show: remove trailing space in output

Requested reviews:
  cloud-init Commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373228
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:whitespace into cloud-init:master.
diff --git a/cloudinit/analyze/show.py b/cloudinit/analyze/show.py
index 511b808..fb152b1 100644
--- a/cloudinit/analyze/show.py
+++ b/cloudinit/analyze/show.py
@@ -349,7 +349,7 @@ def generate_records(events, blame_sort=False,
 if event_name(event) == event_name(prev_evt):
 record = event_record(start_time, prev_evt, event)
 records.append(format_record("Finished stage: "
- "(%n) %d seconds ",
+ "(%n) %d seconds",
  record) + "\n")
 total_time += record.get('delta')
 else:
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ci_fix into cloud-init:master

2019-09-25 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ci_fix into cloud-init:master.

Commit message:
.pylintrc: ignore missing enter_context members

Pylint has started (incorrectly) warning about these in some test cases,
so unbreak the build by ignoring them.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/373221
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ci_fix into cloud-init:master.
diff --git a/.pylintrc b/.pylintrc
index e376b48..365c8c8 100644
--- a/.pylintrc
+++ b/.pylintrc
@@ -67,5 +67,5 @@ ignored-classes=argparse.Namespace,optparse.Values,thread._local
 # List of members which are set dynamically and missed by pylint inference
 # system, and so shouldn't trigger E1101 when accessed. Python regular
 # expressions are accepted.
-generated-members=types,http.client,command_handlers,m_.*
+generated-members=types,http.client,command_handlers,m_.*,enter_context
 
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into cloud-init:ubuntu/devel

2019-09-17 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.

Requested reviews:
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1844191 in cloud-init: "azure advanced networking sometimes triggers 
duplicate mac detection"
  https://bugs.launchpad.net/cloud-init/+bug/1844191

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372864
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.
diff --git a/.gitignore b/.gitignore
index 80c509e..b9b98e7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -12,3 +12,14 @@ stage
 *.snap
 *.cover
 .idea/
+
+# Ignore packaging artifacts
+cloud-init.dsc
+cloud-init_*.build
+cloud-init_*.buildinfo
+cloud-init_*.changes
+cloud-init_*.deb
+cloud-init_*.dsc
+cloud-init_*.orig.tar.gz
+cloud-init_*.tar.xz
+cloud-init_*.upload
diff --git a/Makefile b/Makefile
index 4ace227..2c6d0c8 100644
--- a/Makefile
+++ b/Makefile
@@ -106,7 +106,9 @@ deb-src:
 		  echo sudo apt-get install devscripts; exit 1; }
 	$(PYVER) ./packages/bddeb -S -d
 
+doc:
+	tox -e doc
 
 .PHONY: test pyflakes pyflakes3 clean pep8 rpm srpm deb deb-src yaml
 .PHONY: check_version pip-test-requirements pip-requirements clean_pyc
-.PHONY: unittest unittest3 style-check
+.PHONY: unittest unittest3 style-check doc
diff --git a/cloudinit/atomic_helper.py b/cloudinit/atomic_helper.py
index 587b994..1f61faa 100644
--- a/cloudinit/atomic_helper.py
+++ b/cloudinit/atomic_helper.py
@@ -1,11 +1,13 @@
 # This file is part of cloud-init. See LICENSE file for license information.
 
 import json
+import logging
 import os
 import stat
 import tempfile
 
 _DEF_PERMS = 0o644
+LOG = logging.getLogger(__name__)
 
 
 def write_file(filename, content, mode=_DEF_PERMS,
@@ -23,6 +25,10 @@ def write_file(filename, content, mode=_DEF_PERMS,
 try:
 tf = tempfile.NamedTemporaryFile(dir=os.path.dirname(filename),
  delete=False, mode=omode)
+LOG.debug(
+"Atomically writing to file %s (via temporary file %s) - %s: [%o]"
+" %d bytes/chars",
+filename, tf.name, omode, mode, len(content))
 tf.write(content)
 tf.close()
 os.chmod(tf.name, mode)
diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
index ea707c0..5de5c6d 100644
--- a/cloudinit/net/__init__.py
+++ b/cloudinit/net/__init__.py
@@ -109,6 +109,127 @@ def is_bond(devname):
 return os.path.exists(sys_dev_path(devname, "bonding"))
 
 
+def has_master(devname):
+return os.path.exists(sys_dev_path(devname, path="master"))
+
+
+def is_netfailover(devname, driver=None):
+""" netfailover driver uses 3 nics, master, primary and standby.
+this returns True if the device is either the primary or standby
+as these devices are to be ignored.
+"""
+if driver is None:
+driver = device_driver(devname)
+if is_netfail_primary(devname, driver) or is_netfail_standby(devname,
+ driver):
+return True
+return False
+
+
+def get_dev_features(devname):
+""" Returns a str from reading /sys/class/net//device/features."""
+features = ''
+try:
+features = read_sys_net(devname, 'device/features')
+except Exception:
+pass
+return features
+
+
+def has_netfail_standby_feature(devname):
+""" Return True if VIRTIO_NET_F_STANDBY bit (62) is set.
+
+https://github.com/torvalds/linux/blob/ \
+089cf7f6ecb266b6a4164919a2e69bd2f938374a/ \
+include/uapi/linux/virtio_net.h#L60
+"""
+features = get_dev_features(devname)
+if not features or len(features) < 64:
+return False
+return features[62] == "1"
+
+
+def is_netfail_master(devname, driver=None):
+""" A device is a "netfail master" device if:
+
+- The device does NOT have the 'master' sysfs attribute
+- The device driver is 'virtio_net'
+- The device has the standby feature bit set
+
+Return True if all of the above is True.
+"""
+if has_master(devname):
+return False
+
+if driver is None:
+driver = device_driver(devname)
+
+if driver != "virtio_net":
+return False
+
+if not has_netfail_standby_feature(devname):
+return False
+
+return True
+
+
+def is_netfail_primary(devname, driver=None):
+""" A device is a "netfail primary" device if:
+
+- the device has a 'master' sysfs file
+- the device driver is not 'virtio_net'
+- the 'master' sysfs file points to devic

Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master

2019-09-09 Thread Dan Watkins
Updated.

(CI failures are due to an infra problem, which we've hopefully just fixed.)
-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372491
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master

2019-09-09 Thread Dan Watkins
OK, I've updated the log message to indicate we're writing to a temporary file, 
and added a log message for the rename (because I think failures could happen 
in either of those steps, and this allows us to distinguish between those).  
(We will also have the different logger name to distinguish between the two log 
messages, which is why I didn't push even more context into the log message 
itself.)

Thanks for the review, Scott!
-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372491
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master

2019-09-09 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master.

Commit message:
atomic_helper: add DEBUG logging to write_file

Fixes LP: #1843276

Requested reviews:
  Server Team CI bot (server-team-bot): continuous-integration
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1843276 in cloud-init: "cloudinit.atomic_helper.write_file should have 
the same logging as util.write_file"
  https://bugs.launchpad.net/cloud-init/+bug/1843276

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372491
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master.
diff --git a/cloudinit/atomic_helper.py b/cloudinit/atomic_helper.py
index 587b994..077a780 100644
--- a/cloudinit/atomic_helper.py
+++ b/cloudinit/atomic_helper.py
@@ -1,11 +1,13 @@
 # This file is part of cloud-init. See LICENSE file for license information.
 
 import json
+import logging
 import os
 import stat
 import tempfile
 
 _DEF_PERMS = 0o644
+LOG = logging.getLogger(__name__)
 
 
 def write_file(filename, content, mode=_DEF_PERMS,
@@ -23,9 +25,12 @@ def write_file(filename, content, mode=_DEF_PERMS,
 try:
 tf = tempfile.NamedTemporaryFile(dir=os.path.dirname(filename),
  delete=False, mode=omode)
+LOG.debug("Writing to temporary file %s - %s: [%s] %s bytes/chars",
+  tf.name, omode, mode, len(content))
 tf.write(content)
 tf.close()
 os.chmod(tf.name, mode)
+LOG.debug("Renaming temporary file %s to %s", tf.name, filename)
 os.rename(tf.name, filename)
 except Exception as e:
 if tf is not None:
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master

2019-09-09 Thread Dan Watkins
Oh yeah, great points.  Will fix.
-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372491
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master

2019-09-09 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master.

Commit message:
atomic_helper: add DEBUG logging to write_file

Fixes LP: #1843276

Requested reviews:
  cloud-init Commiters (cloud-init-dev)
Related bugs:
  Bug #1843276 in cloud-init: "cloudinit.atomic_helper.write_file should have 
the same logging as util.write_file"
  https://bugs.launchpad.net/cloud-init/+bug/1843276

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372491
-- 
Your team cloud-init Commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lp1843276 into cloud-init:master.
diff --git a/cloudinit/atomic_helper.py b/cloudinit/atomic_helper.py
index 587b994..a59739b 100644
--- a/cloudinit/atomic_helper.py
+++ b/cloudinit/atomic_helper.py
@@ -1,11 +1,13 @@
 # This file is part of cloud-init. See LICENSE file for license information.
 
 import json
+import logging
 import os
 import stat
 import tempfile
 
 _DEF_PERMS = 0o644
+LOG = logging.getLogger(__name__)
 
 
 def write_file(filename, content, mode=_DEF_PERMS,
@@ -23,6 +25,8 @@ def write_file(filename, content, mode=_DEF_PERMS,
 try:
 tf = tempfile.NamedTemporaryFile(dir=os.path.dirname(filename),
  delete=False, mode=omode)
+LOG.debug("Writing to %s - %s: [%s] %s bytes/chars",
+  tf.name, omode, mode, len(content))
 tf.write(content)
 tf.close()
 os.chmod(tf.name, mode)
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master

2019-09-04 Thread Dan Watkins
The proposal to merge ~daniel-thewatkins/cloud-init/+git/cloud-init:networking 
into cloud-init:master has been updated.

Status: Work in progress => Rejected

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372289
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master

2019-09-04 Thread Dan Watkins
On Wed, Sep 04, 2019 at 06:30:07PM -, Ryan Harper wrote:
> If the interface already has an IP, it may not have anything else that
> we'd apply after parsing the lease output (routes, etc).

If the interface already has an IP before we DHCP, it's not 100% clear
to me that we should consider routes etc. from the DHCP response
applicable.

> This sort of feels OK in that we skip tearing down when we don't do
> anything else, but it also seems wrong in that the whole context
> manager should exit if we don't bring up DHCP.

What do you mean by "should exit" here?  The way we use the context
manager (in DataSourceOpenStack, at least), is essentially "ensure I
have network while executing the body", so not needing to configure
networking isn't really an error condition in that usecase.

-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/372289
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~powersj/cloud-init:docs/config-tox into cloud-init:master

2019-09-04 Thread Dan Watkins



Diff comments:

> diff --git a/doc/rtd/topics/docs.rst b/doc/rtd/topics/docs.rst
> new file mode 100644
> index 000..af2a3fb
> --- /dev/null
> +++ b/doc/rtd/topics/docs.rst
> @@ -0,0 +1,83 @@
> +.. _docs:
> +
> +Docs
> +
> +
> +These docs are hosted on Read the Docs. The following will explain how to
> +contribute to and build these docs locally.
> +
> +The documentation is primarily written in reStructuredText.
> +
> +
> +Building
> +
> +
> +There is a makefile target to build the documentation for you:
> +
> +.. code-block:: shell
> +
> +$ make doc
> +
> +This will do two things:
> +
> +- Build the documentation using sphinx
> +- Run doc8 against the documentation source code
> +
> +Once build the HTML files will be viewable in ``doc/rtd_html``. Use your
> +web browser to open ``index.html`` to view and navigate the site.
> +
> +Style Guide
> +===
> +
> +Headings
> +
> +The headings used across the documentation use the following hierarchy:
> +
> +- ``*``: used once atop of a new page
> +- ``=``: each sections on the page
> +- ``-``: subsections
> +- ``^``: sub-subsections
> +- ``"``: paragraphs
> +
> +The top level header ``##`` is reserved for the first page.
> +
> +If under and overline are used, their length must be identical. The length of
> +the underline must be at least as long as the title itself
> +
> +Line Length
> +---
> +Please keep the line lengths to a maximum of **79** characters. This ensures
> +that the pages and tables do not get too wide that side scrolling is 
> required.
> +
> +Header
> +--
> +Adding a link at the top of the page allows for the page to be referenced by
> +other pages. For example for the FAQ page this would be:
> +
> +.. code-block:: rst
> +
> +.. _faq:
> +
> +Footer
> +--
> +The footer should include the textwidth
> +
> +.. code-block:: rst
> +
> +.. vi: textwidth=80

Should this be 79?

> +
> +Vertical Whitespace
> +---
> +One newline between each section helps ensure readability of the 
> documentation
> +source code.
> +
> +Common Words
> +
> +There are some common words that should follow specific usage:
> +
> +- ``cloud-init``: always lower case with a hyphen unless starting a sentence
> +- ``metadata``: one word
> +- ``user data``: two words, not to be combined
> +- ``vendor data``: like user data, it is two words
> +
> +.. vi: textwidth=80


-- 
https://code.launchpad.net/~powersj/cloud-init/+git/cloud-init/+merge/372102
Your team cloud-init commiters is requested to review the proposed merge of 
~powersj/cloud-init:docs/config-tox into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:gitignore into cloud-init:master

2019-08-22 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:gitignore into cloud-init:master.

Commit message:
.gitignore: ignore files produced by package builds

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371683
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:gitignore into cloud-init:master.
diff --git a/.gitignore b/.gitignore
index 80c509e..b9b98e7 100644
--- a/.gitignore
+++ b/.gitignore
@@ -12,3 +12,14 @@ stage
 *.snap
 *.cover
 .idea/
+
+# Ignore packaging artifacts
+cloud-init.dsc
+cloud-init_*.build
+cloud-init_*.buildinfo
+cloud-init_*.changes
+cloud-init_*.deb
+cloud-init_*.dsc
+cloud-init_*.orig.tar.gz
+cloud-init_*.tar.xz
+cloud-init_*.upload
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:dracut into cloud-init:master

2019-08-22 Thread Dan Watkins



Diff comments:

> diff --git a/cloudinit/net/cmdline.py b/cloudinit/net/cmdline.py
> index 556a10f..c05d36c 100755
> --- a/cloudinit/net/cmdline.py
> +++ b/cloudinit/net/cmdline.py
> @@ -137,6 +203,24 @@ def config_from_klibc_net_cfg(files=None, 
> mac_addrs=None):
>  return {'config': entries, 'version': 1}
>  
>  
> +def read_initramfs_config():
> +"""
> +Return v1 network config for initramfs-configured networking (or None)
> +
> +This will consider each _INITRAMFS_CONFIG_SOURCES entry in turn, and 
> return
> +v1 network configuration for the first one that is applicable.  If none 
> are
> +applicable, return None.
> +"""
> +for src_cls in _INITRAMFS_CONFIG_SOURCES:

Yeah, good thinking.  I'll refactor bits up as I find them in the dracut work.  
(Because, for example, the open-iscsi check _doesn't_ apply to dracut.  They do 
use open-iscsi, but that file isn't present.)

> +cfg_source = src_cls()
> +
> +if not cfg_source.is_applicable():
> +continue
> +
> +return cfg_source.render_config()
> +return None
> +
> +
>  def _decomp_gzip(blob, strict=True):
>  # decompress blob. raise exception if not compressed unless strict=False.
>  with io.BytesIO(blob) as iobuf:


-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371673
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:dracut into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:dracut into cloud-init:master

2019-08-22 Thread Dan Watkins



Diff comments:

> diff --git a/cloudinit/net/cmdline.py b/cloudinit/net/cmdline.py
> index 556a10f..c05d36c 100755
> --- a/cloudinit/net/cmdline.py
> +++ b/cloudinit/net/cmdline.py
> @@ -137,6 +203,24 @@ def config_from_klibc_net_cfg(files=None, 
> mac_addrs=None):
>  return {'config': entries, 'version': 1}
>  
>  
> +def read_initramfs_config():
> +"""
> +Return v1 network config for initramfs-configured networking (or None)

At the very least the Oracle DS is relying on this being v1, so it can merge 
secondary NIC config into it.  So this _does_ need to be v1 until we lift up 
everything to v2 (and even then it will always need to be consistent across 
InitramfsNetworkConfigSources).

> +
> +This will consider each _INITRAMFS_CONFIG_SOURCES entry in turn, and 
> return
> +v1 network configuration for the first one that is applicable.  If none 
> are
> +applicable, return None.
> +"""
> +for src_cls in _INITRAMFS_CONFIG_SOURCES:

In KlibcNetworkConfigSource.is_applicable, we check for the klibc-written files 
before we even look at the kernel cmdline.  So if there aren't any 
/run/{net,net6}-*.conf files, we won't select KlibcNetworkConfigSource.

(I'll improve the docstring on KlibcNetworkConfigSource.is_applicable, because 
I _also_ thought this was going to be a problem for a while and clearly it 
wasn't just me. :)

> +cfg_source = src_cls()
> +
> +if not cfg_source.is_applicable():
> +continue
> +
> +return cfg_source.render_config()
> +return None
> +
> +
>  def _decomp_gzip(blob, strict=True):
>  # decompress blob. raise exception if not compressed unless strict=False.
>  with io.BytesIO(blob) as iobuf:


-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371673
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:dracut into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:dracut into cloud-init:master

2019-08-22 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:dracut into cloud-init:master.

Commit message:
net/cmdline: refactor to allow multiple initramfs network config sources

This refactors read_initramfs_config to support multiple different types
of initramfs network configuration.  It introduces an
InitramfsNetworkConfigSource abstract base class.  There is currently a
single sub-class, KlibcNetworkConfigSource, which contains the logic
which previously was directly within read_initramfs_config.

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371673
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:dracut into cloud-init:master.
diff --git a/cloudinit/net/cmdline.py b/cloudinit/net/cmdline.py
index 556a10f..c05d36c 100755
--- a/cloudinit/net/cmdline.py
+++ b/cloudinit/net/cmdline.py
@@ -5,20 +5,86 @@
 #
 # This file is part of cloud-init. See LICENSE file for license information.
 
+import abc
 import base64
 import glob
 import gzip
 import io
 import os
 
-from . import get_devicelist
-from . import read_sys_net_safe
+import six
 
 from cloudinit import util
 
+from . import get_devicelist
+from . import read_sys_net_safe
+
 _OPEN_ISCSI_INTERFACE_FILE = "/run/initramfs/open-iscsi.interface"
 
 
+@six.add_metaclass(abc.ABCMeta)
+class InitramfsNetworkConfigSource(object):
+"""ABC for net config sources that read config written by initramfses"""
+
+@abc.abstractmethod
+def is_applicable(self):
+# type: () -> bool
+"""Is this initramfs config source applicable to the current system?"""
+pass
+
+@abc.abstractmethod
+def render_config(self):
+# type: () -> dict
+"""Render a v1 network config from the initramfs configuration"""
+pass
+
+
+class KlibcNetworkConfigSource(InitramfsNetworkConfigSource):
+"""InitramfsNetworkConfigSource for klibc initramfs (i.e. Debian/Ubuntu)
+
+Has three parameters, but they are intended to make testing simpler, _not_
+for use in production code.  (This is indicated by the prepended
+underscores.)
+"""
+
+def __init__(self, _files=None, _mac_addrs=None, _cmdline=None):
+self._files = _files
+self._mac_addrs = _mac_addrs
+self._cmdline = _cmdline
+
+# Set defaults here, as they require computation that we don't want to
+# do at method definition time
+if self._files is None:
+self._files = _get_klibc_net_cfg_files()
+if self._cmdline is None:
+self._cmdline = util.get_cmdline()
+if self._mac_addrs is None:
+self._mac_addrs = {}
+for k in get_devicelist():
+mac_addr = read_sys_net_safe(k, 'address')
+if mac_addr:
+self._mac_addrs[k] = mac_addr
+
+def is_applicable(self):
+# type: () -> bool
+if self._files:
+if 'ip=' in self._cmdline or 'ip6=' in self._cmdline:
+return True
+if os.path.exists(_OPEN_ISCSI_INTERFACE_FILE):
+# iBft can configure networking without ip=
+return True
+return False
+
+def render_config(self):
+# type: () -> dict
+return config_from_klibc_net_cfg(
+files=self._files, mac_addrs=self._mac_addrs,
+)
+
+
+_INITRAMFS_CONFIG_SOURCES = [KlibcNetworkConfigSource]
+
+
 def _klibc_to_config_entry(content, mac_addrs=None):
 """Convert a klibc written shell content file to a 'config' entry
 When ip= is seen on the kernel command line in debian initramfs
@@ -137,6 +203,24 @@ def config_from_klibc_net_cfg(files=None, mac_addrs=None):
 return {'config': entries, 'version': 1}
 
 
+def read_initramfs_config():
+"""
+Return v1 network config for initramfs-configured networking (or None)
+
+This will consider each _INITRAMFS_CONFIG_SOURCES entry in turn, and return
+v1 network configuration for the first one that is applicable.  If none are
+applicable, return None.
+"""
+for src_cls in _INITRAMFS_CONFIG_SOURCES:
+cfg_source = src_cls()
+
+if not cfg_source.is_applicable():
+continue
+
+return cfg_source.render_config()
+return None
+
+
 def _decomp_gzip(blob, strict=True):
 # decompress blob. raise exception if not compressed unless strict=False.
 with io.BytesIO(blob) as iobuf:
@@ -167,36 +251,6 @@ def _b64dgz(b64str, gzipped="try"):
 return _decomp_gzip(blob, strict=gzipped != "try")
 
 
-def _is_initramfs_netconfig(files, cmdline):
-if files:
-if 'ip=' 

[Cloud-init-dev] [Merge] ~chad.smith/cloud-init:bug/1840080-ubuntu-drivers-emit-latelink-v2 into cloud-init:master

2019-08-22 Thread Dan Watkins
The proposal to merge 
~chad.smith/cloud-init:bug/1840080-ubuntu-drivers-emit-latelink-v2 into 
cloud-init:master has been updated.

Status: Needs review => Approved

For more details, see:
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/371546
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~chad.smith/cloud-init:bug/1840080-ubuntu-drivers-emit-latelink-v2 into 
cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~chad.smith/cloud-init:bug/1840080-ubuntu-drivers-emit-latelink-v2 into cloud-init:master

2019-08-22 Thread Dan Watkins
The proposal to merge 
~chad.smith/cloud-init:bug/1840080-ubuntu-drivers-emit-latelink-v2 into 
cloud-init:master has been updated.

Status: Approved => Needs review

For more details, see:
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/371546
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~chad.smith/cloud-init:bug/1840080-ubuntu-drivers-emit-latelink-v2 into 
cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~chad.smith/cloud-init:bug/1840080-ubuntu-drivers-emit-latelink-v2 into cloud-init:master

2019-08-22 Thread Dan Watkins



Diff comments:

> diff --git a/cloudinit/config/cc_ubuntu_drivers.py 
> b/cloudinit/config/cc_ubuntu_drivers.py
> index 4da34ee..ff56775 100644
> --- a/cloudinit/config/cc_ubuntu_drivers.py
> +++ b/cloudinit/config/cc_ubuntu_drivers.py
> @@ -90,17 +118,28 @@ def install_drivers(cfg, pkg_install_func):
>  if version_cfg:
>  driver_arg += ':{}'.format(version_cfg)
>  
> -LOG.debug("Installing NVIDIA drivers (%s=%s, version=%s)",
> +LOG.debug("Installing and activating NVIDIA drivers (%s=%s, version=%s)",
>cfgpath, nv_acc, version_cfg if version_cfg else 'latest')
>  
> -# Setting NVIDIA latelink confirms acceptance of EULA for the package
> -# linux-restricted-modules
> -# Reference code defining debconf variable is here
> -# https://git.launchpad.net/~ubuntu-kernel/ubuntu/+source/
> -# linux-restricted-modules/+git/eoan/tree/debian/templates/
> -# nvidia.templates.in
> -selections = b'linux-restricted-modules linux/nvidia/latelink boolean 
> true'
> -cc_apt_configure.debconf_set_selections(selections)
> +# Register and set debconf selection linux/nvidia/latelink = true
> +tdir = temp_utils.mkdtemp(needs_exe=True)
> +debconf_file = os.path.join(tdir, 'nvidia.template')
> +debconf_script = os.path.join(tdir, 'nvidia-debconf.sh')
> +try:
> +util.write_file(debconf_file, NVIDIA_DEBCONF_CONTENT)

This indentation looks off to me.

> +util.write_file(
> +debconf_script,
> +util.encode_text(NVIDIA_DRIVER_LATELINK_DEBCONF_SCRIPT),
> +mode=0o755)
> +util.subp([debconf_script, debconf_file])
> +except Exception as e:
> +util.logexc(
> +LOG,
> +"Failed to register NVIDIA debconf template: %s", str(e))
> +raise
> +finally:
> +if os.path.isdir(tdir):
> +util.del_dir(tdir)
>  
>  try:
>  util.subp(['ubuntu-drivers', 'install', '--gpgpu', driver_arg])


-- 
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/371546
Your team cloud-init commiters is requested to review the proposed merge of 
~chad.smith/cloud-init:bug/1840080-ubuntu-drivers-emit-latelink-v2 into 
cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master

2019-08-16 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master.

Commit message:
DataSourceOracle: prefer DS network config over initramfs

DataSourceOracle.network_config merges the initramfs configuration with
configuration from OCI's IMDS, so prefer data source network config over
the initramfs network config.

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371403
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceOracle.py b/cloudinit/sources/DataSourceOracle.py
index 086af79..6e73f56 100644
--- a/cloudinit/sources/DataSourceOracle.py
+++ b/cloudinit/sources/DataSourceOracle.py
@@ -109,6 +109,13 @@ class DataSourceOracle(sources.DataSource):
 dsname = 'Oracle'
 system_uuid = None
 vendordata_pure = None
+network_config_sources = (
+sources.NetworkConfigSource.cmdline,
+sources.NetworkConfigSource.ds,
+sources.NetworkConfigSource.initramfs,
+sources.NetworkConfigSource.system_cfg,
+)
+
 _network_config = sources.UNSET
 
 def __init__(self, sys_cfg, *args, **kwargs):
diff --git a/cloudinit/sources/tests/test_oracle.py b/cloudinit/sources/tests/test_oracle.py
index 3e14677..3ddf7df 100644
--- a/cloudinit/sources/tests/test_oracle.py
+++ b/cloudinit/sources/tests/test_oracle.py
@@ -1,7 +1,7 @@
 # This file is part of cloud-init. See LICENSE file for license information.
 
 from cloudinit.sources import DataSourceOracle as oracle
-from cloudinit.sources import BrokenMetadata
+from cloudinit.sources import BrokenMetadata, NetworkConfigSource
 from cloudinit import helpers
 
 from cloudinit.tests import helpers as test_helpers
@@ -303,6 +303,14 @@ class TestDataSourceOracle(test_helpers.CiTestCase):
 self.assertIn('Failed to fetch secondary network configuration',
   self.logs.getvalue())
 
+def test_ds_network_cfg_preferred_over_initramfs(self):
+"""Ensure that DS net config is preferred over initramfs config"""
+network_config_sources = oracle.DataSourceOracle.network_config_sources
+self.assertLess(
+network_config_sources.index(NetworkConfigSource.ds),
+network_config_sources.index(NetworkConfigSource.initramfs)
+)
+
 
 @mock.patch(DS_PATH + "._read_system_uuid", return_value=str(uuid.uuid4()))
 class TestReadMetaData(test_helpers.HttprettyTestCase):
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:pr_template into cloud-init:master

2019-08-15 Thread Dan Watkins
Good idea, done.
-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371350
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:pr_template into 
cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master

2019-08-15 Thread Dan Watkins
The proposal to merge ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into 
cloud-init:master has been updated.

Commit message changed to:

cloudinit/distros/parsers/sys_conf: add docstring to SysConf

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371363
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master

2019-08-15 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371363
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.
diff --git a/cloudinit/distros/parsers/sys_conf.py b/cloudinit/distros/parsers/sys_conf.py
index c27b5d5..44df17d 100644
--- a/cloudinit/distros/parsers/sys_conf.py
+++ b/cloudinit/distros/parsers/sys_conf.py
@@ -43,6 +43,13 @@ def _contains_shell_variable(text):
 
 
 class SysConf(configobj.ConfigObj):
+"""A configobj.ConfigObj subclass specialised for sysconfig files.
+
+:param contents:
+The sysconfig file to parse, in a format accepted by
+``configobj.ConfigObj.__init__`` (i.e. "a filename, file like object,
+or list of lines").
+"""
 def __init__(self, contents):
 configobj.ConfigObj.__init__(self, contents,
  interpolation=False,
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:pr_template into cloud-init:master

2019-08-15 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:pr_template into 
cloud-init:master.

Commit message:
Add GitHub pull request template to point people at hacking doc

This will hopefully avoid people submitting PRs against our mirror repo.

(Thanks to GitHub user @max06 for this suggestion!)


Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371350
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:pr_template into 
cloud-init:master.
diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md
new file mode 100644
index 000..56cbed4
--- /dev/null
+++ b/.github/pull_request_template.md
@@ -0,0 +1,8 @@
+***This GitHub repo is only a mirror.  Do not submit pull requests
+here!***
+
+Thank you for taking the time to write and submit a change to
+cloud-init!   Please follow [our hacking
+guide](https://cloudinit.readthedocs.io/en/latest/topics/hacking.html)
+to submit your change to cloud-init's Launchpad git repository, where
+cloud-init development happens.
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~florian-mueller-v/cloud-init:docs-fix into cloud-init:master

2019-08-14 Thread Dan Watkins
Hey Florian, thanks for submitting this!  It looks like you haven't signed the 
CLA; please check out the instructions at 
https://cloudinit.readthedocs.io/en/latest/topics/hacking.html for how to do 
so.  Thanks!
-- 
https://code.launchpad.net/~florian-mueller-v/cloud-init/+git/cloud-init/+merge/371298
Your team cloud-init commiters is requested to review the proposed merge of 
~florian-mueller-v/cloud-init:docs-fix into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:useradd into cloud-init:master

2019-08-12 Thread Dan Watkins
On Mon, Aug 12, 2019 at 02:31:18PM -, Ryan Harper wrote:
> The method is call add_user and the variables match the method.

If the variable matched the method, I'd expect it to be add_user_cmd.
I'd be fine with that, if that's what you would prefer?

> So now we have
> 
> def add_user()
>useradd_cmd = 
> 
> Isn't this the same confusion?

I don't think so; the name of the command we do actually use is
hardcoded in a string literal later on that line:

useradd_cmd = ['useradd', ...]

> The command to add a user could be something else.

Not without code changes, which I would expect to also change the
variable name.

> In cloudinit/distros/freebsd.py for example, the adduser_cmd , is 'pw
> useradd'.

Right, I didn't touch the variable name there.

-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371203
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:useradd into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:useradd into cloud-init:master

2019-08-12 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:useradd into cloud-init:master.

Commit message:
distros: fix confusing variable names

Building the subp arguments for a `useradd` call in a variable named
`adduser_cmd` is extremely confusing; let's not do that.

(This also changes the snap variable to something more apropos.)


Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371203
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:useradd into cloud-init:master.
diff --git a/cloudinit/distros/__init__.py b/cloudinit/distros/__init__.py
index 20c994d..00bdee3 100644
--- a/cloudinit/distros/__init__.py
+++ b/cloudinit/distros/__init__.py
@@ -396,16 +396,16 @@ class Distro(object):
 else:
 create_groups = True
 
-adduser_cmd = ['useradd', name]
-log_adduser_cmd = ['useradd', name]
+useradd_cmd = ['useradd', name]
+log_useradd_cmd = ['useradd', name]
 if util.system_is_snappy():
-adduser_cmd.append('--extrausers')
-log_adduser_cmd.append('--extrausers')
+useradd_cmd.append('--extrausers')
+log_useradd_cmd.append('--extrausers')
 
 # Since we are creating users, we want to carefully validate the
 # inputs. If something goes wrong, we can end up with a system
 # that nobody can login to.
-adduser_opts = {
+useradd_opts = {
 "gecos": '--comment',
 "homedir": '--home',
 "primary_group": '--gid',
@@ -418,7 +418,7 @@ class Distro(object):
 "selinux_user": '--selinux-user',
 }
 
-adduser_flags = {
+useradd_flags = {
 "no_user_group": '--no-user-group',
 "system": '--system',
 "no_log_init": '--no-log-init',
@@ -453,32 +453,32 @@ class Distro(object):
 # Check the values and create the command
 for key, val in sorted(kwargs.items()):
 
-if key in adduser_opts and val and isinstance(val, str):
-adduser_cmd.extend([adduser_opts[key], val])
+if key in useradd_opts and val and isinstance(val, str):
+useradd_cmd.extend([useradd_opts[key], val])
 
 # Redact certain fields from the logs
 if key in redact_opts:
-log_adduser_cmd.extend([adduser_opts[key], 'REDACTED'])
+log_useradd_cmd.extend([useradd_opts[key], 'REDACTED'])
 else:
-log_adduser_cmd.extend([adduser_opts[key], val])
+log_useradd_cmd.extend([useradd_opts[key], val])
 
-elif key in adduser_flags and val:
-adduser_cmd.append(adduser_flags[key])
-log_adduser_cmd.append(adduser_flags[key])
+elif key in useradd_flags and val:
+useradd_cmd.append(useradd_flags[key])
+log_useradd_cmd.append(useradd_flags[key])
 
 # Don't create the home directory if directed so or if the user is a
 # system user
 if kwargs.get('no_create_home') or kwargs.get('system'):
-adduser_cmd.append('-M')
-log_adduser_cmd.append('-M')
+useradd_cmd.append('-M')
+log_useradd_cmd.append('-M')
 else:
-adduser_cmd.append('-m')
-log_adduser_cmd.append('-m')
+useradd_cmd.append('-m')
+log_useradd_cmd.append('-m')
 
 # Run the command
 LOG.debug("Adding user %s", name)
 try:
-util.subp(adduser_cmd, logstring=log_adduser_cmd)
+util.subp(useradd_cmd, logstring=log_useradd_cmd)
 except Exception as e:
 util.logexc(LOG, "Failed to create user %s", name)
 raise e
@@ -490,15 +490,15 @@ class Distro(object):
 
 snapuser = kwargs.get('snapuser')
 known = kwargs.get('known', False)
-adduser_cmd = ["snap", "create-user", "--sudoer", "--json"]
+create_user_cmd = ["snap", "create-user", "--sudoer", "--json"]
 if known:
-adduser_cmd.append("--known")
-adduser_cmd.append(snapuser)
+create_user_cmd.append("--known")
+create_user_cmd.append(snapuser)
 
 # Run the command
 LOG.debug("Adding snap user %s", name)
 try:
-(out, err) = util.subp(adduser_cmd, logstring=adduser_cmd,
+(out, err) = util.subp(create_user_cmd, logstring=create_user_cmd,
capture=True)
 LOG.debug("snap cre

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into cloud-init:ubuntu/devel

2019-08-09 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.

Requested reviews:
  cloud-init commiters (cloud-init-dev)
Related bugs:
  Bug #1838794 in cloud-init: "Set Passwords documentation describes incorrect 
behaviour for `password` config key"
  https://bugs.launchpad.net/cloud-init/+bug/1838794

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371135
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.
diff --git a/cloudinit/apport.py b/cloudinit/apport.py
index 22cb7fd..003ff1f 100644
--- a/cloudinit/apport.py
+++ b/cloudinit/apport.py
@@ -23,6 +23,7 @@ KNOWN_CLOUD_NAMES = [
 'CloudStack',
 'DigitalOcean',
 'GCE - Google Compute Engine',
+'Exoscale',
 'Hetzner Cloud',
 'IBM - (aka SoftLayer or BlueMix)',
 'LXD',
diff --git a/cloudinit/config/cc_set_passwords.py b/cloudinit/config/cc_set_passwords.py
index 4585e4d..cf9b5ab 100755
--- a/cloudinit/config/cc_set_passwords.py
+++ b/cloudinit/config/cc_set_passwords.py
@@ -9,27 +9,40 @@
 """
 Set Passwords
 -
-**Summary:** Set user passwords
-
-Set system passwords and enable or disable ssh password authentication.
-The ``chpasswd`` config key accepts a dictionary containing a single one of two
-keys, either ``expire`` or ``list``. If ``expire`` is specified and is set to
-``false``, then the ``password`` global config key is used as the password for
-all user accounts. If the ``expire`` key is specified and is set to ``true``
-then user passwords will be expired, preventing the default system passwords
-from being used.
-
-If the ``list`` key is provided, a list of
-``username:password`` pairs can be specified. The usernames specified
-must already exist on the system, or have been created using the
-``cc_users_groups`` module. A password can be randomly generated using
-``username:RANDOM`` or ``username:R``. A hashed password can be specified
-using ``username:$6$salt$hash``. Password ssh authentication can be
-enabled, disabled, or left to system defaults using ``ssh_pwauth``.
+**Summary:** Set user passwords and enable/disable SSH password authentication
+
+This module consumes three top-level config keys: ``ssh_pwauth``, ``chpasswd``
+and ``password``.
+
+The ``ssh_pwauth`` config key determines whether or not sshd will be configured
+to accept password authentication.  True values will enable password auth,
+false values will disable password auth, and the literal string ``unchanged``
+will leave it unchanged.  Setting no value will also leave the current setting
+on-disk unchanged.
+
+The ``chpasswd`` config key accepts a dictionary containing either or both of
+``expire`` and ``list``.
+
+If the ``list`` key is provided, it should contain a list of
+``username:password`` pairs.  This can be either a YAML list (of strings), or a
+multi-line string with one pair per line.  Each user will have the
+corresponding password set.  A password can be randomly generated by specifying
+``RANDOM`` or ``R`` as a user's password.  A hashed password, created by a tool
+like ``mkpasswd``, can be specified; a regex
+(``r'\\$(1|2a|2y|5|6)(\\$.+){2}'``) is used to determine if a password value
+should be treated as a hash.
 
 .. note::
-if using ``expire: true`` then a ssh authkey should be specified or it may
-not be possible to login to the system
+The users specified must already exist on the system.  Users will have been
+created by the ``cc_users_groups`` module at this point.
+
+By default, all users on the system will have their passwords expired (meaning
+that they will have to be reset the next time the user logs in).  To disable
+this behaviour, set ``expire`` under ``chpasswd`` to a false value.
+
+If a ``list`` of user/password pairs is not specified under ``chpasswd``, then
+the value of the ``password`` config key will be used to set the default user's
+password.
 
 **Internal name:** ``cc_set_passwords``
 
@@ -160,6 +173,8 @@ def handle(_name, cfg, cloud, log, args):
 hashed_users = []
 randlist = []
 users = []
+# N.B. This regex is included in the documentation (i.e. the module
+# docstring), so any changes to it should be reflected there.
 prog = re.compile(r'\$(1|2a|2y|5|6)(\$.+){2}')
 for line in plist:
 u, p = line.split(':', 1)
diff --git a/cloudinit/config/cc_ssh.py b/cloudinit/config/cc_ssh.py
index f8f7cb3..53f6939 100755
--- a/cloudinit/config/cc_ssh.py
+++ b/cloudinit/config/cc_ssh.py
@@ -91,6 +91,9 @@ public keys.
 ssh_authorized_keys:
 - ssh-rsa B3NzaC1yc2EBIwAAAGEA3FSyQwBI6Z+nCSjUU ...
 - ssh-rsa B3NzaC1yc2EBIwAAAQEA3I7VUf2l5gSn5uavROsc5HRDpZ ...
+ssh_publish_hostkeys:
+enabled:  (Defaults to true)
+

Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master

2019-08-08 Thread Dan Watkins
Made some changes locally, still thinking about one other.  (Not pushing them 
up until all comments are addressed, so I don't have to go digging to find the 
current set of comments.)

Diff comments:

> diff --git a/cloudinit/sources/DataSourceOracle.py 
> b/cloudinit/sources/DataSourceOracle.py
> index 76cfa38..f18e5a0 100644
> --- a/cloudinit/sources/DataSourceOracle.py
> +++ b/cloudinit/sources/DataSourceOracle.py
> @@ -28,8 +28,70 @@ import re
>  
>  LOG = logging.getLogger(__name__)
>  
> +BUILTIN_DS_CONFIG = {
> +# Don't use IMDS to configure secondary NICs by default
> +'configure_secondary_nics': False,
> +}
>  CHASSIS_ASSET_TAG = "OracleCloud.com"
>  METADATA_ENDPOINT = "http://169.254.169.254/openstack/;
> +VNIC_METADATA_URL = 'http://169.254.169.254/opc/v1/vnics/'
> +
> +
> +def _network_config_from_opc_imds(network_config):
> +"""
> +Fetch data from Oracle's IMDS and generate secondary NIC config.
> +
> +The primary NIC configuration should not be modified based on the IMDS
> +values, as it should continue to be configured for DHCP.  As such, this
> +takes an existing network_config dict which is expected to have the 
> primary
> +NIC configuration already present.
> +
> +:param network_config:
> +A v1 network config dict with the primary NIC already configured.  
> This
> +dict will be mutated.
> +
> +:raises:
> +Exceptions are not handled within this function.  Likely exceptions 
> are
> +those raised by url_helper.readurl (if communicating with the IMDS
> +fails), ValueError/JSONDecodeError (if the IMDS returns invalid 
> JSON),
> +and KeyError/IndexError (if the IMDS returns valid JSON with 
> unexpected
> +contents).
> +
> +:return:
> +The ``network_config`` dict with secondary NICs added.
> +"""
> +resp = readurl(VNIC_METADATA_URL)
> +vnics = json.loads(str(resp))
> +
> +if 'nicIndex' in vnics[0]:
> +LOG.debug('VNIC metadata indicates this is a bare metal machine;'

I'm now wondering if _network_config_from_opc_imds should return something 
specific (None?) in this case, so that DataSourceOracle.network_config can 
decide what to do with this information (rather than encoding the logging logic 
in here).

(I'm not a huge fan of the mutate-and-return thing that's going on ATM anyway, 
so changing the signature wouldn't be the worst thing in the world.)

I'm going to sleep on this and decide what to do in the morning.

> +  ' skipping secondary VNIC configuration.')
> +return network_config
> +
> +interfaces_by_mac = get_interfaces_by_mac()
> +
> +for vnic_dict in vnics[1:]:
> +mac_address = vnic_dict['macAddr'].lower()
> +if mac_address not in interfaces_by_mac:
> +LOG.info('Interface with MAC %s not found; skipping', 
> mac_address)
> +continue
> +name = interfaces_by_mac[mac_address]
> +subnet = {
> +'type': 'static',
> +'address': vnic_dict['privateIp'],
> +'netmask': vnic_dict['subnetCidrBlock'].split('/')[1],
> +'gateway': vnic_dict['virtualRouterIp'],
> +'control': 'manual',
> +}
> +network_config['config'].append({
> +'name': name,
> +'type': 'physical',
> +'mac_address': mac_address,
> +'mtu': 9000,

Done, locally.

> +'subnets': [subnet],
> +})
> +
> +return network_config
>  
>  
>  class DataSourceOracle(sources.DataSource):
> @@ -121,6 +190,14 @@ class DataSourceOracle(sources.DataSource):
>  self._network_config = cmdline.read_initramfs_config()
>  if not self._network_config:
>  self._network_config = self.distro.generate_fallback_config()
> +if self.ds_cfg.get('configure_secondary_nics'):

Done locally (for this MP, not that bug).

> +try:
> +self._network_config = _network_config_from_opc_imds(
> +self._network_config)
> +except Exception:
> +util.logexc(
> +LOG,
> +"Failed to fetch secondary network configuration!")
>  return self._network_config
>  
>  


-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371053
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master

2019-08-08 Thread Dan Watkins



Diff comments:

> diff --git a/cloudinit/sources/DataSourceOracle.py 
> b/cloudinit/sources/DataSourceOracle.py
> index 76cfa38..f18e5a0 100644
> --- a/cloudinit/sources/DataSourceOracle.py
> +++ b/cloudinit/sources/DataSourceOracle.py
> @@ -121,6 +190,14 @@ class DataSourceOracle(sources.DataSource):
>  self._network_config = cmdline.read_initramfs_config()
>  if not self._network_config:
>  self._network_config = self.distro.generate_fallback_config()
> +if self.ds_cfg.get('configure_secondary_nics'):

https://bugs.launchpad.net/cloud-init/+bug/1839538

> +try:
> +self._network_config = _network_config_from_opc_imds(
> +self._network_config)
> +except Exception:
> +util.logexc(
> +LOG,
> +"Failed to fetch secondary network configuration!")
>  return self._network_config
>  
>  


-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371053
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master

2019-08-08 Thread Dan Watkins
Thanks for the reviews!  Responses inline.

Diff comments:

> diff --git a/cloudinit/sources/DataSourceOracle.py 
> b/cloudinit/sources/DataSourceOracle.py
> index 76cfa38..f18e5a0 100644
> --- a/cloudinit/sources/DataSourceOracle.py
> +++ b/cloudinit/sources/DataSourceOracle.py
> @@ -28,8 +28,70 @@ import re
>  
>  LOG = logging.getLogger(__name__)
>  
> +BUILTIN_DS_CONFIG = {
> +# Don't use IMDS to configure secondary NICs by default
> +'configure_secondary_nics': False,
> +}
>  CHASSIS_ASSET_TAG = "OracleCloud.com"
>  METADATA_ENDPOINT = "http://169.254.169.254/openstack/;
> +VNIC_METADATA_URL = 'http://169.254.169.254/opc/v1/vnics/'
> +
> +
> +def _network_config_from_opc_imds(network_config):
> +"""
> +Fetch data from Oracle's IMDS and generate secondary NIC config.
> +
> +The primary NIC configuration should not be modified based on the IMDS
> +values, as it should continue to be configured for DHCP.  As such, this
> +takes an existing network_config dict which is expected to have the 
> primary
> +NIC configuration already present.
> +
> +:param network_config:
> +A v1 network config dict with the primary NIC already configured.  
> This
> +dict will be mutated.
> +
> +:raises:
> +Exceptions are not handled within this function.  Likely exceptions 
> are
> +those raised by url_helper.readurl (if communicating with the IMDS
> +fails), ValueError/JSONDecodeError (if the IMDS returns invalid 
> JSON),
> +and KeyError/IndexError (if the IMDS returns valid JSON with 
> unexpected
> +contents).
> +
> +:return:
> +The ``network_config`` dict with secondary NICs added.

"This is v1 config, because cloudinit.net.cmdline generates v1 config and we 
need to integrate the secondary NICs into that configuration."  I've filed 
https://bugs.launchpad.net/cloud-init/+bug/1839537 for that migration, and 
included a note about this in there.

(I worked so hard on that commit message and you're telling me you didn't even 
read it? ;)

> +"""
> +resp = readurl(VNIC_METADATA_URL)
> +vnics = json.loads(str(resp))
> +
> +if 'nicIndex' in vnics[0]:
> +LOG.debug('VNIC metadata indicates this is a bare metal machine;'

This is a good question.  Our eventual aim is to have secondary NIC generation 
enabled by default.  In that scenario, I don't think this should be a warning, 
because cloud-init is operating exactly as expected.  However, as it's disabled 
by default at the moment, I agree that a WARNING probably is more appropriate.

(This does make me wonder, though, if this indicates that the configuration 
option name is still too broad.  When we come to implementing this for Bare 
Metal Machines, do we really want the same toggle for both code paths?  I'm 
going to noodle on this some.)

> +  ' skipping secondary VNIC configuration.')
> +return network_config
> +
> +interfaces_by_mac = get_interfaces_by_mac()
> +
> +for vnic_dict in vnics[1:]:
> +mac_address = vnic_dict['macAddr'].lower()
> +if mac_address not in interfaces_by_mac:
> +LOG.info('Interface with MAC %s not found; skipping', 
> mac_address)
> +continue
> +name = interfaces_by_mac[mac_address]
> +subnet = {
> +'type': 'static',
> +'address': vnic_dict['privateIp'],
> +'netmask': vnic_dict['subnetCidrBlock'].split('/')[1],
> +'gateway': vnic_dict['virtualRouterIp'],
> +'control': 'manual',
> +}
> +network_config['config'].append({
> +'name': name,
> +'type': 'physical',
> +'mac_address': mac_address,
> +'mtu': 9000,

That's what The Script uses, and Si-Wei also made sure to call it out.  I've 
also found it in their documentation[0], shall I add a comment pointing there?


[0] 
https://docs.cloud.oracle.com/iaas/Content/Network/Troubleshoot/connectionhang.htm#Overview

> +'subnets': [subnet],
> +})
> +
> +return network_config
>  
>  
>  class DataSourceOracle(sources.DataSource):
> @@ -121,6 +190,14 @@ class DataSourceOracle(sources.DataSource):
>  self._network_config = cmdline.read_initramfs_config()
>  if not self._network_config:
>  self._network_config = self.distro.generate_fallback_config()
> +if self.ds_cfg.get('configure_secondary_nics'):

Yes, I think we should.  Looking around the codebase, there are a few other 
places that should be doing this too, so I'll file a bug for those.  Good catch!

> +try:
> +self._network_config = _network_config_from_opc_imds(
> +self._network_config)
> +except Exception:
> +util.logexc(
> +LOG,
> +"Failed to fetch secondary network configuration!")
>  

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master

2019-08-08 Thread Dan Watkins
You have been requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master.

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371053



-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceOracle.py b/cloudinit/sources/DataSourceOracle.py
index 76cfa38..f18e5a0 100644
--- a/cloudinit/sources/DataSourceOracle.py
+++ b/cloudinit/sources/DataSourceOracle.py
@@ -16,7 +16,7 @@ Notes:
 """
 
 from cloudinit.url_helper import combine_url, readurl, UrlError
-from cloudinit.net import dhcp
+from cloudinit.net import dhcp, get_interfaces_by_mac
 from cloudinit import net
 from cloudinit import sources
 from cloudinit import util
@@ -28,8 +28,70 @@ import re
 
 LOG = logging.getLogger(__name__)
 
+BUILTIN_DS_CONFIG = {
+# Don't use IMDS to configure secondary NICs by default
+'configure_secondary_nics': False,
+}
 CHASSIS_ASSET_TAG = "OracleCloud.com"
 METADATA_ENDPOINT = "http://169.254.169.254/openstack/;
+VNIC_METADATA_URL = 'http://169.254.169.254/opc/v1/vnics/'
+
+
+def _network_config_from_opc_imds(network_config):
+"""
+Fetch data from Oracle's IMDS and generate secondary NIC config.
+
+The primary NIC configuration should not be modified based on the IMDS
+values, as it should continue to be configured for DHCP.  As such, this
+takes an existing network_config dict which is expected to have the primary
+NIC configuration already present.
+
+:param network_config:
+A v1 network config dict with the primary NIC already configured.  This
+dict will be mutated.
+
+:raises:
+Exceptions are not handled within this function.  Likely exceptions are
+those raised by url_helper.readurl (if communicating with the IMDS
+fails), ValueError/JSONDecodeError (if the IMDS returns invalid JSON),
+and KeyError/IndexError (if the IMDS returns valid JSON with unexpected
+contents).
+
+:return:
+The ``network_config`` dict with secondary NICs added.
+"""
+resp = readurl(VNIC_METADATA_URL)
+vnics = json.loads(str(resp))
+
+if 'nicIndex' in vnics[0]:
+LOG.debug('VNIC metadata indicates this is a bare metal machine;'
+  ' skipping secondary VNIC configuration.')
+return network_config
+
+interfaces_by_mac = get_interfaces_by_mac()
+
+for vnic_dict in vnics[1:]:
+mac_address = vnic_dict['macAddr'].lower()
+if mac_address not in interfaces_by_mac:
+LOG.info('Interface with MAC %s not found; skipping', mac_address)
+continue
+name = interfaces_by_mac[mac_address]
+subnet = {
+'type': 'static',
+'address': vnic_dict['privateIp'],
+'netmask': vnic_dict['subnetCidrBlock'].split('/')[1],
+'gateway': vnic_dict['virtualRouterIp'],
+'control': 'manual',
+}
+network_config['config'].append({
+'name': name,
+'type': 'physical',
+'mac_address': mac_address,
+'mtu': 9000,
+'subnets': [subnet],
+})
+
+return network_config
 
 
 class DataSourceOracle(sources.DataSource):
@@ -39,6 +101,13 @@ class DataSourceOracle(sources.DataSource):
 vendordata_pure = None
 _network_config = sources.UNSET
 
+def __init__(self, sys_cfg, *args, **kwargs):
+super(DataSourceOracle, self).__init__(sys_cfg, *args, **kwargs)
+
+self.ds_cfg = util.mergemanydict([
+util.get_cfg_by_path(sys_cfg, ['datasource', self.dsname], {}),
+BUILTIN_DS_CONFIG])
+
 def _is_platform_viable(self):
 """Check platform environment to report if this datasource may run."""
 return _is_platform_viable()
@@ -121,6 +190,14 @@ class DataSourceOracle(sources.DataSource):
 self._network_config = cmdline.read_initramfs_config()
 if not self._network_config:
 self._network_config = self.distro.generate_fallback_config()
+if self.ds_cfg.get('configure_secondary_nics'):
+try:
+self._network_config = _network_config_from_opc_imds(
+self._network_config)
+except Exception:
+util.logexc(
+LOG,
+"Failed to fetch secondary network configuration!")
 return self._network_config
 
 
diff --git a/cloudinit/sources/tests/test_oracle.py b/cloudinit/sources/tests/test_oracle.py
index 282382c..b20238e 100644
--- a/cloudinit/sources/tests/test_oracle.py
+++ b/cloudinit/sources/tests/test_oracle.py
@@ -19,9 +19,30 @@ DS_PATH = "cloudinit.sources.DataSourceOracle"
 MD_VER = "2013-10-17"
 
 
+# `curl -L 

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master

2019-08-08 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.

Commit message:
doc: remove intersphinx extension

We don't use it in our doc builds, and if sphinx.pocoo.org is down then
it causes doc build issues.


Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371090
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.
diff --git a/doc/rtd/conf.py b/doc/rtd/conf.py
index 50eb05c..4174477 100644
--- a/doc/rtd/conf.py
+++ b/doc/rtd/conf.py
@@ -27,16 +27,11 @@ project = 'Cloud-Init'
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
 extensions = [
-'sphinx.ext.intersphinx',
 'sphinx.ext.autodoc',
 'sphinx.ext.autosectionlabel',
 'sphinx.ext.viewcode',
 ]
 
-intersphinx_mapping = {
-'sphinx': ('http://sphinx.pocoo.org', None)
-}
-
 # The suffix of source filenames.
 source_suffix = '.rst'
 
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master

2019-08-07 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master.

Commit message:
WIP: DataSourceOracle: configure secondary NICs on Virtual Machines

Oracle Cloud Infrastructure's Instance Metadata Service provides network
configuration information for non-primary NICs.  This commit introduces
support, on Virtual Machines[0], for fetching that network metadata,
converting it to v1 network-config[1] and combining it into the network
configuration generated for the primary interface.

By default, this behaviour is not enabled.  Configuring the Oracle
datasource to `configure_secondary_nics` enables it:

datasource:
  Oracle:
configure_secondary_nics: True

Failures to fetch and generate secondary NIC configuration will log a
warning, but otherwise will not affect boot.

[0] The expected use of the IMDS-provided network configuration is
substantially different on Bare Metal Machines, so support for that
will be addressed separately.
[1] This is v1 config, because cloudinit.net.cmdline generates v1 config
and we need to integrate the secondary NICs into that configuration.

STILL TODO:
* docs

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/371053
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:oci-vnic into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceOracle.py b/cloudinit/sources/DataSourceOracle.py
index 76cfa38..f18e5a0 100644
--- a/cloudinit/sources/DataSourceOracle.py
+++ b/cloudinit/sources/DataSourceOracle.py
@@ -16,7 +16,7 @@ Notes:
 """
 
 from cloudinit.url_helper import combine_url, readurl, UrlError
-from cloudinit.net import dhcp
+from cloudinit.net import dhcp, get_interfaces_by_mac
 from cloudinit import net
 from cloudinit import sources
 from cloudinit import util
@@ -28,8 +28,70 @@ import re
 
 LOG = logging.getLogger(__name__)
 
+BUILTIN_DS_CONFIG = {
+# Don't use IMDS to configure secondary NICs by default
+'configure_secondary_nics': False,
+}
 CHASSIS_ASSET_TAG = "OracleCloud.com"
 METADATA_ENDPOINT = "http://169.254.169.254/openstack/;
+VNIC_METADATA_URL = 'http://169.254.169.254/opc/v1/vnics/'
+
+
+def _network_config_from_opc_imds(network_config):
+"""
+Fetch data from Oracle's IMDS and generate secondary NIC config.
+
+The primary NIC configuration should not be modified based on the IMDS
+values, as it should continue to be configured for DHCP.  As such, this
+takes an existing network_config dict which is expected to have the primary
+NIC configuration already present.
+
+:param network_config:
+A v1 network config dict with the primary NIC already configured.  This
+dict will be mutated.
+
+:raises:
+Exceptions are not handled within this function.  Likely exceptions are
+those raised by url_helper.readurl (if communicating with the IMDS
+fails), ValueError/JSONDecodeError (if the IMDS returns invalid JSON),
+and KeyError/IndexError (if the IMDS returns valid JSON with unexpected
+contents).
+
+:return:
+The ``network_config`` dict with secondary NICs added.
+"""
+resp = readurl(VNIC_METADATA_URL)
+vnics = json.loads(str(resp))
+
+if 'nicIndex' in vnics[0]:
+LOG.debug('VNIC metadata indicates this is a bare metal machine;'
+  ' skipping secondary VNIC configuration.')
+return network_config
+
+interfaces_by_mac = get_interfaces_by_mac()
+
+for vnic_dict in vnics[1:]:
+mac_address = vnic_dict['macAddr'].lower()
+if mac_address not in interfaces_by_mac:
+LOG.info('Interface with MAC %s not found; skipping', mac_address)
+continue
+name = interfaces_by_mac[mac_address]
+subnet = {
+'type': 'static',
+'address': vnic_dict['privateIp'],
+'netmask': vnic_dict['subnetCidrBlock'].split('/')[1],
+'gateway': vnic_dict['virtualRouterIp'],
+'control': 'manual',
+}
+network_config['config'].append({
+'name': name,
+'type': 'physical',
+'mac_address': mac_address,
+'mtu': 9000,
+'subnets': [subnet],
+})
+
+return network_config
 
 
 class DataSourceOracle(sources.DataSource):
@@ -39,6 +101,13 @@ class DataSourceOracle(sources.DataSource):
 vendordata_pure = None
 _network_config = sources.UNSET
 
+def __init__(self, sys_cfg, *args, **kwargs):
+super(DataSourceOracle, self).__init__(sys_cfg, *args, **kwargs)
+
+self.ds_cfg = util.mergemanydict([
+util.get_cfg_by_path(sys_cfg, ['datasource', self.dsname], {}),
+   

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master

2019-08-06 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.

Commit message:
cc_set_passwords: rewrite documentation

What we had previously was inaccurate in a few respects.

LP: #1838794

Requested reviews:
  Server Team CI bot (server-team-bot): continuous-integration
  cloud-init commiters (cloud-init-dev)
Related bugs:
  Bug #1838794 in cloud-init: "Set Passwords documentation describes incorrect 
behaviour for `password` config key"
  https://bugs.launchpad.net/cloud-init/+bug/1838794

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/370927
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.
diff --git a/cloudinit/config/cc_set_passwords.py b/cloudinit/config/cc_set_passwords.py
index 4585e4d..debdee4 100755
--- a/cloudinit/config/cc_set_passwords.py
+++ b/cloudinit/config/cc_set_passwords.py
@@ -9,27 +9,39 @@
 """
 Set Passwords
 -
-**Summary:** Set user passwords
-
-Set system passwords and enable or disable ssh password authentication.
-The ``chpasswd`` config key accepts a dictionary containing a single one of two
-keys, either ``expire`` or ``list``. If ``expire`` is specified and is set to
-``false``, then the ``password`` global config key is used as the password for
-all user accounts. If the ``expire`` key is specified and is set to ``true``
-then user passwords will be expired, preventing the default system passwords
-from being used.
-
-If the ``list`` key is provided, a list of
-``username:password`` pairs can be specified. The usernames specified
-must already exist on the system, or have been created using the
-``cc_users_groups`` module. A password can be randomly generated using
-``username:RANDOM`` or ``username:R``. A hashed password can be specified
-using ``username:$6$salt$hash``. Password ssh authentication can be
-enabled, disabled, or left to system defaults using ``ssh_pwauth``.
+**Summary:** Set user passwords and enable/disable SSH password authentication
+
+This module consumes three top-level config keys: ``ssh_pwauth``, ``chpasswd``
+and ``password``.
+
+The ``ssh_pwauth`` config key determines whether or not sshd will be configured
+to accept password authentication.  True values will enable password auth,
+false values will disable password auth, and the literal string ``unchanged``
+will leave it unchanged.  Setting no value will also leave the current setting
+on-disk unchanged.
+
+The ``chpasswd`` config key accepts a dictionary containing either or both of
+``expire`` and ``list``.
+
+If the ``list`` key is provided, it should contain a list of
+``username:password`` pairs.  This can be either a YAML list (of strings), or a
+multi-line string with one pair per line.  Each user will have the
+corresponding password set.  A password can be randomly generated by specifying
+``RANDOM`` or ``R`` as a user's password.  A hashed password, created by a tool
+like ``mkpasswd``, can be specified; a regex (``r'\$(1|2a|2y|5|6)(\$.+){2}'``)
+is used to determine if a password value should be treated as a hash.
 
 .. note::
-if using ``expire: true`` then a ssh authkey should be specified or it may
-not be possible to login to the system
+The users specified must already exist on the system.  Users will have been
+created by the ``cc_users_groups`` module at this point.
+
+By default, all users on the system will have their passwords expired (meaning
+that they will have to be reset the next time the user logs in).  To disable
+this behaviour, set ``expire`` under ``chpasswd`` to a false value.
+
+If a ``list`` of user/password pairs is not specified under ``chpasswd``, then
+the value of the ``password`` config key will be used to set the default user's
+password.
 
 **Internal name:** ``cc_set_passwords``
 
@@ -160,6 +172,8 @@ def handle(_name, cfg, cloud, log, args):
 hashed_users = []
 randlist = []
 users = []
+# N.B. This regex is included in the documentation (i.e. the module
+# docstring), so any changes to it should be reflected there.
 prog = re.compile(r'\$(1|2a|2y|5|6)(\$.+){2}')
 for line in plist:
 u, p = line.split(':', 1)
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master

2019-08-06 Thread Dan Watkins



Diff comments:

> diff --git a/cloudinit/config/cc_set_passwords.py 
> b/cloudinit/config/cc_set_passwords.py
> index 4585e4d..10d68ea 100755
> --- a/cloudinit/config/cc_set_passwords.py
> +++ b/cloudinit/config/cc_set_passwords.py
> @@ -9,27 +9,38 @@
>  """
>  Set Passwords
>  -
> -**Summary:** Set user passwords
> -
> -Set system passwords and enable or disable ssh password authentication.
> -The ``chpasswd`` config key accepts a dictionary containing a single one of 
> two
> -keys, either ``expire`` or ``list``. If ``expire`` is specified and is set to
> -``false``, then the ``password`` global config key is used as the password 
> for
> -all user accounts. If the ``expire`` key is specified and is set to ``true``
> -then user passwords will be expired, preventing the default system passwords
> -from being used.
> -
> -If the ``list`` key is provided, a list of
> -``username:password`` pairs can be specified. The usernames specified
> -must already exist on the system, or have been created using the
> -``cc_users_groups`` module. A password can be randomly generated using
> -``username:RANDOM`` or ``username:R``. A hashed password can be specified
> -using ``username:$6$salt$hash``. Password ssh authentication can be
> -enabled, disabled, or left to system defaults using ``ssh_pwauth``.
> +**Summary:** Set user passwords and enable/disable SSH password 
> authentication
> +
> +This module consumes three top-level config keys: ``ssh_pwauth``, 
> ``chpasswd``
> +and ``password``.
> +
> +The ``ssh_pwauth`` config key determines whether or not sshd will be 
> configured
> +to accept password authentication.  True values will enable password auth,
> +false values will disable password auth, and the literal string ``unchanged``
> +will leave it unchanged.  Setting no value will also leave the current 
> setting
> +on-disk unchanged.
> +
> +The ``chpasswd`` config key accepts a dictionary containing either or both of
> +``expire`` and ``list``.
> +
> +If the ``list`` key is provided, it should contain a list of
> +``username:password`` pairs.  This can be either a YAML list (of strings), 
> or a
> +multi-line string with one pair per line.  Each user will have the
> +corresponding password set.  A password can be randomly generated by 
> specifying
> +``RANDOM`` or ``R`` as a user's password.  A hashed password can be specified
> +using ``$6$salt$hash`` as a user's password.

Yep, let me rework this.

>  
>  .. note::
> -if using ``expire: true`` then a ssh authkey should be specified or it 
> may
> -not be possible to login to the system
> +The users specified must already exist on the system.  Users will have 
> been
> +created by the ``cc_users_groups`` module at this point.
> +
> +By default, all users on the system will have their passwords expired 
> (meaning
> +that they will have to be reset the next time the user logs in).  To disable
> +this behaviour, set ``expire`` under ``chpasswd`` to a false value.
> +
> +If a ``list`` of user/password pairs is not specified under ``chpasswd``, 
> then
> +the value of the ``password`` config key will be used to set the default 
> user's
> +password.
>  
>  **Internal name:** ``cc_set_passwords``
>  


-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/370927
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master

2019-08-06 Thread Dan Watkins
The proposal to merge ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into 
cloud-init:master has been updated.

Status: Needs review => Work in progress

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/370927
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master

2019-08-02 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.

Commit message:
cc_set_passwords: rewrite documentation

What we had previously was inaccurate in a few respects.

LP: #1838794

Requested reviews:
  cloud-init commiters (cloud-init-dev)
Related bugs:
  Bug #1838794 in cloud-init: "Set Passwords documentation describes incorrect 
behaviour for `password` config key"
  https://bugs.launchpad.net/cloud-init/+bug/1838794

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/370927
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:doc into cloud-init:master.
diff --git a/cloudinit/config/cc_set_passwords.py b/cloudinit/config/cc_set_passwords.py
index 4585e4d..10d68ea 100755
--- a/cloudinit/config/cc_set_passwords.py
+++ b/cloudinit/config/cc_set_passwords.py
@@ -9,27 +9,38 @@
 """
 Set Passwords
 -
-**Summary:** Set user passwords
-
-Set system passwords and enable or disable ssh password authentication.
-The ``chpasswd`` config key accepts a dictionary containing a single one of two
-keys, either ``expire`` or ``list``. If ``expire`` is specified and is set to
-``false``, then the ``password`` global config key is used as the password for
-all user accounts. If the ``expire`` key is specified and is set to ``true``
-then user passwords will be expired, preventing the default system passwords
-from being used.
-
-If the ``list`` key is provided, a list of
-``username:password`` pairs can be specified. The usernames specified
-must already exist on the system, or have been created using the
-``cc_users_groups`` module. A password can be randomly generated using
-``username:RANDOM`` or ``username:R``. A hashed password can be specified
-using ``username:$6$salt$hash``. Password ssh authentication can be
-enabled, disabled, or left to system defaults using ``ssh_pwauth``.
+**Summary:** Set user passwords and enable/disable SSH password authentication
+
+This module consumes three top-level config keys: ``ssh_pwauth``, ``chpasswd``
+and ``password``.
+
+The ``ssh_pwauth`` config key determines whether or not sshd will be configured
+to accept password authentication.  True values will enable password auth,
+false values will disable password auth, and the literal string ``unchanged``
+will leave it unchanged.  Setting no value will also leave the current setting
+on-disk unchanged.
+
+The ``chpasswd`` config key accepts a dictionary containing either or both of
+``expire`` and ``list``.
+
+If the ``list`` key is provided, it should contain a list of
+``username:password`` pairs.  This can be either a YAML list (of strings), or a
+multi-line string with one pair per line.  Each user will have the
+corresponding password set.  A password can be randomly generated by specifying
+``RANDOM`` or ``R`` as a user's password.  A hashed password can be specified
+using ``$6$salt$hash`` as a user's password.
 
 .. note::
-if using ``expire: true`` then a ssh authkey should be specified or it may
-not be possible to login to the system
+The users specified must already exist on the system.  Users will have been
+created by the ``cc_users_groups`` module at this point.
+
+By default, all users on the system will have their passwords expired (meaning
+that they will have to be reset the next time the user logs in).  To disable
+this behaviour, set ``expire`` under ``chpasswd`` to a false value.
+
+If a ``list`` of user/password pairs is not specified under ``chpasswd``, then
+the value of the ``password`` config key will be used to set the default user's
+password.
 
 **Internal name:** ``cc_set_passwords``
 
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master

2019-07-24 Thread Dan Watkins



Diff comments:

> diff --git a/cloudinit/sources/tests/test_oracle.py 
> b/cloudinit/sources/tests/test_oracle.py
> index 97d6294..282382c 100644
> --- a/cloudinit/sources/tests/test_oracle.py
> +++ b/cloudinit/sources/tests/test_oracle.py
> @@ -145,15 +145,15 @@ class TestDataSourceOracle(test_helpers.CiTestCase):
>  MD_VER: {'system_uuid': self.my_uuid,
>   'meta_data': self.my_md)
>  ncfg = {'version': 1, 'config': [{'a': 'b'}]}
> -m_cmdline_config.return_value = ncfg
> +m_initramfs_config.return_value = ncfg
>  self.assertTrue(ds._get_data())
>  self.assertEqual(ncfg, ds.network_config)
> -m_cmdline_config.assert_called_once_with()
> +self.assertEqual([mock.call()], m_initramfs_config.call_args_list)

This, specifically, is testing that it is called with no parameters 
(`mock.call()`), which is what the replaced line was doing previously[0].  Are 
you asking to confirm that we are doing that (we are!), or are you suggesting 
that we should relax the test here to not check that?

[0] I generally prefer to avoid the assertion methods on mocks, as mistyping 
them is very easy to miss during code review and can lead to tests silently 
passing.  As I was modifying the line anyway, I figured I'd make this change 
also.

>  self.assertFalse(distro.generate_fallback_config.called)
>  
> -@mock.patch(DS_PATH + ".cmdline.read_kernel_cmdline_config")
> +@mock.patch(DS_PATH + ".cmdline.read_initramfs_config")
>  @mock.patch(DS_PATH + "._is_iscsi_root", return_value=True)
> -def test_network_fallback(self, m_is_iscsi_root, m_cmdline_config):
> +def test_network_fallback(self, m_is_iscsi_root, m_initramfs_config):
>  """test that fallback network is generated if no kernel cmdline."""
>  distro = mock.MagicMock()
>  ds, _ = self._get_ds(distro=distro, patches={


-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/370526
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~raharper/cloud-init:fix/debian-ntp-template into cloud-init:master

2019-07-16 Thread Dan Watkins
The proposal to merge ~raharper/cloud-init:fix/debian-ntp-template into 
cloud-init:master has been updated.

Commit message changed to:

templates/ntp.conf.debian.tmpl: fix missing newline for pools

The debian ntp.conf template did not contain a newline for the
comment used to mark the rendered ntp pools configured.  This
resulted in an invalid line:

'# poolspool 0.int.pool.ntp.org iburst'

rather than:

'# pools
 pool 0.int.pool.ntp.org iburst'

This patch fixes the template and updates the unittest to
verify that the rendered templates puts servers and pools
at the beginning of a line.

LP: #1836598

For more details, see:
https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/370213
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~raharper/cloud-init:fix/debian-ntp-template into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:coverity into cloud-init:master

2019-07-10 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:coverity into cloud-init:master.

Commit message:
Fix a couple of issues raised by a coverity scan

* cc_lxd: fix copy/paste error in debug logging
* DataSourceCloudSigma: remove unreachable code
  * This unreachable code was introduced in a refactor (in 2015) which
removed the need for an exception handler, but retained the logging
from the exception handler as an unreachable fall-through.

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369961
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:coverity into cloud-init:master.
diff --git a/cloudinit/config/cc_lxd.py b/cloudinit/config/cc_lxd.py
index 71d13ed..d983077 100644
--- a/cloudinit/config/cc_lxd.py
+++ b/cloudinit/config/cc_lxd.py
@@ -152,7 +152,7 @@ def handle(name, cfg, cloud, log, args):
 
 if cmd_attach:
 log.debug("Setting up default lxd bridge: %s" %
-  " ".join(cmd_create))
+  " ".join(cmd_attach))
 _lxc(cmd_attach)
 
 elif bridge_cfg:
diff --git a/cloudinit/sources/DataSourceCloudSigma.py b/cloudinit/sources/DataSourceCloudSigma.py
index 2955d3f..df88f67 100644
--- a/cloudinit/sources/DataSourceCloudSigma.py
+++ b/cloudinit/sources/DataSourceCloudSigma.py
@@ -42,12 +42,8 @@ class DataSourceCloudSigma(sources.DataSource):
 if not sys_product_name:
 LOG.debug("system-product-name not available in dmi data")
 return False
-else:
-LOG.debug("detected hypervisor as %s", sys_product_name)
-return 'cloudsigma' in sys_product_name.lower()
-
-LOG.warning("failed to query dmi data for system product name")
-return False
+LOG.debug("detected hypervisor as %s", sys_product_name)
+return 'cloudsigma' in sys_product_name.lower()
 
 def _get_data(self):
 """
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~chad.smith/cloud-init:feature/ec2-secondary-nics into cloud-init:master

2019-07-08 Thread Dan Watkins
Thanks!  A couple of follow-ups, and a couple of nits I missed first time 
around.

Diff comments:

> diff --git a/cloudinit/sources/DataSourceEc2.py 
> b/cloudinit/sources/DataSourceEc2.py
> index 5c017bf..9a5ed43 100644
> --- a/cloudinit/sources/DataSourceEc2.py
> +++ b/cloudinit/sources/DataSourceEc2.py
> @@ -536,24 +536,43 @@ def convert_ec2_metadata_network_config(network_md, 
> macs_to_nics=None,
>  @param: fallback_nic: Optionally provide the primary nic interface name.
> This nic will be guaranteed to minimally have a dhcp4 configuration.
>  
> -@return A dict of network config version 1 based on the metadata and 
> macs.
> +@return A dict of network config version 2 based on the metadata and 
> macs.
>  """
> -netcfg = {'version': 1, 'config': []}
> +netcfg = {'version': 2, 'ethernets': {}}
>  if not macs_to_nics:
>  macs_to_nics = net.get_interfaces_by_mac()
>  macs_metadata = network_md['interfaces']['macs']
>  for mac, nic_name in macs_to_nics.items():
> +dev_config = {}
>  nic_metadata = macs_metadata.get(mac)
>  if not nic_metadata:
>  continue  # Not a physical nic represented in metadata
> -nic_cfg = {'type': 'physical', 'name': nic_name, 'subnets': []}
> -nic_cfg['mac_address'] = mac
> +local_ipv4s = nic_metadata.get('local-ipv4s')
>  if (nic_name == fallback_nic or nic_metadata.get('public-ipv4s') or
> -nic_metadata.get('local-ipv4s')):
> -nic_cfg['subnets'].append({'type': 'dhcp4'})
> +local_ipv4s):
> +dev_config['dhcp4'] = True
> +# In version < 2018-09-24 local_ipvs is a str with a single IP

Thanks for this comment!  I have the context of the commit message informing me 
that this about secondary IP addresses, but I wonder if in future people might 
wonder why we aren't doing anything with the one str IP address; is it worth 
explicitly mentioning that the next stanza is handling secondary interfaces?

> +if isinstance(local_ipv4s, list) and len(local_ipv4s) > 1:
> +if dev_config.get('addresses') is None:

If we get here, dev_config is, I think, always going to be {'dhcp4': True}.  So 
do we need this to be conditional any longer?

> +dev_config['addresses'] = []
> +subnet_cidr = nic_metadata.get('subnet-ipv4-cidr-block')
> +if not subnet_cidr or '/' not in subnet_cidr:
> +LOG.warning(
> +'Could not parse subnet-ipv4-cidr-block %s.'
> +' Network config for Secondary IPs default to /32',
> +subnet_cidr)
> +prefix = '32'
> +else:
> +_ip, prefix = subnet_cidr.split('/')

If subnet_cidr has more than one / in it, this will fail with "ValueError: too 
many values to unpack".

(Sorry I didn't catch this the first time around!)

> +for secondary_ip in local_ipv4s[1:]:
> +dev_config['addresses'].append(
> +'{ip}/{prefix}'.format(ip=secondary_ip, 
> prefix=prefix))
>  if nic_metadata.get('ipv6s'):
> -nic_cfg['subnets'].append({'type': 'dhcp6'})
> -netcfg['config'].append(nic_cfg)
> +dev_config['dhcp6'] = True
> +dev_config.update({
> +'match': {'macaddress': mac.lower()},
> +'set-name': nic_name})
> +netcfg['ethernets'][nic_name] = dev_config
>  return netcfg
>  
>  
> diff --git a/tests/unittests/test_datasource/test_ec2.py 
> b/tests/unittests/test_datasource/test_ec2.py
> index 20d59bf..e031fe9 100644
> --- a/tests/unittests/test_datasource/test_ec2.py
> +++ b/tests/unittests/test_datasource/test_ec2.py
> @@ -309,10 +396,40 @@ class TestEc2(test_helpers.HttprettyTestCase):
>  ds.get_data()
>  
>  mac1 = '06:17:04:d7:26:0A'  # IPv4 only in DEFAULT_METADATA
> -expected = {'version': 1, 'config': [
> -{'mac_address': '06:17:04:d7:26:0A', 'name': 'eth9',
> - 'subnets': [{'type': 'dhcp4'}],
> - 'type': 'physical'}]}
> +expected = {'version': 2, 'ethernets': {'eth9': {
> +'match': {'macaddress': '06:17:04:d7:26:0a'}, 'set-name': 'eth9',
> +'dhcp4': True}}}
> +patch_path = (
> +'cloudinit.sources.DataSourceEc2.net.get_interfaces_by_mac')
> +get_interface_mac_path = (
> +'cloudinit.sources.DataSourceEc2.net.get_interface_mac')
> +with mock.patch(patch_path) as m_get_interfaces_by_mac:
> +with mock.patch(find_fallback_path) as m_find_fallback:
> +with mock.patch(get_interface_mac_path) as m_get_mac:
> +m_get_interfaces_by_mac.return_value = {mac1: 'eth9'}
> +m_find_fallback.return_value = 'eth9'
> +m_get_mac.return_value 

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master

2019-07-05 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master.

Commit message:
stages: allow data sources to override network config source order

Currently, if a platform provides any network configuration via the
"cmdline" method (i.e. network-data=... on the kernel command line,
ip=... on the kernel command line, or iBFT config via /run/net-*.conf),
the value of the data source's network_config property is completely
ignored.

This means that on platforms that use iSCSI boot (such as Oracle Compute
Infrastructure), there is no way for the data source to configure any
network interfaces other than those that have already been configured by
the initramfs.

This change allows data sources to specify the order in which network
configuration sources are considered.  Data sources that opt to use this
mechanism will be expected to consume the command line network data and
integrate it themselves.

(The generic merging of network configuration sources was considered,
but we concluded that the single use case we have presently (a) didn't
warrant the increased complexity, and (b) didn't give us a broad enough
view to be sure that our generic implementation would be sufficiently
generic.  This change in no way precludes a merging strategy in future.)

Requested reviews:
  Server Team CI bot (server-team-bot): continuous-integration
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369783
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:networking into cloud-init:master.
diff --git a/cloudinit/sources/__init__.py b/cloudinit/sources/__init__.py
index e6966b3..ce4626b 100644
--- a/cloudinit/sources/__init__.py
+++ b/cloudinit/sources/__init__.py
@@ -153,6 +153,11 @@ class DataSource(object):
 # Track the discovered fallback nic for use in configuration generation.
 _fallback_interface = None
 
+# The network configuration sources that should be considered for this data
+# source.  (The first source in this list that provides network
+# configuration will be used without considering any that follow.)
+network_config_sources = ('cmdline', 'system_cfg', 'ds')
+
 # read_url_params
 url_max_wait = -1   # max_wait < 0 means do not wait
 url_timeout = 10# timeout for each metadata url read attempt
diff --git a/cloudinit/stages.py b/cloudinit/stages.py
index da7d349..2498c69 100644
--- a/cloudinit/stages.py
+++ b/cloudinit/stages.py
@@ -630,18 +630,27 @@ class Init(object):
 if os.path.exists(disable_file):
 return (None, disable_file)
 
-cmdline_cfg = ('cmdline', cmdline.read_kernel_cmdline_config())
-dscfg = ('ds', None)
+available_cfgs = {
+'cmdline': cmdline.read_kernel_cmdline_config(),
+'ds': None,
+'system_cfg': self.cfg.get('network'),
+}
+
 if self.datasource and hasattr(self.datasource, 'network_config'):
-dscfg = ('ds', self.datasource.network_config)
-sys_cfg = ('system_cfg', self.cfg.get('network'))
+available_cfgs['ds'] = self.datasource.network_config
 
-for loc, ncfg in (cmdline_cfg, sys_cfg, dscfg):
+if self.datasource and hasattr(self.datasource,
+   'network_config_sources'):
+order = self.datasource.network_config_sources
+else:
+order = sources.DataSource.network_config_sources
+for cfg_source in order:
+ncfg = available_cfgs[cfg_source]
 if net.is_disabled_cfg(ncfg):
-LOG.debug("network config disabled by %s", loc)
-return (None, loc)
+LOG.debug("network config disabled by %s", cfg_source)
+return (None, cfg_source)
 if ncfg:
-return (ncfg, loc)
+return (ncfg, cfg_source)
 return (self.distro.generate_fallback_config(), "fallback")
 
 def apply_network_config(self, bring_up):
diff --git a/cloudinit/tests/test_stages.py b/cloudinit/tests/test_stages.py
index 94b6b25..8f0be28 100644
--- a/cloudinit/tests/test_stages.py
+++ b/cloudinit/tests/test_stages.py
@@ -95,6 +95,23 @@ class TestInit(CiTestCase):
  self.logs.getvalue())
 
 @mock.patch('cloudinit.stages.cmdline.read_kernel_cmdline_config')
+def test__find_networking_config_uses_datasrc_order(self, m_cmdline):
+"""find_networking_config should check sources in DS defined order"""
+# cmdline, which would normally be preferred over other sources,
+# disables networking; in this case, though, the DS moves cmdline later
+# so its own config is preferred
+m_cmdline.return_value = 

Re: [Cloud-init-dev] [Merge] ~tribaal/cloud-init:feat/datasource-exoscale into cloud-init:master

2019-07-02 Thread Dan Watkins
Quick drive-by review to ask if we can reuse the password fetching from the 
CloudStack DS.

Diff comments:

> diff --git a/cloudinit/sources/DataSourceExoscale.py 
> b/cloudinit/sources/DataSourceExoscale.py
> new file mode 100644
> index 000..4588c9d
> --- /dev/null
> +++ b/cloudinit/sources/DataSourceExoscale.py
> @@ -0,0 +1,126 @@
> +import time
> +
> +from cloudinit import ec2_utils as ec2
> +from cloudinit import log as logging
> +from cloudinit import sources
> +from cloudinit import url_helper
> +
> +API_VERSION = "latest"
> +LOG = logging.getLogger(__name__)
> +SERVICE_ADDRESS = "http://169.254.169.254;
> +
> +
> +class DataSourceExoscale(sources.DataSource):
> +
> +dsname = 'Exoscale'
> +url_timeout = 10
> +url_retries = 6
> +url_max_wait = 60
> +
> +def __init__(self, sys_cfg, distro, paths):
> +sources.DataSource.__init__(self, sys_cfg, distro, paths)
> +LOG.info("Initializing the Exoscale datasource")
> +self.extra_config = {}
> +
> +def get_password(self):

DataSourceCloudStack.py has the CloudStackPasswordServerClient class which 
encapsulates this logic already; could it be reused?

> +"""Return the VM's passwords."""
> +LOG.info("Fetching password from metadata service")
> +password_url = "{}:8080".format(SERVICE_ADDRESS)
> +response = url_helper.read_file_or_url(
> +password_url,
> +ssl_details=None,
> +headers={"DomU_Request": "send_my_password"},
> +timeout=self.url_timeout,
> +retries=self.url_retries)
> +password = response.contents.decode('utf-8')
> +# the password is empty or already saved
> +if password in ['', 'saved_password']:
> +LOG.info("Password is missing or already saved")
> +return None
> +LOG.info("Found the password in metadata service")
> +# save the password
> +url_helper.read_file_or_url(
> +password_url,
> +ssl_details=None,
> +headers={"DomU_Request": "saved_password"},
> +timeout=self.url_timeout,
> +retries=self.url_retries)
> +LOG.info("password saved")
> +return password
> +
> +def wait_for_metadata_service(self):
> +"""Wait for the metadata service to be reachable."""
> +LOG.info("waiting for the metadata service")
> +start_time = time.time()
> +
> +metadata_url = "{}/{}/meta-data/instance-id".format(
> +SERVICE_ADDRESS,
> +API_VERSION)
> +
> +start_time = time.time()
> +url = url_helper.wait_for_url(
> +urls=[metadata_url],
> +max_wait=self.url_max_wait,
> +timeout=self.url_timeout,
> +status_cb=LOG.critical)
> +
> +if url:
> +LOG.info("metadata service ok")
> +return True
> +else:
> +wait_time = int(time.time() - start_time)
> +LOG.critical(("Giving up on waiting for the metadata from %s"
> +  " after %s seconds"),
> + url,
> + wait_time)
> +return False
> +
> +def _get_data(self):
> +"""Fetch the user data, the metadata and the VM password
> +from the metadata service."""
> +LOG.info("fetching data")
> +if not self.wait_for_metadata_service():
> +return False
> +start_time = time.time()
> +self.userdata_raw = ec2.get_instance_userdata(API_VERSION,
> +  SERVICE_ADDRESS,
> +  
> timeout=self.url_timeout,
> +  
> retries=self.url_retries)
> +self.metadata = ec2.get_instance_metadata(API_VERSION,
> +  SERVICE_ADDRESS,
> +  timeout=self.url_timeout,
> +  retries=self.url_retries)
> +password = self.get_password()
> +if password:
> +self.extra_config = {
> +'ssh_pwauth': True,
> +'password': password,
> +'chpasswd': {
> +'expire': False,
> +},
> +}
> +get_data_time = int(time.time() - start_time)
> +LOG.info("finished fetching the metadata in %s seconds",
> + get_data_time)
> +return True
> +
> +def get_config_obj(self):
> +return self.extra_config
> +
> +def get_instance_id(self):
> +return self.metadata['instance-id']
> +
> +@property
> +def availability_zone(self):
> +return self.metadata['availability-zone']
> +
> +
> +# Used to match classes to dependencies
> +datasources = [
> +(DataSourceExoscale, (sources.DEP_FILESYSTEM, 

Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:docs into cloud-init:master

2019-06-25 Thread Dan Watkins
On Tue, Jun 25, 2019 at 07:54:20PM -, Ryan Harper wrote:
> Nice.  Let's add a link to netplan.io in there.

Done.

-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369312
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:docs into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:docs into cloud-init:master

2019-06-25 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:docs into cloud-init:master.

Commit message:
doc: indicate that netplan is default in Ubuntu now

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/369312
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:docs into cloud-init:master.
diff --git a/doc/rtd/topics/network-config.rst b/doc/rtd/topics/network-config.rst
index 1e99455..05966c6 100644
--- a/doc/rtd/topics/network-config.rst
+++ b/doc/rtd/topics/network-config.rst
@@ -163,10 +163,11 @@ found in Ubuntu and Debian.
 
 - **Netplan**
 
-Since Ubuntu 16.10, codename Yakkety, the ``netplan`` project has been an
-optional network configuration tool which consumes :ref:`network_config_v2`
-input and renders network configuration for supported backends such as
-``systemd-networkd`` and ``NetworkManager``.
+Introduced in Ubuntu 16.10 (Yakkety Yak), ``netplan`` has been the default
+network configuration tool in Ubuntu since 17.10 (Artful Aardvark).
+``netplan`` consumes :ref:`network_config_v2` input and renders network
+configuration for supported backends such as ``systemd-networkd`` and
+``NetworkManager``.
 
 - **Sysconfig**
 
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~pzakha/cloud-init:userdata into cloud-init:master

2019-05-28 Thread Dan Watkins
Hi Pavel, thanks for the contribution to cloud-init!  I'm flitting between 
meetings, so I'm not able to assess whether these changes are broadly 
appropriate.  That said, I do have an inline comment on how the changes could 
be expressed a little more concisely.  I'll revisit this later today for a more 
full review.

Diff comments:

> diff --git a/cloudinit/stages.py b/cloudinit/stages.py
> index da7d349..94b3f0f 100644
> --- a/cloudinit/stages.py
> +++ b/cloudinit/stages.py
> @@ -548,7 +548,17 @@ class Init(object):
>  with events.ReportEventStack("consume-user-data",
>   "reading and applying user-data",
>   parent=self.reporter):
> -self._consume_userdata(frequency)
> +cfg = self.cfg

>From a Python code review POV, this could be much simpler:

if self.cfg.get('allow_userdata', True):
LOG.debug('allow_userdata = True: consuming user-data')
self._consume_userdata(frequency)
else:
LOG.debug('allow_userdata = False: discarding user-data')

I believe an analogous change could also be made to cc_ssh above.

> +if 'allow_userdata' in cfg:
> +allow_userdata = cfg['allow_userdata']
> +else:
> +allow_userdata = True
> +
> +if allow_userdata:
> +LOG.debug('allow_userdata = True: consuming user-data')
> +self._consume_userdata(frequency)
> +else:
> +LOG.debug('allow_userdata = False: discarding user-data')
>  with events.ReportEventStack("consume-vendor-data",
>   "reading and applying vendor-data",
>   parent=self.reporter):


-- 
https://code.launchpad.net/~pzakha/cloud-init/+git/cloud-init/+merge/367721
Your team cloud-init commiters is requested to review the proposed merge of 
~pzakha/cloud-init:userdata into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~med/cloud-init:typo_verbatim into cloud-init:master

2019-05-22 Thread Dan Watkins
Review: Needs Information

Hey David,

Thanks for the fix!  To contribute to cloud-init, you must sign the Canonical 
Contributor License Agreement (CLA) [1].

If you have already signed it as an individual, your Launchpad user will be 
listed in the contributor-agreement-canonical launchpad group [2]. 
Unfortunately there is no easy way to check if an organization or company you 
are doing work for has signed. If you are unsure or have questions, email 
josh.pow...@canonical.com or ping powersj in #cloud-init channel via freenode.

For information on how to sign, please see the HACKING document [3].

Thanks again, and please feel free to reach out with any questions.

–
[1] http://www.canonical.com/contributors
[2] https://launchpad.net/~contributor-agreement-canonical/+members
[3] http://cloudinit.readthedocs.io/en/latest/topics/hacking.html
-- 
https://code.launchpad.net/~med/cloud-init/+git/cloud-init/+merge/367722
Your team cloud-init commiters is requested to review the proposed merge of 
~med/cloud-init:typo_verbatim into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:warn into cloud-init:master

2019-05-07 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:warn into cloud-init:master.

Requested reviews:
  cloud-init commiters (cloud-init-dev)
Related bugs:
  Bug #1508442 in cloud-init: "LOG.warn is deprecated"
  https://bugs.launchpad.net/cloud-init/+bug/1508442

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/367054
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:warn into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceCloudStack.py b/cloudinit/sources/DataSourceCloudStack.py
index d4b758f..f185dc7 100644
--- a/cloudinit/sources/DataSourceCloudStack.py
+++ b/cloudinit/sources/DataSourceCloudStack.py
@@ -95,7 +95,7 @@ class DataSourceCloudStack(sources.DataSource):
 start_time = time.time()
 url = uhelp.wait_for_url(
 urls=urls, max_wait=url_params.max_wait_seconds,
-timeout=url_params.timeout_seconds, status_cb=LOG.warn)
+timeout=url_params.timeout_seconds, status_cb=LOG.warning)
 
 if url:
 LOG.debug("Using metadata source: '%s'", url)
diff --git a/cloudinit/sources/DataSourceEc2.py b/cloudinit/sources/DataSourceEc2.py
index ac28f1d..5c017bf 100644
--- a/cloudinit/sources/DataSourceEc2.py
+++ b/cloudinit/sources/DataSourceEc2.py
@@ -208,7 +208,7 @@ class DataSourceEc2(sources.DataSource):
 start_time = time.time()
 url = uhelp.wait_for_url(
 urls=urls, max_wait=url_params.max_wait_seconds,
-timeout=url_params.timeout_seconds, status_cb=LOG.warn)
+timeout=url_params.timeout_seconds, status_cb=LOG.warning)
 
 if url:
 self.metadata_address = url2base[url]
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~raharper/cloud-init:fix/add-netplan-fixed-grat-arp-spelling into cloud-init:master

2019-05-03 Thread Dan Watkins



Diff comments:

> diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py
> index 4d19f56..91046b5 100644
> --- a/cloudinit/net/network_state.py
> +++ b/cloudinit/net/network_state.py
> @@ -707,6 +707,13 @@ class NetworkStateInterpreter(object):
>  item_params = dict((key, value) for (key, value) in
> item_cfg.items() if key not in
> NETWORK_V2_KEY_FILTER)
> +if 'gratuitious-arp' in item_params['parameters']:
> +params = item_params['parameters']

This might as well move out of the if, so save the duplicate lookup the line 
above.

> +old_key = 'gratuitious-arp'
> +new_key = 'gratuitous-arp'
> +params[new_key] = copy.copy(params[old_key])
> +del params[old_key]
> +
>  v1_cmd = {
>  'type': cmd_type,
>  'name': item_name,


-- 
https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/366935
Your team cloud-init commiters is requested to review the proposed merge of 
~raharper/cloud-init:fix/add-netplan-fixed-grat-arp-spelling into 
cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~raharper/cloud-init:fix/add-netplan-fixed-grat-arp-spelling into cloud-init:master

2019-05-03 Thread Dan Watkins
Do we need to handle older versions of netplan that _don't_ understand the new 
key?
-- 
https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/366935
Your team cloud-init commiters is requested to review the proposed merge of 
~raharper/cloud-init:fix/add-netplan-fixed-grat-arp-spelling into 
cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:snap into cloud-init:master

2019-04-26 Thread Dan Watkins
This is what I've done in terms of testing:

```
lxc launch ubuntu:b c1
lxc file push *.snap c1/tmp/
lxc exec c1 /bin/bash

snap install /tmp/*.snap
/snap/bin/cloud-init clean --logs
/snap/bin/cloud-init init --local
/snap/bin/cloud-init init
```

http://paste.ubuntu.com/p/TXrh2n4mpk/ is /var/log/cloud-init.log from the 
container after doing the above.
-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/366572
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:snap into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~liaoishere/cloud-init:sysconfig-bond-options into cloud-init:master

2019-04-24 Thread Dan Watkins
The proposal to merge ~liaoishere/cloud-init:sysconfig-bond-options into 
cloud-init:master has been updated.

Commit message changed to:

sysconfig: support more bonding options

Currently, only a few bonding parameters can be configured on
sysconfig systems. This patch aims to support more parameters
documented on the docs site.

For more details, see:
https://code.launchpad.net/~liaoishere/cloud-init/+git/cloud-init/+merge/366377
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~liaoishere/cloud-init:sysconfig-bond-options into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~liaoishere/cloud-init:sysconfig-bond-options into cloud-init:master

2019-04-24 Thread Dan Watkins
I've just kicked off a CI run, the results will be reported in here once 
complete.
-- 
https://code.launchpad.net/~liaoishere/cloud-init/+git/cloud-init/+merge/366377
Your team cloud-init commiters is requested to review the proposed merge of 
~liaoishere/cloud-init:sysconfig-bond-options into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~goneri/cloud-init:build-on-freebsd into cloud-init:master

2019-04-23 Thread Dan Watkins
This looks like a reasonable set of changes to me; is there any good way for me 
to test it?  (Perhaps in a container/public cloud?)

Also, do we need to open a bug for converting this script to Python 3 in 
preparation for dropping Python 2 support from cloud-init[0]?


[0] https://lists.launchpad.net/cloud-init/msg00170.html
-- 
https://code.launchpad.net/~goneri/cloud-init/+git/cloud-init/+merge/366333
Your team cloud-init commiters is requested to review the proposed merge of 
~goneri/cloud-init:build-on-freebsd into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lintian into cloud-init:master

2019-04-19 Thread Dan Watkins
I've tested that this (a) fixes the package build on CentOS, and (b) bash 
completion still works from the installed package.

(I attempted to build the package on SuSE too, but I ran in to issues that I 
believe are unrelated to this change.)
-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/366306
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lintian into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lintian into cloud-init:master

2019-04-19 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:lintian into cloud-init:master.

Commit message:
packages: update rpm specs for new bash completion path

LP: #1825444

Requested reviews:
  cloud-init commiters (cloud-init-dev)
Related bugs:
  Bug #1825444 in cloud-init (Ubuntu): ""make rpm" fails on CentOS after commit 
947d3c208"
  https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1825444

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/366306
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lintian into cloud-init:master.
diff --git a/packages/redhat/cloud-init.spec.in b/packages/redhat/cloud-init.spec.in
index 6b2022b..057a578 100644
--- a/packages/redhat/cloud-init.spec.in
+++ b/packages/redhat/cloud-init.spec.in
@@ -205,7 +205,9 @@ fi
 %dir%{_sysconfdir}/cloud/templates
 %config(noreplace)  %{_sysconfdir}/cloud/templates/*
 %config(noreplace) %{_sysconfdir}/rsyslog.d/21-cloudinit.conf
-%{_sysconfdir}/bash_completion.d/cloud-init
+
+# Bash completion script
+%{_datadir}/bash-completion/completions/cloud-init
 
 %{_libexecdir}/%{name}
 %dir %{_sharedstatedir}/cloud
diff --git a/packages/suse/cloud-init.spec.in b/packages/suse/cloud-init.spec.in
index 26894b3..004b875 100644
--- a/packages/suse/cloud-init.spec.in
+++ b/packages/suse/cloud-init.spec.in
@@ -120,7 +120,9 @@ version_pys=$(cd "%{buildroot}" && find . -name version.py -type f)
 %config(noreplace) %{_sysconfdir}/cloud/cloud.cfg.d/README
 %dir   %{_sysconfdir}/cloud/templates
 %config(noreplace) %{_sysconfdir}/cloud/templates/*
-%{_sysconfdir}/bash_completion.d/cloud-init
+
+# Bash completion script
+%{_datadir}/bash-completion/completions/cloud-init
 
 %{_sysconfdir}/dhcp/dhclient-exit-hooks.d/hook-dhclient
 %{_sysconfdir}/NetworkManager/dispatcher.d/hook-network-manager
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:lintian into cloud-init:master

2019-04-18 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:lintian into cloud-init:master.

Commit message:
setup.py: install bash completion script in new location

Per lintian, this is the path at which bash completion scripts should
now be installed.

Requested reviews:
  Server Team CI bot (server-team-bot): continuous-integration
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/366283
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:lintian into cloud-init:master.
diff --git a/setup.py b/setup.py
index 186e215..fcaf26f 100755
--- a/setup.py
+++ b/setup.py
@@ -245,13 +245,14 @@ if not in_virtualenv():
 INITSYS_ROOTS[k] = "/" + INITSYS_ROOTS[k]
 
 data_files = [
-(ETC + '/bash_completion.d', ['bash_completion/cloud-init']),
 (ETC + '/cloud', [render_tmpl("config/cloud.cfg.tmpl")]),
 (ETC + '/cloud/cloud.cfg.d', glob('config/cloud.cfg.d/*')),
 (ETC + '/cloud/templates', glob('templates/*')),
 (USR_LIB_EXEC + '/cloud-init', ['tools/ds-identify',
 'tools/uncloud-init',
 'tools/write-ssh-key-fingerprints']),
+(USR + '/share/bash-completion/completions',
+ ['bash_completion/cloud-init']),
 (USR + '/share/doc/cloud-init', [f for f in glob('doc/*') if is_f(f)]),
 (USR + '/share/doc/cloud-init/examples',
 [f for f in glob('doc/examples/*') if is_f(f)]),
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into cloud-init:ubuntu/devel

2019-04-10 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/365823
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.
diff --git a/cloudinit/sources/DataSourceScaleway.py b/cloudinit/sources/DataSourceScaleway.py
index 54bfc1f..b573b38 100644
--- a/cloudinit/sources/DataSourceScaleway.py
+++ b/cloudinit/sources/DataSourceScaleway.py
@@ -171,11 +171,10 @@ def query_data_api(api_type, api_address, retries, timeout):
 
 class DataSourceScaleway(sources.DataSource):
 dsname = "Scaleway"
+update_events = {'network': [EventType.BOOT_NEW_INSTANCE, EventType.BOOT]}
 
 def __init__(self, sys_cfg, distro, paths):
 super(DataSourceScaleway, self).__init__(sys_cfg, distro, paths)
-self.update_events = {
-'network': {EventType.BOOT_NEW_INSTANCE, EventType.BOOT}}
 
 self.ds_cfg = util.mergemanydict([
 util.get_cfg_by_path(sys_cfg, ["datasource", "Scaleway"], {}),
diff --git a/cloudinit/sources/__init__.py b/cloudinit/sources/__init__.py
index 1604932..e6966b3 100644
--- a/cloudinit/sources/__init__.py
+++ b/cloudinit/sources/__init__.py
@@ -164,6 +164,9 @@ class DataSource(object):
 # A datasource which supports writing network config on each system boot
 # would call update_events['network'].add(EventType.BOOT).
 
+# Default: generate network config on new instance id (first boot).
+update_events = {'network': set([EventType.BOOT_NEW_INSTANCE])}
+
 # N-tuple listing default values for any metadata-related class
 # attributes cached on an instance by a process_data runs. These attribute
 # values are reset via clear_cached_attrs during any update_metadata call.
@@ -188,9 +191,6 @@ class DataSource(object):
 self.vendordata = None
 self.vendordata_raw = None
 
-# Default: generate network config on new instance id (first boot).
-self.update_events = {'network': {EventType.BOOT_NEW_INSTANCE}}
-
 self.ds_cfg = util.get_cfg_by_path(
 self.sys_cfg, ("datasource", self.dsname), {})
 if not self.ds_cfg:
diff --git a/cloudinit/sources/tests/test_init.py b/cloudinit/sources/tests/test_init.py
index cb1912b..6378e98 100644
--- a/cloudinit/sources/tests/test_init.py
+++ b/cloudinit/sources/tests/test_init.py
@@ -575,21 +575,6 @@ class TestDataSource(CiTestCase):
 " events: New instance first boot",
 self.logs.getvalue())
 
-def test_data_sources_cant_mutate_update_events_for_others(self):
-"""update_events shouldn't be changed for other DSes (LP: #1819913)"""
-
-class ModifyingDS(DataSource):
-
-def __init__(self, sys_cfg, distro, paths):
-# This mirrors what DataSourceAzure does which causes LP:
-# #1819913
-DataSource.__init__(self, sys_cfg, distro, paths)
-self.update_events['network'].add(EventType.BOOT)
-
-before_update_events = copy.deepcopy(self.datasource.update_events)
-ModifyingDS(self.sys_cfg, self.distro, self.paths)
-self.assertEqual(before_update_events, self.datasource.update_events)
-
 
 class TestRedactSensitiveData(CiTestCase):
 
diff --git a/debian/changelog b/debian/changelog
index f869278..0630854 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,10 @@
+cloud-init (18.5-62-g6322c2dd-0ubuntu1) disco; urgency=medium
+
+  * New upstream snapshot.
+- Revert "DataSource: move update_events from a class to an instance..."
+
+ -- Daniel Watkins   Wed, 10 Apr 2019 16:23:25 -0400
+
 cloud-init (18.5-61-gb76714c3-0ubuntu1) disco; urgency=medium
 
   * New upstream snapshot.
diff --git a/tests/unittests/test_datasource/test_scaleway.py b/tests/unittests/test_datasource/test_scaleway.py
index 3bfd752..f96bf0a 100644
--- a/tests/unittests/test_datasource/test_scaleway.py
+++ b/tests/unittests/test_datasource/test_scaleway.py
@@ -7,7 +7,6 @@ import requests
 
 from cloudinit import helpers
 from cloudinit import settings
-from cloudinit.event import EventType
 from cloudinit.sources import DataSourceScaleway
 
 from cloudinit.tests.helpers import mock, HttprettyTestCase, CiTestCase
@@ -404,9 +403,3 @@ class TestDataSourceScaleway(HttprettyTestCase):
 
 netcfg = self.datasource.network_config
 self.assertEqual(netcfg, '0xdeadbeef')
-
-def test_update_events_is_correct(self):
-"""ensure update_events contains correct data"""
-self.assertEqual(
-{'network': {EventType.BOOT_NEW_INSTANCE, EventType.BOOT}},
-  

[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:revert into cloud-init:master

2019-04-10 Thread Dan Watkins
The proposal to merge ~daniel-thewatkins/cloud-init/+git/cloud-init:revert into 
cloud-init:master has been updated.

Commit message changed to:

Revert "DataSource: move update_events from a class to an instance attribute"

Moving update_events from a class attribute to an instance attribute
means that it doesn't exist on DataSource objects that are unpickled,
causing tracebacks on cloud-init upgrade.

As this change is only required for cloud-init installations which don't
utilise ds-identify, we're backing it out to be reintroduced once the
upgrade path bug has been addressed.

This reverts commit f2fd6eac4407e60d0e98826ab03847dda4cde138.

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/365820
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:revert into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into cloud-init:ubuntu/devel

2019-04-10 Thread Dan Watkins
Dan Watkins has proposed merging 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.

Requested reviews:
  Server Team CI bot (server-team-bot): continuous-integration
  cloud-init commiters (cloud-init-dev)
Related bugs:
  Bug #1801364 in cloud-init: "persisting OpenStack metadata fails"
  https://bugs.launchpad.net/cloud-init/+bug/1801364
  Bug #1815109 in cloud-init: "cloud-final.service: "cloud-init modules --mode 
final" exit with "KeyError: 'modules-init'" after upgrade to version 18.2"
  https://bugs.launchpad.net/cloud-init/+bug/1815109
  Bug #1823084 in cloud-init: "DataSourceAzure doesn't rebuild network-config 
after reboot"
  https://bugs.launchpad.net/cloud-init/+bug/1823084

For more details, see:
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/365803
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:ubuntu/devel into 
cloud-init:ubuntu/devel.
diff --git a/cloudinit/cmd/main.py b/cloudinit/cmd/main.py
index 933c019..a5446da 100644
--- a/cloudinit/cmd/main.py
+++ b/cloudinit/cmd/main.py
@@ -632,13 +632,14 @@ def status_wrapper(name, args, data_d=None, link_d=None):
 'start': None,
 'finished': None,
 }
+
 if status is None:
 status = {'v1': {}}
-for m in modes:
-status['v1'][m] = nullstatus.copy()
 status['v1']['datasource'] = None
-elif mode not in status['v1']:
-status['v1'][mode] = nullstatus.copy()
+
+for m in modes:
+if m not in status['v1']:
+status['v1'][m] = nullstatus.copy()
 
 v1 = status['v1']
 v1['stage'] = mode
diff --git a/cloudinit/config/cc_ubuntu_advantage.py b/cloudinit/config/cc_ubuntu_advantage.py
index 5e082bd..f488123 100644
--- a/cloudinit/config/cc_ubuntu_advantage.py
+++ b/cloudinit/config/cc_ubuntu_advantage.py
@@ -1,150 +1,143 @@
-# Copyright (C) 2018 Canonical Ltd.
-#
 # This file is part of cloud-init. See LICENSE file for license information.
 
-"""Ubuntu advantage: manage ubuntu-advantage offerings from Canonical."""
+"""ubuntu_advantage: Configure Ubuntu Advantage support services"""
 
-import sys
 from textwrap import dedent
 
-from cloudinit import log as logging
+import six
+
 from cloudinit.config.schema import (
 get_schema_doc, validate_cloudconfig_schema)
+from cloudinit import log as logging
 from cloudinit.settings import PER_INSTANCE
-from cloudinit.subp import prepend_base_command
 from cloudinit import util
 
 
-distros = ['ubuntu']
-frequency = PER_INSTANCE
+UA_URL = 'https://ubuntu.com/advantage'
 
-LOG = logging.getLogger(__name__)
+distros = ['ubuntu']
 
 schema = {
 'id': 'cc_ubuntu_advantage',
 'name': 'Ubuntu Advantage',
-'title': 'Install, configure and manage ubuntu-advantage offerings',
+'title': 'Configure Ubuntu Advantage support services',
 'description': dedent("""\
-This module provides configuration options to setup ubuntu-advantage
-subscriptions.
-
-.. note::
-Both ``commands`` value can be either a dictionary or a list. If
-the configuration provided is a dictionary, the keys are only used
-to order the execution of the commands and the dictionary is
-merged with any vendor-data ubuntu-advantage configuration
-provided. If a ``commands`` is provided as a list, any vendor-data
-ubuntu-advantage ``commands`` are ignored.
-
-Ubuntu-advantage ``commands`` is a dictionary or list of
-ubuntu-advantage commands to run on the deployed machine.
-These commands can be used to enable or disable subscriptions to
-various ubuntu-advantage products. See 'man ubuntu-advantage' for more
-information on supported subcommands.
-
-.. note::
-   Each command item can be a string or list. If the item is a list,
-   'ubuntu-advantage' can be omitted and it will automatically be
-   inserted as part of the command.
+Attach machine to an existing Ubuntu Advantage support contract and
+enable or disable support services such as Livepatch, ESM,
+FIPS and FIPS Updates. When attaching a machine to Ubuntu Advantage,
+one can also specify services to enable.  When the 'enable'
+list is present, any named service will be enabled and all absent
+services will remain disabled.
+
+Note that when enabling FIPS or FIPS updates you will need to schedule
+a reboot to ensure the machine is running the FIPS-compliant kernel.
+See :ref:`Power State Change` for information on how to configure
+cloud-init to perform this reboot.
 """),
 'distros': distros,
 'examples': [dedent(""

Re: [Cloud-init-dev] [Merge] ~daniel-thewatkins/cloud-init/+git/cloud-init:feature/cc-uaclient into cloud-init:master

2019-04-01 Thread Dan Watkins
This is a re-submission of Chad's branch (which was previously reviewed at 
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/362161).
  I've addressed much of the review feedback there (generally in separate 
commits, so review my new commits to understand the changes I've made); the 
remaining unaddressed item I have inserted inline here for discussion.

Diff comments:

> diff --git a/cloudinit/config/cc_ubuntu_advantage.py 
> b/cloudinit/config/cc_ubuntu_advantage.py
> index 5e082bd..9732ffa 100644
> --- a/cloudinit/config/cc_ubuntu_advantage.py
> +++ b/cloudinit/config/cc_ubuntu_advantage.py
> @@ -159,14 +134,26 @@ def maybe_install_ua_tools(cloud):
>  
>  
>  def handle(name, cfg, cloud, log, args):
> -cfgin = cfg.get('ubuntu-advantage')
> -if cfgin is None:
> +if 'ubuntu-advantage' in cfg:
> +msg = ('Deprecated configuration key "ubuntu-advantage" provided.'

Ryan said:
> do we really want to raise runtime error on deprecated config?
> I think logging a warning and doing the ua attach is a better outcome, right?

> +   ' Expected underscore delimited "ubuntu_advantage"')
> +LOG.error(msg)
> +raise RuntimeError(msg)
> +ua_section = cfg.get('ubuntu_advantage')
> +if ua_section is None:
>  LOG.debug(("Skipping module named %s,"
> -   " no 'ubuntu-advantage' key in configuration"), name)
> +   " no 'ubuntu_advantage' configuration found"), name)
>  return
> -
>  validate_cloudconfig_schema(cfg, schema)
> +if 'commands' in ua_section:
> +msg = (
> +'Deprecated configuration "ubuntu_advantage: commands" provided.'
> +' Expected "token"')
> +LOG.error(msg)
> +raise RuntimeError(msg)
> +
>  maybe_install_ua_tools(cloud)
> -run_commands(cfgin.get('commands', []))
> +configure_ua(token=ua_section.get('token'),
> + entitlements=ua_section.get('entitlements'))
>  
>  # vi: ts=4 expandtab


-- 
https://code.launchpad.net/~daniel-thewatkins/cloud-init/+git/cloud-init/+merge/365366
Your team cloud-init commiters is requested to review the proposed merge of 
~daniel-thewatkins/cloud-init/+git/cloud-init:feature/cc-uaclient into 
cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


  1   2   >