Re: [Cloud-init-dev] [Merge] ~raharper/cloud-init:fix/ds_update_events into cloud-init:master

2019-03-13 Thread Ryan Harper
I've done initial testing on EC2 with vpc on Xenial; this fixes the issue where 
the Azure datasource would modify the base-class update_events attribute.

I need to validate:

1) ec-classic still works as expected (xenial, bionic, cosmic, disco)
2) azure still works as expected (xenial, bionic, cosmic, disco)
3) scaleway still works as expected (xenial, bionic, cosmic, disco)
-- 
https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/364403
Your team cloud-init commiters is requested to review the proposed merge of 
~raharper/cloud-init:fix/ds_update_events into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~raharper/cloud-init:fix/ds_update_events into cloud-init:master

2019-03-13 Thread Ryan Harper
The proposal to merge ~raharper/cloud-init:fix/ds_update_events into 
cloud-init:master has been updated.

Status: Needs review => Work in progress

For more details, see:
https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/364403
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~raharper/cloud-init:fix/ds_update_events into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~raharper/cloud-init:fix/ds_update_events into cloud-init:master

2019-03-13 Thread Ryan Harper
The proposal to merge ~raharper/cloud-init:fix/ds_update_events into 
cloud-init:master has been updated.

Commit message changed to:

DataSource: use class prop for subclass override of update_events

Subclasses of DataSource, like DataSourceAzure would update
the .update_events attribute of the base-class; this value
is propagated to any future subclass that gets instantiated.
Therefore cloud-init would *leak* these additional update_events
into subclasses which do not use them.

Convert the class attribute into a property and allow subclasses
to override the property to append/modify the attribute but only
for the subclass.

LP: #1819913

For more details, see:
https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/364403
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~raharper/cloud-init:fix/ds_update_events into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~raharper/cloud-init:fix/ds_update_events into cloud-init:master

2019-03-13 Thread Ryan Harper
Ryan Harper has proposed merging ~raharper/cloud-init:fix/ds_update_events into 
cloud-init:master.

Commit message:
DataSource: use class property to allow subclasses to override update_events

Subclasses of DataSource, like DataSourceAzure would update the .update_events
attribute of the base-class; this value is propagated to any future subclass
that gets instantiated.  Therefore cloud-init would *leak* these additional
update_events into subclasses which do not use them.

Convert the class attribute into a property and allow subclasses to
override the property to append/modify the attribute but only for
the subclass.

LP: #1819913


Requested reviews:
  cloud-init commiters (cloud-init-dev)
Related bugs:
  Bug #1819913 in cloud-init: "cloud-init on xenial may generate network config 
on every boot"
  https://bugs.launchpad.net/cloud-init/+bug/1819913

For more details, see:
https://code.launchpad.net/~raharper/cloud-init/+git/cloud-init/+merge/364403
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~raharper/cloud-init:fix/ds_update_events into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index eccbee5..cfe2d69 100644
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -6,6 +6,7 @@
 
 import base64
 import contextlib
+import copy
 import crypt
 from functools import partial
 import json
@@ -279,8 +280,6 @@ class DataSourceAzure(sources.DataSource):
 BUILTIN_DS_CONFIG])
 self.dhclient_lease_file = self.ds_cfg.get('dhclient_lease_file')
 self._network_config = None
-# Regenerate network config new_instance boot and every boot
-self.update_events['network'].add(EventType.BOOT)
 self._ephemeral_dhcp_ctx = None
 
 def __str__(self):
@@ -656,6 +655,15 @@ class DataSourceAzure(sources.DataSource):
 return
 
 @property
+def update_events(self):
+if not self._update_events:
+events = copy.deepcopy(super().update_events)
+# Regenerate network_config on each boot
+events['network'].add(EventType.BOOT)
+self._update_events = events
+return self._update_events
+
+@property
 def network_config(self):
 """Generate a network config like net.generate_fallback_network() with
the following exceptions.
diff --git a/cloudinit/sources/DataSourceEc2.py b/cloudinit/sources/DataSourceEc2.py
index ac28f1d..3bbfcc3 100644
--- a/cloudinit/sources/DataSourceEc2.py
+++ b/cloudinit/sources/DataSourceEc2.py
@@ -8,6 +8,7 @@
 #
 # This file is part of cloud-init. See LICENSE file for license information.
 
+import copy
 import os
 import time
 
@@ -283,6 +284,21 @@ class DataSourceEc2(sources.DataSource):
 return None
 
 @property
+def update_events(self):
+if not self._update_events:
+events = copy.deepcopy(super().update_events)
+# Non-VPC (aka Classic) Ec2 instances need to rewrite the
+# network config file every boot due to MAC address change.
+# RELEASE_BLOCKER: Xenial debian/postinst needs to add
+# EventType.BOOT on upgrade path for classic.
+if self.is_classic_instance():
+LOG.debug('Ec2: Classic Instance, adding update_event: %s',
+  EventType.BOOT)
+events['network'].add(EventType.BOOT)
+self._update_events = events
+return self._update_events
+
+@property
 def region(self):
 if self.cloud_name == CloudNames.AWS:
 region = self.identity.get('region')
@@ -334,6 +350,7 @@ class DataSourceEc2(sources.DataSource):
 if isinstance(net_md, dict):
 result = convert_ec2_metadata_network_config(
 net_md, macs_to_nics=macs_to_nics, fallback_nic=iface)
+<<< cloudinit/sources/DataSourceEc2.py
 
 # RELEASE_BLOCKER: xenial should drop the below if statement,
 # because the issue being addressed doesn't exist pre-netplan.
@@ -345,6 +362,8 @@ class DataSourceEc2(sources.DataSource):
 # network config file every boot due to MAC address change.
 if self.is_classic_instance():
 self.update_events['network'].add(EventType.BOOT)
+===
+>>> cloudinit/sources/DataSourceEc2.py
 else:
 LOG.warning("Metadata 'network' key not valid: %s.", net_md)
 self._network_config = result
diff --git a/cloudinit/sources/DataSourceScaleway.py b/cloudinit/sources/DataSourceScaleway.py
index b573b38..57cad13 100644
--- a/cloudinit/sources/DataSourceScaleway.py
+++ b/cloudinit/sources/DataSourceScaleway.py
@@ -5,6 +5,7 @@
 # Scaleway API:
 # https://developer.scaleway.com/#metadata
 
+import copy
 import json
 import os
 import socket
@@ -171,7 +172,6 @@ def query_data_api(api_type, api_address, retries, timeout):
 
 class