Re: [Cloud-init-dev] [Merge] ('~chad.smith/cloud-init', ':', 'feature/azure-to-network-v2') into ('cloud-init', ':', 'master')

2019-08-05 Thread Server Team CI bot
Review: Approve continuous-integration

PASSED: Continuous integration, rev:850c0d90bdf26f45b5d771e77caee3d32d73ba74
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1026/
Executed test runs:
SUCCESS: Checkout
SUCCESS: Unit & Style Tests
SUCCESS: Ubuntu LTS: Build
SUCCESS: Ubuntu LTS: Integration
IN_PROGRESS: Declarative: Post Actions


Click here to trigger a rebuild:
https://jenkins.ubuntu.com/server/job/cloud-init-ci/1026//rebuild
-- 
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/370970
Your team cloud-init commiters is requested to review the proposed merge of 
~chad.smith/cloud-init:feature/azure-to-network-v2 into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~chad.smith/cloud-init:feature/azure-to-network-v2 into cloud-init:master

2019-08-05 Thread Chad Smith
Chad Smith has proposed merging 
~chad.smith/cloud-init:feature/azure-to-network-v2 into cloud-init:master.

Commit message:
azure: provide fallback network config v2 for fallback nic

To enable Azure to send network v2, net.generate_fallback_config now
accepts a network_version param to specify whether version 1 or version
2 network configuration is desired. DataSourceAzure.network_config now
requests network_version=2 when generating fallback configuration.


This branch avoids moving generate_fallback_config to v2 exclusively because we 
first need NetworkState to be able to parse v2 as fallback config is passed 
directly into NetworkState.

Requested reviews:
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/370970
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~chad.smith/cloud-init:feature/azure-to-network-v2 into cloud-init:master.
diff --git a/cloudinit/net/__init__.py b/cloudinit/net/__init__.py
index f3cec79..fa15b96 100644
--- a/cloudinit/net/__init__.py
+++ b/cloudinit/net/__init__.py
@@ -264,7 +264,8 @@ def find_fallback_nic(blacklist_drivers=None):
 return None
 
 
-def generate_fallback_config(blacklist_drivers=None, config_driver=None):
+def generate_fallback_config(
+blacklist_drivers=None, config_driver=None, network_version=1):
 """Determine which attached net dev is most likely to have a connection and
generate network state to run dhcp on that interface"""
 
@@ -272,25 +273,30 @@ def generate_fallback_config(blacklist_drivers=None, config_driver=None):
 config_driver = False
 
 target_name = find_fallback_nic(blacklist_drivers=blacklist_drivers)
-if target_name:
-target_mac = read_sys_net_safe(target_name, 'address')
+if not target_name:
+# can't read any interfaces addresses (or there are none); give up
+return None
+target_mac = read_sys_net_safe(target_name, 'address')
+driver_params = {}
+if config_driver:
+driver = device_driver(target_name)
+if driver:
+driver_params = {'driver': driver,
+ 'device_id': device_devid(target_name)}
+if network_version == 1:
+# TODO(Drop network v1 once NetworkState parses v2)
 nconf = {'config': [], 'version': 1}
 cfg = {'type': 'physical', 'name': target_name,
'mac_address': target_mac, 'subnets': [{'type': 'dhcp'}]}
-# inject the device driver name, dev_id into config if enabled and
-# device has a valid device driver value
-if config_driver:
-driver = device_driver(target_name)
-if driver:
-cfg['params'] = {
-'driver': driver,
-'device_id': device_devid(target_name),
-}
+if driver_params:
+cfg['params'] = driver_params
 nconf['config'].append(cfg)
 return nconf
-else:
-# can't read any interfaces addresses (or there are none); give up
-return None
+cfg = {'dhcp4': True, 'set-name': target_name,
+   'match': {'macaddress': target_mac.lower()}}
+cfg['match'].update(driver_params)
+nconf = {'ethernets': {target_name: cfg}, 'version': 2}
+return nconf
 
 
 def extract_physdevs(netcfg):
diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py
index 0ca576b..1e6ead2 100644
--- a/cloudinit/net/network_state.py
+++ b/cloudinit/net/network_state.py
@@ -343,7 +343,6 @@ class NetworkStateInterpreter(object):
  ]
 }
 '''
-
 interfaces = self._network_state.get('interfaces', {})
 iface = interfaces.get(command['name'], {})
 for param, val in command.get('params', {}).items():
diff --git a/cloudinit/sources/DataSourceAzure.py b/cloudinit/sources/DataSourceAzure.py
index d2fad9b..ab47507 100755
--- a/cloudinit/sources/DataSourceAzure.py
+++ b/cloudinit/sources/DataSourceAzure.py
@@ -1268,7 +1268,8 @@ def parse_network_config(imds_metadata):
 LOG.debug('Azure: generating fallback configuration')
 # generate a network config, blacklist picking mlx4_core devs
 netconfig = net.generate_fallback_config(
-blacklist_drivers=blacklist, config_driver=True)
+blacklist_drivers=blacklist, config_driver=True,
+network_version=2)
 evt.description = "network config from fallback"
 return netconfig
 
diff --git a/tests/unittests/test_datasource/test_azure.py b/tests/unittests/test_datasource/test_azure.py
index 2de2aea..3ed9e4e 100644
--- a/tests/unittests/test_datasource/test_azure.py
+++ b/tests/unittests/test_datasource/test_azure.py
@@ -997,7 +997,7 @@ scbus-1 on xpt0 bus 0
 netconfig = dsrc.network_config
 self.assertEqual(netconfig, fallback_config)
 

[Cloud-init-dev] [Merge] ~chad.smith/cloud-init:feature/ec2-secondary-nics into cloud-init:master

2019-08-05 Thread Chad Smith
Chad Smith has proposed merging 
~chad.smith/cloud-init:feature/ec2-secondary-nics into cloud-init:master.

Commit message:
ec2: render secondary IPs on primary nic when present in metadata

Parse local-ipv4s and subnet-ipv4-cidr-block on EC2 metadata version
2018-09-24 to obtain secondary nic private IPs and network mask for
the primary nic.

In adding this feature, convert DataSourceEc2.network_config to
emit network version 2 instead of version 1.

To allow for retaining original network config behavior on earlier
distribution series, surface a datasource config option
configure_secondary_ips which defaults to True on tip.

Older/stable distribution series will set configure_secondary_ips
default to False.

Requested reviews:
  Server Team CI bot (server-team-bot): continuous-integration
  cloud-init commiters (cloud-init-dev)

For more details, see:
https://code.launchpad.net/~chad.smith/cloud-init/+git/cloud-init/+merge/369792
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~chad.smith/cloud-init:feature/ec2-secondary-nics into cloud-init:master.
diff --git a/cloudinit/sources/DataSourceEc2.py b/cloudinit/sources/DataSourceEc2.py
index 5c017bf..f39b73f 100644
--- a/cloudinit/sources/DataSourceEc2.py
+++ b/cloudinit/sources/DataSourceEc2.py
@@ -54,7 +54,7 @@ class DataSourceEc2(sources.DataSource):
 
 # Priority ordered list of additional metadata versions which will be tried
 # for extended metadata content. IPv6 support comes in 2016-09-02
-extended_metadata_versions = ['2016-09-02']
+extended_metadata_versions = ['2018-09-24', '2016-09-02']
 
 # Setup read_url parameters per get_url_params.
 url_max_wait = 120
@@ -332,8 +332,13 @@ class DataSourceEc2(sources.DataSource):
 macs_to_nics = {net.get_interface_mac(iface): iface}
 net_md = self.metadata.get('network')
 if isinstance(net_md, dict):
+# SRU_BLOCKER: xenial, bionic and disco should default
+# configure_secondary_ips to False to retain original behavior on
+# those releases.
 result = convert_ec2_metadata_network_config(
-net_md, macs_to_nics=macs_to_nics, fallback_nic=iface)
+net_md, macs_to_nics=macs_to_nics, fallback_nic=iface,
+config_secondary_ips=util.is_true(
+self.ds_cfg.get('configure_secondary_ips', True)))
 
 # RELEASE_BLOCKER: xenial should drop the below if statement,
 # because the issue being addressed doesn't exist pre-netplan.
@@ -373,7 +378,7 @@ class DataSourceEc2(sources.DataSource):
 if not self.wait_for_metadata_service():
 return {}
 api_version = self.get_metadata_api_version()
-crawled_metadata = {}
+crawled_metadata = {'_metadata_api_version': api_version}
 try:
 crawled_metadata['user-data'] = ec2.get_instance_userdata(
 api_version, self.metadata_address)
@@ -388,7 +393,6 @@ class DataSourceEc2(sources.DataSource):
 LOG, "Failed reading from metadata address %s",
 self.metadata_address)
 return {}
-crawled_metadata['_metadata_api_version'] = api_version
 return crawled_metadata
 
 
@@ -523,8 +527,9 @@ def _collect_platform_data():
 return data
 
 
-def convert_ec2_metadata_network_config(network_md, macs_to_nics=None,
-fallback_nic=None):
+def convert_ec2_metadata_network_config(
+network_md, macs_to_nics=None, fallback_nic=None,
+config_secondary_ips=True):
 """Convert ec2 metadata to network config version 1 data dict.
 
 @param: network_md: 'network' portion of EC2 metadata.
@@ -535,25 +540,52 @@ def convert_ec2_metadata_network_config(network_md, macs_to_nics=None,
not provided, get_interfaces_by_mac is called to get it from the OS.
 @param: fallback_nic: Optionally provide the primary nic interface name.
This nic will be guaranteed to minimally have a dhcp4 configuration.
+@param: config_secondary_ips: Boolean set True to configure any
+   secondary IPs described by the metadata service.
 
-@return A dict of network config version 1 based on the metadata and macs.
+@return A dict of network config version 2 based on the metadata and macs.
 """
-netcfg = {'version': 1, 'config': []}
+netcfg = {'version': 2, 'ethernets': {}}
 if not macs_to_nics:
 macs_to_nics = net.get_interfaces_by_mac()
 macs_metadata = network_md['interfaces']['macs']
 for mac, nic_name in macs_to_nics.items():
+dev_config = {}
 nic_metadata = macs_metadata.get(mac)
 if not nic_metadata:
 continue  # Not a physical nic represented in metadata
-nic_cfg = {'type': 'physical', 'name': nic_name, 'subnets': []}
-nic_cfg['mac_address'] = mac
-if (nic_name == fallback_nic or 

Re: [Cloud-init-dev] [Merge] ~jasonzio/cloud-init:disableIMDSnetcfg into cloud-init:master

2019-08-05 Thread Chad Smith
This content will not be merged because there is an active fix for systemd that 
has been pushed upstream and will be landing imminently in Ubuntu Bionic, Disco 
and Eoan.


https://bugs.launchpad.net/ubuntu/+source/systemd/+bug/1835581

it requires no changes to cloud-init to support this. I have already verified 
the fix for cloud-init, expectation is that the fix will be published this week 
for Bionic/Disco and likely next week for Eoan.
-- 
https://code.launchpad.net/~jasonzio/cloud-init/+git/cloud-init/+merge/364012
Your team cloud-init commiters is requested to review the proposed merge of 
~jasonzio/cloud-init:disableIMDSnetcfg into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp


[Cloud-init-dev] [Merge] ~jasonzio/cloud-init:disableIMDSnetcfg into cloud-init:master

2019-08-05 Thread Chad Smith
The proposal to merge ~jasonzio/cloud-init:disableIMDSnetcfg into 
cloud-init:master has been updated.

Status: Needs review => Rejected

For more details, see:
https://code.launchpad.net/~jasonzio/cloud-init/+git/cloud-init/+merge/364012
-- 
Your team cloud-init commiters is requested to review the proposed merge of 
~jasonzio/cloud-init:disableIMDSnetcfg into cloud-init:master.

___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help   : https://help.launchpad.net/ListHelp