Public bug reported:
This bug was first reported at https://github.com/kubernetes-sigs/image-
builder/issues/712 and occurs when the v2 network configuration
directive "set-name" is used in conjunction with interface specific DNS
settings.
Cloud-Provider: VMware, but does not matter as this bug is distro and DS
agnostic
Config: The metadata was set to the following:
instance-id: "wlan-1-md-0-775d8846bf-9bfrd"
local-hostname: "wlan-1-md-0-775d8846bf-9bfrd"
wait-on-network:
ipv4: false
ipv6: false
network:
version: 2
ethernets:
id0:
match:
macaddress: "00:50:56:a1:d8:a7"
set-name: "eth0"
wakeonlan: true
addresses:
- "10.196.27.122/28"
gateway4: "10.196.27.126"
nameservers:
addresses:
- "10.102.102.132"
- "10.102.102.133"
- "10.90.24.1"
search:
- "refsa1.bn.schiff.telekom.de"
Again though, the platform is likely irrelevant as this seems to be a
bug introduced with https://github.com/canonical/cloud-
init/commit/abd2da5777195e7e432b0d53a3f7f29d071dd50e, and can occur on
any platform with and datasource as long as network v2 config is used
with "set-name" and interface specific DNS.
The bug can be surfaced via unit test by patching the v21.3 version of
Cloud-Init with the following:
diff --git a/cloudinit/net/tests/test_network_state.py
b/cloudinit/net/tests/test_network_state.py
index 84e8308a..c0aa78a0 100644
--- a/cloudinit/net/tests/test_network_state.py
+++ b/cloudinit/net/tests/test_network_state.py
@@ -52,6 +52,7 @@ network:
eth1:
match:
macaddress: '66:77:88:99:00:11'
+ set-name: "eth2"
nameservers:
search: [foo.local, bar.local]
addresses: [4.4.4.4]
Next, run the affected test from the root of the Cloud-Init source tree:
$ make clean_pyc && \
PYTHONPATH="$(pwd)" \
python3 -m pytest -v cloudinit/net/tests/test_network_state.py
The output will resemble the following:
====================================================================== test
session starts
======================================================================
platform darwin -- Python 3.9.7, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 --
/usr/local/opt/[email protected]/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/akutz/Projects/cloud-init, configfile: tox.ini
collected 10 items
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_empty_v1_config_gets_network_state
PASSED [ 10%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_empty_v2_config_gets_network_state
PASSED [ 20%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_missing_version_returns_none
PASSED [ 30%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_unknown_versions_returns_none
PASSED [ 40%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_valid_config_gets_network_state
PASSED [ 50%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_version_2_passes_self_as_config
PASSED [ 60%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfigV2::test_version_2_ignores_renderer_key
PASSED [ 70%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseNameservers::test_v1_nameservers_valid
PASSED [ 80%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseNameservers::test_v1_nameservers_invalid
PASSED [ 90%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseNameservers::test_v2_nameservers
FAILED [100%]
===========================================================================
FAILURES
============================================================================
_____________________________________________________
TestNetworkStateParseNameservers.test_v2_nameservers
______________________________________________________
self =
<cloudinit.net.tests.test_network_state.TestNetworkStateParseNameservers object
at 0x10a7db9d0>
def test_v2_nameservers(self):
> config =
self._parse_network_state_from_config(V2_CONFIG_NAMESERVERS)
cloudinit/net/tests/test_network_state.py:139:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _
cloudinit/net/tests/test_network_state.py:114: in
_parse_network_state_from_config
return network_state.parse_net_config_data(yaml['network'])
cloudinit/net/network_state.py:1074: in parse_net_config_data
nsi.parse_config(skip_broken=skip_broken)
cloudinit/net/network_state.py:261: in parse_config
self.parse_config_v2(skip_broken=skip_broken)
cloudinit/net/network_state.py:310: in parse_config_v2
self._v2_common(command)
cloudinit/net/network_state.py:722: in _v2_common
self._handle_individual_nameserver(name_cmd, iface)
cloudinit/net/network_state.py:91: in decorator
return func(self, command, *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _
self = <cloudinit.net.network_state.NetworkStateInterpreter object at
0x10a7e4f10>
command = {'address': ['4.4.4.4'], 'search': ['foo.local', 'bar.local'],
'type': 'nameserver'}, iface = 'eth1'
@ensure_command_keys(['address'])
def _handle_individual_nameserver(self, command, iface):
_iface = self._network_state.get('interfaces')
nameservers, search = self._parse_dns(command)
> _iface[iface]['dns'] = {'nameservers': nameservers, 'search':
search}
E KeyError: 'eth1'
cloudinit/net/network_state.py:546: KeyError
-----------------------------------------------------------------------
Captured log call
-----------------------------------------------------------------------
2021-10-08 00:57:41 DEBUG
cloudinit.net.network_state:network_state.py:670 v2(ethernets) -> v1(physical):
{'type': 'physical', 'name': 'eth0', 'mac_address': '00:11:22:33:44:55',
'match': {'macaddress': '00:11:22:33:44:55'}}
2021-10-08 00:57:41 DEBUG
cloudinit.net.network_state:network_state.py:670 v2(ethernets) -> v1(physical):
{'type': 'physical', 'name': 'eth2', 'mac_address': '66:77:88:99:00:11',
'match': {'macaddress': '66:77:88:99:00:11'}}
2021-10-08 00:57:41 DEBUG
cloudinit.net.network_state:network_state.py:711 v2_common: handling config:
{'eth0': {'match': {'macaddress': '00:11:22:33:44:55'}, 'nameservers':
{'search': ['spam.local', 'eggs.local'], 'addresses': ['8.8.8.8']}}, 'eth1':
{'match': {'macaddress': '66:77:88:99:00:11'}, 'set-name': 'eth2',
'nameservers': {'search': ['foo.local', 'bar.local'], 'addresses':
['4.4.4.4']}}}
=======================================================================
warnings summary
========================================================================
../../../../usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py:1183
/usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py:1183:
PytestDeprecationWarning: The --strict option is deprecated, use
--strict-markers instead.
self.issue_config_time_warning(
conftest.py:68
/Users/akutz/Projects/cloud-init/conftest.py:68:
PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.
@pytest.yield_fixture(autouse=True)
conftest.py:169
/Users/akutz/Projects/cloud-init/conftest.py:169:
PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.
def httpretty():
-- Docs: https://docs.pytest.org/en/stable/warnings.html
==================================================================== short
test summary info
====================================================================
FAILED
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseNameservers::test_v2_nameservers
- KeyError: 'eth1'
============================================================ 1 failed, 9
passed, 3 warnings in 0.60s
============================================================
This is is occurring because the code to iterate over the interfaces
when configuring interface-specific DNS is using the original interface
name, not the one from the "set-name" directive. The following patch
corrects the issue:
diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py
index 95b064f0..06ff8e96 100644
--- a/cloudinit/net/network_state.py
+++ b/cloudinit/net/network_state.py
@@ -543,7 +543,11 @@ class
NetworkStateInterpreter(metaclass=CommandHandlerMeta):
def _handle_individual_nameserver(self, command, iface):
_iface = self._network_state.get('interfaces')
nameservers, search = self._parse_dns(command)
- _iface[iface]['dns'] = {'nameservers': nameservers, 'search':
search}
+ try:
+ _iface[iface]['dns'] = {'nameservers': nameservers, 'search':
search}
+ except:
+ print("original iface name: %s\niface dict: %s\n" % (iface,
_iface))
+ raise
@ensure_command_keys(['destination'])
def handle_route(self, command):
diff --git a/cloudinit/net/tests/test_network_state.py
b/cloudinit/net/tests/test_network_state.py
index 84e8308a..45e99171 100644
--- a/cloudinit/net/tests/test_network_state.py
+++ b/cloudinit/net/tests/test_network_state.py
@@ -52,6 +52,7 @@ network:
eth1:
match:
macaddress: '66:77:88:99:00:11'
+ set-name: "ens92"
nameservers:
search: [foo.local, bar.local]
addresses: [4.4.4.4]
Now the above test passes. A PR will be opened on Cloud-Init's GitHub
repository with the above patch.
** Affects: cloud-init
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to cloud-init.
https://bugs.launchpad.net/bugs/1946493
Title:
Using "set-name" with interface specific DNS with a v2 network config
causes a KeyError
Status in cloud-init:
New
Bug description:
This bug was first reported at https://github.com/kubernetes-
sigs/image-builder/issues/712 and occurs when the v2 network
configuration directive "set-name" is used in conjunction with
interface specific DNS settings.
Cloud-Provider: VMware, but does not matter as this bug is distro and DS
agnostic
Config: The metadata was set to the following:
instance-id: "wlan-1-md-0-775d8846bf-9bfrd"
local-hostname: "wlan-1-md-0-775d8846bf-9bfrd"
wait-on-network:
ipv4: false
ipv6: false
network:
version: 2
ethernets:
id0:
match:
macaddress: "00:50:56:a1:d8:a7"
set-name: "eth0"
wakeonlan: true
addresses:
- "10.196.27.122/28"
gateway4: "10.196.27.126"
nameservers:
addresses:
- "10.102.102.132"
- "10.102.102.133"
- "10.90.24.1"
search:
- "refsa1.bn.schiff.telekom.de"
Again though, the platform is likely irrelevant as this seems to be a
bug introduced with https://github.com/canonical/cloud-
init/commit/abd2da5777195e7e432b0d53a3f7f29d071dd50e, and can occur on
any platform with and datasource as long as network v2 config is used
with "set-name" and interface specific DNS.
The bug can be surfaced via unit test by patching the v21.3 version of
Cloud-Init with the following:
diff --git a/cloudinit/net/tests/test_network_state.py
b/cloudinit/net/tests/test_network_state.py
index 84e8308a..c0aa78a0 100644
--- a/cloudinit/net/tests/test_network_state.py
+++ b/cloudinit/net/tests/test_network_state.py
@@ -52,6 +52,7 @@ network:
eth1:
match:
macaddress: '66:77:88:99:00:11'
+ set-name: "eth2"
nameservers:
search: [foo.local, bar.local]
addresses: [4.4.4.4]
Next, run the affected test from the root of the Cloud-Init source
tree:
$ make clean_pyc && \
PYTHONPATH="$(pwd)" \
python3 -m pytest -v cloudinit/net/tests/test_network_state.py
The output will resemble the following:
======================================================================
test session starts
======================================================================
platform darwin -- Python 3.9.7, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
-- /usr/local/opt/[email protected]/bin/python3.9
cachedir: .pytest_cache
rootdir: /Users/akutz/Projects/cloud-init, configfile: tox.ini
collected 10 items
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_empty_v1_config_gets_network_state
PASSED [ 10%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_empty_v2_config_gets_network_state
PASSED [ 20%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_missing_version_returns_none
PASSED [ 30%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_unknown_versions_returns_none
PASSED [ 40%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_valid_config_gets_network_state
PASSED [ 50%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfig::test_version_2_passes_self_as_config
PASSED [ 60%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseConfigV2::test_version_2_ignores_renderer_key
PASSED [ 70%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseNameservers::test_v1_nameservers_valid
PASSED [ 80%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseNameservers::test_v1_nameservers_invalid
PASSED [ 90%]
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseNameservers::test_v2_nameservers
FAILED [100%]
===========================================================================
FAILURES
============================================================================
_____________________________________________________
TestNetworkStateParseNameservers.test_v2_nameservers
______________________________________________________
self =
<cloudinit.net.tests.test_network_state.TestNetworkStateParseNameservers object
at 0x10a7db9d0>
def test_v2_nameservers(self):
> config =
self._parse_network_state_from_config(V2_CONFIG_NAMESERVERS)
cloudinit/net/tests/test_network_state.py:139:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _
cloudinit/net/tests/test_network_state.py:114: in
_parse_network_state_from_config
return network_state.parse_net_config_data(yaml['network'])
cloudinit/net/network_state.py:1074: in parse_net_config_data
nsi.parse_config(skip_broken=skip_broken)
cloudinit/net/network_state.py:261: in parse_config
self.parse_config_v2(skip_broken=skip_broken)
cloudinit/net/network_state.py:310: in parse_config_v2
self._v2_common(command)
cloudinit/net/network_state.py:722: in _v2_common
self._handle_individual_nameserver(name_cmd, iface)
cloudinit/net/network_state.py:91: in decorator
return func(self, command, *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _
self = <cloudinit.net.network_state.NetworkStateInterpreter object at
0x10a7e4f10>
command = {'address': ['4.4.4.4'], 'search': ['foo.local', 'bar.local'],
'type': 'nameserver'}, iface = 'eth1'
@ensure_command_keys(['address'])
def _handle_individual_nameserver(self, command, iface):
_iface = self._network_state.get('interfaces')
nameservers, search = self._parse_dns(command)
> _iface[iface]['dns'] = {'nameservers': nameservers, 'search':
search}
E KeyError: 'eth1'
cloudinit/net/network_state.py:546: KeyError
-----------------------------------------------------------------------
Captured log call
-----------------------------------------------------------------------
2021-10-08 00:57:41 DEBUG
cloudinit.net.network_state:network_state.py:670 v2(ethernets) -> v1(physical):
{'type': 'physical', 'name': 'eth0', 'mac_address': '00:11:22:33:44:55',
'match': {'macaddress': '00:11:22:33:44:55'}}
2021-10-08 00:57:41 DEBUG
cloudinit.net.network_state:network_state.py:670 v2(ethernets) -> v1(physical):
{'type': 'physical', 'name': 'eth2', 'mac_address': '66:77:88:99:00:11',
'match': {'macaddress': '66:77:88:99:00:11'}}
2021-10-08 00:57:41 DEBUG
cloudinit.net.network_state:network_state.py:711 v2_common: handling config:
{'eth0': {'match': {'macaddress': '00:11:22:33:44:55'}, 'nameservers':
{'search': ['spam.local', 'eggs.local'], 'addresses': ['8.8.8.8']}}, 'eth1':
{'match': {'macaddress': '66:77:88:99:00:11'}, 'set-name': 'eth2',
'nameservers': {'search': ['foo.local', 'bar.local'], 'addresses':
['4.4.4.4']}}}
=======================================================================
warnings summary
========================================================================
../../../../usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py:1183
/usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py:1183:
PytestDeprecationWarning: The --strict option is deprecated, use
--strict-markers instead.
self.issue_config_time_warning(
conftest.py:68
/Users/akutz/Projects/cloud-init/conftest.py:68:
PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.
@pytest.yield_fixture(autouse=True)
conftest.py:169
/Users/akutz/Projects/cloud-init/conftest.py:169:
PytestDeprecationWarning: @pytest.yield_fixture is deprecated.
Use @pytest.fixture instead; they are the same.
def httpretty():
-- Docs: https://docs.pytest.org/en/stable/warnings.html
====================================================================
short test summary info
====================================================================
FAILED
cloudinit/net/tests/test_network_state.py::TestNetworkStateParseNameservers::test_v2_nameservers
- KeyError: 'eth1'
============================================================ 1 failed, 9
passed, 3 warnings in 0.60s
============================================================
This is is occurring because the code to iterate over the interfaces
when configuring interface-specific DNS is using the original
interface name, not the one from the "set-name" directive. The
following patch corrects the issue:
diff --git a/cloudinit/net/network_state.py
b/cloudinit/net/network_state.py
index 95b064f0..06ff8e96 100644
--- a/cloudinit/net/network_state.py
+++ b/cloudinit/net/network_state.py
@@ -543,7 +543,11 @@ class
NetworkStateInterpreter(metaclass=CommandHandlerMeta):
def _handle_individual_nameserver(self, command, iface):
_iface = self._network_state.get('interfaces')
nameservers, search = self._parse_dns(command)
- _iface[iface]['dns'] = {'nameservers': nameservers, 'search':
search}
+ try:
+ _iface[iface]['dns'] = {'nameservers': nameservers,
'search': search}
+ except:
+ print("original iface name: %s\niface dict: %s\n" % (iface,
_iface))
+ raise
@ensure_command_keys(['destination'])
def handle_route(self, command):
diff --git a/cloudinit/net/tests/test_network_state.py
b/cloudinit/net/tests/test_network_state.py
index 84e8308a..45e99171 100644
--- a/cloudinit/net/tests/test_network_state.py
+++ b/cloudinit/net/tests/test_network_state.py
@@ -52,6 +52,7 @@ network:
eth1:
match:
macaddress: '66:77:88:99:00:11'
+ set-name: "ens92"
nameservers:
search: [foo.local, bar.local]
addresses: [4.4.4.4]
Now the above test passes. A PR will be opened on Cloud-Init's GitHub
repository with the above patch.
To manage notifications about this bug go to:
https://bugs.launchpad.net/cloud-init/+bug/1946493/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
