[Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
The proposal to merge ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master has been updated. Status: Needs review => Work in progress For more details, see: https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 -- Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
For in-tree testing there are two main options; first writting unittests; under tests/unitests/test_handlers/test_.py are most of the config module tests. For an integration test, the next best way is to build the package, via ./package/[brpm|bddeb] which can create a .rpm or deb you can install to a target system. We typically use LXD to create a system container where we install the new package. You can run a single module like list: cloud-init --debug single --name cc_name_of_module --frequency always -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Hey Ryan, I'm having trouble finding the best way to test the code, I can always launch a full stack for this but that feels a bit overpowered for testing just one module. Is there any way I could test this on my development environment, for the network in my first commit I could test it with the `cloud-init devel` command: cloud-init devel net-convert --debug -D centos -O sysconfig --network-data sysconfig_test.yaml -k yaml -d . -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
cloud-init really relies on networking in almost all cases; unless networking config is disable, cloud-init is going to generate a network config for the system and as I understand sysconfig, we will need to set NETWORKING=yes to ensure that the OS networking service brings up networking. However, instead of *rendering* a new file, cloudinit.net.sysconfig would instead utilize whatever primitive class cc_sysconfig would use to read existing content and then toggle the specified value. This would avoid networking from clobbering values it doesn't care about. -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
The best way to update the /etc/sysconfig/network contents with custom values is via the write_files in append mode. A more robust solution would be to implement a config module: cc_sysconfig.py which could read in existing files from /etc/sysconfig, parse them with util.load_shell_content() which returns a dict of key=values; and let users provide new key/value: sysconfig: network: mode: append # content: NTPSERVERARGS: "minpoll 3 maxpoll 3" RES_OPTION: "rotate" Append mode would merely append the KEY=VALUE pairs from the content dictionary in the file specified (/etc/sysconfig/network). Update would replace the existing KEY with the new value, if KEY wasn't present, it would append it. overwrite would allow writing a completely new file with the KEY/VALUE pairs provided. Some challenges include retaining existing comments, indicating where cloud-init made changes within the file, handling interactions from cloudinit.net whose's renderers currently update/write values. -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Hi Ryan, > There is no direct integration in the module for dhcp provided ntp servers at > this time. It's possible but there isn't a great interface (AFAIK) to query > your OS for the dhcp response for a given interface. sorry but I don't understand your comment. The proposed change has nothing to do with what the OS stores from a DHCP response, neither with NTP specifically, in fact the point here is being able to have options defined which cannot come from a DHCP response. The related bug may be misleading, but the OP in the related bug just wants to permanently define NTP options in the `/etc/sysconfig/network` file, not save options from a DHCP response. Cheers! -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
There is no direct integration in the module for dhcp provided ntp servers at this time. It's possible but there isn't a great interface (AFAIK) to query your OS for the dhcp response for a given interface. -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Hi, @Ryan: the `NTPSERVERARGS` was just an example from the original issue #1819966, what got me there in the first place was that when I wrote anything to `/etc/sysconfig/network` it was replaced by static content generated by cloud-init's `render_network_state` function on reboot. >From what I can see in the `cc_write_files` module, using that to generate the >`/etc/sysconfig/network` file will only work on instance creation (default >`frequency = PER_INSTANCE` for that module), and only if it comes before the >network handlers. If you look into `cloudinit/net/sysconfig.py` the original >code uses `util.write_file` method which overwrites the contents of the >parameter file by default. @Brendan: yes the ntp servers are obtained via DHCP and that is not an issue here, just the case that we may want persistent controllable settings independent of the DHCP data received. At the time there is not really a clean way to configure the `/etc/sysconfig/network` with arbitrary options, which I guess is desirable. The rationale for this is to be able to specify persistent network options that are not obtainable from DHCP (i.e: resolver option `rotate`), either because the information received from the DHCP server is incomplete, there are independent networks for each of my NICs thus one DHCP information taking priority on the other, because you may not have in control of what information in send from the DHCP server, or even just because you want to =) That said, is there a better suited place in the code to add this functionality? Cheers! -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
does the cc_ntp module work alongside dhcp? what if your ntp servers are obtained via dhcp? -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Hi Manual,
Thanks for creating a potential fix. The various network-config formats that
cloud-init handles don't lend themselves to writing arbitrary configuration
values to renderer specific files, in your case, the NTPSERVERARGS is a
sysconfig renderer specific value that doesn't have an equivalent in other
backends, such as eni (etc/network/interfaces) nor netplan.
To address the bug I think the best option is use cloud-init write_files config
module to append the string you're interested in.
#cloud-config
write_files:
- content: |
NTPSERVERARGS="minpoll 3 maxpoll 3"
RES_OPTIONS="rotate"
path: /etc/sysconfig/network
append: true
There is also the cc_ntp module, which controls enabling ntp service, and
configuring
but this would only cover your ntp configuration using a custom template.
#cloud-config
ntp:
enabled: true
config:
template: |
## template:jinja
# My NTP Client config
{% if pools -%}# pools{% endif %}
{% for pool in pools -%}
pool {{pool}} iburst minpoll 3 maxpoll 4
{% endfor %}
{%- if servers %}# servers
{% endif %}
{% for server in servers -%}
server {{server}} iburst
{% endfor %}
--
https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948
Your team cloud-init commiters is requested to review the proposed merge of
~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master.
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
The proposal to merge ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master has been updated. Description changed to: Added a new handle (handle_sysconfig) to deal with the new key. Using the options passed in the cloud-init config file a /etc/sysconfig/network file is generated contemplating those options. For more details, see: https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 -- Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
[Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Manuel Torrinha has proposed merging
~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master.
Commit message:
Added support for arbitrary options in sysconfig
These options should be added within the `sysconfig` key, as such:
```
network:
version: 2
sysconfig:
NTPSERVERARGS: "minpoll 3 maxpoll 4"
RES_OPTIONS: "rotate"
```
LP: #1819966
Signed-off-by: Manuel Torrinha
Requested reviews:
cloud-init commiters (cloud-init-dev)
For more details, see:
https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948
I do not feel comfortable with having a dummy handler for this, I guess
the purpose of the `network` key is exclusive for network interfaces and
network interface interaction. Having a handle_ function is perhaps not
the best choice.
For now this does what is intended, will eventually discuss this with the
cloud-init team and maybe this will be done in some other way.
On a final note, when running tox tests I get several errors/warnings, however:
```
...
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
```
--
Your team cloud-init commiters is requested to review the proposed merge of
~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master.
diff --git a/cloudinit/net/network_state.py b/cloudinit/net/network_state.py
index c0c415d..a15df67 100644
--- a/cloudinit/net/network_state.py
+++ b/cloudinit/net/network_state.py
@@ -652,6 +652,14 @@ class NetworkStateInterpreter(object):
LOG.debug('v2(ethernets) -> v1(physical):\n%s', phy_cmd)
self.handle_physical(phy_cmd)
+def handle_sysconfig(self, command):
+'''
+sysconfig:
+ NTPSERVERARGS: "minpoll 3 maxpoll 4"
+ RES_OPTIONS: "rotate"
+'''
+LOG.debug('[handle_sysconfig] %s', command.items())
+
def handle_vlans(self, command):
'''
v2_vlans = {
diff --git a/cloudinit/net/sysconfig.py b/cloudinit/net/sysconfig.py
index be5dede..c608d68 100644
--- a/cloudinit/net/sysconfig.py
+++ b/cloudinit/net/sysconfig.py
@@ -724,6 +724,10 @@ class Renderer(renderer.Renderer):
if network_state.use_ipv6:
netcfg.append('NETWORKING_IPV6=yes')
netcfg.append('IPV6_AUTOCONF=no')
+if network_state.config and network_state.config['sysconfig']:
+for opt, val in network_state.config['sysconfig'].items():
+netcfg.append("{}=\"{}\"".format(opt, val))
+
util.write_file(sysconfig_path,
"\n".join(netcfg) + "\n", file_mode)
___
Mailing list: https://launchpad.net/~cloud-init-dev
Post to : cloud-init-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~cloud-init-dev
More help : https://help.launchpad.net/ListHelp
