Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
For in-tree testing there are two main options; first writting unittests; under tests/unitests/test_handlers/test_.py are most of the config module tests. For an integration test, the next best way is to build the package, via ./package/[brpm|bddeb] which can create a .rpm or deb you can install to a target system. We typically use LXD to create a system container where we install the new package. You can run a single module like list: cloud-init --debug single --name cc_name_of_module --frequency always -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Hey Ryan, I'm having trouble finding the best way to test the code, I can always launch a full stack for this but that feels a bit overpowered for testing just one module. Is there any way I could test this on my development environment, for the network in my first commit I could test it with the `cloud-init devel` command: cloud-init devel net-convert --debug -D centos -O sysconfig --network-data sysconfig_test.yaml -k yaml -d . -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
cloud-init really relies on networking in almost all cases; unless networking config is disable, cloud-init is going to generate a network config for the system and as I understand sysconfig, we will need to set NETWORKING=yes to ensure that the OS networking service brings up networking. However, instead of *rendering* a new file, cloudinit.net.sysconfig would instead utilize whatever primitive class cc_sysconfig would use to read existing content and then toggle the specified value. This would avoid networking from clobbering values it doesn't care about. -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Ryan, thanks for the suggestions. Using write_files module with append mode seems counter-intuitive, although is a reasonable workaround. I will look into implementing that cc_sysconfig module. What is your thought on making cloudinit.net renderer to leave the `/etc/sysconfig/network` file as is, i.e: not writing the `NETWORKING=yes` at all, if the user does not specify anything with the sysconfig module? Would there be any downside to this? -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
The best way to update the /etc/sysconfig/network contents with custom values is via the write_files in append mode. A more robust solution would be to implement a config module: cc_sysconfig.py which could read in existing files from /etc/sysconfig, parse them with util.load_shell_content() which returns a dict of key=values; and let users provide new key/value: sysconfig: network: mode: append # content: NTPSERVERARGS: "minpoll 3 maxpoll 3" RES_OPTION: "rotate" Append mode would merely append the KEY=VALUE pairs from the content dictionary in the file specified (/etc/sysconfig/network). Update would replace the existing KEY with the new value, if KEY wasn't present, it would append it. overwrite would allow writing a completely new file with the KEY/VALUE pairs provided. Some challenges include retaining existing comments, indicating where cloud-init made changes within the file, handling interactions from cloudinit.net whose's renderers currently update/write values. -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Hi Ryan, > There is no direct integration in the module for dhcp provided ntp servers at > this time. It's possible but there isn't a great interface (AFAIK) to query > your OS for the dhcp response for a given interface. sorry but I don't understand your comment. The proposed change has nothing to do with what the OS stores from a DHCP response, neither with NTP specifically, in fact the point here is being able to have options defined which cannot come from a DHCP response. The related bug may be misleading, but the OP in the related bug just wants to permanently define NTP options in the `/etc/sysconfig/network` file, not save options from a DHCP response. Cheers! -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init Commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
There is no direct integration in the module for dhcp provided ntp servers at this time. It's possible but there isn't a great interface (AFAIK) to query your OS for the dhcp response for a given interface. -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Hi, @Ryan: the `NTPSERVERARGS` was just an example from the original issue #1819966, what got me there in the first place was that when I wrote anything to `/etc/sysconfig/network` it was replaced by static content generated by cloud-init's `render_network_state` function on reboot. >From what I can see in the `cc_write_files` module, using that to generate the >`/etc/sysconfig/network` file will only work on instance creation (default >`frequency = PER_INSTANCE` for that module), and only if it comes before the >network handlers. If you look into `cloudinit/net/sysconfig.py` the original >code uses `util.write_file` method which overwrites the contents of the >parameter file by default. @Brendan: yes the ntp servers are obtained via DHCP and that is not an issue here, just the case that we may want persistent controllable settings independent of the DHCP data received. At the time there is not really a clean way to configure the `/etc/sysconfig/network` with arbitrary options, which I guess is desirable. The rationale for this is to be able to specify persistent network options that are not obtainable from DHCP (i.e: resolver option `rotate`), either because the information received from the DHCP server is incomplete, there are independent networks for each of my NICs thus one DHCP information taking priority on the other, because you may not have in control of what information in send from the DHCP server, or even just because you want to =) That said, is there a better suited place in the code to add this functionality? Cheers! -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
does the cc_ntp module work alongside dhcp? what if your ntp servers are obtained via dhcp? -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
Hi Manual, Thanks for creating a potential fix. The various network-config formats that cloud-init handles don't lend themselves to writing arbitrary configuration values to renderer specific files, in your case, the NTPSERVERARGS is a sysconfig renderer specific value that doesn't have an equivalent in other backends, such as eni (etc/network/interfaces) nor netplan. To address the bug I think the best option is use cloud-init write_files config module to append the string you're interested in. #cloud-config write_files: - content: | NTPSERVERARGS="minpoll 3 maxpoll 3" RES_OPTIONS="rotate" path: /etc/sysconfig/network append: true There is also the cc_ntp module, which controls enabling ntp service, and configuring but this would only cover your ntp configuration using a custom template. #cloud-config ntp: enabled: true config: template: | ## template:jinja # My NTP Client config {% if pools -%}# pools{% endif %} {% for pool in pools -%} pool {{pool}} iburst minpoll 3 maxpoll 4 {% endfor %} {%- if servers %}# servers {% endif %} {% for server in servers -%} server {{server}} iburst {% endfor %} -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp
Re: [Cloud-init-dev] [Merge] ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master
I do not feel comfortable with having a dummy handler for this, I guess the purpose of the `network` key is exclusive for network interfaces and network interface interaction. Having a handle_ function is perhaps not the best choice. For now this does what is intended, will eventually discuss this with the cloud-init team and maybe this will be done in some other way. On a final note, when running tox tests I get several errors/warnings, however: ``` ... Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00) ``` -- https://code.launchpad.net/~t0rrant/cloud-init/+git/cloud-init/+merge/371948 Your team cloud-init commiters is requested to review the proposed merge of ~t0rrant/cloud-init:1819966-sysconfig-options into cloud-init:master. ___ Mailing list: https://launchpad.net/~cloud-init-dev Post to : cloud-init-dev@lists.launchpad.net Unsubscribe : https://launchpad.net/~cloud-init-dev More help : https://help.launchpad.net/ListHelp