Hi RubenThank you very much, I have watched this and all other related online videos to best assist with this. I can 100% confirm when I curl the virtual router from the VM that I see the userdata perfectly fine, but something keeps overwriting my custom changes when I boot the VM. Something else keeps setting the VM hostname, to the "Name" specified when the VM is created in Cloudstack. Its as if another datasource is triggering it each time.
It would be nice if someone could assist with their cloud-init configuration and potential userdata that helps set custom values when a VM is booted, specifically the hostname.
On 8/10/24 21:11, Ruben Bosch wrote:
Hi Granwille, Wido has done a great talk about cloudinit. Check it out here: https://www.youtube.com/watch?v=seknX44Ktfc. You can check the details of your supplied userdata in the virtual router to see whether what you stored in the UI/API is what is supplied to the VM. On Fri, Aug 9, 2024 at 11:39 AM Granwille Strauss <granwi...@namhost.com.invalid> wrote:Hi Guys Any help on this? Surely I am using *userdata* right via the UI? Or should I log this on GitHub instead? On 8/8/24 10:32, Granwille Strauss wrote: Hi I push *userdata* manually when I deploy a VM and still it does not want to work: I have this in place now: #cloud-config hostname: hostname.example.com fqdn: hostname.example.com prefer_fqdn_over_hostname: true create_hostname_file: true preserve_hostname: true Screenshot: https://www.awesomescreenshot.com/image/49925105?key=a51b729871fb9239579c1604c43677a3 Instead, it keeps pulling this instead and not accepting my manual inputs: 2024-08-08 08:21:40,488 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/a3da80ae-1cc3-4c3b-9204-5d2a14f58fef/vendor-data2.txt.i - wb: [600] 308 bytes 2024-08-08 08:21:40,489 - cc_set_hostname.py[DEBUG]: Setting the hostname to VMNAME FROM UI (VMNAME FROM UI) 2024-08-08 08:21:40,489 - subp.py[DEBUG]: Running command ['hostnamectl', 'set-hostname', 'VMNAME FROM UI'] with allowed return codes [0] (shell=False, capture=True) 2024-08-08 08:21:40,512 - __init__.py[DEBUG]: Non-persistently setting the system hostname to VMNAME FROM UI 2024-08-08 08:21:40,512 - subp.py[DEBUG]: Running command ['hostname', 'VMNAME FROM UI'] with allowed return codes [0] (shell=False, capture=True) 2024-08-08 08:21:40,522 - atomic_helper.py[DEBUG]: Atomically writing to file /var/lib/cloud/data/set-hostname (via temporary file /var/lib/cloud/data/tmpskrfsk7k) - w: [644] 53 bytes/chars 2024-08-08 08:21:40,523 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/a3da80ae-1cc3-4c3b-9204-5d2a14f58fef/sem/consume_data - wb: [644] 24 bytes 2024-08-08 08:21:40,524 - helpers.py[DEBUG]: Running consume_data using lock (<FileLock using file '/var/lib/cloud/instances/a3da80ae-1cc3-4c3b-9204-5d2a14f58fef/sem/consume_data'>) 2024-08-08 08:21:40,524 - handlers.py[DEBUG]: start: init-network/consume-user-data: reading and applying user-data 2024-08-08 08:21:40,525 - stages.py[DEBUG]: Added default handler for {'text/cloud-config', 'text/cloud-config-jsonp'} from CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] Anyone that can guide me with how to use *userdata* via the Cloudstack UI correctly, please? On 8/8/24 01:01, Granwille Strauss wrote: Hi Thank you, you were right. After creating a new template and having the root last, the growfs worked 100%. Thank you for this. This now brings me to *userdata* in cloudstack that's not working. I registered a new userdata in the UI: #cloud-config hostname: {{ ds.meta_data.custom_hostname }} fqdn: {{ ds.meta_data.custom_hostname }} prefer_fqdn_over_hostname: true create_hostname_file: true Declared the *custom_hostname *variable and when booting a VM it still fails: 2024-08-07 22:46:56,673 - stages.py[DEBUG]: Added default handler for {'text/jinja2'} from JinjaTemplatePartHandler: [['text/jinja2']] 2024-08-07 22:46:56,673 - __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (__begin__, None, 3) with frequency once-per-instance 2024-08-07 22:46:56,673 - __init__.py[DEBUG]: Calling handler ShellScriptPartHandler: [['text/x-shellscript']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler ShellScriptByFreqPartHandler: [['text/x-shellscript-per-boot']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler ShellScriptByFreqPartHandler: [['text/x-shellscript-per-instance']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler ShellScriptByFreqPartHandler: [['text/x-shellscript-per-once']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler BootHookPartHandler: [['text/cloud-boothook']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler JinjaTemplatePartHandler: [['text/jinja2']] (__begin__, None, 3) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: {'MIME-Version': '1.0', 'Content-Type': 'text/cloud-config', 'Content-Disposition': 'attachment; filename="part-001"'} 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (text/cloud-config, part-001, 3) with frequency once-per-instance 2024-08-07 22:46:56,674 - util.py[DEBUG]: Attempting to load yaml from string of length 158 with allowed root types (<class 'dict'>,) 2024-08-07 22:46:56,676 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 2 column 11: "while constructing a mapping in "<unicode string>", line 2, column 11: hostname: {{ ds.meta_data.custom_hostname }} ^ found unhashable key in "<unicode string>", line 2, column 12: hostname: {{ ds.meta_data.custom_hostname }} ^" 2024-08-07 22:46:56,676 - cloud_config.py[WARNING]: Failed at merging in cloud config part from part-001: empty cloud config 2024-08-07 22:46:56,676 - __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (__end__, None, 3) with frequency once-per-instance 2024-08-07 22:46:56,677 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt - wb: [600] 77 bytes 2024-08-07 22:46:56,677 - util.py[DEBUG]: Restoring selinux mode for /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt (recursive=False) 2024-08-07 22:46:56,678 - util.py[DEBUG]: Restoring selinux mode for /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt (recursive=False) 2024-08-07 22:46:56,679 - __init__.py[DEBUG]: Calling handler ShellScriptPartHandler: [['text/x-shellscript']] (__end__, None, 2) with frequency once-per-instance What am I doing wrong in this case? And why does the VM hostname keep defaulting to the instance name specified in the UI instead? I also created the following in */etc/cloud/cloud.cfg.d*, which injects my SSH key and updated packages and installs a few additional ones, but this too seems to fail: #cloud-config ssh_authorized_keys: - <my pub key> package_update: true package_upgrade: true packages: - epel-release - neofetch - htop - gcc - gcc-c++ - perl Here's the error: 2024-08-07 22:47:00,315 - rhel.py[DEBUG]: Using DNF for package management 2024-08-07 22:47:00,315 - subp.py[DEBUG]: Running command ['dnf', '-y', 'makecache'] with allowed return codes [0] (shell=False, capture=False) 2024-08-07 22:47:05,331 - rhel.py[DEBUG]: Using DNF for package management 2024-08-07 22:47:05,331 - subp.py[DEBUG]: Running command ['dnf', '-y', 'upgrade'] with allowed return codes [0] (shell=False, capture=False) 2024-08-07 22:47:07,103 - rhel.py[DEBUG]: Using DNF for package management 2024-08-07 22:47:07,103 - subp.py[DEBUG]: Running command ['dnf', '-y', 'install', 'epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] with allowed return codes [0] (shell=False, capture=False) 2024-08-07 22:47:08,573 - util.py[WARNING]: Failed to install packages: ['epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] 2024-08-07 22:47:08,575 - util.py[DEBUG]: Failed to install packages: ['epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 121, in handle cloud.distro.install_packages(pkglist) File "/usr/lib/python3.6/site-packages/cloudinit/distros/rhel.py", line 59, in install_packages self.package_command("install", pkgs=pkglist) File "/usr/lib/python3.6/site-packages/cloudinit/distros/rhel.py", line 201, in package_command subp.subp(cmd, capture=False) File "/usr/lib/python3.6/site-packages/cloudinit/subp.py", line 323, in subp stdout=out, stderr=err, exit_code=rc, cmd=args cloudinit.subp.ProcessExecutionError: Unexpected error while running command. Command: ['dnf', '-y', 'install', 'epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] Exit code: 1 Reason: - Stdout: - Stderr: - 2024-08-07 22:47:08,582 - cc_package_update_upgrade_install.py[WARNING]: 1 failed with exceptions, re-raising the last one 2024-08-07 22:47:08,582 - handlers.py[DEBUG]: finish: modules-final/config-package_update_upgrade_install: FAIL: running config-package_update_upgrade_install with frequency once-per-instance 2024-08-07 22:47:08,582 - util.py[WARNING]: Running module package_update_upgrade_install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py'>) failed 2024-08-07 22:47:08,582 - util.py[DEBUG]: Running module package_update_upgrade_install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py'>) failed My main concern is the hostname that I cannot seem to set no matter what I try, if I can get this at least right that would be the ultimate. How can I pass the update_hostname userdata so that I can manually enter custom hostnames when deploying VM via UI? On 8/7/24 22:52, Alexandru Stan wrote: Hello, Can’t say about everything you mentioned, but regarding the disk resizinng I can share my experience, I also had some issues before I managed to get it working. So, from what I’ve searched, growpart does not seem to work with lvm (could be wrong here though), so basic partitioning it is. Then, it seems that it extends the partition only if it’s the last one on the disk, seems to be your issue since the last partition is the swap partition. Try with the root partition being the last one, it should work fine. From: Granwille Strauss<granwi...@namhost.com.INVALID> <granwi...@namhost.com.INVALID> Sent: 7 August, 2024 20:35 To:users@cloudstack.apache.org Subject: Cloud-Init not working as per Cloudstack documentation Evening, Please note, I do not use any API service and solely rely on the Cloudstack UI. I deployed a new clean template and I followed the cloudstack documentation thoroughly to use the basics of cloud-init with cloudstack. But when I deployed my VM nothing seems to trigger at all, including userdata I registered. pwd /etc/cloud/cloud.cfg.d [root@hostname cloud.cfg.d]# ll total 32 -rw-r--r--. 1 root root 2070 Dec 4 2023 05_logging.cfg -rw-r--r--. 1 root root 22 Aug 7 17:43 49_hostkeys.cfg -rw-r--r--. 1 root root 88 Aug 7 17:45 50_growpartion.cfg -rw-r--r--. 1 root root 209 Aug 7 17:52 51_extend_volume.cfg -rw-r--r--. 1 root root 143 Aug 7 17:55 52_upgrade_packages.cfg -rw-r--r--. 1 root root 355 Aug 7 18:00 53_install_packages.cfg -rw-r--r--. 1 root root 357 Aug 7 17:42 80_user.cfg -rw-r--r--. 1 root root 167 Dec 4 2023 README For example, I deployed a new VM with its root disk overwrite from the template, which was 10 GB to 60 GB and the 50_growpartion.cfg did nothing: [root@hostname cloud.cfg.d]# cat 50_growpartion.cfg growpart: mode: auto devices: - \"/dev/vda2\" ignore_growroot_disabled: false cat 51_extend_volume.cfg runcmd: - [ cloud-init-per, always, grow_FS, resize2fs, /dev/vda2 ] grep "50_growpartion.cfg" /var/log/cloud-init.log 2024-08-07 16:50:45,881 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False) 2024-08-07 16:50:45,881 - util.py[DEBUG]: Read 88 bytes from /etc/cloud/cloud.cfg.d/50_growpartion.cfg 2024-08-07 16:50:45,928 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False) 2024-08-07 16:50:45,928 - util.py[DEBUG]: Read 88 bytes from /etc/cloud/cloud.cfg.d/50_growpartion.cfg 2024-08-07 16:50:46,046 - util.py[DEBUG]: Reading from /etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False) 2024-08-07 16:50:46,046 - util.py[DEBUG]: Read 88 bytes from /etc/cloud/cloud.cfg.d/50_growpartion.cfg lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sr0 11:0 1 1024M 0 rom vda 253:0 0 60G 0 disk ├─vda1 253:1 0 512M 0 part /boot ├─vda2 253:2 0 7.5G 0 part / └─vda3 253:3 0 2G 0 part [SWAP] The same happens with the userdata specified, because I don't use an API, I need to set custom value, such as a hostname when I boot a VM I created the following userdata in the UI as per doc: #cloud-config runcmd: - echo 'hostname {{ ds.meta_data.hostname1 }}' > /tmp/hostname - echo '{{ ds.meta_data.hostname1 }} > /etc/hostname' > /tmp/hostname - echo 'hostnamectl set-hostname {{ ds.meta_data.hostname1 }}' > /tmp/hostname And I declared the hostname1 variable, which allows me to specify the hostname under "Advanced" settings when I create a new instance. But nothing, for some reason, it keeps setting the VM name as the server hostname. But periods (.) are not allowed in the Name field for me to continue with this method and I need to be able to have special characters in the hostname for FQDNS spmetimes. The only cloud-init command that ran was the 80_user.cfg that creates a managed user, that was the only one that worked. My upgrade and install configs none ran: cat 52_upgrade_packages.cfg #cloud-config # Upgrade the instance on first boot # (ie run apt-get upgrade) # # Default: false # Aliases: apt_upgrade cat 53_install_packages.cfg #cloud-config # Install additional packages on first boot # # Default: none # # if packages are specified, this apt_update will be set to true # # packages may be supplied as a single package name or as a list # with the format [<package>, <version>] wherein the specifc # package version will be installed. packages: - epel-release - neofetch - htop And here's my cloud.cfg: cat cloud.cfg datasource_list: ["CloudStack"] datasource: CloudStack: max_wait: 120 timeout: 50 network: config: disabled users: - default system_info: default_user: name: root disable_root: false ssh_deletekeys: true cloud_init_modules: - seed_random - bootcmd - write-files - growpart - resizefs - disk_setup - mounts - set_hostname - update_hostname - update_etc_hosts - [ssh, always] - users-groups cloud_config_modules: - ssh-import-id - locale - [ set-passwords, always ] - ntp - timezone - disable-ec2-metadata - [runcmd, always] cloud_final_modules: - scripts-vendor - scripts-per-once - scripts-per-boot - scripts-per-instance - [scripts-user, always] - puppet warnings: dsid_missing_source: off Am I missing something with this? Any assistance will be greatly appreciated. -- Regards / Groete [https://www.namhost.com/t/namhost/images/mail/namhost-email-sig-logo.png]<https://www.namhost.com/> <https://www.namhost.com/> Granwille Strauss // Senior Systems Admin e:granwi...@namhost.com<mailto:granwi...@namhost.com> <granwi...@namhost.com> m: +264 81 323 1260<tel:+264813231260> <+264813231260> w:www.namhost.com<https://www.namhost.com/> <https://www.namhost.com/> [https://www.namhost.com/t/namhost/images/mail/facebook.png]<https://www.facebook.com/namhost> <https://www.facebook.com/namhost>[https://www.namhost.com/t/namhost/images/mail/twitter.png]<https://twitter.com/namhost> <https://twitter.com/namhost>[https://www.namhost.com/t/namhost/images/mail/instagram.png]<https://www.instagram.com/namhostinternetservices/> <https://www.instagram.com/namhostinternetservices/>[https://www.namhost.com/t/namhost/images/mail/linkedin.png]<https://www.linkedin.com/company/namhost> <https://www.linkedin.com/company/namhost>[https://www.namhost.com/t/namhost/images/mail/youtube.png]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> [https://www.namhost.com/t/namhost/images/mail/yourmove-email-banner.png]<https://www.namhost.com/your-move> <https://www.namhost.com/your-move> Namhost Internet Services (Pty) Ltd , 24 Black Eagle Rd, Hermanus, 7210, RSA The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go tohttps://www.namhost.com/privacy-policy -- Regards / Groete <https://www.namhost.com> Granwille Strauss // Senior Systems Admin *e:*granwi...@namhost.com *m:* +264 81 323 1260 <+264813231260> *w:*www.namhost.com <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://www.instagram.com/namhostinternetservices/> <https://www.linkedin.com/company/namhost> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.namhost.com/your-move> Namhost Internet Services (Pty) Ltd , 24 Black Eagle Rd, Hermanus, 7210, RSA The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go to https://www.namhost.com/privacy-policy -- Regards / Groete <https://www.namhost.com> Granwille Strauss // Senior Systems Admin *e:*granwi...@namhost.com *m:* +264 81 323 1260 <+264813231260> *w:*www.namhost.com <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://www.instagram.com/namhostinternetservices/> <https://www.linkedin.com/company/namhost> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.namhost.com/your-move> Namhost Internet Services (Pty) Ltd , 24 Black Eagle Rd, Hermanus, 7210, RSA The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go to https://www.namhost.com/privacy-policy -- Regards / Groete <https://www.namhost.com> Granwille Strauss // Senior Systems Admin *e:*granwi...@namhost.com *m:* +264 81 323 1260 <+264813231260> *w:*www.namhost.com <https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://www.instagram.com/namhostinternetservices/> <https://www.linkedin.com/company/namhost> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> <https://www.namhost.com/your-move> Namhost Internet Services (Pty) Ltd , 24 Black Eagle Rd, Hermanus, 7210, RSA The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go to https://www.namhost.com/privacy-policy
-- Regards / Groete <https://www.namhost.com> Granwille Strauss // Senior Systems Admin *e:* granwi...@namhost.com *m:* +264 81 323 1260 <tel:+264813231260> *w:* www.namhost.com <https://www.namhost.com/><https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://www.instagram.com/namhostinternetservices/> <https://www.linkedin.com/company/namhost> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>
<https://www.namhost.com/your-move> Namhost Internet Services (Pty) Ltd , 24 Black Eagle Rd, Hermanus, 7210, RSA
The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go to https://www.namhost.com/privacy-policy
smime.p7s
Description: S/MIME Cryptographic Signature
