Hi Granwille, Wido has done a great talk about cloudinit. Check it out here: https://www.youtube.com/watch?v=seknX44Ktfc. You can check the details of your supplied userdata in the virtual router to see whether what you stored in the UI/API is what is supplied to the VM.
On Fri, Aug 9, 2024 at 11:39 AM Granwille Strauss <granwi...@namhost.com.invalid> wrote: > Hi Guys > > Any help on this? Surely I am using *userdata* right via the UI? Or > should I log this on GitHub instead? > On 8/8/24 10:32, Granwille Strauss wrote: > > Hi > > I push *userdata* manually when I deploy a VM and still it does not want > to work: > > I have this in place now: > > #cloud-config > hostname: hostname.example.com > fqdn: hostname.example.com > prefer_fqdn_over_hostname: true > create_hostname_file: true > preserve_hostname: true > > > Screenshot: > https://www.awesomescreenshot.com/image/49925105?key=a51b729871fb9239579c1604c43677a3 > > Instead, it keeps pulling this instead and not accepting my manual inputs: > > 2024-08-08 08:21:40,488 - util.py[DEBUG]: Writing to > /var/lib/cloud/instances/a3da80ae-1cc3-4c3b-9204-5d2a14f58fef/vendor-data2.txt.i > - wb: [600] 308 bytes > 2024-08-08 08:21:40,489 - cc_set_hostname.py[DEBUG]: Setting the hostname > to VMNAME FROM UI (VMNAME FROM UI) > 2024-08-08 08:21:40,489 - subp.py[DEBUG]: Running command ['hostnamectl', > 'set-hostname', 'VMNAME FROM UI'] with allowed return codes [0] > (shell=False, capture=True) > 2024-08-08 08:21:40,512 - __init__.py[DEBUG]: Non-persistently setting the > system hostname to VMNAME FROM UI > 2024-08-08 08:21:40,512 - subp.py[DEBUG]: Running command ['hostname', 'VMNAME > FROM UI'] with allowed return codes [0] (shell=False, capture=True) > 2024-08-08 08:21:40,522 - atomic_helper.py[DEBUG]: Atomically writing to > file /var/lib/cloud/data/set-hostname (via temporary file > /var/lib/cloud/data/tmpskrfsk7k) - w: [644] 53 bytes/chars > 2024-08-08 08:21:40,523 - util.py[DEBUG]: Writing to > /var/lib/cloud/instances/a3da80ae-1cc3-4c3b-9204-5d2a14f58fef/sem/consume_data > - wb: [644] 24 bytes > 2024-08-08 08:21:40,524 - helpers.py[DEBUG]: Running consume_data using > lock (<FileLock using file > '/var/lib/cloud/instances/a3da80ae-1cc3-4c3b-9204-5d2a14f58fef/sem/consume_data'>) > 2024-08-08 08:21:40,524 - handlers.py[DEBUG]: start: > init-network/consume-user-data: reading and applying user-data > 2024-08-08 08:21:40,525 - stages.py[DEBUG]: Added default handler for > {'text/cloud-config', 'text/cloud-config-jsonp'} from > CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] > > > Anyone that can guide me with how to use *userdata* via the Cloudstack UI > correctly, please? > On 8/8/24 01:01, Granwille Strauss wrote: > > Hi > > Thank you, you were right. After creating a new template and having the > root last, the growfs worked 100%. Thank you for this. > > This now brings me to *userdata* in cloudstack that's not working. I > registered a new userdata in the UI: > > #cloud-config hostname: {{ ds.meta_data.custom_hostname }} fqdn: {{ > ds.meta_data.custom_hostname }} prefer_fqdn_over_hostname: true > create_hostname_file: true > > Declared the *custom_hostname *variable and when booting a VM it still > fails: > > 2024-08-07 22:46:56,673 - stages.py[DEBUG]: Added default handler for > {'text/jinja2'} from JinjaTemplatePartHandler: [['text/jinja2']] > 2024-08-07 22:46:56,673 - __init__.py[DEBUG]: Calling handler > CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] > (__begin__, None, 3) with frequency once-per-instance > 2024-08-07 22:46:56,673 - __init__.py[DEBUG]: Calling handler > ShellScriptPartHandler: [['text/x-shellscript']] (__begin__, None, 2) with > frequency once-per-instance > 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler > ShellScriptByFreqPartHandler: [['text/x-shellscript-per-boot']] (__begin__, > None, 2) with frequency once-per-instance > 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler > ShellScriptByFreqPartHandler: [['text/x-shellscript-per-instance']] > (__begin__, None, 2) with frequency once-per-instance > 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler > ShellScriptByFreqPartHandler: [['text/x-shellscript-per-once']] (__begin__, > None, 2) with frequency once-per-instance > 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler > BootHookPartHandler: [['text/cloud-boothook']] (__begin__, None, 2) with > frequency once-per-instance > 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler > JinjaTemplatePartHandler: [['text/jinja2']] (__begin__, None, 3) with > frequency once-per-instance > 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: {'MIME-Version': '1.0', > 'Content-Type': 'text/cloud-config', 'Content-Disposition': 'attachment; > filename="part-001"'} > 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler > CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] > (text/cloud-config, part-001, 3) with frequency once-per-instance > 2024-08-07 22:46:56,674 - util.py[DEBUG]: Attempting to load yaml from > string of length 158 with allowed root types (<class 'dict'>,) > 2024-08-07 22:46:56,676 - util.py[WARNING]: Failed loading yaml blob. > Invalid format at line 2 column 11: "while constructing a mapping > in "<unicode string>", line 2, column 11: > hostname: {{ ds.meta_data.custom_hostname }} > ^ > found unhashable key > in "<unicode string>", line 2, column 12: > hostname: {{ ds.meta_data.custom_hostname }} > ^" > 2024-08-07 22:46:56,676 - cloud_config.py[WARNING]: Failed at merging in > cloud config part from part-001: empty cloud config > 2024-08-07 22:46:56,676 - __init__.py[DEBUG]: Calling handler > CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] > (__end__, None, 3) with frequency once-per-instance > 2024-08-07 22:46:56,677 - util.py[DEBUG]: Writing to > /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt > - wb: [600] 77 bytes > 2024-08-07 22:46:56,677 - util.py[DEBUG]: Restoring selinux mode for > /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt > (recursive=False) > 2024-08-07 22:46:56,678 - util.py[DEBUG]: Restoring selinux mode for > /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt > (recursive=False) > 2024-08-07 22:46:56,679 - __init__.py[DEBUG]: Calling handler > ShellScriptPartHandler: [['text/x-shellscript']] (__end__, None, 2) with > frequency once-per-instance > > > What am I doing wrong in this case? And why does the VM hostname keep > defaulting to the instance name specified in the UI instead? > > I also created the following in */etc/cloud/cloud.cfg.d*, which injects > my SSH key and updated packages and installs a few additional ones, but > this too seems to fail: > > #cloud-config > ssh_authorized_keys: > - <my pub key> > package_update: true > package_upgrade: true > packages: > - epel-release > - neofetch > - htop > - gcc > - gcc-c++ > - perl > > > Here's the error: > > 2024-08-07 22:47:00,315 - rhel.py[DEBUG]: Using DNF for package management > 2024-08-07 22:47:00,315 - subp.py[DEBUG]: Running command ['dnf', '-y', > 'makecache'] with allowed return codes [0] (shell=False, capture=False) > 2024-08-07 22:47:05,331 - rhel.py[DEBUG]: Using DNF for package management > 2024-08-07 22:47:05,331 - subp.py[DEBUG]: Running command ['dnf', '-y', > 'upgrade'] with allowed return codes [0] (shell=False, capture=False) > 2024-08-07 22:47:07,103 - rhel.py[DEBUG]: Using DNF for package management > 2024-08-07 22:47:07,103 - subp.py[DEBUG]: Running command ['dnf', '-y', > 'install', 'epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] > with allowed return codes [0] (shell=False, capture=False) > 2024-08-07 22:47:08,573 - util.py[WARNING]: Failed to install packages: > ['epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] > 2024-08-07 22:47:08,575 - util.py[DEBUG]: Failed to install packages: > ['epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] > Traceback (most recent call last): > File > "/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", > line 121, in handle > cloud.distro.install_packages(pkglist) > File "/usr/lib/python3.6/site-packages/cloudinit/distros/rhel.py", line > 59, in install_packages > self.package_command("install", pkgs=pkglist) > File "/usr/lib/python3.6/site-packages/cloudinit/distros/rhel.py", line > 201, in package_command > subp.subp(cmd, capture=False) > File "/usr/lib/python3.6/site-packages/cloudinit/subp.py", line 323, in > subp > stdout=out, stderr=err, exit_code=rc, cmd=args > cloudinit.subp.ProcessExecutionError: Unexpected error while running > command. > Command: ['dnf', '-y', 'install', 'epel-release', 'neofetch', 'htop', > 'gcc', 'gcc-c++', 'perl'] > Exit code: 1 > Reason: - > Stdout: - > Stderr: - > 2024-08-07 22:47:08,582 - cc_package_update_upgrade_install.py[WARNING]: 1 > failed with exceptions, re-raising the last one > 2024-08-07 22:47:08,582 - handlers.py[DEBUG]: finish: > modules-final/config-package_update_upgrade_install: FAIL: running > config-package_update_upgrade_install with frequency once-per-instance > 2024-08-07 22:47:08,582 - util.py[WARNING]: Running module > package_update_upgrade_install (<module > 'cloudinit.config.cc_package_update_upgrade_install' from > '/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py'>) > failed > 2024-08-07 22:47:08,582 - util.py[DEBUG]: Running module > package_update_upgrade_install (<module > 'cloudinit.config.cc_package_update_upgrade_install' from > '/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py'>) > failed > > > My main concern is the hostname that I cannot seem to set no matter what I > try, if I can get this at least right that would be the ultimate. How can I > pass the update_hostname userdata so that I can manually enter custom > hostnames when deploying VM via UI? > On 8/7/24 22:52, Alexandru Stan wrote: > > Hello, > > Can’t say about everything you mentioned, but regarding the disk resizinng I > can share my experience, I also had some issues before I managed to get it > working. So, from what I’ve searched, growpart does not seem to work with lvm > (could be wrong here though), so basic partitioning it is. Then, it seems > that it extends the partition only if it’s the last one on the disk, seems to > be your issue since the last partition is the swap partition. Try with the > root partition being the last one, it should work fine. > > > From: Granwille Strauss <granwi...@namhost.com.INVALID> > <granwi...@namhost.com.INVALID> > Sent: 7 August, 2024 20:35 > To: users@cloudstack.apache.org > Subject: Cloud-Init not working as per Cloudstack documentation > > > Evening, > > Please note, I do not use any API service and solely rely on the Cloudstack > UI. > > I deployed a new clean template and I followed the cloudstack documentation > thoroughly to use the basics of cloud-init with cloudstack. But when I > deployed my VM nothing seems to trigger at all, including userdata I > registered. > pwd > /etc/cloud/cloud.cfg.d > [root@hostname cloud.cfg.d]# ll > total 32 > -rw-r--r--. 1 root root 2070 Dec 4 2023 05_logging.cfg > -rw-r--r--. 1 root root 22 Aug 7 17:43 49_hostkeys.cfg > -rw-r--r--. 1 root root 88 Aug 7 17:45 50_growpartion.cfg > -rw-r--r--. 1 root root 209 Aug 7 17:52 51_extend_volume.cfg > -rw-r--r--. 1 root root 143 Aug 7 17:55 52_upgrade_packages.cfg > -rw-r--r--. 1 root root 355 Aug 7 18:00 53_install_packages.cfg > -rw-r--r--. 1 root root 357 Aug 7 17:42 80_user.cfg > -rw-r--r--. 1 root root 167 Dec 4 2023 README > > > For example, I deployed a new VM with its root disk overwrite from the > template, which was 10 GB to 60 GB and the 50_growpartion.cfg did nothing: > [root@hostname cloud.cfg.d]# cat 50_growpartion.cfg > growpart: > mode: auto > devices: > - \"/dev/vda2\" > ignore_growroot_disabled: false > cat 51_extend_volume.cfg > runcmd: > - [ cloud-init-per, always, grow_FS, resize2fs, /dev/vda2 ] > > grep "50_growpartion.cfg" /var/log/cloud-init.log > 2024-08-07 16:50:45,881 - util.py[DEBUG]: Reading from > /etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False) > 2024-08-07 16:50:45,881 - util.py[DEBUG]: Read 88 bytes from > /etc/cloud/cloud.cfg.d/50_growpartion.cfg > 2024-08-07 16:50:45,928 - util.py[DEBUG]: Reading from > /etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False) > 2024-08-07 16:50:45,928 - util.py[DEBUG]: Read 88 bytes from > /etc/cloud/cloud.cfg.d/50_growpartion.cfg > 2024-08-07 16:50:46,046 - util.py[DEBUG]: Reading from > /etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False) > 2024-08-07 16:50:46,046 - util.py[DEBUG]: Read 88 bytes from > /etc/cloud/cloud.cfg.d/50_growpartion.cfg > lsblk > NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT > sr0 11:0 1 1024M 0 rom > vda 253:0 0 60G 0 disk > ├─vda1 253:1 0 512M 0 part /boot > ├─vda2 253:2 0 7.5G 0 part / > └─vda3 253:3 0 2G 0 part [SWAP] > The same happens with the userdata specified, because I don't use an API, I > need to set custom value, such as a hostname when I boot a VM I created the > following userdata in the UI as per doc: > > > #cloud-config runcmd: - echo 'hostname {{ ds.meta_data.hostname1 }}' > > /tmp/hostname - echo '{{ ds.meta_data.hostname1 }} > /etc/hostname' > > /tmp/hostname - echo 'hostnamectl set-hostname {{ ds.meta_data.hostname1 }}' > > /tmp/hostname > And I declared the hostname1 variable, which allows me to specify the > hostname under "Advanced" settings when I create a new instance. But nothing, > for some reason, it keeps setting the VM name as the server hostname. But > periods (.) are not allowed in the Name field for me to continue with this > method and I need to be able to have special characters in the hostname for > FQDNS spmetimes. > > The only cloud-init command that ran was the 80_user.cfg that creates a > managed user, that was the only one that worked. My upgrade and install > configs none ran: > cat 52_upgrade_packages.cfg > #cloud-config > > # Upgrade the instance on first boot > # (ie run apt-get upgrade) > # > # Default: false > # Aliases: apt_upgrade > cat 53_install_packages.cfg > #cloud-config > > # Install additional packages on first boot > # > # Default: none > # > # if packages are specified, this apt_update will be set to true > # > # packages may be supplied as a single package name or as a list > # with the format [<package>, <version>] wherein the specifc > # package version will be installed. > packages: > - epel-release > - neofetch > - htop > And here's my cloud.cfg: > cat cloud.cfg > datasource_list: ["CloudStack"] > > datasource: > CloudStack: > max_wait: 120 > timeout: 50 > > network: > config: disabled > > users: > - default > > system_info: > default_user: > name: root > > disable_root: false > > ssh_deletekeys: true > > cloud_init_modules: > - seed_random > - bootcmd > - write-files > - growpart > - resizefs > - disk_setup > - mounts > - set_hostname > - update_hostname > - update_etc_hosts > - [ssh, always] > - users-groups > > cloud_config_modules: > - ssh-import-id > - locale > - [ set-passwords, always ] > - ntp > - timezone > - disable-ec2-metadata > - [runcmd, always] > > cloud_final_modules: > - scripts-vendor > - scripts-per-once > - scripts-per-boot > - scripts-per-instance > - [scripts-user, always] > - puppet > > warnings: > dsid_missing_source: off > > Am I missing something with this? Any assistance will be greatly appreciated. > -- > Regards / Groete > [https://www.namhost.com/t/namhost/images/mail/namhost-email-sig-logo.png]<https://www.namhost.com/> > <https://www.namhost.com/> > Granwille Strauss // Senior Systems Admin > > e: granwi...@namhost.com<mailto:granwi...@namhost.com> <granwi...@namhost.com> > m: +264 81 323 1260<tel:+264813231260> <+264813231260> > w: www.namhost.com<https://www.namhost.com/> <https://www.namhost.com/> > > [https://www.namhost.com/t/namhost/images/mail/facebook.png]<https://www.facebook.com/namhost> > > <https://www.facebook.com/namhost>[https://www.namhost.com/t/namhost/images/mail/twitter.png]<https://twitter.com/namhost> > > <https://twitter.com/namhost>[https://www.namhost.com/t/namhost/images/mail/instagram.png]<https://www.instagram.com/namhostinternetservices/> > > <https://www.instagram.com/namhostinternetservices/>[https://www.namhost.com/t/namhost/images/mail/linkedin.png]<https://www.linkedin.com/company/namhost> > > <https://www.linkedin.com/company/namhost>[https://www.namhost.com/t/namhost/images/mail/youtube.png]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> > <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> > > [https://www.namhost.com/t/namhost/images/mail/yourmove-email-banner.png]<https://www.namhost.com/your-move> > <https://www.namhost.com/your-move> > Namhost Internet Services (Pty) Ltd , > > 24 Black Eagle Rd, Hermanus, 7210, RSA > > > > The content of this message is confidential. If you have received it by > mistake, please inform us by email reply and then delete the message. It is > forbidden to copy, forward, or in any way reveal the contents of this message > to anyone without our explicit consent. The integrity and security of this > email cannot be guaranteed over the Internet. Therefore, the sender will not > be held liable for any damage caused by the message. For our full privacy > policy and disclaimers, please go to https://www.namhost.com/privacy-policy > > -- > Regards / Groete > > <https://www.namhost.com> Granwille Strauss // Senior Systems Admin > > *e:* granwi...@namhost.com > *m:* +264 81 323 1260 <+264813231260> > *w:* www.namhost.com > > <https://www.facebook.com/namhost> <https://twitter.com/namhost> > <https://www.instagram.com/namhostinternetservices/> > <https://www.linkedin.com/company/namhost> > <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> > > <https://www.namhost.com/your-move> > > Namhost Internet Services (Pty) Ltd , > > 24 Black Eagle Rd, Hermanus, 7210, RSA > > > > The content of this message is confidential. If you have received it by > mistake, please inform us by email reply and then delete the message. It is > forbidden to copy, forward, or in any way reveal the contents of this > message to anyone without our explicit consent. The integrity and security > of this email cannot be guaranteed over the Internet. Therefore, the sender > will not be held liable for any damage caused by the message. For our full > privacy policy and disclaimers, please go to > https://www.namhost.com/privacy-policy > > > -- > Regards / Groete > > <https://www.namhost.com> Granwille Strauss // Senior Systems Admin > > *e:* granwi...@namhost.com > *m:* +264 81 323 1260 <+264813231260> > *w:* www.namhost.com > > <https://www.facebook.com/namhost> <https://twitter.com/namhost> > <https://www.instagram.com/namhostinternetservices/> > <https://www.linkedin.com/company/namhost> > <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> > > <https://www.namhost.com/your-move> > > Namhost Internet Services (Pty) Ltd , > > 24 Black Eagle Rd, Hermanus, 7210, RSA > > > > The content of this message is confidential. If you have received it by > mistake, please inform us by email reply and then delete the message. It is > forbidden to copy, forward, or in any way reveal the contents of this > message to anyone without our explicit consent. The integrity and security > of this email cannot be guaranteed over the Internet. Therefore, the sender > will not be held liable for any damage caused by the message. For our full > privacy policy and disclaimers, please go to > https://www.namhost.com/privacy-policy > > > -- > Regards / Groete > > <https://www.namhost.com> Granwille Strauss // Senior Systems Admin > > *e:* granwi...@namhost.com > *m:* +264 81 323 1260 <+264813231260> > *w:* www.namhost.com > > <https://www.facebook.com/namhost> <https://twitter.com/namhost> > <https://www.instagram.com/namhostinternetservices/> > <https://www.linkedin.com/company/namhost> > <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA> > > <https://www.namhost.com/your-move> > > Namhost Internet Services (Pty) Ltd , > > 24 Black Eagle Rd, Hermanus, 7210, RSA > > > > The content of this message is confidential. If you have received it by > mistake, please inform us by email reply and then delete the message. It is > forbidden to copy, forward, or in any way reveal the contents of this > message to anyone without our explicit consent. The integrity and security > of this email cannot be guaranteed over the Internet. Therefore, the sender > will not be held liable for any damage caused by the message. For our full > privacy policy and disclaimers, please go to > https://www.namhost.com/privacy-policy > >
