Re: Cloud-Init not working as per Cloudstack documentation

Granwille Strauss Wed, 07 Aug 2024 16:02:30 -0700

Hi

Thank you, you were right. After creating a new template and having the root last, the growfs worked 100%. Thank you for this.

This now brings me to *userdata* in cloudstack that's not working. I registered a new userdata in the UI:

#cloud-config hostname: {{ ds.meta_data.custom_hostname }} fqdn: {{ ds.meta_data.custom_hostname }} prefer_fqdn_over_hostname: true create_hostname_file: true

Declared the *custom_hostname *variable and when booting a VM it still fails:

2024-08-07 22:46:56,673 - stages.py[DEBUG]: Added default handler for {'text/jinja2'} from JinjaTemplatePartHandler: [['text/jinja2']] 2024-08-07 22:46:56,673 - __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (__begin__, None, 3) with frequency once-per-instance 2024-08-07 22:46:56,673 - __init__.py[DEBUG]: Calling handler ShellScriptPartHandler: [['text/x-shellscript']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler ShellScriptByFreqPartHandler: [['text/x-shellscript-per-boot']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler ShellScriptByFreqPartHandler: [['text/x-shellscript-per-instance']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler ShellScriptByFreqPartHandler: [['text/x-shellscript-per-once']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler BootHookPartHandler: [['text/cloud-boothook']] (__begin__, None, 2) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler JinjaTemplatePartHandler: [['text/jinja2']] (__begin__, None, 3) with frequency once-per-instance 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: {'MIME-Version': '1.0', 'Content-Type': 'text/cloud-config', 'Content-Disposition': 'attachment; filename="part-001"'} 2024-08-07 22:46:56,674 - __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (text/cloud-config, part-001, 3) with frequency once-per-instance 2024-08-07 22:46:56,674 - util.py[DEBUG]: Attempting to load yaml from string of length 158 with allowed root types (<class 'dict'>,) 2024-08-07 22:46:56,676 - util.py[WARNING]: Failed loading yaml blob. Invalid format at line 2 column 11: "while constructing a mapping
  in "<unicode string>", line 2, column 11:
    hostname: {{ ds.meta_data.custom_hostname }}
              ^
found unhashable key
  in "<unicode string>", line 2, column 12:
    hostname: {{ ds.meta_data.custom_hostname }}
               ^"
2024-08-07 22:46:56,676 - cloud_config.py[WARNING]: Failed at merging in cloud config part from part-001: empty cloud config 2024-08-07 22:46:56,676 - __init__.py[DEBUG]: Calling handler CloudConfigPartHandler: [['text/cloud-config', 'text/cloud-config-jsonp']] (__end__, None, 3) with frequency once-per-instance 2024-08-07 22:46:56,677 - util.py[DEBUG]: Writing to /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt - wb: [600] 77 bytes 2024-08-07 22:46:56,677 - util.py[DEBUG]: Restoring selinux mode for /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt (recursive=False) 2024-08-07 22:46:56,678 - util.py[DEBUG]: Restoring selinux mode for /var/lib/cloud/instances/c366e71a-220c-48ed-aba1-bdd8a932064b/cloud-config.txt (recursive=False) 2024-08-07 22:46:56,679 - __init__.py[DEBUG]: Calling handler ShellScriptPartHandler: [['text/x-shellscript']] (__end__, None, 2) with frequency once-per-instance

What am I doing wrong in this case? And why does the VM hostname keep defaulting to the instance name specified in the UI instead?

I also created the following in */etc/cloud/cloud.cfg.d*, which injects my SSH key and updated packages and installs a few additional ones, but this too seems to fail:

#cloud-config
ssh_authorized_keys:
  - <my pub key>
package_update: true
package_upgrade: true
packages:
  - epel-release
  - neofetch
  - htop
  - gcc
  - gcc-c++
  - perl


Here's the error:

2024-08-07 22:47:00,315 - rhel.py[DEBUG]: Using DNF for package management
2024-08-07 22:47:00,315 - subp.py[DEBUG]: Running command ['dnf', '-y', 'makecache'] with allowed return codes [0] (shell=False, capture=False)
2024-08-07 22:47:05,331 - rhel.py[DEBUG]: Using DNF for package management
2024-08-07 22:47:05,331 - subp.py[DEBUG]: Running command ['dnf', '-y', 'upgrade'] with allowed return codes [0] (shell=False, capture=False)
2024-08-07 22:47:07,103 - rhel.py[DEBUG]: Using DNF for package management
2024-08-07 22:47:07,103 - subp.py[DEBUG]: Running command ['dnf', '-y', 'install', 'epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] with allowed return codes [0] (shell=False, capture=False) 2024-08-07 22:47:08,573 - util.py[WARNING]: Failed to install packages: ['epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl'] 2024-08-07 22:47:08,575 - util.py[DEBUG]: Failed to install packages: ['epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl']
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py", line 121, in handle
    cloud.distro.install_packages(pkglist)
File "/usr/lib/python3.6/site-packages/cloudinit/distros/rhel.py", line 59, in install_packages
    self.package_command("install", pkgs=pkglist)
File "/usr/lib/python3.6/site-packages/cloudinit/distros/rhel.py", line 201, in package_command
    subp.subp(cmd, capture=False)
File "/usr/lib/python3.6/site-packages/cloudinit/subp.py", line 323, in subp
    stdout=out, stderr=err, exit_code=rc, cmd=args
cloudinit.subp.ProcessExecutionError: Unexpected error while running command. Command: ['dnf', '-y', 'install', 'epel-release', 'neofetch', 'htop', 'gcc', 'gcc-c++', 'perl']
Exit code: 1
Reason: -
Stdout: -
Stderr: -
2024-08-07 22:47:08,582 - cc_package_update_upgrade_install.py[WARNING]: 1 failed with exceptions, re-raising the last one 2024-08-07 22:47:08,582 - handlers.py[DEBUG]: finish: modules-final/config-package_update_upgrade_install: FAIL: running config-package_update_upgrade_install with frequency once-per-instance 2024-08-07 22:47:08,582 - util.py[WARNING]: Running module package_update_upgrade_install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py'>) failed 2024-08-07 22:47:08,582 - util.py[DEBUG]: Running module package_update_upgrade_install (<module 'cloudinit.config.cc_package_update_upgrade_install' from '/usr/lib/python3.6/site-packages/cloudinit/config/cc_package_update_upgrade_install.py'>) failed

My main concern is the hostname that I cannot seem to set no matter what I try, if I can get this at least right that would be the ultimate. How can I pass the update_hostname userdata so that I can manually enter custom hostnames when deploying VM via UI?


On 8/7/24 22:52, Alexandru Stan wrote:

Hello,

Can’t say about everything you mentioned, but regarding the disk resizinng I 
can share my experience, I also had some issues before I managed to get it 
working. So, from what I’ve searched, growpart does not seem to work with lvm 
(could be wrong here though), so basic partitioning it is. Then, it seems that 
it extends the partition only if it’s the last one on the disk, seems to be 
your issue since the last partition is the swap partition. Try with the root 
partition being the last one, it should work fine.


From: Granwille Strauss<granwi...@namhost.com.INVALID>
Sent: 7 August, 2024 20:35
To:users@cloudstack.apache.org
Subject: Cloud-Init not working as per Cloudstack documentation


Evening,

Please note, I do not use any API service and solely rely on the Cloudstack UI.

I deployed a new clean template and I followed the cloudstack documentation 
thoroughly to use the basics of cloud-init with cloudstack. But when I deployed 
my VM nothing seems to trigger at all, including userdata I registered.
pwd
/etc/cloud/cloud.cfg.d
[root@hostname cloud.cfg.d]# ll
total 32
-rw-r--r--. 1 root root 2070 Dec  4  2023 05_logging.cfg
-rw-r--r--. 1 root root   22 Aug  7 17:43 49_hostkeys.cfg
-rw-r--r--. 1 root root   88 Aug  7 17:45 50_growpartion.cfg
-rw-r--r--. 1 root root  209 Aug  7 17:52 51_extend_volume.cfg
-rw-r--r--. 1 root root  143 Aug  7 17:55 52_upgrade_packages.cfg
-rw-r--r--. 1 root root  355 Aug  7 18:00 53_install_packages.cfg
-rw-r--r--. 1 root root  357 Aug  7 17:42 80_user.cfg
-rw-r--r--. 1 root root  167 Dec  4  2023 README


For example, I deployed a new VM with its root disk overwrite from the 
template, which was 10 GB to 60 GB and the 50_growpartion.cfg did nothing:
[root@hostname cloud.cfg.d]# cat 50_growpartion.cfg
growpart:
   mode: auto
   devices:
     - \"/dev/vda2\"
   ignore_growroot_disabled: false
cat 51_extend_volume.cfg
runcmd:
   - [ cloud-init-per, always, grow_FS, resize2fs, /dev/vda2 ]

grep "50_growpartion.cfg" /var/log/cloud-init.log
2024-08-07 16:50:45,881 - util.py[DEBUG]: Reading from 
/etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False)
2024-08-07 16:50:45,881 - util.py[DEBUG]: Read 88 bytes from 
/etc/cloud/cloud.cfg.d/50_growpartion.cfg
2024-08-07 16:50:45,928 - util.py[DEBUG]: Reading from 
/etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False)
2024-08-07 16:50:45,928 - util.py[DEBUG]: Read 88 bytes from 
/etc/cloud/cloud.cfg.d/50_growpartion.cfg
2024-08-07 16:50:46,046 - util.py[DEBUG]: Reading from 
/etc/cloud/cloud.cfg.d/50_growpartion.cfg (quiet=False)
2024-08-07 16:50:46,046 - util.py[DEBUG]: Read 88 bytes from 
/etc/cloud/cloud.cfg.d/50_growpartion.cfg
lsblk
NAME   MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sr0     11:0    1 1024M  0 rom
vda    253:0    0   60G  0 disk
├─vda1 253:1    0  512M  0 part /boot
├─vda2 253:2    0  7.5G  0 part /
└─vda3 253:3    0    2G  0 part [SWAP]
The same happens with the userdata specified, because I don't use an API, I 
need to set custom value, such as a hostname when I boot a VM I created the 
following userdata in the UI as per doc:


#cloud-config runcmd: - echo 'hostname {{ ds.meta_data.hostname1 }}' > /tmp/hostname - 
echo '{{ ds.meta_data.hostname1 }} > /etc/hostname' > /tmp/hostname - echo 
'hostnamectl set-hostname {{ ds.meta_data.hostname1 }}' > /tmp/hostname
And I declared the hostname1 variable, which allows me to specify the hostname under 
"Advanced" settings when I create a new instance. But nothing, for some reason, 
it keeps setting the VM name as the server hostname. But periods (.) are not allowed in 
the Name field for me to continue with this method and I need to be able to have special 
characters in the hostname for FQDNS spmetimes.

The only cloud-init command that ran was the 80_user.cfg that creates a managed 
user, that was the only one that worked. My upgrade and install configs none 
ran:
cat 52_upgrade_packages.cfg
#cloud-config

# Upgrade the instance on first boot
# (ie run apt-get upgrade)
#
# Default: false
# Aliases: apt_upgrade
cat  53_install_packages.cfg
#cloud-config

# Install additional packages on first boot
#
# Default: none
#
# if packages are specified, this apt_update will be set to true
#
# packages may be supplied as a single package name or as a list
# with the format [<package>, <version>] wherein the specifc
# package version will be installed.
packages:
  - epel-release
  - neofetch
  - htop
And here's my cloud.cfg:
cat cloud.cfg
datasource_list: ["CloudStack"]

datasource:
   CloudStack:
     max_wait: 120
     timeout: 50

network:
   config: disabled

users:
   - default

system_info:
   default_user:
     name: root

disable_root: false

ssh_deletekeys: true

cloud_init_modules:
  - seed_random
  - bootcmd
  - write-files
  - growpart
  - resizefs
  - disk_setup
  - mounts
  - set_hostname
  - update_hostname
  - update_etc_hosts
  - [ssh, always]
  - users-groups

cloud_config_modules:
  - ssh-import-id
  - locale
  - [ set-passwords, always ]
  - ntp
  - timezone
  - disable-ec2-metadata
  - [runcmd, always]

cloud_final_modules:
  - scripts-vendor
  - scripts-per-once
  - scripts-per-boot
  - scripts-per-instance
  - [scripts-user, always]
  - puppet

warnings:
   dsid_missing_source: off

Am I missing something with this? Any assistance will be greatly appreciated.
--
Regards / Groete
[https://www.namhost.com/t/namhost/images/mail/namhost-email-sig-logo.png]<https://www.namhost.com/>
Granwille Strauss  //  Senior Systems Admin

e:granwi...@namhost.com<mailto:granwi...@namhost.com>
m: +264 81 323 1260<tel:+264813231260>
w:www.namhost.com<https://www.namhost.com/>

[https://www.namhost.com/t/namhost/images/mail/facebook.png]<https://www.facebook.com/namhost>[https://www.namhost.com/t/namhost/images/mail/twitter.png]<https://twitter.com/namhost>[https://www.namhost.com/t/namhost/images/mail/instagram.png]<https://www.instagram.com/namhostinternetservices/>[https://www.namhost.com/t/namhost/images/mail/linkedin.png]<https://www.linkedin.com/company/namhost>[https://www.namhost.com/t/namhost/images/mail/youtube.png]<https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>

[https://www.namhost.com/t/namhost/images/mail/yourmove-email-banner.png]<https://www.namhost.com/your-move>
Namhost Internet Services (Pty) Ltd ,

                     24 Black Eagle Rd, Hermanus, 7210, RSA



The content of this message is confidential. If you have received it by 
mistake, please inform us by email reply and then delete the message. It is 
forbidden to copy, forward, or in any way reveal the contents of this message 
to anyone without our explicit consent. The integrity and security of this 
email cannot be guaranteed over the Internet. Therefore, the sender will not be 
held liable for any damage caused by the message. For our full privacy policy 
and disclaimers, please go tohttps://www.namhost.com/privacy-policy

--
Regards / Groete

<https://www.namhost.com>         Granwille Strauss  // Senior Systems Admin

*e:* granwi...@namhost.com
*m:* +264 81 323 1260 <tel:+264813231260>
*w:* www.namhost.com <https://www.namhost.com/>

<https://www.facebook.com/namhost> <https://twitter.com/namhost> <https://www.instagram.com/namhostinternetservices/> <https://www.linkedin.com/company/namhost> <https://www.youtube.com/channel/UCTd5v-kVPaic_dguGur15AA>


<https://www.namhost.com/your-move>

Namhost Internet Services (Pty) Ltd ,

                    24 Black Eagle Rd, Hermanus, 7210, RSA

The content of this message is confidential. If you have received it by mistake, please inform us by email reply and then delete the message. It is forbidden to copy, forward, or in any way reveal the contents of this message to anyone without our explicit consent. The integrity and security of this email cannot be guaranteed over the Internet. Therefore, the sender will not be held liable for any damage caused by the message. For our full privacy policy and disclaimers, please go to https://www.namhost.com/privacy-policy

smime.p7s
Description: S/MIME Cryptographic Signature

Re: Cloud-Init not working as per Cloudstack documentation

Reply via email to