This is an automated email from the ASF dual-hosted git repository.
nvazquez pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/cloudstack-documentation.git
The following commit(s) were added to refs/heads/main by this push:
new d6984e7 Modernizing "Create linux template" guide + Cloud-init
integration steps (#215)
d6984e7 is described below
commit d6984e77078d4686771b463c014860d0a8f0fd69
Author: dredknight <[email protected]>
AuthorDate: Mon Jun 6 18:57:36 2022 +0300
Modernizing "Create linux template" guide + Cloud-init integration steps
(#215)
* up
* major refurbishment of the linux template guide
* up
* up
* up
* finishing touches on first release
* finishing touches
* Update _cloud_init.rst
* Update _create_linux.rst
* Update _cloud_init.rst
* Update _cloud_init.rst
* Update _cloud_init.rst
* cloud-init - ssh keys fix
When Cloud-init SSH module is configured to work on every boot instead of
once-per-instance it will also regenerate the unique SSH host fingerprint
(certifacte bundles) every time. This commit adds a configuration adjustment to
the guide that disables this peculiarity.
* up
* up
* up
* Update source/adminguide/templates/_cloud_init.rst
Co-authored-by: Wei Zhou <[email protected]>
* Update source/adminguide/templates/_cloud_init.rst
Co-authored-by: Wei Zhou <[email protected]>
* Restore images
Co-authored-by: dredknight <[email protected]>
Co-authored-by: Wei Zhou <[email protected]>
Co-authored-by: nvazquez <[email protected]>
---
source/adminguide/templates/_cloud_init.rst | 241 ++++++++++++++++++++++
source/adminguide/templates/_create_linux.rst | 280 +++++++++++---------------
2 files changed, 359 insertions(+), 162 deletions(-)
diff --git a/source/adminguide/templates/_cloud_init.rst
b/source/adminguide/templates/_cloud_init.rst
new file mode 100644
index 0000000..70acd9a
--- /dev/null
+++ b/source/adminguide/templates/_cloud_init.rst
@@ -0,0 +1,241 @@
+.. Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information#
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+ http://www.apache.org/licenses/LICENSE-2.0
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+
+
+Cloud-init integration
+-------------------------
+
+Cloudstack and cloud-init integration provide instances with advanced
management features such as:
+
+* Password management
+* SSH keys management
+* Partition management
+* User-data input
+* `Other modules
<https://cloudinit.readthedocs.io/en/latest/topics/modules.html>`_
+
+
+
+Examples for relevant features are listed for different distributions.
+
+~~~~~~~~~~~~~~~~~~~~~~
+
+Linux with Cloud-init
+----------------------
+
+These features can be implemented in `“Linux template creation process”
<_create_linux.html#creating-a-linux-template>`_. where they are executed just
before the **Template cleanup** step.
+
+#. **Install and configure cloud-init**
+
+ Install cloud-init and mentioned auxiliary packages.
+
+ ~ CentOS
+
+ .. code:: bash
+
+ yum install -y cloud-init wget
+
+ ~ Ubuntu
+
+ .. code:: bash
+
+ apt-get install -y cloud-init wget
+
+ Configure cloud-init to detect Cloudstack data source during runtime.
+
+ ~ CentOS
+
+ CentOS cloud-init looks for datasources in /etc/cloud/ds-indetify.cfg
during boot. Add the following config in that file.
+
+ .. code:: bash
+
+ datasource: CloudStack
+
+ ~ Ubuntu
+
+ Ubuntu cloud-init data sources can be specified in /etc/cloud/cloud.cfg.d/
directory. Add the following config in /etc/cloud/cloud.cfg.d/99_cloudstack.cfg.
+
+ .. code:: bash
+
+ datasource_list: [ ConfigDrive, CloudStack, None ]
+ datasource:
+ CloudStack: {}
+ None: {}
+
+#. **Password management**
+
+ Cloudstack integration with cloud-init `set-passwords module
<https://cloudinit.readthedocs.io/en/latest/topics/modules.html?highlight=ssh_pwauth#set-passwords>`_
will enable the platform to set a password for each instance created from the
Main Template. Additionally it will allow to reset the user password through
the GUI.
+
+ - **Enable set-passwords module on every boot**
+
+ By default the set-passwords module runs only on first boot of the
instance, change that to run on every boot.
+
+ .. code:: bash
+
+ sudo sed -i s/" - set-passwords"/" - [set-passwords, always]"/g
/etc/cloud/cloud.cfg
+
+ - **Specify the managed user**
+
+ Cloudstack will create the user, set a password and reset it when
requested. To do that set the following configuration in
/etc/cloud/cloud.cfg.d/80_user.cfg
+
+ .. code:: bash
+
+ system_info:
+ default_user:
+ name: cloud-user
+ lock_passwd: false # disable user password login -
true/false
+ sudo: [\"ALL=(ALL) ALL\"] # User permissions
+ disable_root: 0 # root remote login is 0 - enabled, 1
- disabled
+ ssh_pwauth: 1 # password login is 0 - disabled,
1- enabled
+
+#. **SSH keys management**
+
+ Cloud-init `ssh module
<https://cloudinit.readthedocs.io/en/latest/topics/modules.html#ssh>`_ can
automatically install new SSH keys when set or reset from Cloudstack GUI.
+ By default the module runs once during instance creation and will fetch
Cloudstack keys without any additional configuration.
+ To enable Cloudstack reset SSH keys feature configure cloud-init ssh module
to run on every boot.
+
+ .. code:: bash
+
+ sudo sed -i s/" - ssh$"/" - [ssh, always]"/g /etc/cloud/cloud.cfg
+
+=======
+
+ .. warning::
+
+ Cloud-init ssh module runs on every boot and will regenerate the
certificate fingerprint of the host. This will cause a warning to anyone that
logs in the system and also bring trouble to anyone trying to automate ssh
access.
+
+ Disable cloud-init regenerating host certificates on boot. If template
certificates are deleted they will be regenerated by the OS on instnace first
boot.
+
+ .. code:: bash
+
+ echo "ssh_deletekeys: false" > /etc/cloud/cloud.cfg.d/49_hostkeys.cfg
+
+#. **Partition management**
+
+ Cloud-init can detect and resize one or more existing partitions
automatically after reboot. This guide will cover root partition and volume.
+ First install the `Growpart module
<https://cloudinit.readthedocs.io/en/latest/topics/modules.html#growpart>`_ as
it is not shipped with cloud-init.
+
+ ~ Centos
+
+ .. code:: bash
+
+ yum install cloud-utils-growpart -y
+
+ ~ Ubuntu
+
+ .. code:: bash
+
+ apt-get install cloud-initramfs-growroot -y
+
+ - **Detect and extend MBR partitions**
+
+ Locate the root partition.
+
+ .. note::
+
+ Root partition can differ per OS type, version and partition setup.
+
+ .. code:: bash
+
+ [root@localhost ~]# lvs
+ LV VG Attr LSize Pool Origin Data% Meta% Move Log
Cpy%Sync Convert
+ root centos -wi-ao---- <17.00g
+ swap centos -wi-ao---- 2.00g
+ [root@localhost ~]# vgs
+ VG #PV #LV #SN Attr VSize VFree
+ centos 1 2 0 wz--n- <19.00g 0
+ [root@localhost ~]# pvs
+ PV VG Fmt Attr PSize PFree
+ /dev/xvda2 centos lvm2 a-- <19.00g 0
+
+ On the current setup root is on /dev/xvda2 partition. Define the
configuration below in /etc/cloud/cloud.cfg.d/50_growpartion.cfg
+
+ .. code:: bash
+
+ growpart:
+ mode: auto
+ devices:
+ - \"/dev/xvda2\"
+ ignore_growroot_disabled: false
+
+ Now on every boot growpart will check and extend /dev/xvda2 if there is
change in size.
+
+ - **Extend Physical volume, Volume group and root lvm**
+
+ After parition is extended the upper layers should also be resized. This
can be automated with cloud-init `runcmd module
<https://cloudinit.readthedocs.io/en/latest/topics/modules.html?highlight=runcmd#runcmd>`_
. Set the configuration below in /etc/cloud/cloud.cfg.d/51_extend_volume.cfg.
+
+ ~ CentOS
+
+ Centos 7 root volume is /dev/centos/root if no changes are done during
installation. Change the value accordingly if setup is different.
+
+ .. code:: bash
+
+ runcmd:
+ - [ cloud-init-per, always, grow_VG, pvresize, /dev/xvda2 ]
+ - [ cloud-init-per, always, grow_LV, lvresize, -l, '+100%FREE',
/dev/centos/root ]
+ - [ cloud-init-per, always, grow_FS, xfs_growfs, /dev/centos/root ]
+
+ ~ Ubuntu
+
+ Ubuntu 20 root volume is /dev/ubuntu-vg/ubuntu-lv if no changes are done
during installation. Change the value accordingly if setup is different.
+
+ .. code:: bash
+
+ runcmd:
+ - [ cloud-init-per, always, grow_VG, pvresize, /dev/xvda3 ]
+ - [ cloud-init-per, always, grow_LV, lvresize, -l, '+100%FREE',
/dev/ubuntu-vg/ubuntu-lv ]
+ - [ cloud-init-per, always, grow_FS, xfs_growfs,
/dev/ubuntu-vg/ubuntu-lv ]
+
+ .. warning::
+
+ The example code above is based on XFS parition type. If ext4
partitioning is utilized replace **xfs_growfs** with **resize2fs** in the last
code line.
+ It is possible to also use cloud-init `resize2fs module
<https://cloudinit.readthedocs.io/en/latest/topics/modules.html#resizefs>`_ .
+
+ - **Enable autoresize on every boot**
+
+ By default cloud-init **runcmd** module executes defined commands on
first boot only.
+ Commands will run on every boot only if both **runcmd** and
**user-scripts** modules are configured to run on every boot.
+
+ .. code:: bash
+
+ sudo sed -i s/" - runcmd"/" - [runcmd, always]"/g /etc/cloud/cloud.cfg
+ sudo sed -i s/" - scripts-user"/" - [scripts-user, always]"/g
/etc/cloud/cloud.cfg
+
+#. **User-data**
+
+ Cloud-init can parse and execute user-data form Cloud-stack during instance
creation. This feature works as is without additional configuration.
+
+#. **Cleanup**
+
+ Once desired cloud-init features are implemented, clean cloud-init tracker
files.
+
+ .. code:: bash
+
+ cloud-init clean
+
+ Or do it manually.
+
+ .. code:: bash
+
+ rm -rf /var/lib/cloud/*
+
+ If **Password management** feature is used clean /etc/sudoers from any
cloud-init user setups.
+
+ .. code:: bash
+
+ rm -rf /etc/sudoers.d/*
+
+#. **Finalize template**
+
+ Proceed with `“Linux template creation process” <_create_linux.html>`_
continuing with **Template cleanup** step.
diff --git a/source/adminguide/templates/_create_linux.rst
b/source/adminguide/templates/_create_linux.rst
index 217556f..f8cee78 100644
--- a/source/adminguide/templates/_create_linux.rst
+++ b/source/adminguide/templates/_create_linux.rst
@@ -20,9 +20,10 @@ Creating a Linux Template
Linux templates should be prepared using this documentation in order to
prepare your linux VMs for template deployment. For ease of
documentation, the VM which you are configuring the template on will be
-referred to as "Template Primary". This guide currently covers legacy
-setups which do not take advantage of UserData and cloud-init and
-assumes openssh-server is installed during installation.
+referred to as "Main Template". The final product, as created and usable
+for deplyoment in Cloudstack, will be referred as "Final Template".
+This guide will cover cloud-init setup and scripted setups where available.
It is assumed that openssh-server
+is installed during installation.
An overview of the procedure is as follow:
@@ -45,206 +46,165 @@ An overview of the procedure is as follow:
System preparation for Linux
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+----------------------------
+
+The following steps will provide basic Linux installation for
+templating of Centos and Ubuntu.
+
+#. **Update OS**
+
+ The next step update the packages on the Main Template.
+
+ ~ CentOS
+
+ .. code:: bash
+
+ yum update -y
+ reboot
+
+ ~ Ubuntu
+
+ .. code:: bash
+
+ sudo -i
+ apt-get update
+ apt-get upgrade -y
+ apt-get install -y acpid ntp
+ reboot
+
+#. **Networking**
+
+ Set template network interface configuration to DHCP so Cloudstack
infrastructure can assign one on boot.
+
+ .. warning::
+
+ For CentOS, it is mandatory to take unique identification out of the
+ interface configuration file /etc/sysconfig/network-scripts/ifcfg-eth0.
Any entries starting with <VALUE> should be removed.
+
+ ~ Centos
+
+ .. code:: bash
+
+ echo "DEVICE=eth0
+ TYPE=Ethernet
+ BOOTPROTO=dhcp
+ ONBOOT=yes" > /etc/sysconfig/network-scripts/ifcfg-eth0
-The following steps will prepare a basic Linux installation for
-templating.
-
-#. **Installation**
+#. **Hostname Management**
- It is good practice to name your VM something generic during
- installation, this will ensure components such as LVM do not appear
- unique to a machine. It is recommended that the name of "localhost"
- is used for installation.
-
- .. warning::
- For CentOS, it is necessary to take unique identification out of the
- interface configuration file, for this edit
- /etc/sysconfig/network-scripts/ifcfg-eth0 and change the content to
- the following.
+ Set a generic name to the template VM during installation, this will ensure
components such as LVM do not appear unique to a machine. It is recommended
that the name of "localhost" is used for installation.
.. code:: bash
- DEVICE=eth0
- TYPE=Ethernet
- BOOTPROTO=dhcp
- ONBOOT=yes
-
- The next steps updates the packages on the Template Primary.
-
- - Ubuntu
-
- .. code:: bash
-
- sudo -i
- apt-get update
- apt-get upgrade -y
- apt-get install -y acpid ntp
- reboot
-
- - CentOS
-
- .. code:: bash
-
- ifup eth0
- yum update -y
- reboot
+ hostname localhost
+ echo "localhost" > /etc/hostname
#. **Password management**
-
+
.. note::
- If preferred, custom users (such as ones created during the Ubuntu
- installation) should be removed. First ensure the root user account
- is enabled by giving it a password and then login as root to continue.
-
- .. code:: bash
-
- sudo passwd root
- logout
+
+ It is a good practice to remove any non root users that come with the OS
(such as ones created during the Ubuntu
+ installation). First ensure the root user account is enabled by giving it
a password and then login as root to continue.
- As root, remove any custom user accounts created during the
- installation process.
+ Once logged in as root, any custom user can be removed.
.. code:: bash
- deluser myuser --remove-home
-
- See :ref:`adding-password-management-to-templates` for
- instructions to setup the password management script, this will allow
- CloudStack to change your root password from the web interface.
-
-#. **Hostname Management**
-
- CentOS configures the hostname by default on boot. Unfortunately
- Ubuntu does not have this functionality, for Ubuntu installations use
- the following steps.
-
- - Ubuntu
-
- The hostname of a Templated VM is set by a custom script in
- `/etc/dhcp/dhclient-exit-hooks.d`, this script first checks if the
- current hostname is localhost, if true, it will get the host-name,
- domain-name and fixed-ip from the DHCP lease file and use those
- values to set the hostname and append the `/etc/hosts` file for
- local hostname resolution. Once this script, or a user has changed
- the hostname from localhost, it will no longer adjust system files
- regardless of its new hostname. The script also recreates
- openssh-server keys, which should have been deleted before
- templating (shown below). Save the following script to
- `/etc/dhcp/dhclient-exit-hooks.d/sethostname`, and adjust the
- permissions.
-
- .. code:: bash
-
- #!/bin/sh
- # dhclient change hostname script for Ubuntu
- oldhostname=$(hostname -s)
- if [ $oldhostname = 'localhost' ]
- then
- sleep 10 # Wait for configuration to be written to disk
- hostname=$(cat /var/lib/dhcp/dhclient.eth0.leases | awk '
/host-name/ { host = $3 } END { printf host } ' | sed 's/[";]//g' )
- fqdn="$hostname.$(cat /var/lib/dhcp/dhclient.eth0.leases | awk
' /domain-name/ { domain = $3 } END { printf domain } ' | sed 's/[";]//g')"
- ip=$(cat /var/lib/dhcp/dhclient.eth0.leases | awk '
/fixed-address/ { lease = $2 } END { printf lease } ' | sed 's/[";]//g')
- echo "cloudstack-hostname: Hostname _localhost_ detected.
Changing hostname and adding hosts."
- printf " Hostname: $hostname\n FQDN: $fqdn\n IP: $ip"
- # Update /etc/hosts
- awk -v i="$ip" -v f="$fqdn" -v h="$hostname" "/^127/{x=1} !/^127/
&& x { x=0; print i,f,h; } { print $0; }" /etc/hosts > /etc/hosts.dhcp.tmp
- mv /etc/hosts /etc/hosts.dhcp.bak
- mv /etc/hosts.dhcp.tmp /etc/hosts
- # Rename Host
- echo $hostname > /etc/hostname
- hostname -b -F /etc/hostname
- echo $hostname > /proc/sys/kernel/hostname
- # Recreate SSH2
- export DEBIAN_FRONTEND=noninteractive
- dpkg-reconfigure openssh-server
- fi
- ### End of Script ###
-
- chmod 774 /etc/dhcp/dhclient-exit-hooks.d/sethostname
-
+ deluser myuser --remove-home
+
+ User password management and reset cappabilities in GUI are available with:
+
+ * `Cloud-init integration <_cloud_init.html#linux-with-cloud-init>`_
+ * `Adding Password Management to Your Templates
<_password.html#adding-password-management-to-templates>`_ /Legacy for non
systemd systems only/
+
+#. **SSH keys management**
+
+ Cloudstack can create key pair and push certificates to instances. This
feature is available with:
+
+ * `Cloud-init integration <_cloud_init.html#linux-with-cloud-init>`_
+ * `Implementing a SSH-Key bash script
<http://docs.cloudstack.apache.org/en/latest/adminguide/virtual_machines.html#creating-an-instance-template-that-supports-ssh-keys>`_
+
+#. **Partition management**
+
+ Volumes can autorextend after reboot when partition is extended in the GUI.
+ This feature is possible with `Cloud-init integration
<_cloud_init.html#linux-with-cloud-init>`_.
+
+#. **User-data**
+
+ Cloudstack can push user-data during instance creation.
+ This feature is possible with `Cloud-init integration
<_cloud_init.html#linux-with-cloud-init>`_.
+
+#. **Template cleanup**
+
.. warning::
- The following steps should be run when you are ready to template
- your Template Primary. If the Template Primary is rebooted during
- these steps you will have to run all the steps again. At the end
- of this process the Template Primary should be shutdown and the
- template created in order to create and deploy the final template.
-
-#. **Remove the udev persistent device rules**
-
- This step removes information unique to your Template Primary such as
- network MAC addresses, lease files and CD block devices, the files
- are automatically generated on next boot.
-
- - Ubuntu
+
+ Cleanup steps should be run when all Main Template configuration
+ is done and just before the shutdown step. After shut down Final
+ template should be created. If the Main Template is started or
+ rebooted before Final template creation all cleanup steps have to be rerun.
+
+ - **Remove the udev persistent device rules**
+
+ This step removes information unique to the Main Template such as
+ network MAC addresses, lease files and CD block devices, the files
+ are automatically generated on next boot.
+
+ ~ CentOS
.. code:: bash
- rm -f /etc/udev/rules.d/70*
- rm -f /var/lib/dhcp/dhclient.*
-
- - CentOS
+ rm -f /etc/udev/rules.d/70*
+ rm -f /var/lib/dhclient/*
+
+ ~ Ubuntu
.. code:: bash
- rm -f /etc/udev/rules.d/70*
- rm -f /var/lib/dhclient/*
+ rm -f /etc/udev/rules.d/70*
+ rm -f /var/lib/dhcp/dhclient.*
-#. **Remove SSH Keys**
+ - **Remove SSH Keys**
- This step is to ensure all your Templated VMs do not have the same
- SSH keys, which would decrease the security of the machines
- dramatically.
+ This step is to ensure all Templated VMs do not have the same
+ SSH keys, which would decrease the security of the machines
+ dramatically.
- .. code:: bash
+ .. code:: bash
rm -f /etc/ssh/*key*
-#. **Cleaning log files**
+ - **Cleaning log files**
- It is good practice to remove old logs from the Template Primary.
+ It is good practice to remove old logs from the Main Template.
- .. code:: bash
+ .. code:: bash
cat /dev/null > /var/log/audit/audit.log 2>/dev/null
cat /dev/null > /var/log/wtmp 2>/dev/null
logrotate -f /etc/logrotate.conf 2>/dev/null
rm -f /var/log/*-* /var/log/*.gz 2>/dev/null
-#. **Setting hostname**
-
- In order for the Ubuntu DHCP script to function and the CentOS
- dhclient to set the VM hostname they both require the Template
- Primary's hostname to be "localhost", run the following commands to
- change the hostname.
-
- .. code:: bash
-
- hostname localhost
- echo "localhost" > /etc/hostname
+ - **Set user password to expire**
-#. **Set user password to expire**
+ This step forces the user to change the password of the VM after the
+ template has been deployed.
- This step forces the user to change the password of the VM after the
- template has been deployed.
-
- .. code:: bash
+ .. code:: bash
passwd --expire root
-#. **Clearing User History**
+ - **Clearing User History**
- The next step clears the bash commands you have just run.
+ The next step clears the bash commands you have just run.
- .. code:: bash
+ .. code:: bash
history -c
unset HISTFILE
#. **Shutdown the VM**
- Your now ready to shutdown your Template Primary and create a
- template!
+ Shutdown the Main Template.
.. code:: bash
@@ -252,10 +212,6 @@ templating.
#. **Create the template!**
- You are now ready to create the template, for more information see
+ You are now ready to create the Final Template, for more information see
`“Creating a Template from an Existing Virtual
Machine” <#creating-a-template-from-an-existing-virtual-machine>`_.
-
-.. note::
- Templated VMs for both Ubuntu and CentOS may require a reboot after
- provisioning in order to pickup the hostname.
