GitHub user GutoVeronezi closed the discussion with a comment: CloudStack fails to start more VMs
Just for future reference, as I haven't seen it mentioned here or in #10205: Apache HTTPd's file `/var/www/html/latest/.htaccess` is used to change permission rules for directories. This file is loaded to memory for each request that expects a document/file as a return (see https://httpd.apache.org/docs/2.4/howto/htaccess.html), which is the case of the requests made by CloudInit when the VM starts. Due to a bug in the Python script that performs operations in the VR, every time a VM is started, a command is executed in the VR to add a set of entries to the file `/var/www/html/latest/.htaccess`. Thus, the more VMs are started, the larger this file will be, to the point where Apache HTTPd consumes all the memory allocated in the VR to respond to CloudInit requests, generating a kernel panic (out-of-memory) and rebooting the VR; as the reboot is not triggered by ACS, the VR will be misconfigured and will not work. When restarting a network or VPC with the clean-up option, a new VR is created with only a few entries in the `.htaccess` file, thus making it less l ikely that Apache HTTPd out-of-memory will occur in this scenario. Furthermore, the more lines this file has, the more CPU and memory the Python script will allocate to process it; thus, making the deployment of new VMs slow. As already validated and mentioned in this discussion, the bug is fixed with patch #10254. GitHub link: https://github.com/apache/cloudstack/discussions/10184#discussioncomment-13411538 ---- This is an automatically sent email for users@cloudstack.apache.org. To unsubscribe, please send an email to: users-unsubscr...@cloudstack.apache.org
