On 01.05.2011 17:56, Knut St. Osmundsen wrote:
Hi Zak!
The VMM imposes a limit of 1023 VMs on 64-bit hosts and 127 VMs on
32-bit hosts. See:
http://www.virtualbox.org/browser/trunk/src/VBox/VMM/VMMR0/GVMMR0.cpp#L121
That means your plan to have 1800 VMs will not work without bumping the
hardcoded limits, and in this area it needs much more thinking than just
changing a #define.
Anyway, let's get you to that limit first...
If you run "ipcs -ls" you'll get a "max number of arrays = 128" line,
which is what you're most likely bumping into.
You can read the kernel defaults with "sysctl kernel.sem" as well, and
the last number needs to be increased to a bit more than 1023, just to
be on the safe side if some other app also needs a few semaphores.
"sysctl -w kernel.sem=250 32000 32 1200" should eliminate this limit.
You can also put this in /etc/sysctl.conf, so that it is set straight on
system boot, as otherwise these settings are lost on a reboot.
So... how many VMs can you start now?
Klaus
Seeing you have 2TB of memory and 128 cores, I would assume you're
running 64-bit Ubuntu. So, it is probably some other limit that you're
hitting. It could be SysV semaphores, it could be memory available below
4GB, ... Could you please provide some more details regarding what
happens when you are unable to launch more VMs? For instance, does a new
VBox.log file get created for the VM? If it does, please zip it up and
send it to me (or Klaus) per private mail and we'll work it out.
I think the 64bit host OS assumption is safe... I would seriously ask
people to have their brain inspected if they'd use a 32bit OS for this.
On Apr 29, 2011, at 7:43 PM, Zak Perschon wrote:
Ah, sorry forgot to include the original support thread. The limit I
hit was 128 active VMs at once. Once I hit that number, I was unable
to launch any more unless I stopped one that was already running. My
original thread can be found here (though there is little information
contained in
it):http://forums.virtualbox.org/viewtopic.php?f=7&t=40955
<http://forums.virtualbox.org/viewtopic.php?f=7&t=40955>
-Zak
> Date: Fri, 29 Apr 2011 16:25:44 +0200
> From:[email protected] <mailto:[email protected]>
> To:[email protected] <mailto:[email protected]>
> Subject: Re: [vbox-dev] Linux Host VM Cap?
>
> On 25.04.2011 18:00, Zak Perschon wrote:
> >
> > I've come across what is seemingly a hard cap for the amount of
> > concurrent VMs I can run on a Linux based machine (after posting
on the
> > support forums, I was directed here). Apparently this same cap was
> > encountered on a solaris host and a fix was deployed to increase the
> > amount of VMs that were allowed to run at the same time. Was this fix
> > supposed to be included in other OS releases? Any information on this
> > would be greatly appreciated.
>
> That's a rather vague statement, and if you could tell what number of
> concurrently running VMs you can achieve I might be able to infer what
> limit is causing trouble for you.
>
> It could be the SysV semaphore count, the number of open files per
> process, and so on. All these defaults depend on the kernel
> configuration and/or distribution, so it's hard to generalize.
>
> Solaris is easy compared to that, as it is very uniform compared to
Linux.
>
> Klaus
>
> >
> > Here is the information on the hardware/VM profiles I am using
> >
> > Host Machine:
> > 128 Processor Cores (64 dual core Xenon processors [email protected] GHz)
> > 2.0 TB Memory (1.8~ Usable)
> > 1.8 TB RAID 0 (6 x 300GB HDD running at 10k RPM)
> > 16TB SAS
> >
> > VM Profile
> > Compiled TinyCore Linux 2.6
> > Required Memory: 150~MB
> > Required HDD Space: 50MB
> >
> > -Zak
>
> _______________________________________________
> vbox-dev mailing list
>[email protected] <mailto:[email protected]>
>http://vbox.innotek.de/mailman/listinfo/vbox-dev
_______________________________________________
vbox-dev mailing list
[email protected] <mailto:[email protected]>
http://vbox.innotek.de/mailman/listinfo/vbox-dev
--
Kind regards / Mit freundlichen Gruessen / Vennlig hilsen,
bird
--
ORACLE Deutschland B.V. & Co. KG Knut St. Osmundsen
Werkstrasse 24 Senior Staff Engineer, VirtualBox
71384 Weinstadt, Germany mailto:[email protected]
Hauptverwaltung: Riesstr. 25, D-80992 Muenchen
Registergericht: Amtsgericht Muenchen, HRA 95603
Komplementaerin: ORACLE Deutschland Verwaltung B.V.
Rijnzathe 6, 3454PV De Meern, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschaeftsfuehrer: J. Kunz, M. van de Molen, A. van der Ven
_______________________________________________
vbox-dev mailing list
[email protected]
http://vbox.innotek.de/mailman/listinfo/vbox-dev
--
Oracle <http://www.oracle.com>
Dr. Klaus Espenlaub | Snr. Manager Software Development Desktop
Virtualization
Phone: +49 7151 60405 205 <tel:+49715160405205>
Oracle VM VirtualBox
ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | 71384 Weinstadt
ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstr. 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603
Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Niederlande, Nr. 30143697
Geschäftsführer: Jürgen Kunz, Marcel van de Molen, Alexander van der Ven
Green Oracle <http://www.oracle.com/commitment> Oracle is committed to
developing practices and products that help protect the environment
_______________________________________________
vbox-dev mailing list
[email protected]
http://vbox.innotek.de/mailman/listinfo/vbox-dev
