I added a "cat /proc/meminfo | grep Slab" to go-lxc-run.sh and found this:
$ sysctl fs.inotify fs.inotify.max_queued_events = 65536 fs.inotify.max_user_instances = 1024 fs.inotify.max_user_watches = 524288 $ ulimit -a ... open files (-n) 1048576 ... $ go-lxc-run.sh [0] x-001:[13]......7 Slab: 432176 kB ... [1365] x-071:[15].......8 Slab: 9639484 kB [1387] x-072:[15].......8 Slab: 9603648 kB [1409] x-073:[14].......8 Slab: 9621936 kB [1431] x-074:[14].......8 Slab: 9673000 kB [1453] x-075:[13]........9 Slab: 9680444 kB [1475] x-076:[14]........9 Slab: 8223312 kB [1501] x-077:[16]......7 Slab: 5095880 kB ... 1812] x-093:[12]......7 Slab: 6163512 kB [1831] x-094:[12]......7 Slab: 6271272 kB [1850] x-095:[13]..................false Slab: 6371956 kB x-095 failed to boot. keeping x-095. So kernel memory seemed to peak at around 10GB, and then somehow dropped down to 5GB to allow 20 more containers to be created. (on a 16GB machine, that's a fair bit allotted to just the kernel). However, 95 seems to be the limit for 1024 fs.inotify.max_user_instances. But that still means setting it to 1M is silly. I'll do another run with max_user_instances=2048 and see what happens. Since I know with very-high max values, I can get to 230 containers, that sounds like 2048 is sufficient for it. I'll also play around next with putting Juju into the loop and see where I get. I really wonder about max-open-files for Root vs User. LXD is running as Root, but if the containers are all running User level, shouldn't that be the constraint? -- You received this bug notification because you are a member of Ubuntu Bugs, which is subscribed to Ubuntu. https://bugs.launchpad.net/bugs/1602192 Title: when starting many LXD containers, they start failing to boot with "Too many open files" To manage notifications about this bug go to: https://bugs.launchpad.net/juju/+bug/1602192/+subscriptions -- ubuntu-bugs mailing list [email protected] https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
