Re: [Xen-devel] [PATCH v5] tools/xenconsoled: Increase file descriptor limit
On Mon, 2015-03-02 at 14:22 +, Ian Campbell wrote: On Fri, 2015-02-27 at 17:53 +, Andrew Cooper wrote: XenServer's VM density testing uncovered a regression when moving from sysvinit to systemd where the file descriptor limit dropped from 4096 to 1024. (XenServer had previously inserted a ulimit statement into its initscripts.) One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the lost ulimit, but that is only a stopgap solution. As Xenconsoled genuinely needs a large number of file descriptors if a large number of domains are running, attempt to increase the limit. Signed-off-by: Andrew Cooper andrew.coop...@citrix.com CC: Ian Campbell ian.campb...@citrix.com CC: Ian Jackson ian.jack...@eu.citrix.com CC: Wei Liu wei.l...@citrix.com I tried to apply but I'm afraid that for 32-bit userspace this gives me: daemon/main.c: In function 'increase_fd_limit': daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 5 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 6 has type 'rlim_t' [-Werror=format] I've no idea how one is formally supposed to print and rlim_r. The Internet(tm) seems to think by casting to long long is the answer :-/ Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v5] tools/xenconsoled: Increase file descriptor limit
On 02/03/15 14:22, Ian Campbell wrote: On Fri, 2015-02-27 at 17:53 +, Andrew Cooper wrote: XenServer's VM density testing uncovered a regression when moving from sysvinit to systemd where the file descriptor limit dropped from 4096 to 1024. (XenServer had previously inserted a ulimit statement into its initscripts.) One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the lost ulimit, but that is only a stopgap solution. As Xenconsoled genuinely needs a large number of file descriptors if a large number of domains are running, attempt to increase the limit. Signed-off-by: Andrew Cooper andrew.coop...@citrix.com CC: Ian Campbell ian.campb...@citrix.com CC: Ian Jackson ian.jack...@eu.citrix.com CC: Wei Liu wei.l...@citrix.com I tried to apply but I'm afraid that for 32-bit userspace this gives me: daemon/main.c: In function 'increase_fd_limit': daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 5 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 6 has type 'rlim_t' [-Werror=format] I've no idea how one is formally supposed to print and rlim_r. Ian. Urgh - that would be why all the examples I found had an explicit (unsigned long long) cast for the values for printf(). I can respin if you wish. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v5] tools/xenconsoled: Increase file descriptor limit
On Fri, 2015-02-27 at 17:53 +, Andrew Cooper wrote: XenServer's VM density testing uncovered a regression when moving from sysvinit to systemd where the file descriptor limit dropped from 4096 to 1024. (XenServer had previously inserted a ulimit statement into its initscripts.) One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the lost ulimit, but that is only a stopgap solution. As Xenconsoled genuinely needs a large number of file descriptors if a large number of domains are running, attempt to increase the limit. Signed-off-by: Andrew Cooper andrew.coop...@citrix.com CC: Ian Campbell ian.campb...@citrix.com CC: Ian Jackson ian.jack...@eu.citrix.com CC: Wei Liu wei.l...@citrix.com I tried to apply but I'm afraid that for 32-bit userspace this gives me: daemon/main.c: In function 'increase_fd_limit': daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 5 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 6 has type 'rlim_t' [-Werror=format] I've no idea how one is formally supposed to print and rlim_r. Ian. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v5] tools/xenconsoled: Increase file descriptor limit
On Mon, 2015-03-02 at 14:24 +, Andrew Cooper wrote: On 02/03/15 14:22, Ian Campbell wrote: On Fri, 2015-02-27 at 17:53 +, Andrew Cooper wrote: XenServer's VM density testing uncovered a regression when moving from sysvinit to systemd where the file descriptor limit dropped from 4096 to 1024. (XenServer had previously inserted a ulimit statement into its initscripts.) One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the lost ulimit, but that is only a stopgap solution. As Xenconsoled genuinely needs a large number of file descriptors if a large number of domains are running, attempt to increase the limit. Signed-off-by: Andrew Cooper andrew.coop...@citrix.com CC: Ian Campbell ian.campb...@citrix.com CC: Ian Jackson ian.jack...@eu.citrix.com CC: Wei Liu wei.l...@citrix.com I tried to apply but I'm afraid that for 32-bit userspace this gives me: daemon/main.c: In function 'increase_fd_limit': daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 3 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 4 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 5 has type 'rlim_t' [-Werror=format] daemon/main.c:89:10: error: format '%lu' expects argument of type 'long unsigned int', but argument 6 has type 'rlim_t' [-Werror=format] I've no idea how one is formally supposed to print and rlim_r. Ian. Urgh - that would be why all the examples I found had an explicit (unsigned long long) cast for the values for printf(). I can respin if you wish. yes, please. ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
[Xen-devel] [PATCH v5] tools/xenconsoled: Increase file descriptor limit
XenServer's VM density testing uncovered a regression when moving from
sysvinit to systemd where the file descriptor limit dropped from 4096 to
1024. (XenServer had previously inserted a ulimit statement into its
initscripts.)
One solution is to use LimitNOFILE=4096 in xenconsoled.service to match the
lost ulimit, but that is only a stopgap solution.
As Xenconsoled genuinely needs a large number of file descriptors if a large
number of domains are running, attempt to increase the limit.
Signed-off-by: Andrew Cooper andrew.coop...@citrix.com
CC: Ian Campbell ian.campb...@citrix.com
CC: Ian Jackson ian.jack...@eu.citrix.com
CC: Wei Liu wei.l...@citrix.com
---
v5:
* Drop system maximum checking
* Unify set paths
v4:
* Calculate fd limit based on domid ABI - result is 132008 fds
* Warn if sufficient fds are not available.
v3:
* Hide Linux specific bits in #ifdef __linux__
v2:
* Always increase soft limit to hard limit
* Correct commment regarding number of file descriptors
* long - unsigned long as that appears to be the underlying type of an rlim_t
---
tools/console/daemon/main.c | 36
1 file changed, 36 insertions(+)
diff --git a/tools/console/daemon/main.c b/tools/console/daemon/main.c
index 92d2fc4..6e84f5a 100644
--- a/tools/console/daemon/main.c
+++ b/tools/console/daemon/main.c
@@ -26,6 +26,7 @@
#include string.h
#include signal.h
#include sys/types.h
+#include sys/resource.h
#include xenctrl.h
@@ -55,6 +56,39 @@ static void version(char *name)
printf(Xen Console Daemon 3.0\n);
}
+static void increase_fd_limit(void)
+{
+ /*
+* We require many file descriptors:
+* - per domain: pty master, pty slave, logfile and evtchn
+* - misc extra: hypervisor log, privcmd, gntdev, std...
+*
+* Allow a generous 1000 for misc, and calculate the maximum possible
+* number of fds which could be used.
+*/
+ unsigned min_fds = (DOMID_FIRST_RESERVED * 4) + 1000;
+ struct rlimit lim, new = { min_fds, min_fds };
+
+ if (getrlimit(RLIMIT_NOFILE, lim) 0) {
+ fprintf(stderr, Failed to obtain fd limit: %s\n,
+ strerror(errno));
+ exit(1);
+ }
+
+ /* Do we already have sufficient? Great! */
+ if (lim.rlim_cur = min_fds)
+ return;
+
+ /* Try to increase our limit. */
+ if (setrlimit(RLIMIT_NOFILE, new) 0)
+ syslog(LOG_WARNING,
+ Unable to increase fd limit from {%lu, %lu} to
+ {%lu, %lu}: (%s) - May run out with lots of domains,
+ lim.rlim_cur, lim.rlim_max,
+ new.rlim_cur, new.rlim_max,
+ strerror(errno));
+}
+
int main(int argc, char **argv)
{
const char *sopts = hVvit:o:;
@@ -154,6 +188,8 @@ int main(int argc, char **argv)
openlog(xenconsoled, syslog_option, LOG_DAEMON);
setlogmask(syslog_mask);
+ increase_fd_limit();
+
if (!is_interactive) {
daemonize(pidfile ? pidfile : /var/run/xenconsoled.pid);
}
--
1.7.10.4
___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v5] tools/xenconsoled: Increase file descriptor limit
Andrew Cooper writes ([PATCH v5] tools/xenconsoled: Increase file descriptor limit): XenServer's VM density testing uncovered a regression when moving from sysvinit to systemd where the file descriptor limit dropped from 4096 to 1024. (XenServer had previously inserted a ulimit statement into its initscripts.) ... Thanks, and sorry to be pernickety. Acked-by: Ian Jackson ian.jack...@eu.citrix.com ___ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
