Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Tom Lane
Andres Freund writes: > On 2017-08-07 17:30:13 -0400, Tom Lane wrote: >> Meh. The lack of field complaints about this doesn't indicate to me that >> we have a huge problem, and in any case, just increasing NUM_RESERVED_FDS >> would do nothing for the system-wide limits. > Howso? Via count_usable

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Andres Freund
Hi, On 2017-08-07 17:30:13 -0400, Tom Lane wrote: > Meh. The lack of field complaints about this doesn't indicate to me that > we have a huge problem, and in any case, just increasing NUM_RESERVED_FDS > would do nothing for the system-wide limits. Howso? Via count_usable_fds() we test for max_fi

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Tom Lane
Andres Freund writes: > On 2017-08-07 17:05:06 -0400, Tom Lane wrote: >> Probably the best we can hope for there is to have fd.c provide a function >> "close an FD please", which postgres_fdw could call if libpq fails because >> of ENFILE/EMFILE, and then retry. > Unless that takes up a slot in f

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Peter Geoghegan
On Mon, Aug 7, 2017 at 1:40 PM, Andres Freund wrote: > Given how close max_files_per_process is to the default linux limit of > 1024 fds, I wonder if we shouldn't increase NUM_RESERVED_FDS by quite a > bit? Personally, any time I've seen a problem with this it was because an extension leaked FDs,

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Andres Freund
On 2017-08-07 17:05:06 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2017-08-07 16:52:42 -0400, Tom Lane wrote: > >> No, I don't think so. If you're depending on the NUM_RESERVED_FDS > >> headroom for anything meaningful, *you're doing it wrong*. You should be > >> getting an FD via fd.c

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Tom Lane
Andres Freund writes: > On 2017-08-07 16:52:42 -0400, Tom Lane wrote: >> No, I don't think so. If you're depending on the NUM_RESERVED_FDS >> headroom for anything meaningful, *you're doing it wrong*. You should be >> getting an FD via fd.c, so that there is an opportunity to free up an FD >> (b

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Andres Freund
On 2017-08-07 16:52:42 -0400, Tom Lane wrote: > Andres Freund writes: > > These days there's a number of other consumers of > > fds. E.g. postgres_fdw, epoll, ... All these aren't accounted for by > > fd.c. > > > Given how close max_files_per_process is to the default linux limit of > > 1024 fds

Re: [HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Tom Lane
Andres Freund writes: > These days there's a number of other consumers of > fds. E.g. postgres_fdw, epoll, ... All these aren't accounted for by > fd.c. > Given how close max_files_per_process is to the default linux limit of > 1024 fds, I wonder if we shouldn't increase NUM_RESERVED_FDS by quit

[HACKERS] max_files_per_processes vs others uses of file descriptors

2017-08-07 Thread Andres Freund
Hi, currently the default max_files_per_process is 1000. fd.c uses close to that many (- NUM_RESERVED_FDS/10). count_usable_fds() makes sure that at server start there's at most that many fds available, but that doesn't mean that much for runtime. These days there's a number of other consumers of