Nathan Henrie <n8hen...@gmail.com> added the comment:
I've continued looking into this.
If you have your limits configured higher than default, as I did (and which
seems to be working fine):
```
sudo launchctl limit maxfiles 64000 524288
ulimit -Sn 64000
```
then you'll find that having a soft limit (`NUM_FDS`, prior to the `-=32`)
greater than 10273 causes the error I was seeing.
First, I found this number by passing in NUM_FDS as an envvar to the unittest
and looping in a bash script, but I have since extracted some of the relevant
portions to a separate python script:
```python
import selectors
import socket
import sys
def main(NUM_FDS):
s = selectors.PollSelector()
NUM_FDS -= 32
sockets = []
try:
for i in range(NUM_FDS // 2):
try:
rd, wr = socket.socketpair()
sockets.extend([rd, wr])
except OSError as e:
print(f"err 2: {type(e)} {e}")
# too many FDs, skip - note that we should only catch EMFILE
# here, but apparently *BSD and Solaris can fail upon connect()
# or bind() with EADDRNOTAVAIL, so let's be safe
print("FD limit reached")
try:
s.register(rd, selectors.EVENT_READ)
s.register(wr, selectors.EVENT_WRITE)
except OSError as e:
print(f"err 3: {type(e)} {e}")
if e.errno == errno.ENOSPC:
# this can be raised by epoll if we go over
# fs.epoll.max_user_watches sysctl
print("FD limit reached")
raise
try:
slen = len(s.select())
except OSError as e:
print(f"err 4: {type(e)} {e}")
raise
print(f"asserting {NUM_FDS // 2} == {slen}")
assert NUM_FDS // 2 == slen
finally:
for sock in sockets:
sock.close()
s.close()
print("Closed")
if __name__ == "__main__":
main(int(sys.argv[1]))
```
```shellsession
$ ./python.exe tester.py 10273
asserting 5120 == 5120
Closed
$ ./python.exe tester.py 10274
err 4: <class 'OSError'> [Errno 22] Invalid argument
Closed
Traceback (most recent call last):
File "tester.py", line 50, in <module>
main(int(sys.argv[1]))
File "tester.py", line 36, in main
slen = len(s.select())
File "cpython/Lib/selectors.py", line 376, in select
fd_event_list = self._poll.poll(timeout)
OSError: [Errno 22] Invalid argument
```
Tested and confirmed that I can provoke the error with the above launchctl /
ulimit settings on my wife's Macbook Air.
I wonder if the 10273 limit I'm running up against has something to do with the
10240 number I keep running into as a default kern.maxfilesperproc
(https://unix.stackexchange.com/a/350824/77904), given the `NUM_FDS -= 32` in
the test.
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue33036>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
