Ok, i have found why:
In that commit there is a new templated implementation for newSelector()
(0.17.2) to newSelector[T]() (0.17.3+) in linux specifically it is using epoll.
proc newSelector*[T](): Selector[T] =
# Retrieve the maximum fd count (for current OS) via getrlimit()
var a = RLimit()
if getrlimit(RLIMIT_NOFILE, a) != 0:
raiseOsError(osLastError())
var maxFD = int(a.rlim_max)
doAssert(maxFD > 0)
var epollFD = epoll_create(MAX_EPOLL_EVENTS)
if epollFD < 0:
raiseOsError(osLastError())
when hasThreadSupport:
result = cast[Selector[T]](allocShared0(sizeof(SelectorImpl[T])))
result.epollFD = epollFD
result.maxFD = maxFD
result.fds = allocSharedArray[SelectorKey[T]](maxFD)
else:
result = Selector[T]()
result.epollFD = epollFD
result.maxFD = maxFD
result.fds = newSeq[SelectorKey[T]](maxFD)
As you see, it is allocating newSeq[SelectorKey[T]](maxFD)
in my system (docker alpine) maxFD is set to 1M and SelectorKey is defined as:
SelectorKey[T] = object
ident: int
events: set[Event]
param: int
data: T
sizeof SelectorKey[AsyncData] is 40 so a simple math 40*1M = 40M
A simple workaround that uses 10x less memory:
proc newSelector*[T](): Selector[T] =
# Retrieve the maximum fd count (for current OS) via getrlimit()
var a = RLimit()
if getrlimit(RLIMIT_NOFILE, a) != 0:
raiseOsError(osLastError())
var maxFD = int(a.rlim_max)
doAssert(maxFD > 0)
var epollFD = epoll_create(MAX_EPOLL_EVENTS)
if epollFD < 0:
raiseOsError(osLastError())
when defined(maxFileDescriptors):
const maxFileDescriptors {.intdefine.}: int = 132072
if maxFD > maxFileDescriptors:
maxFD = maxFileDescriptors
when hasThreadSupport:
result = cast[Selector[T]](allocShared0(sizeof(SelectorImpl[T])))
result.epollFD = epollFD
result.maxFD = maxFD
result.fds = allocSharedArray[SelectorKey[T]](maxFD)
else:
result = Selector[T]()
result.epollFD = epollFD
result.maxFD = maxFD
result.fds = newSeq[SelectorKey[T]](maxFD)
For a proper solution we should dinamically increase fds if we are requesting
more than 132072 file descriptors
