Understanding an address sanitizer message for a memory leak caused by global dispatcher (?)

[mratsim]

> Does anybody actually use address-sanitizers or valgrind or heaptrack and 
> eradicates potential memory leaks to the point that nothing shows up while 
> they run?

Yes, I use sanitizers when developing:

  * Weave
  * nim-taskpools
  * Constantine and Constantine's threadpool



However:

  * it's incompatible with Nim `refc` (the conservative stack scanning will 
trigger an alert when reading unallocated stack memory)
  * it needs `useMalloc` to avoid Nim allocator and provides precise heap 
checks.
  * it needs low-level understanding of implementation details when globals are 
involved.



In your case you call asyncdispatch `getGlobalDispatcher` and it's possible 
that global ref are not freed at the end of the program. The C codegen should 
be checked here, in NimMain, PreMainInner and the exit procs.

In your case the leak points to 
<https://github.com/nim-lang/Nim/blob/v2.0.2/lib/pure/ioselects/ioselectors_epoll.nim#L101>
    
    
    proc newSelector*[T](): Selector[T] =
      proc initialNumFD(): int {.inline.} =
        when defined(nuttx):
          result = NEPOLL_MAX
        else:
          result = 1024
      # Retrieve the maximum fd count (for current OS) via getrlimit()
      var maxFD = maxDescriptors()
      doAssert(maxFD > 0)
      # Start with a reasonable size, checkFd() will grow this on demand
      let numFD = initialNumFD()
      
      var epollFD = epoll_create1(O_CLOEXEC)
      if epollFD < 0:
        raiseOSError(osLastError())
      
      when hasThreadSupport:
        result = cast[Selector[T]](allocShared0(sizeof(SelectorImpl[T])))
        result.epollFD = epollFD
        result.maxFD = maxFD
        result.numFD = numFD
        result.fds = allocSharedArray[SelectorKey[T]](numFD)
      else:
        result = Selector[T]()
        result.epollFD = epollFD
        result.maxFD = maxFD
        result.numFD = numFD
        result.fds = newSeq[SelectorKey[T]](numFD)
    
    
    Run

This is indeed instantiated from asyncdispatch: 
<https://github.com/nim-lang/Nim/blob/v2.0.2/lib/pure/asyncdispatch.nim#L1207-L1209>
    
    
    proc newDispatcher*(): owned(PDispatcher) =
        new result
        result.selector = newSelector[AsyncData]()
    
    
    Run

And related to the global dispatcher: 
<https://github.com/nim-lang/Nim/blob/v2.0.2/lib/pure/asyncdispatch.nim#L1228-L1239>
    
    
    proc setGlobalDispatcher*(disp: owned PDispatcher) =
        if not gDisp.isNil:
          assert gDisp.callbacks.len == 0
        gDisp = disp
        initCallSoonProc()
      
      proc getGlobalDispatcher*(): PDispatcher =
        if gDisp.isNil:
          setGlobalDispatcher(newDispatcher())
          when defined(nuttx):
            addFinalyzer()
        result = gDisp
    
    
    Run

And if we look further how the global dispatcher is defined:

<https://github.com/nim-lang/Nim/blob/v2.0.2/lib/pure/asyncdispatch.nim#L356>
    
    
    var gDisp{.threadvar.}: owned PDispatcher ## Global dispatcher
    
    
    Run

It's a thread-local variable.

> Or rather, gut feeling says that this is address sanitizer not seeing the 
> destruction of the global dispatcher and thus falsely claiming a memory leak.

It is a memory leak, if the gDisp thread is joined, memory becomes unclaimable.

> 2 If it is a leak, what is supposed to be done here?

Raising a bug.

I'm not too sure what's the correct fix here.

Maybe having a magic `destroyThreadLocalVariables` that can be called before 
`joinThread` or exiting a program.