Re: Why is my HTTP handler not gcsafe?
Weirdly, after I did some more work on my code I tried to reproduce the original problems I described (by backing out my tweaked versions of news and asynchttpserver) ... and they went away. Adding one single `gcsafe` pragma to the top-level HTTP server callback is sufficient. This is very strange, but I'm happy that the problem is gone.
Re: Why is my HTTP handler not gcsafe?
You shouldn't need to do that. Passing `--threadAnalysis:off` to the compiler should work too.
Re: Why is my HTTP handler not gcsafe?
I am not saying that you are using thread. This problem is that in standard
library the callback function in asynchttpserver.serve() is marked as gcsafe,
which (in my understanding) force all its inner-descendent function call to
follow the gcsafe rule, no matter you compile with --threads or not.
In your case, add {.gcsafe.} at the end of type SendProc should solve your
problem.
Re: Why is my HTTP handler not gcsafe?
I ended up copying asynchttpserver.nim into my source tree, renaming it, and
deleting all the `{.gcsafe.}` pragmas. Everything now compiles, and it seems to
be working fine aside from the bugs in my own code that I'm ironing out.
¯_(ツ)_/¯
Re: Why is my HTTP handler not gcsafe?
> In short, no declaration of non-primitives global variable. There seems
> something declared at module level in blip that you didn't show.
My code has zero global variables.The global referred to in the error is
`nameIterVar`gensym21340263` which is obviously a compiler-generated name, not
anything I did myself.
I've also run into the error Warning: 'send' is not GC-safe as it performs an
indirect call here [GcUnsafe2] referring to this code:
type
MessageBuf* = object
# ...
sendProc: SendProc
SendProc* = proc(msg: sink MessageBuf): Future[MessageIn]
proc send*(buf: sink MessageBuf): Future[MessageIn] {.gcsafe.} =
return buf.sendProc(buf)
Run
So it appears that calling a closure isn't GC-safe?!
> Nim current approach force threads to talk explicitly.
I'm not using threads. This is all with asynchttpserver, which AFAIK is
single-threaded.
Re: Why is my HTTP handler not gcsafe?
> If I add the gcsafe pragma to my callback proc, it now compiles without > errors! Where's my "extended error information"? There is a phase ordering problem in today's compiler so that in rare cases `.gcsafe` isn't inferred the way it should be. (Known problem.) The good news is that when the code compiles with `.gcsafe`, it really is GC safe and nothing to worry about. There is also [https://github.com/nim-lang/RFCs/issues/142](https://github.com/nim-lang/RFCs/issues/142) for how to evolve the "GC safety" issues.
Re: Why is my HTTP handler not gcsafe?
btw, if you need to pass some state to callback in server(), wrap it will closure. Nim current approach force threads to talk explicitly. I recall my memory that I have written a database connection pool and logger for a web server, which they are supposed to be globally accessible. And then I have to fight with the unexpected gcsafe for a long time... dealing with threads in nim is another story, I encountered unexpected exits without any messages if I did something wrong in ITC...
Re: Why is my HTTP handler not gcsafe?
In short, no declaration of non-primitives global variable. There seems
something declared at module level in blip that you didn't show.
When a proc is annotated with gcsafe, any proc called inside the proc has to be
gcsafe also, regardless of whether you are compiling with --threads. And if any
proc marked as gcsafe ever touched a module level non-primitive variable. An
error will throw. The first proc annotated with gcsafe here is the callback
function in serve().
Other people on this forum will probably explain better me. But here my 3-yo
attempt, if a proc is annotated with {.gcsafe.}, the proc is supposed to be run
in multi-threaded environment. And in current nim memory management model, each
thread has its own gc and they work on their own, no knowledge of each other.
This raises a problem: which gc should deal with the shared memory allocation
and freeing? Nim current answer is to avoid the problem at its first place: not
allow existence of non-primitives global variable touched at compile time.
gcsafe was one of the most annoying in my nim-experience... I had a long time
didn't know why my single threaded web server has to be complained with
non-gcsafe. Generally speaking, avoid global variable at all, this make testing
easier and common practice also.
Re: Why is my HTTP handler not gcsafe?
I've added some more code, and now I'm getting actual errors that seem to
explain the cause of the GC-safe violation ... if I could understand them.
/Users/snej/Code/nim/blip/src/server.nim(14, 25) template/generic
instantiation of `async` from here
/usr/local/Cellar/nim/1.2.0/nim/lib/pure/asyncmacro.nim(318, 24) Warning:
'receiveLoopNimAsyncContinue' is not GC-safe as it accesses
'nameIterVar`gensym21340263' which is a global using GC'ed memory [GcUnsafe2]
/Users/snej/Code/nim/blip/src/server.nim(14, 25) template/generic
instantiation of `async` from here
/Users/snej/Code/nim/blip/src/blip.nim(126, 6) Warning: 'receiveLoop' is
not GC-safe as it calls 'receiveLoopNimAsyncContinue' [GcUnsafe2]
/Users/snej/Code/nim/blip/src/server.nim(14, 25) template/generic
instantiation of `async` from here
/Users/snej/Code/nim/blip/src/blip.nim(136, 6) Warning: 'run' is not
GC-safe as it calls 'receiveLoop' [GcUnsafe2]
/Users/snej/Code/nim/blip/src/server.nim(14, 25) template/generic
instantiation of `async` from here
/usr/local/Cellar/nim/1.2.0/nim/lib/pure/asyncmacro.nim(278, 31) Warning:
'cbIter' is not GC-safe as it calls 'run' [GcUnsafe2]
Run
`receiveLoop` is pretty simple (see below), but `receiveLoopNimAsyncContinue`
and `nameIterVar`gensym21340263` must be internal functions created by the
macros behind the `{.async.}` pragma, so I don't really have a clue what's
going on...
proc receiveLoop(blip: Blip) {.async.} =
while blip.socket.readyState == Open:
let packet = await blip.socket.receivePacket()
case packet.kind
of Binary: blip.handleFrame(cast[seq[byte]](packet.data))
of Close: return
else: continue
Run
Re: Why is my HTTP handler not gcsafe?
...also, at a higher level, why is gcsafe even an issue at all, when all this async stuff is supposed to be running on the same thread?!
Why is my HTTP handler not gcsafe?
I've now got some slightly less trivial async code I'm trying to get to run,
using the 'news' WebSockets module. The compiler complains that my HTTP handler
isn't gcsafe, and suggests "Annotate the proc with {.gcsafe.} to get extended
error information."
However,
1. I can't see how my code isn't GC-safe, since it has no global variables
whatsoever.
2. If I add the `gcsafe` pragma to my callback proc, it now compiles without
errors! Where's my "extended error information"?
import blip
import news, asyncdispatch, asynchttpserver
when isMainModule:
echo("Starting Blip server on port 9001...")
var server = newAsyncHttpServer()
proc cb(req: Request) {.async.} =
if req.url.path == "/blipsync":
var ws = await newWebsocket(req)
echo "Creating new Blip"
var blip = newBlip(ws)
await blip.run() # <-- this line triggers the gcsafe error
echo "...Closed Blip"
await req.respond(Http404, "Nope", newHttpHeaders())
waitFor server.serve(Port(9001), cb)
Run
If I take out the line `await blip.run()`, the error goes away. But neither
that function nor anything it calls appears to involve any global state. (It's
too much code to post here in the forum, unless someone really wants me to!)
Did adding the `{.gcsafe.}` pragma silence a false alarm? Or is it suppressing
a genuine problem that will cause thread-safety issues?
Thanks!
—Jens
