Re: Experimenting with a FreeRTOS OS Port
> So it's a pain to rebase... Anyone know what's required to get the base PR > merged? Where is your PR?
Re: Experimenting with a FreeRTOS OS Port
I was able to auto-merge your branch with upstream devel with no immediate problems, but only tested a hello world, didn't test all the POSIX stuff. The question of how to separate the different options is fraught... It would be great to be able to specify --os:esp-idf and have it work as smoothly as, e.g. --os:nintendoswitch Or are you thinking that since there's so many variants, better to specify each separately? How do you specify --lwip when compiling? I've got lots more bikeshedding, but I'll put it on your github
Re: Experimenting with a FreeRTOS OS Port
That's a good list! But the LwIP and FreeRTOS <=> POSIX brigde requires support in the Nim standard library to make using Nim sockets/selects/etc libraries possible. The port on my branch with networking supports the LwIP, and some of the VFS via the normal POSIX API's (uart, select, etc). The core part to make Nim usable with ESP-IDF is: * the FreeRTOS <=> POSIX bridges * Nim compiler constants for 'os:freertos' * LwIP wrappers * Modifications to the standard libraries. Once those are in place the VFS, FreeRTOS tasks, etc can be handled readily as libraries, or small PR's. Unfortunately the standard libraries include number of when ... blocks to avoid functions that don't make sense on FreeRTOS slimmed POSIX. Since you need to space the rest of the proc with the when block, any upstream changes in those files cause conflicts. So it's a pain to rebase... Anyone know what's required to get the base PR merged? Then it'd be much easier to start working on testing out the full system and expanding out the features.
Re: Experimenting with a FreeRTOS OS Port
The other option is that I may re-apply my changes to 1.2.4 branch and just develop from there as the "stable" version.
Re: Experimenting with a FreeRTOS OS Port
@dponyatov suggests a list of features for embedded [https://github.com/nim-lang/needed-libraries/issues/81](https://github.com/nim-lang/needed-libraries/issues/81) suggests it support newlib+FreeRTOS+FAT32+lwip esp-idf specifically provides newlib,freertos,lwip and 'vfs' which is a filesystem wrapper
Re: Experimenting with a FreeRTOS OS Port
> Have you given the selectors module a try? It should provide an even nicer > API on top of the raw POSIX select. Oh actually that is what I'm using, not the raw posix.select. I meant to say that wrapping the esp32 libraries in Nim was far enough along to provide access to the posix.select, which then let me use the Nim selectors. Since the esp-idf provides a POSIX compatible select shim it worked like a charm!
Re: Experimenting with a FreeRTOS OS Port
Excellent! It'd be great to get some help. I've got a number of changes for wrapping a few ESP libraries that could probably be brought out into another package. Also the branch is pretty out of date and needs some rebasing.
Re: Experimenting with a FreeRTOS OS Port
> Currently I'm using Nim on an ESP32 with posix.select for a custom TCP > protocol. Have you given the selectors module a try? It should provide an even nicer API on top of the raw POSIX select.
Re: Experimenting with a FreeRTOS OS Port
this is fabulous, thanks for your hard work! I work a lot with the esp, and it would be such a dream to have first class nim support. I'll check this out and contribute as much as i'm able
Re: Experimenting with a FreeRTOS OS Port
Nice! I'll check it out a bit. Async should be ok on many embedded platforms. Especially if async is the standard for asynchttp and other protocols. ESP32's have 328kB of ram and dual 240 MHz cores. Plenty enough for async http servers, even if it requires ORC that'd be fine. Though nimcsp could be handy for optimized or more memory limited devices. :-) Currently I'm using Nim on an ESP32 with posix.select for a custom TCP protocol. It's working pretty well! Sooo much better than writing embedded C.
Re: Experimenting with a FreeRTOS OS Port
There is also [https://github.com/zevv/nimcsp](https://github.com/zevv/nimcsp) which is better suited for embedded development, it deserves more attention and development.
Re: Experimenting with a FreeRTOS OS Port
I've add a bit of documentation to my main example project, with networking (ethernet): [https://github.com/elcritch/esp32_nim_net_example](https://github.com/elcritch/esp32_nim_net_example)
Re: Experimenting with a FreeRTOS OS Port
> For stdlib's async you need --gc:orc and Nim devel, otherwise it leaks > memory. Sorry, I hope we can get 1.2.2 out soon! The good news is that we're > also planning an async more suited for embedded. Ah, that makes sense. I look forward to 1.2.2! A better async for embedded would be great. I tried the devel branch with ORC running on Linux and it still looks like async has a memory leak. I'll try pulling in more recent changes from devel and see if that fixes it.
Re: Experimenting with a FreeRTOS OS Port
> P.S. while the "hello world" http server is working, it currently blows up > after about ~22 requests. Anyone got suggestions? For stdlib's async you need `--gc:orc` and Nim devel, otherwise it leaks memory. Sorry, I hope we can get 1.2.2 out soon! The good news is that we're also planning an async more suited for embedded.
Re: Experimenting with a FreeRTOS OS Port
P.S. while the "hello world" http server is working, it currently blows up after about ~22 requests. Anyone got suggestions? @Clonk \-- Debugging on JTAG should work with Nim sources as the stack traces are using Nim source maps. Though sometimes I disable it as tracking at the C code level is more useful. 0x40107ba7: newSeqPayload at $HOME/.asdf/installs/nim/devel/lib/system/seqs_v2.nim:38 (discriminator 8) 0x40107c1d: newSeq__0du5Us2v5bU6GIaJAPx4mw at $HOME/.asdf/installs/nim/devel/lib/system.nim:644 0x4010a470: auxWriteStackTrace__cEBhPS13dF9bIXoO5ipUApw at $HOME/.asdf/installs/nim/devel/lib/system/excpt.nim:224 0x4010a4ca: rawWriteStackTrace__zo0sKsXK7oXYz0ERc41mPw at $HOME/.asdf/installs/nim/devel/lib/system/excpt.nim:328 0x4010d302: getStackTraceEntries__9bQRI0oawDvYWdmWCeQkCQw at $HOME/.asdf/installs/nim/devel/lib/system/excpt.nim:537 0x400e3593: newFuture__VtQuu6AalFf9ctR9cpEfL1rQ at $HOME/.asdf/installs/nim/devel/lib/pure/asyncfutures.nim:112 0x400e3692: newFutureVar__XDPsxWPxi0l6fpPZSAM3yw at $HOME/.asdf/installs/nim/devel/lib/pure/asyncfutures.nim:131 0x400f2169: processClientIter__EWkU6XVqweduW9a0tak9cXyg_2 at $HOME/.asdf/installs/nim/devel/lib/pure/asynchttpserver.nim:284 0x400e7b1e: processClientNimAsyncContinue__MeADiwFdYVlucEKpa8KHdg_2 at $HOME/.asdf/installs/nim/devel/lib/pure/asyncmacro.nim:23 0x400f259d: processClient__hc4zfpq4yysbMo9bAsMN1xA at $HOME/.asdf/installs/nim/devel/lib/pure/asyncmacro.nim:246 (discriminator 1) 0x400f2761: serveIter__EWkU6XVqweduW9a0tak9cXyg at $HOME/.asdf/installs/nim/devel/lib/pure/asynchttpserver.nim:313 0x400e7ed2: serveNimAsyncContinue__MeADiwFdYVlucEKpa8KHdg_3 at $HOME/.asdf/installs/nim/devel/lib/pure/asyncmacro.nim:23 0x400d92f1: processPendingCallbacks__xdICRvJ1ns9byYHPjM07uJQ at $HOME/.asdf/installs/nim/devel/lib/pure/asyncdispatch.nim:214 (discriminator 1) 0x400da5a9: runOnce__nEnn7ilZ6aSkcXTtLEKnJA at $HOME/.asdf/installs/nim/devel/lib/pure/asyncdispatch.nim:1348 (discriminator 2) 0x400da649: poll__wcvh7vf5dGU1VYto9aiAnpA at $HOME/.asdf/installs/nim/devel/lib/pure/asyncdispatch.nim:1589 0x400da6ec: waitFor__zz9aVsJN0aTuFUXvzKrFo9cA at $HOME/.asdf/installs/nim/devel/lib/pure/asyncdispatch.nim:1899 0x400d64bd: run_http_server at $HOME/projects/pt/third-party/esp32_nim_net_example/nim/example_server.nim:12 0x400d4e56: app_main at $HOME/projects/pt/third-party/esp32_nim_net_example/build/../main/ethernet_example_main.c:145 0x400d361e: main_task at $HOME/projects/pt/esp-idf/components/esp32/cpu_start.c:580 0x40084ac1: vPortTaskWrapper at $HOME/projects/pt/esp-idf/components/freertos/xtensa/port.c:143 Run
Re: Experimenting with a FreeRTOS OS Port
@Yardanico oh thanks! @Clonk, it would be nice to support those but are outside the scope of my efforts. The ESP32 is my current target so that's why I'm going for FreeRTOS + LwIP. Though, I've been toying with using a "embeededos" or "oslite" so other embedded RTOS'es could use the work... but that'll be tricky as I'd suspect each RTOS would support slightly different subsets of POSIX. _sigh_ Still getting a working FreeRTOS target should make it a _lot_ easier for others to copy and add Zephyr/Contiki/etc support. Particularly it's been a bit gnarly to get the os module working with embedded, but it'd not be too bad to re-use that work for other RTOS'es!
Re: Experimenting with a FreeRTOS OS Port
NuttX and Zephyr could be also good candidates, they support a lot of hardware. Since they provide a decent posix API it might be easier. Implementing [CMSIS](https://os.mbed.com/handbook/CMSIS-RTOS) , could also be worthwhile as it is an ARM standard API implemented by several RTOS (now I just need a board). > Got an example asynchttpserver running on a wESP32 Out of curiosity, does debugging using the JTAG connector points to the correct Nim source code ?
Re: Experimenting with a FreeRTOS OS Port
FYI: you don't need --exceptions:goto --dead_code_elim:on Run Because: 1. ARC sets goto exceptions by default 2. Dead code elimination is always enabled nowadays
Re: Experimenting with a FreeRTOS OS Port
Got an example asynchttpserver running on an ESP32! curl 192.168.1.22:8080 Hello World% Run You can find the example here: [https://github.com/elcritch/esp32_nim_net_example](https://github.com/elcritch/esp32_nim_net_example) The test example is an ESP32 project which starts the asynchttpserver (see nim/run_http_server.nim and main/ethernet_example_main.c). @foldl, @dom96, @rayman22201 this also follows up on the discussion at: [https://forum.nim-lang.org/t/5522#34460](https://forum.nim-lang.org/t/5522#34460) This example relies on ARC. I haven't done any memory profiling but ARC appears to do a great job keeping memory usage very low on tests I've done. I'll try and cleanup my branch for a PR later this week.
Re: Experimenting with a FreeRTOS OS Port
@araq, that's great! I was hoping to do a PR, but wasn't sure if there would be interest. It seems, LwIP and FreeRTOS support could be split. Most of what I've done is actually port Nim sockets to LwIP. That'd mean it'd be easy to combine Nim & LwIP on many embedded devices. Once I get networking tested I'll create a PR. Would appreciate a code review! There are lots of areas of Nim & FreeRTOS I'm not familiar with. Not really sure how to do the CI, but it'd be great. There is QEMU support for the ESP32.
Re: Experimenting with a FreeRTOS OS Port
I'm _very_ interested in this port and would appreciate PRs. Ideally even CI support.
Re: Experimenting with a FreeRTOS OS Port
This is great! ESP32 is a nice platform; I've used it a bit in the past, with C++. It'd be good to have Nim running on it. I don't have the time to contribute right now; just offering moral support and thumbs-up
Re: Experimenting with a FreeRTOS OS Port
I've made a bit more progress. I've got an example esp32 Ethernet server to compile. But ran into something that's not quite clear: proc newSelectEvent*(): SelectEvent = var fds: array[2, cint] if posix.pipe(fds) != 0: raiseIOSelectorsError(osLastError()) ... Run There are no pipe's on FreeRTOS/ESP32 so I excluded them. It seems this just provides custom user event's and doesn't seem necessary.
Re: Experimenting with a FreeRTOS OS Port
Also, it'd be cool to see how many standard Nim libraries on threads could be extended to the ESP32 as well. That's far down the list, but how would one integrate vTasks*** with Nim?
