Hello, I originally starting working on a "game engine" in Common Lisp (quotations as it's actually a visualization tool). Eventually, I ran into memory management/GC issues so I sought out another Lisp-based language with a more hackable implementation. In my own system, one of the goals is to maximize expressiveness and dynamicness, however, balanced with pragmatism and simplicity. I found PicoLisp which has similar principles and is an awesome system! It's exactly what I've been looking except for a few things which seem feasible to implement.
Because my system has soft deadlines, I would like to implement per-frame allocation which gives more deterministic/bounded execution. Allocation is simply pointer bumping (as is now) and freeing would be reseting a pointer to the memory base giving O(1) performance. Nothing would be shared between frames (anything needed is copied). Of course, I would need to handle the typical pitfalls associated with manual memory management, however, the nature of my system limits those. I was reading about memory management in Lisp Machines back in the day, and they used a concept called "areas" consisting of "regions" (the meaning of those terms, including "arena" and "pool", has become conflated these days), which gives much more control over memory management, while still involving the GC: * https://hanshuebner.github.io/lmman/areas.xml * http://www.unlambda.com/lmman/lmman_21.html Initially, I thought about implementing a similar system into PicoLisp, but I feel that would be restrictive and complex; however, if implemented more fundamental functionality, it would provide greater flexibility for memory management schemes, albeit at the lisp level (performance not withstanding). Thus, I propose a few changes to the memory system: * Expose *Heaps and *Avail pointers * Or more formally, expose Heaps as a list of lists of cons cells, and Avail as a single cons cell. * Create/expose global "*GC" to enable/disable GC * cons* functions would need to check this * The current functionality could be retain and force GC anyways if there is no space, i.e. trust the user but provide fallback mechanism * Heap size can already be controlled with gc, however, could make heap resizing a separate function so GC need not be performed with the resize. * This makes sense with automatic collection, but adds unnecessary work if doing manual management. * If Heap/Avail pointers exposed, then this could also be done manually in lisp. In my situation, I would simply allocate a heap of sufficient memory at the beginning of the program/frame, then allocate as usual, and finally reset *Avail at the start/end of the frame. A few other questions: * Is there any particular reason, such as for convenience or preventing fragmentation, that CELLS is defined rather than a variable? * Is it possible to add a hook for when the GC is called? This would assist in profiling and diagnostics. I believe memory management need not be a dichotomy between malloc/free and GC; tools can be provided to work in the grey space, which typically is some form of arenas/regions/pools. Many implementations also lack said tools as they strictly adhere to one or the other, which results in inadequate workarounds and/or increasing complexity, had the implementation had support in the first place. Last but not least, it could help make PicoLisp more compelling for real-time scenarios; especially for situations where GC tuning is insufficient, but complete manual memory management is not feasible (see Java's various GC implementations). Sincerely, Kevin Ednalino
