I'm quite new to the world of Racket, and this made for some nice reading! If I understand it correctly, this will be quite a massive change. I'm sure a lot of folks, including myself will be happy to pitch in whichever way we can (documentation, testing, coding, etc.) in this process!
On Wednesday, February 15, 2017 at 8:37:19 PM UTC+5:30, mflatt wrote: > > TL;DR: I expect the main Racket distribution to run on Chez Scheme > instead of the current Racket VM sometime in the next couple of years. > > > Background > ---------- > > The core Racket implementation relies on a lot of C code, and that's > a problem for maintenance, for porting it to new platforms (say, > JavaScript), and for improving performance. > > My long-term plan has been to rewrite it all in Racket (except for > things like GC). Last summer, I started by rewriting Racket's macro > and module system from scratch. Although I was able to get a fresh > implementation working, the new implementation's performance was not > good enough to drop in as a replacement for our current > implementation. > > After making sure that the new expander is really working with data > structures and algorithms that are at least as good as the old ones, > the new implementation is still off by up to a factor of two in time > and 25% or so in space. I'm convinced that Racket's compiler and > runtime system will have to be better to support a reimplementation of > the core in Racket. > > Although I have specific ideas on how the current Racket VM could be > improved, there are already other compilers and runtime systems that > perform better. Notably, Chez Scheme has recently become available > under an open-source license (Apache). > > As a first experiment, I have been able to use the new Racket expander > to compile a new Racket regexp implementation to run on Chez. As > expected, Chez runs the matcher about 2 times as fast as Racket, > putting it generally on par with the C implementation that's currently > built into Racket. > > So far, so good. After more experiments along these lines, I see no > scenario where it's easier or more effective to improve Racket than to > build on Chez. Although there are some mismatches between the primitive > features that Chez and Racket provide, it will mostly work; in rare > cases where there's no other solution, we can contribute pull requests > to Chez. > > A Racket on Chez won't be compatible with the current Racket VM at the > level of the C API. For that reason and others, I expect the current > Racket VM to stick around for projects that use the C API, that need > to embed Racket, or so on. The goal of rewriting Racket in Racket has > always been to support multiple VMs, so there's no conflict with > keeping the current Racket VM, too. Other VM efforts, like Pycket and > RacketScript, should also benefit. Still, I expect mainline Racket > development to move to Chez as the primary backend. > > This shift in direction will affect many projects planned to use or > improve Racket. Although I expect to implement most of the conversion > myself, I've talked with others who are invested in the current Racket > direction. So far, everyone seems to be on board. > > Rewriting the Racket core has always seemed prohibitively difficult > and time-consuming, but I think now's the time, especially with the > expander and module system out of the way. The other subsystems look > easy compared to the part that is done, but there's plenty of room for > surprises. > > > Status > ------ > > I've reimplemented the Racket expander, reader, and regexp matcher in > Racket, and I've started on the port and filesystem path subsystems. > The new subsystem implementations have been developed in the "linklet" > branch here: > > https://github.com/mflatt/racket/tree/linklet > > See "expander" and "regexp" in the "pkgs" directory, for example. I > expect this branch to eventually move to a branch or repo in the > "racket" account on GitHub. > > Here's my work-in-progress toward making Racket reimplementations run > on Chez: > > https://github.com/mflatt/not-a-box/ > > For example, you can run the regexp implementation in Chez and compare > to Racket's using its built-in regexp implemenation in C or to Racket > running the new regexp implementation. > > > Plan > ---- > > My immediate plan is to make Racket's reader + expander + module > system run on Chez. It could be two weeks or two months. From there, > I'll continue rewriting all of the Racket subsystems in Racket, > compiling them down to layers of Chez. See "README.txt" in the > "not-a-box" repo for a status report and thoughts on various Racket > subsystems. > > When the current Racket VM becomes the legacy Racket VM, I expect that > it will retain most of its C implementation, which means that it will > keep its performance for backward compatibility. Even this legacy VM, > however, will use the new reader, expander, and module system --- > essentially as the linklet branch is now. The expander performs well > enough to be practical without affecting the runtime performance of > the programs that it expands. > > Currently, I see no obstacle to getting everything running on Chez by > the end of the year, but we'll see how it goes. > > -- You received this message because you are subscribed to the Google Groups "Racket Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-dev+unsubscr...@googlegroups.com. To post to this group, send email to racket-dev@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/racket-dev/1dbd8dce-f198-4b47-bc07-887dcb1ab3e7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
