Hello All;
I am so sorry to have been unavailable due to a Day Job Crisis (you have
*no* *idea* how sorry). Hopefully, I should be more available for the
next few weeks. I shall be responding to queued-up kernel emails in
reverse chronological order...
[EMAIL PROTECTED] wrote:
> I'm currently doing design work / back-of-envelope-scribbling for what I
> hope will be the next version of the garbage collector for jjos / decaf.
> Here's what I'm planning, roughly in the order that I intend to do the
> work:
>
> 1. Rewrite in C++. Avoid assumptions about 32-bit pointers to simplify
> later ports.
> 2. Rework it to be a generational collector. This should decrease the
> average pause time by performing lots of little garbage collects, rather
> than the occasional big one (these would still be needed, but less
> frequently).
> 3. Tune and optimise.
> 4. Add support for finalisation.
> 5. Add support for weak references.
This all sounds pretty sensible to me, although I confess I am not as
familiar with the weak references as applied to Java. For my own
edification, can anybody point me at some web-available (or otherwise
available, I guess) info on them? Are these references identifiable to
the JVM (e.g., special markings in constant table, derived from special
Object subclass)?
> As part of this work, I'll need to put in simple support for virtual memory.
> This is so that the generational collector can find dirty pages that might
> have had inter-generational pointers stored into them. Also, the extra
> breathing space offered by a 4Gb virtual address space might simplify some
> aspects of heap layout. I'll put out a brief 'spec' before carrying out this
> work so that people can comment. Of course, if someone wanted to beat me to
> it...
I have looked at the VM stuff extensively (in this and past lives).
However, I had been punting on putting support for it in JJOS -- at no
time in the past did it seem like the hottest fire. This situation has
changed radically due to your success with the GC system. Thus, I would
be more than happy to help. I even (at one point) had a Plan for doing
this, the first item of which would be identity-mapping the currently
physical memory. (I do not have the Plan notes in front of me now, but
I think one of the soon-to-follow items was mapping the ramdisk way the
hell out into left field, and mapping the single stack out into a place
it wouldn't be likely to grow down into anything important. I may have
wanted to put a "guard page" to trap that unlikely event, too.) I think
there were also some issues with DMA memory address spaces, but I know
that I'd have to crack the books to figure out exactly what they were...
> One thing to ponder is whether all the above work is worthwhile - the
> Boehm-Demers garbage collector does all the above and more (eg concurrent gc
> apparently), is freely available and usable, and has been developed and
> widely used for over a decade, so is very stable. Integrating that might be
> a better way to get JOS to progress (though perhaps less fun for me :-).
ICBWB (I Could Be Wrong But) I do not believe the Boehm gc software, as
distributed, is amenable to interfacing with the VM subsystem. ICBWB I
think that is essential here. In particular, I should like to GC pages
AS/BEFORE they get paged out so as to avoid physical I/O to the maximum
extent possible. It is my understanding (from reading the GC book you
mentioned) that this feature alone constituted a lot of the performance
advantages experienced by LispM users -- gc, no matter how slow, is
always faster than hardware I/O.
There is also some question in my mind as to the relationship between
the GC native code and any GC thread that we might want (thinking out
loud -- there is a certain appeal to having a GC thread that can run in
the background, and stroll through memory, and use the same paging
daemon machinery everybody else does).
BTW, I think we also need to worry about "wiring down" (a.k.a., "locking
down") certain important Java text and data for things like the VM
paging machinery. Wouldn't want that paging stuff paged out, now, would
we? Perhaps we can simplify any Java-language API to "wire down"
classes and/or instance of a given class by using the
pointer-identification algorithms of the gc class you're writing.
(I.e., we could just do:
jos.PagingDaemon pd = new PagingDaemon(...);
jos.platform.wireDown(pd); // wires down class text and all reachable
data
instead of having to write a lot of "wireDown" methods for all the
reachable data and text in terms of memory addresses -- just a thought.)
> Some other musings:
>
> � There should be another mail of mine floating around on the arch list
> containing thoughts on how to deal with real-time aspects of device drivers.
> Does anyone have any idea if what I propose there will be sufficient, or
> does more thought need to be given to it?
I couldn't find the article -- (actually it looks like mailman is hosed
when I try to view the November archives) -- would you please forward me
a copy?
> � The current GC and the one proposed above are conservative (see the FAQ
> below for what this means). An upshot of this is that objects can't be moved
> once allocated, and this restricts the type of gc algorithms that can be
> used. There are a number of possibilities for making the garbage collector
> more accurate and thus able to move objects:
> 1. Separate C++ heap from which objects are explicitly freed.
> 2. Separate C++ heap which is conservatively gc'd.
These are all fine ideas, ...
> 3. Get hold of C++ object and stack layout from the compiler. I haven't
> fully investigated this, but it would seem that the stabs debug information
> produced by g++ should contain all the information needed. Whether this is
> true when the optimisation levels are cranked up is not clear.
... and this is easily computable ...
> Todd - didn't you let slip in a mail recently that you'd had some thoughts
> about Java v C++ gc? Care to elaborate?
... but I strongly urge (I would insist if I had that kind of leverage)
that we not undertake any such optimizations and/or improvements until
we actually have enough of the system hung together and run some
profiling. I have yet to work on any non-trivial application for which
I was able to clairvoyantly determine which optimizations I should
design into the system.
For example, even if our JVM is dog-slow (e.g., 10 times worse than
competing JVMs), if we can run some threads whilst others page (unlike
any other JVM I know of which runs under UNIX to date), and avoid
sleeping the whole system whilst the disk drive turns, our JVM will in
fact perform superior to any other for large Java system behavior...
> Anything else I should be considering? Any other comments?
Let me know how I can help. At a minimum, I have some really good
protected/virtual x86 references handy (well, at home, anyways).
Keep up the good work!
-jm
--
==== John Morrison
==== MaK Technologies Inc.
==== 185 Alewife Brook Parkway, Cambridge, MA 02138
==== http://www.mak.com/
==== vox:617-876-8085 x115
==== fax:617-876-9208
==== [EMAIL PROTECTED]
_______________________________________________
Kernel maillist - [EMAIL PROTECTED]
http://jos.org/mailman/listinfo/kernel
