Eliot, this is absolutely wonderful news. Thank you and everyone at Teleplace who made this possible.
Best wishes, Michael On Sun, Jun 20, 2010 at 10:11 PM, Eliot Miranda <[email protected]> wrote: > Hi All, > > it gives me great pleasure to announce that the Teleplace Cog VMs are now > available. Huge thanks to all at Teleplace who have given me the > opportunity to build Cog and release it as open source, been willing guinea > pigs braving its bugs, and providing indispensable participation in getting > Cog to its current state. Huge thanks are also due to the original Back To > The Future team whose VMMaker Cog extends to write the VM, and to Peter > Deutsch from whom I've taken many ideas. > > This release contains two VMs. The Stack VM, is a cross-platform > interpreter that uses context-to-stack mapping to achieve modest performance > gains. The Cog VM is a just-in-time compiler that currently supports only > x86 that builds upon the Stack VM to achieve substantial performance > improvements. The release is in the form of a Monticello package containing > the VMMaker source and a tarball containing the platform sources, the > generated sources and a Squeak 4.1 image containing the VMMaker sources. > Download both at > > http://ftp.squeak.org/Cog/VMMaker-oscog.11.mcz > > http://ftp.squeak.org/Cog/OpenSourceCog.tar.gz > > Cog VMs: > > The Cog VMs are Squeak/Croquet VMs that run closure > Squeak/Croquet/Pharo/Cuis images. The VMs support existing plugin source but > will require plugins to be recompiled as the VM_PROXY_MAJOR plugin api has > been extended. > > This release contains two distinct VMs, the StackInterpreter and the Cogit. > The StackInterpreter is a fully-portable plug-in replacement for the current > closure Squeak VMs and images. The Stack VM uses context-to-stack mapping > and a somewhat improved garbage collector to achieve modest but useful > performance gains in the 10% to 15% range. The StackInterpreter is intended > to supersede the Squeak VM on platforms where the Cogit cannot be used. The > Cogit extends the StackInterpreter with a just-in-time compiler that uses > aggressive inline caching techniques to deliver substantial performance > gains in the 3x to 15x range, depending on benchmark. The Cogit currently > supports only x86 and the floating-point primitives and parts of the > platform support code depend on SSE2. I hope members of the community will > attempt to port it, e.g. to ARM, PowerPC and x86-64. The Cogit (excuse the > pun) is so named because it is both an interpreter and a JIT, choosing not > to generate machine code for large methods, interpreting them instead, the > default policy being not to JIT methods with more than 60 literals. > > The Cog VM requires a few minor image changes all in > image/NecessaryImageChangesForCogToWork.1.cs. The JIT's machine-code > SmallInteger primitives insist on a SmallInteger receiver so the primitives > in LargePositiveInteger = ~= bitAnd: bitOr: butShift: and bitXor: cannot be > used and these methods must be deleted. The Cogit inlines the address of > the Character instance table, Smalltalk specialObjectsArray at: 25, into the > machine-code at: primitive for faster ByteString>>at: and so the table > cannot be rebuilt in SmalltalkImage>>recreateSpecialObjectsArray. The new > version preserves the existing table. Both VMs maintain floats in platform > order to ease implementation of machine code floating-point primitives, and > hence internally are in little-endian order instead of big-endian in current > Squeak images. While the VMs convert float order automatically on load they > do require special accessing primitives Float>>basicAt: & > Float>>basicAt:put: that undo the reversal and answer Float contents in > big-endian order so that e.g. Float>>hash is unchanged. The methods assume > these primitives can fail, allowing the code to be used on current Squeak > VMs. > > The image/VMMaker-Squeak4.1.image is a Squeak 4.1 image, runnable with the > current Squeak VMs, that contains these changes, and can hence also be run > with a Cog VM. But beware, once an image has been saved on Cog it cannot be > run by an existing Squeak VM, because existing VMs cannot undo the Float > order change. > > Platform Subsystem: > > Most of the platform subsystem is unchanged but there are some important > changes that need description. The biggest change is the heartbeat and the > clock in platforms/unix/vm/sqUnixHeartbeat.c and > platforms/win32/vm/sqWin32Heartbeat.c. The Cog VMs avoid the slow and > variable interruptCheckCounter, folding the event check into the stack > overflow check on frame build. The heartbeat, typically 500Hz or 1KHz, > changes the stackLimit to a value that will always fail. On the next frame > building send the VM will enter stack overflow handling that, as a side > effect, will also check for events. This is more efficient than the update > of interruptCheckCounter and much more regular. If one is running code that > executes long-running primitives (e.g. large integer arithmetic) the counter > approach will result in too low an interrupt check frequency, and conversely > if one is running normal code the interrupt check frequency can be very > high. > > The heartbeat also maintains a 64-bit microsecond clock, UTC microseconds > from 1901, from which the backward-compatible millisecond and second clocks > are derived. Primitives exist to answer UTC microseconds and local > microseconds. Updating the clock in the heartbeat results in a 1 or 2 > millisecond resolution but avoids the cost of accessing the OS time on every > prim tie which we've found important for performance at Teleplace. The > 64-bit microsecond clocks provide a unified time basis and eliminate > wrapping (for the next 54,000 years at least). I hope community images will > move to these clocks. It's worked well in VisualWorks. > > Another significant change is in the external semaphore table support code. > This is now lock-free at the cost of having to specify a maximum number of > external semaphores at start-up (default 256). The support code for the > lock-free data structures are processor-specific and is currently > implemented only for x86 and gcc-compatible compilers; see > platforms/Cross/vm/{sqAtomicOps.h,sqMemoryFence.h}. > > There is also improved crash reporting code that prints a primitive log and > a C backtrace in addition to the Smalltalk backtrace. See platforms/Mac > OS/vm/sqMacMain.c, platforms/unix/vm/sqUnixMain.c, > platforms/win32/vm/sqWin32Intel.c & platforms/win32/vm/sqWin32Backtrace.c. > > Finally there is support for the QVMProfiler, a pc-sampling profiler for > profiling at the VM level. See platforms/unix/vm/sqUnixVMProfile.c and > platforms/win32/vm/sqWin32VMProfile.c. The profiler itself is in the > VMMaker image described below in Qwaq-VMProfiling. > > There are also changes to do with Teleplace-specific extensions to the > HostWindowPlugin but these are not essential to Cog. > > VMMaker and Slang: > > The image/VMMaker-Squeak4.1.image Squeak 4.1 image contains the complete Cog > VMMaker with necessary support code for simulation. This image was used to > generate the sources in the src and stacksrc directories. > > Cog's VMMaker is substantially revised and extended from the current > VMMaker. It supports multiple classes, not just Interpreter and > superclasses, because both context-to-stack mapping and the Cogit are too > complex to write monolithically. Classes can specify ancilliaryClasses and > ancilliaryStructClasses, such as CoInterpreterStackPage, CogMethod and > CogAbstractInstruction. The Monticello package version is included in the > header of all generated files and constitutes the version stamp for > generated code. Code is generated in sorted order so that minor changes in > the Smalltalk source produce correspondingly minor changes in the generated > code. The gnuification step is built-in to VMMaker. No effort has been > made to maintain 64-bit compatibility. Apologies, this was unaffordable. > > The VMMaker generates a single source tree used by all platforms. Instead > of deciding at generation time whether to use the Interpreter struct the > generated code depends on the SQ_USE_GLOBAL_STRUCT define which can be > overridden in platform makefiles. All plugins live in src/plugins and > platform makefiles along with plugins.int and plugins.ext files in the build > subdirectories decide which plugins are built as external or internal. The > VM Generation Workspace from Workspace.text workspace contains dots to > generate the sources. We no longer use the VMMakerTool since there should > be nothing platform-specific in the generated sources (if we add ports to > other ISAs all their source can be included and selected as required by the > platform makefiles). > > Since the Cogit generates x86 machine code simulation is much more complex. > There is a support plugin, platforms/Cross/plugins/BochsIA32Plugin that > depends on a large simulation of the x86 family implemented in C++ (see > processors/IA32/bochs) and on Alien. I use the simulator frequently (but > note that I haven't had time to build a working version for Squeak 4.1). I > have tested Cog simulation in this image, running on the > image/VMMaker-Squeak4.1.image itself. The VM Simulation Workspace in the > VMMaker image contains an example doit that starts the simulator. Be > patient, even on a fast machine unhibernating the Squeak display background > image takes nearly a minute. Native fonts do not (yet) simulate correctly, > but the system runs. But note that I have only attempted to build and run > the simulator on Mac OS X. I expect Bochs can be built on linux and win32 > but I have not tried. By the way, I've not described how to run the Bochs > simulator on the current Squeak VM. That's because the plugin depends on > the heartbeat to break out of simulation occasionally via a new > interpreterProxy entry point setInterruptCheckChain. As this isn't > supported by the current Squeak VMs the plugin would require modification. > So to simulate first build either of the Cog VMs and then run the simulation > with it. > > There are a number of unpublished changes to the base other than those in > NecessaryImageChangesForCogToWork.1.cs. This is partly laziness on my part, > partly avoiding publishing things in advance of Cog. These changes are > better motivated once Cog is in use. There are changes to the "translated > primitives" (see implementors of translatedPrimitives) which replace > messages with method tags for generation directives. The Cog VMMaker uses > Object>>perform:with:with:with:with: & > Object>>perform:with:with:with:with:with: during simulation, and > Collection>>#fold: & SquenceableCollection>>#copyUpThrough: during > generation. Object>>inline: and Object var:declareC:, which are mispackaged > in Kernel in Squeak 4.1 are obsolete (method tags being used instead) and > have been removed. I have changed Integer>>hex and Integer>>hex8 back to > their original semantics as of 3.8. Backward compatibility is important and > one can easily add new selectors if one wants different functionality. > VMMaker was here first ;) > > Tarball: > > The top-level directories in the tarball are > > src > > the tree for the Cog generated sources including all plugins > > stacksrc/vm > > the directory containing the Stack VM source (plugins can be taken from > above) > > platforms > > the usual svn platform tree but including Cog specific changes such as the > heartbeat > > processors > > the tree containing simulation support code, i.e. the bochs C++ x86 > simulation library, along with a potential ARM, PowerPC & MIPS simulator, > Skeye. > > image > > the Cog-prepared Squeak 4.1 VMMaker image > > scripts > > some svn scripts to revert unchanged plugins that haven't really changed > > cygwinbuild > > the win32 build directory > > winbuild > > the old win32 build directory for minnow gcc 2.95. Not entirely obsolete as > the cygwin build as yet fails to generate a functional FFIPlugin > > macbuild > > the CoreVM.xcodeproj and support build projects for Mac OS X 10.5 or better > > unixbuild > > the build directory for linux > > Building Cog: > > Each build directory above contains a HowToBuild file that describes > building in more detail. The build directories only contain Cogit > makefiles. f you want to build a Stack VM you're on your own but this is > very close to the existing Squeak VM build. > > Status: > > The Cogit VM has been our sole VM at Teleplace for nearly a year. We do > occasionally find bugs and there are almost certainly areas of functionality > that we have not touched (for example I know that co-routining does not yet > work). If you find a bug please try and create a reproducible test case and > let me know. I can't promise to take a look or fix it but I am motivated to > do so and will try my best as time allows. Better still if you find and fix > bugs be sure to let me know. > > License (MIT): > > All contributions from Teleplace in this release are > > Copyright (c) 2010 Teleplace, Inc. > > Permission is hereby granted, free of charge, to any person obtaining a copy > > of this software and associated documentation files (the "Software"), to > deal > > in the Software without restriction, including without limitation the rights > > to use, copy, modify, merge, publish, distribute, sublicense, and/or sell > > copies of the Software, and to permit persons to whom the Software is > > furnished to do so, subject to the following conditions: > > The above copyright notice and this permission notice shall be included in > > all copies or substantial portions of the Software. > > THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > > IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > > FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE > > AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER > > LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING > FROM, > > OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN > > THE SOFTWARE. > > Eliot Miranda > > June 2010 > > > > _______________________________________________ Pharo-project mailing list [email protected] http://lists.gforge.inria.fr/cgi-bin/mailman/listinfo/pharo-project
