Re: Re: [drlvm] string interning in java
On 7/29/06, Nathan Beyer [EMAIL PROTECTED] wrote: -Original Message- From: Robert Lougher [mailto:[EMAIL PROTECTED] Sent: Friday, July 28, 2006 7:40 PM To: harmony-dev@incubator.apache.org Subject: Re: Re: [drlvm] string interning in java Hi, On 7/29/06, Alex Blewitt [EMAIL PROTECTED] wrote: On 28/07/06, Salikh Zakirov [EMAIL PROTECTED] wrote: The interned String table should be garbage-collected. If the only reference to an interned String is from the class constant pool it will be collected when the class is unloaded. If the intern table is preventing Strings from being collected it's a bug in the VM! Can the String from a class constant pool be collected if the class is still in use? I'm assuming it can't, so the above is an incomplete response to Alex's points, as I imagine it's very possible to have many Strings in constant pools that are never or rarely used, but the class objects are used quite frequently and would never get collected. For example, the Strings used in exceptions that rarely, if every get called in some classes. It wasn't meant to be a complete response. I was responding to the following point: Furthermore, because it's a dynamically read in string, it doesn't pollute the intern() pool, and thus when the class is unloaded, the string is unloaded too. Once a String is intern()'d, it's like a memory leak -- you'll never see that memory again. Which implies that freeing of the dynamically loaded string when the class is unloaded is something special. It is not. Nor is an interned string a memory leak. It will be collected when there are no more references to it like any other object. Rob. -Nathan Rob. Alex. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Re: [drlvm] string interning in java
Hi, On 7/29/06, Alex Blewitt [EMAIL PROTECTED] wrote: On 28/07/06, Salikh Zakirov [EMAIL PROTECTED] wrote: Alex Blewitt wrote: Importantly, it explains why the Eclipse NLS class uses static string variables to refer to messages, and not to String literals for exactly this reason. If you use a static string variable that is dynamically initialised from a properties file, you only take the memory up when you need to refer to that message (e.g. in the constructor of an exception in an exceptional condition). Furthermore, because it's a dynamically read in string, it doesn't pollute the intern() pool, and thus when the class is unloaded, the string is unloaded too. Once a String is intern()'d, it's like a memory leak -- you'll never see that memory again. The interned String table should be garbage-collected. If the only reference to an interned String is from the class constant pool it will be collected when the class is unloaded. If the intern table is preventing Strings from being collected it's a bug in the VM! Rob. Alex. - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vmi] JNI spec interpretation?
The word pragmatic springs to mind. FWIW, JamVM will print nothing if no exception is pending. It didn't do this originally -- it blew up with a SEGV. I changed it because a user reported an application which didn't work with JamVM but it did with Suns VM (can't remember which application, it was a long time ago). It's all very well bombing out with an assertion failure, but to the average end-user it's still the VMs fault, especially if it works with other runtimes (i.e. Suns). Rob. On 4/5/06, Archie Cobbs [EMAIL PROTECTED] wrote: Tim Ellison wrote: Understood -- my point is that blowing up and corrupting internal data structures is not something you would do by design. Agreed. By using assertions you get the best of both worlds. Assertions are especially useful for detecting badly behaving JNI native code, which can otherwise result in very hard to track down errors. -Archie __ Archie Cobbs *CTO, Awarix* http://www.awarix.com - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [vmi] JNI spec interpretation?
Hi, One last point. We're talking about a single if statement here, in a call which outputs to the screen. The overhead is miniscule. Rob. On 4/5/06, Robert Lougher [EMAIL PROTECTED] wrote: The word pragmatic springs to mind. FWIW, JamVM will print nothing if no exception is pending. It didn't do this originally -- it blew up with a SEGV. I changed it because a user reported an application which didn't work with JamVM but it did with Suns VM (can't remember which application, it was a long time ago). It's all very well bombing out with an assertion failure, but to the average end-user it's still the VMs fault, especially if it works with other runtimes (i.e. Suns). Rob. On 4/5/06, Archie Cobbs [EMAIL PROTECTED] wrote: Tim Ellison wrote: Understood -- my point is that blowing up and corrupting internal data structures is not something you would do by design. Agreed. By using assertions you get the best of both worlds. Assertions are especially useful for detecting badly behaving JNI native code, which can otherwise result in very hard to track down errors. -Archie __ Archie Cobbs *CTO, Awarix* http://www.awarix.com - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - Terms of use : http://incubator.apache.org/harmony/mailing.html To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
JamVM 1.4.2 released
Hi, I'm pleased to announce the release of JamVM 1.4.2 (http://jamvm.sourceforge.net). This release adds support for class garbage-collection and unloading. A couple of other changes and bug-fixes have also been made. The full list of changes are here: http://sourceforge.net/project/shownotes.php?release_id=387586 Thanks, Rob.
Re: half-baked idea? j2me
Hi, J2ME is split into two things, the VM specification and the profile. MIDP is the profile, not the VM. The VM follows either the CDC specification (Connected Device) or the CLDC specification (Connected Limited Device). Here again, we now have two versions, 1.0 and 1.1 (CLDC1.1 is specified in JSR 139). Most phones were originally CLDC1.0/MIDP1.0. However, newer devices are all MIDP2.0. CLDC 1.0 was integer only, but 1.1 supports floating-point. It also supports weak references -- this is a subset of the J2SE weak reference classes, so the gap is getting smaller. Rob. On 11/1/05, David N. Welton [EMAIL PROTECTED] wrote: Geir Magnusson Jr. wrote: Do you really mean J2ME for a specific purpose? Yes, 'j2me' is vague... Since MIDP is the lowest common denominator, I'd start with that. What we want to do here is a modular system to which you can easily replace parts as appropriate for your usage, either to port to a new platform or usage/performance profile MIDP has a lot of things missing - you need to make sure you don't have any of that deeply mixed up in your core, or else ripping it out gets to be difficult. No floats, no doubles, no reflection, just off the top of my head. -- David N. Welton - http://www.dedasys.com/davidw/ Linux, Open Source Consulting - http://www.dedasys.com/
Re: Classpath .18 release
On 9/7/05, Mark Wielaard [EMAIL PROTECTED] wrote: It should. If not, please file a bug! Most classpath/harmony sister projects will release an updated version soon. For GCC/GCJ the release should be imported asap, the big merge was one of the primary focuses of 0.18. GCC/GCJ should be much more in sync with upstream classpath development from now on. Kaffe CVS is mostly up to date, but not completely merged yet. IKVM just made a first 0.20 RC1 based on 0.18 and will most probably release a new stable in a couple of days. Cacao has been updated in CVS, but is dealing with the import of the new interpreter based on vmgen so might take a while to stabilize. And JikesRVM is shooting for a new 2.4.1 next Monday assuming they get things switched over to classpath 0.18 in the next few days. It's odd you never mentioned JamVM's status in the list. JamVM from CVS works with classpath 0.18 out of the box on Linux now. I'll make a new release once I've fixed a library naming problem on Mac OS X (extension change between 0.17 and 0.18). This should be in the next day or so. Regards, Rob.
Re: [modules] classloader/jit interface
On 6/26/05, Rafal Lewczuk [EMAIL PROTECTED] wrote: Hi, Newbie's thoughts below. On 6/25/05, Ahmed Saad [EMAIL PROTECTED] wrote: 2. which current VM implemention would we start refining as a core for Harmony? (or we would write it from scratch) Newbie's random thought: start with some simple-as-hell implementation (JamVM may be a good candidate) and refactor it into a modular one (kind of 'stretching' it onto a 'framework' set of interfaces, extracting GC, execution engine, class loader etc. one by one). After my initial posting to this list I've gone very much back into lurker mode. Just as I'm not into promoting JamVM, I'm not much into defending it either (though some people may disagree with this). You can take it or leave it. However, I'm probably one of the few people who has written a non-trivial VM from scratch, and when I started I already was an experienced VM engineer. So my thoughts may be useful/interesting or annoying. First of all, just because JamVM is small does not mean it is trivial. As I was interested in targetting embedded platforms, I put in a large amount of design effort _from the start_ to minimise code size and runtime memory usage. As in many other situations, smallness can come from triviality or from careful design. Of course, many parts of JamVM are simplistic, but if code size == quality we'd all be using Microsoft Windows. It is the last trap I would have thought open-source people would fall into. The continued assumption that JamVM is small == simple has begun to affect my coding style, but this is unfair to users on embedded platforms. The interpreter in particular I like to think of as state-of-the-art. It is certainly not trivial, and it is more optimised than most commercial VM interpreters (in many tests it is 2x faster than HotSpots' interpreter under Mac OS X). This in itself has taken many months of work, and I have substantially rewritten it twice, so it is two iterations beyond my first interpreter, which also included several advanced techniques. It now does direct-threading, static and dynamic stack-caching, prefetching and makes use of super-instructions. One of the advantages of being a one man team is that you know the code intimately. While this can lead to spaghetti-like code with many inter-module dependencies if you're not careful, it can also lead to compact code, as you're not afraid to re-factor modules and their interfaces when the time is right. Having modules written by separate teams can result in in-efficiency and code duplication, as each module implements its own utilities, e.g. hash tables, lists, etc., or ends up marshalling arguments for an inappropriate interface. Trying to guess every need up front in a neat module/interface definition is doomed to failure. I believe it is better to start off with a minimal interface, and then re-factor as experience dictates. Of course, there are some very experienced VM implementors on this list, and several module definitions already, but I like to factor through experience not anticipation. For the record, I believe JamVM to be fairly well modularised, each distinct component is in a separate file, with a defined interface. There is very little duplicate code, and no private utility implementations. The biggest problem is that as yet, I have no abstraction for stack-walking. Please note, I'm not putting JamVM up as an example of a module definition. I'm sure there are many, many problems if you were to look at it in detail towards that end. Regards, Rob. Robert Lougher (Dr.) Upsides: - it should be easy for newcomers to get in; - while designing, there is still a working implementation, hopefully passing many of Mauve tests; - having many pieces in place at start; - JVM simplicity causes design work to be actually easier (than a bigger one) by causing refactoring less painful; (albeit harder than designing and implementing from scratch); Downsides: - minimal JVM usually is compact and its compactness causes code to be very interlinked in many places, so module extraction can be sometimes irritating and a bit painful; - dealing with legacy code causes some extra work (and bugs resulting in misunderstanding legacy code, but working thing along with some good set of tests makes it actuallly easier to squash bugs early); - assumptions about object/class/thread/stack/isolate structure layout are almost surely to be changed and may be hardwired in many places in the legacy code; this is the harder part of refactoring work; - minimal JVM design may not have some issues in mind (for example, JIT, class verifier etc.); Some other general comments (some are truisms, I know): - it is important to distinct between innovation/research and good engineering. While designing framework I'd suggest to use proven solutions rather than great innovations; after all, a reliable/production VM has to be released; well engineered framework
Re: [modules] classloader/jit interface
Hi Steve, First of all, I'd like to say I've found the discussions on this list very interesting. I don't want anybody to take my comments the wrong way. I'm not annoyed or offended in the slightest by people saying JamVM is simple or trivial -- I simply wanted to put the record straight. Many parts of JamVM _are_ relatively simplistic, but some parts I like to think are relatively advanced, following my own interests. These include the interpreter and the thin-locks implementation. Writing a VM from scratch requires a wide breadth of knowledge. I totally agree that simplistic implementations do not scale, and retro-fitting advanced techniques can be as much work as implementing it from scratch. But with JamVM I made the choice to implement parts simplisticly to get a complete VM finished and released. As I'm sure every VM implementor has experienced, after release I spent a lot of time fixing bugs and boundary cases, and I didn't make any major modifications for quite a few releases. And then once you have users you're stuck in the bind that you have a relatively stable release, which acts as a dead-hand on making major modifications. I've made major modifications to the interpreter, but only after extensive testing, and with great trepidation. If I have a point, is that testing should be made an integral part of any design decisions. Getting a VM out there early enables users to test it with a wide-range of applications.But simplistic solutions as in JamVM can prove problems in the future, but complex solutions can delay getting a VM out there, for real user testing. (For the record, I actually worked on JamVM for over a year before I made a first release). JikesRVM is a very complex research-oriented (and I'm not saying that in a derogatory way) VM. I'm only giving my opinions on what it's like implementing a VM from scratch, as a one man team. I'm extremely impressed by JikesRVM. I wrote JamVM initially as a learning exercise and (pleasurably) ended up with a user base. I guess JikesRVM was initially a research vehicle, but is now intended as a wider VM. We fit different ends of the spectrum, but I hope we can both co-exist. After a big preamble, I've also given comments below... On 6/26/05, Steve Blackburn [EMAIL PROTECTED] wrote: Hi Rob, We'd obviously be shooting ourselves in the foot if we did not make the most of JamVM... Thanks. I'm pleased that people think JamVM is a worth-while object of study. I believe its main claim to fame is its small size rather than the complexity of any technique (apart from the interpreter). But I couldn't say in more than general terms how it achieves it, apart from saying that I spent many years squeezing complex programs into the ZX Spectrum using Z80 assembler (total of 48K). However, I'm probably one of the few people who has written a non-trivial VM from scratch, and when I started I already was an experienced VM engineer. So my thoughts may be useful/interesting or annoying. Useful. First of all, just because JamVM is small does not mean it is trivial. As I was interested in targetting embedded platforms, I put in a large amount of design effort _from the start_ to minimise code size and runtime memory usage. The goals of harmony are broader, but this is just where we are right now---trying to get rolling on the up-front design effort. Yes, and I think the discussions so far have been very useful. I personally would like to see code sooner rather than later. If not for the simple reason that engineers are happier reading code rather than design documents. Code is unambiguous. As in many other situations, smallness can come from triviality or from careful design. Of course, many parts of JamVM are simplistic, but if code size == quality we'd all be using Microsoft Windows. It is the last trap I would have thought open-source people would fall into. I think often they do, but I sure hope we're not going to go down that path. I really hope so as well. Maybe it's impossible for one VM to span the spectrum from embedded platforms to multi-processor SMP machines, and to scale appropriately (best commercial example is J9?). Pluggable components might achieve this, but pluggability adds an overhead in itself (unless it's compile time). The interpreter in particular I like to think of as state-of-the-art. It is certainly not trivial, and it is more optimised than most commercial VM interpreters (in many tests it is 2x faster than HotSpots' interpreter under Mac OS X). This in itself has taken many months of work, and I have substantially rewritten it twice, so it is two iterations beyond my first interpreter, which also included several advanced techniques. It now does direct-threading, static and dynamic stack-caching, prefetching and makes use of super-instructions. The obvious question is how Harmony can use this experience? Can we take JamVM in its entireity as
Re: Other interesting papers and research
Hi, On 6/6/05, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Hi Dave, From: David P Grove [EMAIL PROTECTED] One thing to note is that a threaded interpreter would see something like a 2-4x expansion over normal bytecodes when it converts from bytecodes to its internal form (arrays of function pointers). Direct threading interpreters like JDK's one work on plain Java bytecode and they do not need to expand normal bytecode instructions. Such expansion may have been required if Java bytecode is not linear and rather a tree or other complicated form. According to my understanding, an indirect threaded interpreter uses the original bytecode stream. It's indirect because the handler address must be looked up via the bytecode. A direct threaded interpreter removes this step by placing the actual handler addresses in the rewritten instruction stream. For what it's worth, JamVM supports both direct and indirect threading, with static and dynamic stack-caching respectively. I seem to recall an average code size increase of ~4x for JamVM's internal instruction format but I'll need to recheck my figures to be sure. Note, this is assuming a 32-bit architecture. Handler addresses will be double on a 64-bit machine, and the code increase over bytecodes therefore larger. Rob.
Re: JIRA and SVN
Hi List, Now that I've been explicitly asked, I'll forego my current observer status and give some input :) First off, I'm not comfortable pushing JamVM, but I'll give a summary for people who are unfamiliar with it. As has already been alluded to, JamVM's main claim is its small size (currently, its stripped executable weighs in at ~135K on PowerPC and 100K on Intel). This makes it suitable for use on platforms ranging from embedded ARM processors, up to desktop machines and SMP systems (it has appropriate memory barriers for PowerPC and Intel), and it could also be used as a bootstrap VM. Its small size is a result of conscious design decisions and effort. It may not be the most sophisticated VM out there, but it has proved to be complete, stable and fast enough for many of the GNU Classpath developers to adopt it as their development VM. The interpreter in particular, is state-of-the-art, supporting direct-threading, stack-caching, prefetching and common super-instructions. See http://www.csc.uvic.ca/~csc586a/slides/StackCaching-4.pdf and http://www.csc.uvic.ca/~csc586a/Ass1.pdf for an advanced academic course on virtual machines using JamVM. JamVM has currently been built and tested on Linux for the ARM, PowerPC and IA32 architectures, and on Mac OS X/Darwin for PowerPC. Porting to a new architecture/platform involves writing a couple of assembler macros for atomic operations (used in the thin-locking implementation) and for memory barriers. A separate function must also be implemented which handles the construction of a native C call stack frame and perform parameter passing for invoking native JNI methods. This could, however, be simplified in the future by providing the option to use libffi instead. JamVM also includes the option to use an internal native interface which is much more efficient than JNI. The current latest version of JamVM is 1.3.0 (see http://jamvm.sf.net). This works out-of-the-box with GNU Classpath-0.15 (the latest snapshot) and CVS head. I'm currently finishing off JamVM 1.3.1, this is a maintenance release which includes a couple of minor bug-fixes and some performance optimisations for object and array allocation. It will also include the patches enabling JamVM to boot the GNU Classpath generics branch (the branch for Java 1.5 features). See http://lists.gnu.org/archive/html/classpath/2005-04/msg5.html. The major problem with JamVM as regards Harmony is that it is currently licensed under the GPL. I originally had some specific reasons for doing this, however, I am open to suggestions about different licensing models, e.g. dual-licensing, adoption of LGPL or a linking exception as in GNU Classpath. Assuming the license issue could be sorted out, and if (and it's a big if) JamVM was adopted as one of the seed VMs my next concern would be what happens next. Would it be a fork? Would I continue to work on JamVM as now? Of course, change is not necessarily a bad thing. Hope this helps, Rob. On 5/21/05, Raffaele Castagno [EMAIL PROTECTED] wrote: 2005/5/21, Davanum Srinivas [EMAIL PROTECTED]: David, please feel free to ping Rob. It would be great! thanks, dims On 5/21/05, David Griffiths [EMAIL PROTECTED] wrote: On 20 May 2005 17:54:11 -0600, Tom Tromey [EMAIL PROTECTED] wrote: This is too vague -- we don't know much about the unexpected. Plus, in most cases, the core part of the VM is simply not very important. There just isn't much code there -- JamVM is 20KLOC, anybody could comfortably rewrite this. Hmmm, well I used to work with the author of JamVM (Rob Lougher) and he's one of the brightest guys I know. I think you'll find that the low LOC figure is testament to his ability to write lean code rather than an indication of how easy it is to knock off a JVM on a wet Sunday afternoon. BTW has anyone asked Rob about donating JamVM to Harmony? As the (currently) sole owner he should have no problem with switching licenses. Cheers, Dave -- Davanum Srinivas - http://webservices.apache.org/~dims/ -- If you want a GMail account, send me an E-Mail.
