Hi Bill,
With the risk this gets too much a discussion Erlang versus Scala
versus Java, nevertheless I think it ain't that harmful to have these
discussions here given the fact that distribution and parallelization
are subjects that go hand in hand with Jini.
Bill Venners wrote:
Hi Dan,
On Oct 1, 2007, at 12:26 PM, Dan Creswell wrote:
Hi Bill,
I think your idea is an interesting one and certainly and I think the
lessons learnt from Jini would be valuable in this area though I'm not
sure that Jini in its current incarnation is the correct base to start
from. That of course might provide us with a target to evolve
towards.....or perhaps we start a side-project for this work, who knows?
I spoke with Allen Holub a couple hours ago about actors, and one of the
things I learned was that actors don't really have a parallel processing
story. Actors have little message queues, and they sit there processing
and sending messages to other actors, so if you were to, say, model
compute workers as actors, you'd still have the problems of dividing the
work up and figuring out how to send jobs as messages to the worker
actors, deal with failure, etc.
From the above and my reading of actors in Scala it seems a messaging
library can be utilized in Java for this purpose and just using queues
and immutable objects ain't that different, although the Scala syntax
probably providers a smoother integration. But if Java gets closures it
will be possible to make a lot of code look more beautiful, the huge
amount of boilerplate code in my code e.g. to restore the security
context, perform privileged operations, introduce final variables to
access variables outside anonymous classes is indeed appalling at times.
One thing that I consider a drawback in Scala compared to Erlang e.g. is
that actors are not cheap in Scala either as they are implemented on top
of a thread pool. In the Armstrong thesis I referred to earlier it seems
the ATM switch was modeled in a way that each connection (of which can
be over a 100000) had a large number of processes associated with it
(they even ran into the limitations that earlier versions of Erlang/OTP
didn't handle over a million processes) while that would not be possible
with Scala or Java for that matter.
To speed up unit testing, my idea has been that I keep a SuiteRunner
server VM running at all time while developing, and when I run tests
what I actually do is feed jobs to that server. This way I could avoid
starting up a new JVM each time I want to run a test. So what seems to
make sense is to embed a JavaSpace inside that SuiteRunner server.
Assuming you keep the SuiteRunner JVM running and submit suites through
a JavaSpace you will likely run into what I think a lot of people also
consider the hard parts of working with Jini/RMI (and often overlooked)
namely that you must make sure that you will be able to provide as a
codebase annotation all classes your test suites will depend on.
Assuming people are able to do that you will likely run into the fact
the JVM that is about to (re)execute the testsuites has those JAR files
cached (you can also prevent from using JAR files though).
There are ways to solve some of these problems by making sure you end up
with a different codebase annotation each time or configuring e.g. a
smart jar: protocol handler (you can find one in the Seven source code)
for your JVM that doesn't caches JAR files but is able to verify
whether a newer JAR file is available and will use the newer JAR files
for new class loaders created.
I do wonder whether in general people have unit tests that really take
a long time to run, I have only a few that take a long time to complete
and these are for testing concurrency utilities such a thread pools,
blocking queues, etc. Parallelism helps a lot here because the tests
themselves are not consuming any significant CPU most of the time, but
there is no need to distribute these, local parallelization is good enough.
BTW I'm still using your modified SuiteRunner code to make it use
PreferredClassProvider/PreferredClassLoader, control the codebase
annotation for serving JAR files of mobile code from an integrated HTTP
server and have written a Ant task for it, as well as a JUnit XML
reporter so you can use the Ant junitreport. I needed those changes to
have it act as a proper Jini client that was able to serve mobile code
to the services I test through SuiteRunner. Everything is available
through the Cheiron project under the ALv2, if you have any questions
you know where to find me for I think I've solved some of the problems
you might run into.
--
Mark
