Hi Gus:
Sorry for the delayed reply - I’m on vacation and my internet access is a
little spotty.
PreferredClassLoader is important in the long run, but you probably won’t
notice the lack of it until you get to some more complicated service
scenarios. In the short term, you could probably do without, however It
might be a little complicated to get rid of.
Unfortunately, to explain further, we need some more background info.
Bear with me for a few minutes, we’ll come back to the problem, I promise….
Jini is a “mobile-code” style of network architecture, although that may
not be entirely obvious. When you pass arguments to a service call, or
receive return values, what’s actually being passed is a serialized, or
“marshalled” version of the object as it exists inside the virtual machine
(VM). This includes “looking up a service” - what you’re receiving is a
marshalled version of the proxy to the that service.
Serialization is typically performed by ‘net.jini.io.MarshallInputStream’
and ‘net.jini.io.MarshallOutputStream’. These classes extend
ObjectInputStream and ObjectOutputStream, respectively. The marshalled
form includes a copy of all the data inside the instance, plus the name of
the class, and a “codebase annotation” that provides a url that points to
where the class can be downloaded from (hence mobile code).
When a marshalled object is loaded into a local virtual machine, the
MarshallInputStream needs to create an instance of the object’s class, and
then put the instance data into it. It will create a new class loader that
loads bytecode from the “codebase annotation url”, and then use that class
loader to create the local instance of the marshalled object.
Keep in mind that “Foo” loaded by class loader CL-B can implement an
interface “Bar”, or extend a class “Bas” that is loaded by class loader
CL-A. So it’s entirely possible that the user of “Foo” has no knowledge
of, or access to the actual implementation class (e.g. by declaring a
reference to “Bar” or “Bas” and then storing a reference to “Foo” in it).
Also, this applies to fields of an instance as well.
But what happens if the same class name exists in the local VM’s class
path? In most cases, we’d rather use the locally-defined byte code.
Partly for efficiency reasons (avoid downloading the jar file) but also
because if we want to “use” the class locally, it needs to come from the
same class loader (in other words, “Foo” loaded by class loader CL-A is
actually a different class than “Foo” loaded by class loader CL-B, even if
the actual byte code is the same). So the default behaviour is to use a
locally-available class if there is one, and only use a different class
loader if the required class is not available locally.
If we load a marshalled object, and we already have the same class
available locally, what happens to the codebase annotation that was on the
marshalled object? It’s gone, since the codebase annotation goes with the
codebase, which clearly is associated with the class loader. Usually
that’s OK, but what if the service really wants to use the code that it’s
providing remotely? That’s where PreferredClassLoader comes in.
PreferredClassLoader takes a look at a file in the first “jar” file in the
codebase annotation (META-INF/Prefered.list), and treats it as a list of
“Preferred” classes. These are classes where the downloaded code is
“Preferred” to the locally-available class. That way the codebase
annotation sticks, not to mention that the service provides the byte code
that it’s expecting.
That’s not a really common use case, so as I say, you might not notice the
lack of PreferredClassLoader. It’s plugged in using an SPI provider
(META-INF.services/java.rmi.server.RMIClassLoaderSpi) file that’s in
jsk-resources.jar. So to remove PreferredClassLoader properly, you need to
put a different version of jsk-resources.jar into your class path.
There’s another possible problem, although I can’t prove it - just a bad
feeling that I get. Having read through all the above, you might notice
that there’s a lot of class loader hocus-pocus going on. Remote code, and
by extension, Jini, is pretty fussy about class loading, and class loader
hierarchy. In addition to on-demand loading (mobile code), the library
uses things like the Preferred list, and Spi configuration files, that are
also loaded through the class loaders. Although there’s nothing illegal,
it’s an advanced use of the class loading mechanism, so it wouldn’t shock
me if the class loader used in your “one-jar” implementation isn’t complete
enough to support it. I’ve come across other class loader implementations
(e.g. Apache Virtual File System) that didn’t work properly, and you
mentioned that your ‘one-jar’ loader has a known bug around loading classes
by name (which, if you think about it, is what needs to happen for just
about all the remote class loading). Also, you need to be able to set the
codebase annotation on a given class loader.
A “one-jar” library is really a custom class loader that gets it’s byte
code from files that are inside the jar file. So, it’s plausible that any
given “one-jar” approach could cause problems.
That’s why a container approach is usually better. There’s a rudimentary
container provided with River, ‘com.sun.jini.start.ServiceStarter' which is
unfortunately not used in the “getting started” doc (not sure who wrote it,
but I’m afraid that I personally don’t have time to re-write it just now).
You can see an example of its use in the service registrar startup that’s
mentioned in that doc. More advanced containers are available in the
River-Container project (https://github.com/trasukg/river-container) or
the Rio project (http://rio-project.org).
Essentially, Jini needs class loaders that are designed for Jini.
(Really, it’s not Jini - mobile code needs mobile-code-aware class
loaders). My experience has been that it’s possible to put simple service
consumers into non-Jini-aware class loaders (e.g. web application
containers like Tomcat, or command-line class paths) with a few
limitations. Hosting service implementations without ServiceStarter,
River-Container, Rio, or some other Jini-aware environment will be a
complete exercise in frustration.
So, have a look at the way the JavaSpaces implementation gets started up
(it’s the closes to a plain service implementation) and copy that. Or have
a look at River-Container.
And I suppose we ought to renovate the “getting started” page. You might
also want to have a look at the “hello” example, as mentioned in
http://river.apache.org/doc/info-index.html#example.
Cheers,
Greg.
On Jul 7, 2014, at 8:19 PM, Gus Heck <[email protected]> wrote:
Sorry, sometimes I have a habit of over explaining and obscuring the
important question... How critical is PreferredClassProvider? Is it
really
needed? Can I do without it?
On Mon, Jul 7, 2014 at 10:23 AM, Gus Heck <[email protected]> wrote:
Yes the build tool is gradle. It pulls resources from maven central just
like maven. It's worth checking out if you haven't. It uses the cool dep
resolution stuff from maven but ditches the pom straight-jacket.
I found the class, in the jar, overlooked it the first time. I searched
on
loader when I opened the jar with emacs and didn't notice that the
package
I found was com/sun/jini not net/jini (oops!). So that's resolved, but
this
only makes things muddier. The jar is being packed with one-jar (which
is a
tool that allows you to produce a jar containing both your classes and
their dependencies in the same jar). Clearly many other classes are
loading
and so I turned on one-jar's logging and got this:
guss-mbp:ingest gus$ j7a -Done-jar.info=true -jar
build/libs/ingest-node.jar foo bar 2>&1 | grep Preferred
[JarClassLoader] WARN:
com/sun/jini/loader/pref/internal/PreferredResources.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassLoader$1.class
in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassLoader$2.class
in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassLoader$3.class
in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassLoader$4.class
in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN: net/jini/loader/pref/PreferredClassLoader.class
in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassProvider$1.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassProvider$2.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassProvider$3.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassProvider$4.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassProvider$5.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassProvider$LoaderEntry.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassProvider$LoaderKey.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredClassProvider.class
in lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
[JarClassLoader] WARN:
net/jini/loader/pref/PreferredFactoryClassLoader.class in
lib/jsk-platform-2.2.2.jar is hidden by lib/jini-ext-2.1.jar (with
different bytecode)
java.lang.NoClassDefFoundError:
net.jini.loader.pref.PreferredClassProvider
So I took the jini-ext jar out, and all I get is
uss-mbp:ingest gus$ j7a -Done-jar.info=true -jar
build/libs/ingest-node.jar foo bar 2>&1 | grep Preferred
java.lang.NoClassDefFoundError:
net.jini.loader.pref.PreferredClassProvider
So it's there, it's found but something odd is going on after that that
results in NoClassDefFoundError. This may relate to the JarClassLoader
that
gets installed, but clearly that works most of the time...
Are you attempting to manually load this class by looking up the .class
as
file? (one-jar has a known bug along these lines. It loads classes just
fine from the included jars, and files from your project just fine. I
think
it loads files from dep jars just fine too (otherwise lots of stuff
would
have issues) but it seems to confused if you try to load classes as if
they
are files: https://sourceforge.net/p/one-jar/bugs/73/)
How critical is PreferredClassProvider to river's operation?
-Gus
On Mon, Jul 7, 2014 at 9:39 AM, Greg Trasuk <[email protected]>
wrote:
PreferredClassProvider should be in jsk-platform.jar (which I think you
confirmed with your ‘grep -r’). It is there in the 2.2.2 build that I
have. I’m not familiar with your build tool (Gradle?) but you might
want
to check the class path entry in the jar manifest that’s generated, to
see
if jsk-platform is actually in the runtime class path.
Are you starting up the system using the ServiceStarter approach or
winging it yourself? If ServiceStarter, could you post the starter
configuration file?
Cheers,
Greg Trasuk.
On Jul 6, 2014, at 8:27 PM, Gus Heck <[email protected]> wrote:
I've now spent several hours hunting for this class. What jar contains
it?
Presently I'm building with the following (shotgun, messy non
minimized)
dependencies:
dependencies {
compile 'com.google.guava:guava:15.0'
compile 'com.google.code.findbugs:jsr305:2.0.3'
compile 'jini:jini-core:2.1'
compile 'org.apache.river:reggie:2.2.2'
compile 'net.jini:jini-ext:2.1'
compile 'net.jini:jsk-lib:2.2.2'
compile 'net.jini:jsk-dl:2.2.2'
compile 'net.jini:jsk-platform:2.2.2'
compile 'net.jini:jsk-resources:2.2.2'
compile 'net.jini:jini-core:2.1'
compile 'net.jini:jini-ext:2.1'
compile 'net.jini:jsk-policy:2.2.2'
// compile 'net.jini:jini-starterkit:2.1-beta2' // 404 in maven
central
compile 'org.apache.logging.log4j:log4j-core:2.0-rc2'
testCompile 'junit:junit:4.11'
}
I basically got frustrated and just threw everything I could find at
it
and
it still does this:
guss-mbp:ingest gus$ alias j7a
alias
j7a='/Library/Java/JavaVirtualMachines/jdk1.7.0_60.jdk/Contents/Home/jre/bin/java
-Djava.security.policy=/Users/gus/tools/jpolicy/all.policy
-Djava.rmi.server.RMIClassLoaderSpi=net.jini.loader.pref.PreferredClassProvider'
guss-mbp:ingest gus$ j7a -Done-jar.silent=true -jar
build/libs/ingest-node.jar foo bar
Starting injester node...
Jul 06, 2014 7:57:31 PM
net.jini.discovery.LookupDiscovery$UnicastDiscoveryTask run
INFO: exception occurred during unicast discovery to
guss-mbp.lan:54290
with constraints InvocationConstraints[reqs: {}, prefs: {}]
java.lang.NoClassDefFoundError:
net.jini.loader.pref.PreferredClassProvider
at
java.rmi.server.RMIClassLoader.initializeProvider(RMIClassLoader.java:687)
at java.rmi.server.RMIClassLoader.access$000(RMIClassLoader.java:110)
at java.rmi.server.RMIClassLoader$1.run(RMIClassLoader.java:120)
at java.rmi.server.RMIClassLoader$1.run(RMIClassLoader.java:119)
at java.security.AccessController.doPrivileged(Native Method)
at java.rmi.server.RMIClassLoader.<clinit>(RMIClassLoader.java:117)
at net.jini.loader.ClassLoading.loadClass(ClassLoading.java:138)
at
net.jini.io.MarshalInputStream.resolveClass(MarshalInputStream.java:296)
at
java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1612)
at
java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1517)
at
java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1771)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1350)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at net.jini.io.MarshalledInstance.get(MarshalledInstance.java:358)
at
com.sun.jini.discovery.DiscoveryV1.doUnicastDiscovery(DiscoveryV1.java:397)
at
net.jini.discovery.LookupDiscovery$13.run(LookupDiscovery.java:3327)
at java.security.AccessController.doPrivileged(Native Method)
at
net.jini.discovery.LookupDiscovery.doUnicastDiscovery(LookupDiscovery.java:3324)
at
net.jini.discovery.LookupDiscovery.doUnicastDiscovery(LookupDiscovery.java:3355)
at
net.jini.discovery.LookupDiscovery.access$3900(LookupDiscovery.java:696)
at
net.jini.discovery.LookupDiscovery$UnicastDiscoveryTask.run(LookupDiscovery.java:1744)
at
com.sun.jini.thread.TaskManager$TaskThread.run(TaskManager.java:331)
No Service Registries found
I'm passing in the option for this class on the advice of your getting
started page <
http://river.apache.org/user-guide-basic-river-services.html>.
It would be really nice if that page listed what dependency I needed
for
this.
I tried grepping the entire distribution and only found it in two
places
guss-mbp:apache-river-2.2.2 gus$ grep -r PreferredClassProvider *
<*snip* javadoc matches>
Binary file lib/jini-ext.jar matches
Binary file lib/jsk-platform.jar matches
in both jars I see only:
drwxr-xr-x 0 10-Nov-2013 22:49:20 com/sun/jini/loader/
drwxr-xr-x 0 10-Nov-2013 22:49:20 com/sun/jini/loader/pref/
drwxr-xr-x 0 10-Nov-2013 22:49:20
com/sun/jini/loader/pref/internal/
-rw-r--r-- 7218 10-Nov-2013 22:49:20
com/sun/jini/loader/pref/internal/PreferredResources.class
But in any case as you can see both of these libs are already in my
dependency list.
This puzzles me since it's clearly in your repository...
http://svn.apache.org/viewvc/river/jtsk/trunk/src/net/jini/loader/pref/
shouldn't it be in at least one of the distributed jars, if not one of
the
packages on mavencentral?
-Gus
--
http://www.the111shift.com
--
http://www.the111shift.com
--
http://www.the111shift.com
