I like the fact that Gregg's pushed the envelope, he's pushed Jini
beyond it's current capability.
Now I'm wondering if there's a way around the Preferred Classes problem.
In Reggie's proxy implementation, during registration, fields in Entry
classes, if they can be resolved to standard java classes, are, however
they are sent as a MarshalledWrapper (wrapping a MarshalledInstance), if
they can't.
For MarshalledServiceItem, we don't have to honour preferred classes,
since we can unmarshall everything correctly when the unmarshall method
is called, in other words we can unmarshall the Entry's again, using
preferred class loading and downloaded codebases.
For implementation of MarshalledServiceItem, the reggie implementation
EntryClass can be first used to identify an entry we're interested in,
first we check if the name matches an Entry's Class name we're
interested in, secondly we resolve all fields of these Entry's locally
by providing a get() method on MarshalledInstance that ignores codebase
annotations, leaving any field that cannot be unresolved as null.
So what we get at the client is a number of Entry's that may be fully
unmarshalled, or partly so, which should give us enough information to
determine if the service is interesting.
When the client finds a field in an Entry in the ServiceItem, set to
null, and it wants to know more, it must then unmarshall the
MarshalledServiceItem, a process which then retrieves all codebases and
correctly unmarshalls all Entry's and the Service, the original array of
Entry's contained in the MarshalledServiceItem is discarded, so the fact
that these were previously resolved locally doesn't matter.
Then we could leave Preferred classes alone.
Thoughts?
Peter.
Gregg Wonderly wrote:
I am finding my self being slightly defensive sounding and I'm not
wanting to be defensive about my implementation. I want to defend the
features that I need and thus made changes in lookup to provide.
Hopefully the inline below is not degenerating into to much
reiteration of the issues. There are some additional thoughts about
other ways that lookup could be changed to provide the "string"
information without a proxy service.
See below...
On 2/2/2011 3:26 AM, Dan Creswell wrote:
Inline and leaving context for now.....
On 1 February 2011 19:12, Gregg Wonderly<ge...@cox.net> wrote:
On 2/1/2011 3:17 AM, Dan Creswell wrote:
So...
On 1 February 2011 01:29, Gregg Wonderly<gr...@wonderly.org> wrote:
One of the important things for my use of my changes to the
ServiceRegistrar interface was separating deserialization of the
Entrys
from
the service object.
Note "separating" used here. I was trying to say that I need to be
able to
"pick" which Entry objects to deserialize, and deserialize them
independent
of deserializing the service proxy/object.
Being able to defer downloading was also controlled by the changes to
ClassLoading to include the "neverPrefer" settings on a class
name. I
can
do that now through the recent RMIClassLoaderSPI override
capabilities.
But
I still need to be able to control which Entry is deserialized. I
need
to
be able to ask what classes, by name, are in the namespace of each
Entry
object as my implementation allowed. I had made changes to Reggie
marshalling to use reflection to get the class hierarchy, and then
provided
access to the list of classnames as part of the returned, marshalled
value
in my API.
Digging through classes belonging to a service implementation
that is
designed to be encapsulated/not your concern? Does that sound right? I
gotta
say, it sounds horribly invasive, very hacky and speaks of an overly
complex
solution as the result of treating symptoms, not problems.
Okay, I want to go through the problems that I was dealing with, the
attributes of the current mechanisms, and how I inserted points of
control
to manage these issues.
First, let's look at the simple client mechanisms I wanted to have.
I have
a desktop environment, which does lookup, and then shows treeviews of
services based on "machine"->"group"->Service Instance structure. The
deployed services contain Entry objects that provide values for
these three
name spaces. Since I need to look at these, they can not be
preferred by
the service or I can't read them. There may be a subclass that the
client
has used, and that can be preferred no problem. There are some
other items
in the Entrys that I need too, such as icons. Any of the Entry
objects that
I need access to, and any classes which they are dependent on, and
which are
publicly visible classes/interfaces, because of that attribute
(public) can
not be realistically preferred.
These Entry types are part of the basic infrastructure of your system
aren't
they? Why download them at all? I guess because you have to "teach"
it to
the LUS via code-download? Or you could just put them on all classpaths
including LUS? S'far as I can see you've got an extended LUS
infrastructure
that everyone shares thus codebase seems like the wrong choice for
these.
Sure user's can subclass Entry types but do they? Do you even want
them to?
Why? Thus far I've seen many subclass Entry's for JavaSpaces and, I
think,
zero for LUS type annotations.
See my other thread with Peter but it seems to me that you have some
core
additions to LUS infrastructure and a need to be able to filter what you
don't want to see/is outside of a client's knowledge base. Because
it's that
inability to filter that then leads to lots of additional downloading.
The basic issue is that I need to show users what services exist. The
LUS maintains a very useful view of the health of the network. Asking
it what services exist has to happen by some 'client'. Remember that
my customers techs are using the client out in the field, on a slow
network. So whether they enumerate something from the LUS or from
another "service", there will be network traffic.
It might be possible to enumerate services at a aggregation service,
but then you have to unmarshall all of the services there to get the
"data" that you'd send to a client. I don't want to have 100s of
proxies active on the network for this mechanism, really. The
deferred unmarshalling in discussion with Peter would help deal with
this by allowing such an aggregation service to not have to unmarshall
the proxy. That was my initial intent. It was only after I got that
implemented that I found out how the PREFERRED.LIST really kept it
from working at all, and so I had to do something to keep the
PREFERRED.LIST from being accessed when not needed yet.
The service aggregation becomes a single point of failure and I'd need
to replicate it to every server that had an LUS on it. Then, it
really becomes more like a secondary look up service, and so I'd still
have to ask, is the LUS API really in the right form if there is a
"mirror" of it needed? Note that I made it possible to pick through a
lot of the class information for the Entry and the service Proxy in
reef's work because I was trying to get around having to mess with the
never-preferred business. But in the end, I just could not find any
other way, because as soon as I wanted to unmarshall an Entry to get a
"Name" and a "Group" and an "Icon" for my UI, PreferredClassLoading
went after the PREFERRED.LIST, even though not a single class that I
was using would be resolved from the codebase.
I'm not sure how I would "not activate the codebase" in some client.
It needs the Entry object, fully resolved. I want the codebase to
work right, so I'm not gonna just throw in a proxy context class
loader that resolves everything locally. If there is content in an
Entry that is an unknown class, I will suffer the download to make
sure that it resolves to the right codebase.
Note that I am not wanting to focus on the never-prefer issue as
something that I need. What I actually need is to defer download of
any part of the codebase until the service is "activated".
What might be interesting is to figure out how to stuff the preferred
list into the MarshalledObject annotation so that PreferredClassLoader
would not need to retrieve the codebase elements to resolve how to
perform class loading.
Currently, there is a lot of backward compatibility involved in how
the annotation is managed.
The namespace change that we are discussing for com.sun.jini, might be
the right moment to introduce any compatibility issues with
MarshalledObject and the use of the RMIClassLoader annotation.
So my first assertion is that Entry objects will hardly ever be
preferred,
and thus downloading of the PREFERRED.LIST is not a necessary step to
correctly resolve them for my use.
The next issues is this PREFERRED.LIST file. The implementation of
that
technology is that the file is retrieved from the first jar file
listed in
the codebase. So, in order to resolve any class in the proxy or Entry
values, there must be a download from the associated codebase server.
Now, one of the first things that I think about with distributed
computing
is the business of partial failure and limiting that impact to the
overall
performance of the system. My deployments at my customers' sites
consist of
copies of the same service platform with the same services on it.
There are
copies for both redundancy and for capacity.
Each instance of my platform has as a starting point 20 some
services each
with a separate codebase, but with many common jars such as
jsk-dk.jar. If
a lookup service responds with a list of services but the codebase
server
doesn't respond with jars, then any attempt to download limits the
performance of the display of available services, and I considered an
"incorrect list of available services" less important than
non-display or
slow-display of available services.
As Michal says elsewhere, are you sure the solution is to try and
leverage
LUS to handle display challenges like this?
I need the data with minimal network traffic. The LUS lookup is even
something that generates some delays. But, the UI, as it populates,
feels largely like a web page load. I've reached a point of
acceptable performance that works for now.
Service lookup and selection has always been a point of contention as
you well know. I had always been on the side of the fence that said
the LUS works fine the way it is. It wasn't until I tried to figure
out how to manage 100s of services visible and only 1 being
interesting that I suddenly found myself setting on the fence asking,
"If it almost does the right thing, is there a way to maintain the
same interface, but add a new interface with more functionality that
would keep me from having to completely ignore the LUS?"
So, I tried something out. I find that given how Reggie works and is
implemented, that the addition of the functionality as shown in Reef
does provide some new opportunities. But, delayed unmarshalling
alone, does not provide for no-download until needed.
PreferredClassLoading is still in the mix. My solution to that, which
I think does need to be revisited. We need to decide if the
PREFERRED.LIST is really a problem because it causes a download.
Service registration data will always require download, and so just to
use the Entry values in a UI or some other client will still require
download of all the elements of the codebase needed to resolve the
classes.
In one explanation of how things are currently working with River,
without my changes, I suggested that using Reggie and preferred class
loading would be like opening up a Google session, running a query,
and then having to download every matched pages content before you
could see the list of search results.
Google provides the type of service that you describe by changing what
the user has to experience to be something that is usable. But we
don't all then go and write a new service on top of google. It does
what is needed to find something to look at. But, it still then has
the struggle that you have to click on a link, and wait for the page
to load in order to validate which content is appropriate to your
needs. The textual summaries don't always provide the right "view" of
what you are looking for, even through they've tried to provide a
textual view of the document fragment that contains what you searched
for.
It is the load-before-need-is-established behavior that I am trying to
raise the red flag about. We need a complete different LUS if there
are more than a handful of services that match a lookup request.
I feel really strongly against forcing clients to resolve 10s of
services much less 100s just to see what they might choose to use.
Entry and service interface based filtering works to trim results
dramatically. But in the case of a ServiceUI discovery where you are
just looking for services that have a
Service registration with a ServiceRegistrar implementation is the "I
am alive" moment, and I think we need to maintain that single point of
information because doing otherwise spreads knowledge out into more
than one place and makes for a distributed database that I don't feel
we don't need to add to the mix.
What you seem to be doing is constructing a navigation structure for
a GUI
direct from the LUS per client. Why wouldn't you have some separate
"service
summary service" that builds up this navigation structure by tracking
service arrivals and such and creating the necessary descriptive text
for
clients to pull down in one big bang?
There is some tangible benefits I can see in doing this, but I don't
like the idea that this is another layer between the client and the
LUS regarding availability and liveness. As I said above, the LUS has
the information, but it's the "unmarshalling" (ServiceRegistrar's
APIs) and PreferredClassLoading that don't allow a client of the LUS
to avoid downloading what they don't need to download. That's why I
went specifically at those two sources.
If I didn't have the sources to mess with, I am sure I would have had
a different approach or done something without Jini even.
You get caching for staters, no need to download a bunch of services
until
later and thus far fewer downloads especially as things like Entry's
can be
rendered into text reps as part of this process of building navigation
information. (heck you could even insist all Entry's provide a
toString for
use on your platform). No reason too why the summary service couldn't
build
on demand trees for some specified set of Entry types. It could cache
those
and update them too such that if another client requests the same
tree it
gets it straight back.
I guess my summary would be, I feel like you've gone for a "deep"
solution
that builds into existing infrastructure instead of building
extensions atop
that specifically tackle your problems in a more "meta" fashion. Please
don't take offence at that, I'm not for a moment saying it's a
mistake, just
one path to solution and given what I've heard so far I think there's
another path that hasn't been ruled out yet.
I'm good with this discussion. That never happened before, even at
the Jini Community meeting and later when I pushed Reef out onto
java.net.
I want to really stress what I feel are the primary issues for me and
my customers. I also feel that given the larger set of services that
might be visible on a broader network environment, such as the
internet, that these issue will be important to others as well. It
sounds like Chris at least did the caching of jars.
Imagine if ServiceRegistrar included an XML or other string document
that could be used by applications to get "client display"
information, instead of just Entry object values, and that
PreferredClassLoader got the PREFERRED.LIST from the annotation String
instead of from a META-INF entry in the jar file.
Then, the ServiceRegistrar could still be a primary source of service
registration and lookup but would include some flexibility for local
needs. We could even cause the XML to be constructed by the
serialization process so that it looked something like:
<entries>
<entry class="net.jini.lookup.entry.Name">
<field name="name" type="java.lang.String" value="My Service"/>
</entry>
</entries>
All the public fields would be enumerated and specified with values
that would allow complete use of them (when they have appropriate
toString() values) and thus we can get the text from the Entry objects
directly.
`
We'd just provide an API that provided the programmatic iteration over
this information without XML being directly visible.
So the existing lookup would be fine, and ServiceItem could have an
additional XML string field to pass to a new API.
Just thinking out loud here...
Gregg
