1. I am aware of both Remote Services and Remote Services Admin
specifications. I would be happy to see how they deal with dynamic code
downloading but unfortunately they simply don't:
"This specification does not mandate the technique used to proxy an
Endpoint as a service in the OSGi
framework." - taken from "OSGi Compendium Release 6 Specification"
See point 3. below for some inherent issues with OSGi that make it
difficult to implement marshalling/unmarshalling as required by River.
2. It has been a couple of years now since I last checked how Paremus
implemented Jini components. At that time the whole dynamic code
downloading has been based on PreferredClassProvider and
"java.rmi.codebase" property which would suggest it did not really solve
issues with code downloading in Jini at all. I would be happy to be
proven wrong.
3. My comment about OSGi being "non-deterministic" in resolving
dependencies means that the same bundle installed in two different
environments is going to be linked with different dependent bundles. It
basically means the same object graph valid in one VM instance cannot be
deserialized in another VM instance unless:
a) all bundles have their dependencies specified in such a way that
exact same dependency graph is going to be present everywhere (meaning
no Package-Imports, no version ranges etc)
b) all JVMs have the same set of bundles installed (ie - there is no
dynamic code downloading at all - the code is preloaded)
c) classes of serialized objects are not loaded by OSGi framework class
loaders but by other non-related class loading framework (which is what
original old OSGi-Jini spec did)
d) classes of all objects in the object graph are from the same bundle
(special case of a) )
In general all efforts I am aware of regarding "remoting" in Java didn't
really even try to solve issues of dynamically downloaded code. They
were based on all parties having the same code installed and/or having a
central authority providing a shared consistent view of all available
software.
I am not saying it is wrong. I am only saying that IMHO it is not
something River should do - it is simply a solved problem and there is
not point of re-doing it (JERI is cool but it is simply
yet-another-RPC-stack)
Regarding JBoss Modules:
I am not really advocating this particular library - it is just that it
is the only (non-OSGi - see above :) ) implementation of
non-hierarchical class loading that I am aware of which is of good
quality and actively maintained. I wouldn't want to reimplement it by
myself. Thought about ClassWorlds but it doesn't seem to be too active
and had some performance problems in the past. JBoss Modules is Apache
licensed so it can be used by River.
Thanks,
Michal
On Wednesday, 16 November 2016, Richard Nicholson
<richard.nichol...@paremus.com <mailto:richard.nichol...@paremus.com>>
wrote:
Agree with Niclas. I don’t understand the resolution comment.
Not only is OSGi <> Jini integration feasible but it is a historical
fact. Paremus did this - and a pretty good job we did - in …
something like 2006 :-/
If the River community decided that there is interest in OSGi - then
I’d suggest reading the Remote Service and Remote Service Admin
specifications and thinking about how Jini concepts might enhance
that world view. There are well over 10 million OSGi enabled IoT
gateways out there!
Sorry - I see no compelling technical or commercial arguments for
the JBoss Module route.
> On 16 Nov 2016, at 08:11, Niclas Hedhman <nic...@hedhman.org
<javascript:;>> wrote:
>
> I am curious, what do you mean by "non-deterministic dependency
resolution"
> ? You can make it as predictable as you wish with attributes and
> directives.
>
> Cheers
> Niclas
>
> On Wed, Nov 16, 2016 at 4:07 PM, Michał Kłeczek
<michal.klec...@xpro.biz <javascript:;>>
> wrote:
>
>> While non-hierarchical class loading is crucial, OSGI with its
>> non-deterministic dependency resolution is very difficult ( if not
>> impossible ) to target.
>> I'm working on JBoss Module based class loading for River which
I'm going
>> to propose as contribution soon.
>>
>> Thanks,
>> Michal
>>
>> On Wednesday, 16 November 2016, Dawid Loubser
<da...@travellinck.com <javascript:;>>
>> wrote:
>>
>>> +1 for OSGi providing the best solution to the class resolution
problem,
>>> though I think some work will have to be done around trust, as
you say.
>>>
>>>
>>> On 16/11/2016 02:23, Peter wrote:
>>>>
>>>> The conventional alternatives will remain; the existing
ClassLoader
>>> isolation and the complexities surrounding multiple copies of
the same or
>>> different versions of the same classes interacting within the
same jvm.
>>> Maven will present a new alternative of maximum sharing, where
different
>>> service principals will share the same identity.
>>>>
>>>> Clearly, the simplest solution is to avoid code download and
only use
>>> reflection proxy's
>>>>
>>>> An inter process call isn't remote, but there is a question of
how a
>>> reflection proxy should behave when a subprocess is terminated.
>>>>
>>>> UndeclaredThrowableException seems appropriate.
>>>>
>>>> It would plug in via the existing ClassLoading RMIClassLoader
provider
>>> mechanism, it would be a client concern, transparent to the
service or
>>> server.
>>>>
>>>> The existing behaviour would remain default.
>>>>
>>>> So there can be multiple class resolution options:
>>>>
>>>> 1. Existing PrefferedClassProvider.
>>>> 2. Maven class resolution, where maximum class sharing
exists. This
>> may
>>> be preferable in situations where there is one domain of trust,
eg within
>>> one corporation or company. Max performance.
>>>> 3. Process Isolation. Interoperation between trusted
entities, where
>>> code version incompatibilities may exist, because of separate
development
>>> teams and administrators. Each domain of trust has it's own
process
>>> domain. Max compatibility, but slower.
>>>> 4. OSGi.
>>>>
>>>> There may be occassions where simpler (because developers
don't need to
>>> understand ClassLoaders), slow, compatible and reliable wins
over fast
>> and
>>> complex or broken.
>>>>
>>>> A subprocess may host numerous proxy's and codebases from one
principal
>>> trust domain (even a later version of River could be
provisioned using
>>> Maven). A subprocess would exist for each trust domain. So if
there are
>>> two companies, code from each remains isolated and communicates
only
>> using
>>> common api. No unintended code versioning conflicts.
>>>>
>>>> This choice would not prevent or exclude other methods of
>> communication,
>>> the service, even if isolated within it's own process will still
>>> communicate remotely over the network using JERI, JSON etc.
This is
>>> orthogonal to and independant of remote communication protocols.
>>>>
>>>> OSGi would of course be an alternative option, if one wished
to execute
>>> incompatible versions of libraries etc within one process, but
different
>>> trust domains will have a shared identity, again this may not
matter
>>> depending on the use case.
>>>>
>>>> Cheers,
>>>>
>>>> Peter.
>>>>
>>>> ESent from my Samsung device.
>>>>
>>>> Include original message
>>>> ---- Original message ----
>>>> From: "Michał Kłeczek (XPro Sp. z o. o.)"
<michal.klec...@xpro.biz <javascript:;>
>>> <javascript:;>>
>>>> Sent: 15/11/2016 10:30:29 pm
>>>> To: dev@river.apache.org <javascript:;> <javascript:;>
>>>> Subject: Re: Maven Build
>>>>
>>>> While I also thought about out-of-process based mechanism for
execution
>>> of dynamically downloaded code, I came to the conclusion that
in the
>>> context of River/Java in-process mechanism is something that
MUST be done
>>> right. All other things can (and should) be built on that.
>>>>
>>>> I think that the proposal to implement "remote calls on smart
proxy
>>> interfaces that aren't remote" is somewhat a misnomer. The call
is either
>>> remote or local - you cannot have both at the same time. AFAIK Jini
>>> community always believed there is no possibility to have
local/remote
>>> transparency. That is why there exists java.rmi.Remote marker
interface
>> in
>>> the first place.
>>>>
>>>> There is also the question about the level of isolation you
want to
>>> achieve. Simple "out-of-process" is not enough, chroot is not
enough,
>>> CGROUPS/containers/jails/zones might be not enough, virtual
machines
>> might
>>> be not enough :) - going the route you propose opens up the
whole world
>> of
>>> new questions to answer. At the same time you loose the most
important
>>> advantages of in-process execution:
>>>> - simplicity of communication between components (basic
function call,
>>> no need to do anything complicated to implement callbacks etc.)
>>>> - performance
>>>>
>>>> In the end you either standardize on the well known set of
>> communication
>>> protocols (such as JERI) OR you say "end of protocols" by allowing
>>> execution of dynamically downloaded code in-process.
>>>> If River is going to choose the first route - IMHO it is going
to fail
>>> since it does not propose anything competitive comparing to current
>>> mainstream HTTP(S)/REST/JSON stack.
>>>>
>>>> Thanks,
>>>> Michal
>>>> Peter November 15, 2016 at 8:28 AM
>>>> I've been thinking about process isolation (instead of using
>>> ClassLoader's for isolation). Typically, smart proxy's are
isolated in
>>> their own ClassLoader, with their own copies of classes,
however with
>>> Maven, a lot more class sharing occurs. Since River uses codebase
>>> annotations for identity, using maven codebase annotations will
result in
>>> proxy's from different services sharing identity.
>>>>
>>>> A better way to provide for different identities coexisting on
the same
>>> node, would be to use subprocess jvm's for each Service's server
>> principal
>>> identity, to keep classes from different services in different
processes.
>>>>
>>>> This way, each principal would have their own process & Maven
namespace
>>> for their proxy's.
>>>>
>>>> Presently JERI only exports interfaces in reflection proxy's that
>>> implement Remote, so I'd need an endpoint that can export all
>> interfaces,
>>> accross a local interprocess connection to allow remote calls
on smart
>>> proxy interfaces that aren't remote.
>>>>
>>>> This also means that memory resource consumption of smart
proxy's can
>> be
>>> controlled by the client and a smart proxy's process killed without
>> killing
>>> the client jvm.
>>>>
>>>> Cheers,
>>>>
>>>> Peter.
>>>>
>>>>
>>>>
>>>> Dawid Loubser November 15, 2016 at 8:50 AM
>>>> As a very heavy Maven user, I wanted to say that this is great
news.
>>>> This is encouraging indeed!
>>>>
>>>> Dawid
>>>>
>>>>
>>>> Peter November 15, 2016 at 4:08 AM
>>>> Some other news that might encourage participation, I've been
working
>>> on Dennis Reedy's script to modularise the codebase, I haven't
run the
>> test
>>> suites against it and it isn't generating stubs yet, and I'll
need to
>>> modify the platform modules for the IoT effort after the
conversion is
>>> complete.
>>>>
>>>> Here's the output of the River maven build:
>>>>
>>>> Reactor Summary:
>>>>
>>>> River-Internet Project ............................ SUCCESS
[0.689s]
>>>> Module :: River Policy ............................ SUCCESS
[8.395s]
>>>> Module :: River Resources ......................... SUCCESS
[0.607s]
>>>> Module :: River Platform .......................... SUCCESS
[23.521s]
>>>> Module :: River Service DL Library ................ SUCCESS
[8.999s]
>>>> Module :: River Service Library ................... SUCCESS
[8.014s]
>>>> Module :: River Service Starter ................... SUCCESS
[3.930s]
>>>> Module :: River SharedGroup Destroy ............... SUCCESS
[3.018s]
>>>> Module :: Outrigger .............................. SUCCESS
[0.056s]
>>>> Module :: Outrigger Service Download classes ...... SUCCESS
[2.416s]
>>>> Module :: Outrigger Service Implementation ........ SUCCESS
[4.118s]
>>>> Module :: Outrigger Snaplogstore ................. SUCCESS
[3.273s]
>>>> Module :: Lookup Service ......................... SUCCESS
[0.048s]
>>>> Module :: Reggie Service Download classes ........ SUCCESS
[3.966s]
>>>> Module :: Reggie Service Implementation .......... SUCCESS
[3.621s]
>>>> Module :: Mahalo ................................. SUCCESS
[0.436s]
>>>> Module :: Mahalo Service Download classes ......... SUCCESS
[2.059s]
>>>> Module :: Mahalo Service Implementation ........... SUCCESS
[4.175s]
>>>> Module :: Mercury the Event Mailbox ............... SUCCESS
[0.497s]
>>>> Module :: Mercury Service Download classes ........ SUCCESS
[3622s]
>>>> Module :: Mercury Service Implementation .......... SUCCESS
[3.562s]
>>>> Module :: Norm .................................... SUCCESS
[0.013s]
>>>> Module :: Norm Service Download classes ........... SUCCESS
[2.867s]
>>>> Module :: Norm Service Implementation ............. SUCCESS
[6.390s]
>>>> Module :: Group ................................... SUCCESS
[0.025s]
>>>> Module :: Mahalo Service Download classes ......... SUCCESS
[2.877s]
>>>> Module :: Group Service Implementation ............ SUCCESS
[2.037s]
>>>> Module :: Tools ................................... SUCCESS
[0.485s]
>>>> Module :: Check ConfigurationFile ................. SUCCESS
[2.720s]
>>>> Module :: Check serialversionUid .................. SUCCESS
[2.129s]
>>>> Module :: ClassDep ................................ SUCCESS
[4.157s]
>>>> Module :: Class Server ............................. SUCCESS
[3.353s]
>>>> Module :: Compute message digest .................. SUCCESS
[1.734s]
>>>> Module :: Compute httpmd codebase ................. SUCCESS
[2.102s]
>>>> Module :: Environment Check ...................... SUCCESS
[2.837s]
>>>> Module :: Jar wrapper ............................ SUCCESS
[2.179s]
>>>> Module :: Preferred classes list generator ........ SUCCESS
[2.495s]
>>>> Module :: Phoenix Activation ..................... SUCCESS
[0.029s]
>>>> Module :: Phoenix Download ....................... SUCCESS
[2.685s]
>>>> Module :: Phoenix ................................ SUCCESS
[4.095s]
>>>> Module :: Phoenix Group ........................... SUCCESS
[2.445s]
>>>> Module :: Phoenix Init ............................ SUCCESS
[1.740s]
>>>> River Distribution ................................ SUCCESS
[10.523s]
>>>> ------------------------------------------------------------
>> ------------
>>>> BUILD SUCCESS
>>>> ------------------------------------------------------------
>> ------------
>>>> Total time: 2:29.804s
>>>> Finished at: Mon Nov 14 22:22:31 EST 2016
>>>> Final Memory: 145M/247M
>>>> ------------------------------------------------------------
>> ------------
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>
>
>
>
> --
> Niclas Hedhman, Software Developer
> http://zest.apache.org - New Energy for Java
