Continuing this thread to elaborate on what I meant:
When an application developer writes Apex application, and they use binary
incompatible version of jar (using extreme case to simplify elaboration)
than that Hadoop depends on, depending upon how the class path is defined
- JVM will either load the version that application wants or the one that
Hadoop wants and one of the two entities are destined to malfunction. I
faced this situation with an early Apex customer incidentally with the
same jar under discussion - Guava and had no way out. Incidentally
Cloudera¹s revision to their distro included the version of Guava we
wanted and conflict went away.
Another flavor is (which we have fixed both in cli and gateway now) is
when cli is used to launch application1 which has a class a.b.c.classname
and later without terminating the cli is used to launch application2
which requires a different class which incidentally uses the same package
and classname - application2 is launched with the class which was meant
for application1. Although this is more likely to happen with dependency
version changes, the first time I encountered - I was making changes to
the code, recompiling, and creating a new jar.
Servlet Containers apparently solve this problem because they have to load
multiple applications within their own dependency (possibly conflicting)
in the same JVM space. They do some black magic (it seems trivial though)
with ClassLoaders. With Apex we do not do any of this magic (due to tight
coupling among Hadoop/Apex/Application) so the entire stack is vulnerable
to class leaks.
It can be solved in theory and has been proven by Servlet Containers but I
hear one more person saying (Ted - correct me if I mis-interpret) that
it¹s a steep effort.
‹
Chetan
On 11/25/15, 12:31 PM, "Ted Dunning" <[email protected]> wrote:
>On Thu, Nov 26, 2015 at 4:15 AM, Timothy Farkas <[email protected]>
>wrote:
>
>> I thought frameworks such as OSGi are used to isolate classes.
>>Application
>> servers like GlassFish use that technique to isolate application
>> dependencies from platform dependencies. Cask has also implemented a
>> similar technique for their platform
>>
>
>I have tried to use OSGI in the past. It was mind bogglingly difficult to
>get right. My sense is that OSGI has decreased in popularity quite a lot
>in
>recent years. Glassfish is hardly gaining adherents at this point.
