[
https://issues.apache.org/jira/browse/HADOOP-13070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15323708#comment-15323708
]
Sangjin Lee commented on HADOOP-13070:
--------------------------------------
Thanks for the comments [~busbey] and [[email protected]].
{quote}
Ugh. this gives me all kinds of bad feels, though I think I might agree. I know
Steve Loughran has strong feelings on the maintenance burden of this kind of
custom classloader work, so let's ping him early.
{quote}
I completely agree that this is bit of a bitter pill to swallow. But it is also
rather clear to me that something like that is needed to pull off the stricter
isolation (not letting user see parent classes). I worked out a quick working
prototype of the idea recently, and I'll share it soon.
{quote}
If we go down this path, how concerned are we going to be with maintaining
cross-JVM compatibility (vs falling back to some kind of "no isolation"
approach)?
{quote}
That is definitely a concern. This is basically latching onto what
{{ClassLoader.getCallerClassLoader()}} does. It relies on the shape of the call
stack frame to determine what the (non-JDK) "calling" class is. It binds the
JDK implementations to keep that shape uniform across different classloading
patterns. But it is the case that if the JDK ever changes that frame depth we
would likely have to update ours to match it.
{quote}
custom classloaders tend to cause problems downstream, both maintenance and
use. That doesn't mean they don't solve some problems: it's just they are dogs
to work with.
{quote}
Having dabbled a fair amount in classloaders, I am fully aware of the pain that
can happen for implementors and users of custom classloaders. Again, if JDK's
classloading wasn't so leaky to begin with (via things like TCCL or else), it
could have been simpler. The problem with custom classloaders is that if it
breaks it breaks in a pretty painful way and it is hard to work around it.
That said, the current application classloader in hadoop works quite well in
large part with relatively few issues (for disclosure our company pretty much
enables it by default), and whenever an issue arose I was able to fix it in a
fairly straightforward manner. So I would consider the current "working" level
reasonably high.
> classloading isolation improvements for cleaner and stricter dependencies
> -------------------------------------------------------------------------
>
> Key: HADOOP-13070
> URL: https://issues.apache.org/jira/browse/HADOOP-13070
> Project: Hadoop Common
> Issue Type: Improvement
> Components: util
> Reporter: Sangjin Lee
> Assignee: Sangjin Lee
> Priority: Critical
> Attachments: classloading-improvements-ideas-v.3.pdf,
> classloading-improvements-ideas.pdf, classloading-improvements-ideas.v.2.pdf
>
>
> Related to HADOOP-11656, we would like to make a number of improvements in
> terms of classloading isolation so that user-code can run safely without
> worrying about dependency collisions with the Hadoop dependencies.
> By the same token, it should raised the quality of the user code and its
> specified classpath so that users get clear signals if they specify incorrect
> classpaths.
> This will contain a proposal that will include several improvements some of
> which may not be backward compatible. As such, it should be targeted to the
> next major revision of Hadoop.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
