[
https://issues.apache.org/jira/browse/OOZIE-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16261523#comment-16261523
]
Peter Cseh commented on OOZIE-2714:
-----------------------------------
I think this would be a great tool to debug and fix classpath issues of various
kinds. However, I don't think the default behavior should be to throw an error
when a class appears twice on the classpath. Even if there is no performance
hit, we should have this turned off (at least for the moment)
I think it would be better to:
# instead of hardcoding the launcher classname, make it a property.
(oozie.launcher.class.name or similar?)
# Instead of throwing an error at the first conflict, have an option to write
out all of them.
# create some kind of documentation so users can find and use this
> Detect conflicting resources during class loading
> -------------------------------------------------
>
> Key: OOZIE-2714
> URL: https://issues.apache.org/jira/browse/OOZIE-2714
> Project: Oozie
> Issue Type: New Feature
> Components: core
> Reporter: Peter Bacsko
> Assignee: Peter Bacsko
> Attachments: ClassLoaderTest.java, OOZIE-2714-POC01.patch,
> OOZIE-2714-POC02.patch
>
>
> There are a bunch of issues in Oozie which are related to class loading.
> The main problem is that the classpath is constructed in a way which is very
> specific to Oozie:
> - Hadoop lib jars
> - Sharelib jars
> - User-defined jars
> Sometimes there is a conflict between sharelib and hadoop lib version. Also,
> users can add their own jars which sometimes contain a different version of
> popular libraries such as Guava, Apache commons, etc.
> We should be able to detect these conflicts and print exact error message so
> that Oozie users can take appropriate actions to resolve the problem.
> A possible approach is the following:
> * start the execution of an action on a different thread
> * replace the thread's context classloader with a classloader which can
> detect conflicts
> * when the JVM invokes the {{loadClass()}} method of the classloader, it
> scans through the jars (which are available as {{URLClassPath}} objects). If
> it finds the given resource in at least two jars, it can do different things
> depending on the setup:
> ** throws an error immediately, mentioning the conflicting jars (this is
> probably too strict - but still an option)
> ** loads the two resource into a byte array and compares them - it only
> throws an error if there is difference
> ** compares the jars but only emits an error message if there is a conflict
> ** something else (user defined action?)
> Implementing such a classloader is not difficult and would greatly enhance
> the supportability of Oozie. It could work in multiple modes depending on the
> setup - perhaps being able to control it from a workflow config is desirable.
> If there's any problem, we should be able to turn it off completely, too.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
