Github user chinmaykolhatkar commented on a diff in the pull request:
https://github.com/apache/incubator-apex-core/pull/311#discussion_r61535209
--- Diff:
engine/src/main/java/com/datatorrent/stram/plan/logical/LogicalPlan.java ---
@@ -2386,7 +2381,7 @@ public static void write(DAG dag, OutputStream os)
throws IOException
public static LogicalPlan read(InputStream is) throws IOException,
ClassNotFoundException
{
- return (LogicalPlan)new ObjectInputStream(is).readObject();
+ return (LogicalPlan)new
ClassLoaderObjectInputStream(Thread.currentThread().getContextClassLoader(),
is).readObject();
--- End diff --
This is required for Local Mode case.
Lets say a jar path is set to LIBRARY_JARS and this jar contains a POJO
that needs to be set to TUPLE_CLASS attr. Now, one can do that in populateDAG
method, that would cause a problem in LocalMode as follows:
1. populateDAG sets TUPLE_CLASS attr with new POJO class - Works.
2. Local mode serializes the LogicalPlan - Works.
3. Local mode deserialized the LogicalPlan - Fails.
3rd step fails becuase The ObjectInputStream deserializer uses System
ClassLoader which does that have that POJO. ClassLoaderObjectInputStream is an
overridden version of ObjectInputStream which allows to set the custom
classloader.
By this time of callstack, Thread.currentThread().getClassLoader() will
have a classloader which is super of all.
As previously, there was no functionality of LIBRARY_JARS, I guess this was
not tested and so did not surface as an issue.
For cluster mode, there is no difference. i.e. There is only 1 classloader
i.e. System.Classloader and that has new jars as well.
i.e. For cluster mode, Thread.currentThread().getClassLoader() = System
classloader
i.e. ClassLoaderObjectInputStream will behave as if ObjectInputStream.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
