[
https://issues.apache.org/jira/browse/STORM-1435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15387476#comment-15387476
]
Jungtaek Lim commented on STORM-1435:
-------------------------------------
Skimmed Spark code a bit.
(I had look at classloader of Spark before because of bugfix, but it's first
time to look at how to resolve maven artifacts.)
Spark seems to resolve maven artifacts via Ivy (Zeppelin uses Aether), but both
Spark and Zeppelin seems to create local repo and download dependencies.
For handling downloaded jars from another nodes, Spark seems to maintain
fileServer. Not sure how fileServer works, but if it works like http server,
MutableURLClassLoader allows to add URL for jars and load classes if needed. If
fileServer runs for every node, loading jars is fairly simple.
I feel we can do the similar thing via blobstore, but it's just a sketched idea.
> Build a single jar with dependency for StormSQL dependency
> ----------------------------------------------------------
>
> Key: STORM-1435
> URL: https://issues.apache.org/jira/browse/STORM-1435
> Project: Apache Storm
> Issue Type: New Feature
> Reporter: Haohui Mai
>
> Currently StormSQL requires all dependency of the topology to reside in
> either the `lib` or the `extlib` directory. It will greatly improve the
> usability if StormSQL can provide a mechanism to pack all dependency with the
> jar compiled from the topology.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
