[
https://issues.apache.org/jira/browse/STORM-2441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jungtaek Lim updated STORM-2441:
--------------------------------
Description:
There were some interests for having client side artifacts or modularize Storm.
- STORM-30
- STORM-117
I'd like to give credit to [~revans2] since idea of this issue is based on the
discussion with Bobby, especially below link:
http://mail-archives.apache.org/mod_mbox/storm-dev/201703.mbox/%3C986037879.3649881.1490623807299%40mail.yahoo.com%3E
This issue will only focus to extract client from storm-core, not intended to
address hiding local mode or changing base classpath for worker. So we may need
to file new issues for them if it's great to address.
We have been having one huge artifact 'storm-core' for supporting all the
things, and recently we extracted 'storm-drpc-server' due to dependency issue.
We also have been struggling to handle shade & relocation for lots artifacts,
but in fact the artifacts which we need to relocate are just Worker related
thing. So extracting worker and related classes into separate artifact and
assigning different classpath in Storm cluster should help remedying.
We may still need to struggle with relocation, but the count of target
artifacts should be much smaller.
Another thing we need to consider is LocalCluster, which pulls most of daemons
but user topology still need to pull it. So we need to have separate artifact
for LocalCluster (maybe also daemons without webapp).
Last, we need to consider about UI and Logviewer porting. Artifact
'storm-drpc-server' is not generic and I think we can group UI, Logviewer, DRPC
(only http server) into this artifact. Artifact name should be changed then.
was:
There were some interests for having client side artifacts or modularize Storm.
- STORM-30
- STORM-117
We have been having one huge artifact 'storm-core' for supporting all the
things, and recently we extracted 'storm-drpc-server' due to dependency issue.
We also have been struggling to handle shade & relocation for lots artifacts,
but in fact the artifacts which we need to relocate are just Worker related
thing. So extracting worker and related classes into separate artifact and
assigning different classpath in Storm cluster should help remedying.
We may still need to struggle with relocation, but the count of target
artifacts should be much smaller.
Another thing we need to consider is LocalCluster, which pulls most of daemons
but user topology still need to pull it. So we need to have separate artifact
for LocalCluster (maybe also daemons without webapp).
Last, we need to consider about UI and Logviewer porting. Artifact
'storm-drpc-server' is not generic and I think we can group UI, Logviewer, DRPC
(only http server) into this artifact. Artifact name should be changed then.
> Break down 'storm-core' to extract client (worker) artifacts
> ------------------------------------------------------------
>
> Key: STORM-2441
> URL: https://issues.apache.org/jira/browse/STORM-2441
> Project: Apache Storm
> Issue Type: Improvement
> Components: storm-core
> Reporter: Jungtaek Lim
> Assignee: Jungtaek Lim
>
> There were some interests for having client side artifacts or modularize
> Storm.
> - STORM-30
> - STORM-117
> I'd like to give credit to [~revans2] since idea of this issue is based on
> the discussion with Bobby, especially below link:
> http://mail-archives.apache.org/mod_mbox/storm-dev/201703.mbox/%3C986037879.3649881.1490623807299%40mail.yahoo.com%3E
> This issue will only focus to extract client from storm-core, not intended to
> address hiding local mode or changing base classpath for worker. So we may
> need to file new issues for them if it's great to address.
> We have been having one huge artifact 'storm-core' for supporting all the
> things, and recently we extracted 'storm-drpc-server' due to dependency issue.
> We also have been struggling to handle shade & relocation for lots artifacts,
> but in fact the artifacts which we need to relocate are just Worker related
> thing. So extracting worker and related classes into separate artifact and
> assigning different classpath in Storm cluster should help remedying.
> We may still need to struggle with relocation, but the count of target
> artifacts should be much smaller.
> Another thing we need to consider is LocalCluster, which pulls most of
> daemons but user topology still need to pull it. So we need to have separate
> artifact for LocalCluster (maybe also daemons without webapp).
> Last, we need to consider about UI and Logviewer porting. Artifact
> 'storm-drpc-server' is not generic and I think we can group UI, Logviewer,
> DRPC (only http server) into this artifact. Artifact name should be changed
> then.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
