Github user hustfxj commented on a diff in the pull request:
https://github.com/apache/storm/pull/877#discussion_r44796241
--- Diff: STORM_JSTORM_FEATURE_DIFF.md ---
@@ -0,0 +1,30 @@
+ feature | Storm-0.11.0-SNAPSHOT (and expected pull requests) |
JStorm-2.1.0 | Notes | JIRA |
+----------|-----------------------|--------------|-------|----|
+ Security | Basic Authentication/Authorization | N/A | The migration is
not high risk. But JStorm daeons/connections need to be evaluated
+Scheduler | Resource aware scheduling (Work In Progress) |
<ol><li>Make the tasks of same component distributed evenly across the nodes in
cluster.<li>Make number of tasks in a worker is balanced.<li>Try to assign two
tasks which are transferring message directly into a same worker to reduce
network cost.<li>Support user defined assignment and to use the result of last
assignment Different solution between Storm and JStorm.</ol> Actually,
JStorm has ever used cpu/mem/disk/network demond for scheduling. But we faced
the problem of low utilization of cluster. For example, because the
configuration of topologys and resources of hardwares in a cluster are
different. After running a period, it is possible that no topology will be
assigned to the node with enough other resources but just lacking one resource.
| Scheduler interface is pluggable so we should be able to support boht
schedulers for the time being |
+Nimbus HA | Support to configure more than one spare nimbus. When the
master nimbus is down, a most appropriate spare nimbus(topologys in disk mostly
matches the recoreds in zk) will be chosen to be promoted. | Different
solution. | |
+Topology structure | worker/executor/task | worker/task | It might be the
biggest diff between Storm and JStorm. So, not sure if there are some features
related to "executor" can not be merged. (Rebalancing allows us to combine
multiple tasks on a single executor) This may go away in the future once RAS
is stable |
+Topology Master | (Heartbeat server is similar for scaling on a pull
request) | New system bolt "topology master" was added, which is responsible
for collecting task heartbeat info of all tasks and reports to nimbus. Besides
task hb info, it also can be used to dispatch control messages in topology.
Topology master significantly reduce the amout of read/write to zk. Before
this change, zk is the bottleneck to depoly big cluster and topology.
--- End diff --
TM is mainly used to manage topology laterï¼and we hope users can
defines control stream in the future.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
