Github user flisky closed the pull request at:
https://github.com/apache/storm/pull/1419
---
Github user danny0405 closed the pull request at:
https://github.com/apache/storm/pull/2318
---
GitHub user danny0405 opened a pull request:
https://github.com/apache/storm/pull/2319
[STORM-2693] Nimbus assignments promotion
Storm now doesn't support large cluster[ for example thousand of
supervisors] very well, for our production, topology submission/killing is very
Github user revans2 commented on the issue:
https://github.com/apache/storm/pull/2319
@danny0405 conceptually this is a wonderful improvement. I will try to
take some time looking into this patch. One of the big things we will need
though is a corresponding patch for the master
GitHub user hmcc opened a pull request:
https://github.com/apache/storm/pull/2320
STORM-2736: fix o.a.s.b.BlobStoreUtils [ERROR] "Could not update the blob
with key"
Check for empty nimbus info list before attempting to update the blobstore.
You can merge this pull request into a
Github user hmcc commented on the issue:
https://github.com/apache/storm/pull/2320
Thanks! I'll work on a separate PR for master.
---
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138074783
--- Diff: storm-core/src/clj/org/apache/storm/daemon/worker.clj ---
@@ -47,9 +48,30 @@
(defmulti mk-suicide-fn cluster-mode)
+(defn
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138071918
--- Diff: storm-core/pom.xml ---
@@ -331,6 +331,10 @@
commons-codec
+org.rocksdb
+
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138074168
--- Diff: storm-core/src/clj/org/apache/storm/daemon/supervisor.clj ---
@@ -758,6 +752,35 @@
(catch Exception e
(log-error e "Error
Github user danny0405 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138091372
--- Diff: storm-core/src/clj/org/apache/storm/daemon/supervisor.clj ---
@@ -758,6 +752,35 @@
(catch Exception e
(log-error e "Error
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138073574
--- Diff: storm-core/src/clj/org/apache/storm/daemon/supervisor.clj ---
@@ -62,26 +63,17 @@
(defn- assignments-snapshot [storm-cluster-state
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138080750
--- Diff: storm-core/src/clj/org/apache/storm/daemon/nimbus.clj ---
@@ -961,14 +1015,25 @@
;; tasks figure out what tasks to talk to by looking at
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138073996
--- Diff: storm-core/src/clj/org/apache/storm/daemon/supervisor.clj ---
@@ -590,6 +583,7 @@
(set (keys
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138074478
--- Diff: storm-core/src/clj/org/apache/storm/daemon/supervisor.clj ---
@@ -758,6 +752,35 @@
(catch Exception e
(log-error e "Error
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138074935
--- Diff: storm-core/src/clj/org/apache/storm/daemon/worker.clj ---
@@ -351,6 +373,8 @@
(.sendLoadMetrics (:receiver worker) local-pop)
Github user danny0405 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138092651
--- Diff: storm-core/pom.xml ---
@@ -331,6 +331,10 @@
commons-codec
+org.rocksdb
+
Github user danny0405 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138097813
--- Diff: storm-core/src/clj/org/apache/storm/daemon/nimbus.clj ---
@@ -961,14 +1015,25 @@
;; tasks figure out what tasks to talk to by looking at
Github user danny0405 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138103588
--- Diff: storm-core/src/clj/org/apache/storm/daemon/nimbus.clj ---
@@ -961,14 +1015,25 @@
;; tasks figure out what tasks to talk to by looking at
Github user danny0405 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138090333
--- Diff: storm-core/src/clj/org/apache/storm/daemon/supervisor.clj ---
@@ -62,26 +63,17 @@
(defn- assignments-snapshot [storm-cluster-state
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2312#discussion_r138089602
--- Diff: storm-client/src/jvm/org/apache/storm/Config.java ---
@@ -205,6 +205,12 @@
public static final String TOPOLOGY_TASKS = "topology.tasks";
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2312#discussion_r138090061
--- Diff: storm-client/src/jvm/org/apache/storm/Config.java ---
@@ -1193,6 +1199,12 @@
public static final String SUPERVISOR_CPU_CAPACITY =
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2312#discussion_r138090802
--- Diff:
storm-client/src/jvm/org/apache/storm/topology/ComponentConfigurationDeclarer.java
---
@@ -62,4 +69,14 @@
* @return this for chaining.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2312#discussion_r138090901
--- Diff:
storm-client/src/jvm/org/apache/storm/topology/ComponentConfigurationDeclarer.java
---
@@ -62,4 +69,14 @@
* @return this for chaining.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2312#discussion_r138090258
--- Diff:
storm-client/src/jvm/org/apache/storm/topology/BaseConfigurationDeclarer.java
---
@@ -83,4 +83,29 @@ public T setCPULoad(Number amount) {
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2312#discussion_r138090558
--- Diff:
storm-client/src/jvm/org/apache/storm/topology/BaseConfigurationDeclarer.java
---
@@ -83,4 +83,29 @@ public T setCPULoad(Number amount) {
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2312#discussion_r138091077
--- Diff:
storm-client/src/jvm/org/apache/storm/topology/ComponentConfigurationDeclarer.java
---
@@ -62,4 +69,14 @@
* @return this for chaining.
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2312#discussion_r138090750
--- Diff:
storm-client/src/jvm/org/apache/storm/topology/ComponentConfigurationDeclarer.java
---
@@ -28,6 +28,13 @@
T
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138093197
--- Diff: storm-core/pom.xml ---
@@ -331,6 +331,10 @@
commons-codec
+org.rocksdb
+
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/2311
---
Github user Ethanlm commented on the issue:
https://github.com/apache/storm/pull/2315
Thanks @revans2 . I forgot it. Addressed in the latest commit (and
squashed).
---
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/1082
---
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/2315
---
Github user asfgit closed the pull request at:
https://github.com/apache/storm/pull/2316
---
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2315#discussion_r138088942
--- Diff:
external/storm-hdfs/src/main/java/org/apache/storm/hdfs/bolt/AbstractHdfsBolt.java
---
@@ -317,7 +319,18 @@ public WritersMap(long maxWriters) {
Github user revans2 commented on the issue:
https://github.com/apache/storm/pull/1082
Still +1
---
Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2319#discussion_r138099459
--- Diff: storm-core/src/clj/org/apache/storm/daemon/nimbus.clj ---
@@ -961,14 +1015,25 @@
;; tasks figure out what tasks to talk to by looking at
A lot of that depends on the version of storm you have, and it is not a use
case that we have played around with too much.
Just looking at a few of our production clusters the CPU usage is very
small (but trying to determine what the CPU usage on a raspberry pi arm
chip when all you have is the
Github user srdo commented on the issue:
https://github.com/apache/storm/pull/2316
@revans2 It's not the goal itself, it's because the apache-release profile
is activated
https://svn.apache.org/viewvc/maven/pom/tags/apache-10/pom.xml?view=markup#l298.
In the 10 version of
GitHub user raviperi opened a pull request:
https://github.com/apache/storm/pull/2322
Introduce config params to use latest EH client, control request prefetch
size, batch size
of events received per call.
-Refactor the code to group classes more appropriately
-Remove
GitHub user revans2 opened a pull request:
https://github.com/apache/storm/pull/2321
STORM-2733: Better load aware shuffle implementation
I have run several tests that show this works much better at big imbalances
in processing latency than did the previous shuffle implementations.
40 matches
Mail list logo
