[GitHub] storm issue #2647: STORM-3040: Improve scheduler performance
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2647 @danny0405 the failure is a known race condition around netty and is not related to this change. ---
[GitHub] storm issue #2647: STORM-3040: Improve scheduler performance
Github user danny0405 commented on the issue: https://github.com/apache/storm/pull/2647 @revans2 Thx for your work, the storm-core travis check still fails, we should fix that. ---
[GitHub] storm issue #2647: STORM-3040: Improve scheduler performance
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2647 @danny0405 I added in the comments about thread safety like you suggested. ---
[GitHub] storm issue #2647: STORM-3040: Improve scheduler performance
Github user danny0405 commented on the issue: https://github.com/apache/storm/pull/2647 @revans2 I approve with you promotion totally. The only concern are all kinds of cache we use here, now storm has many caches not just for scheduling. I just think if we can make a disk-storage-backend for all of such caches master needs. Disk cache has better fault-tolerance and is much cheeper than memory, but this is another promotion direction, and has nothing to do with this patch. BTW: thx for your nice work. ---
[GitHub] storm issue #2647: STORM-3040: Improve scheduler performance
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2647 Oh I forgot I also added back in something I messed up before and added back in anti-affinity to GRAS. ---
[GitHub] storm issue #2647: STORM-3040: Improve scheduler performance
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2647 @danny0405 @kishorvpatil With some recent changes to master my patch started to fail with some checkstyle issues. I have rebased and fixed all of the issues. Please take a look again, specifically the second commit and let me know. ---
[GitHub] storm issue #2647: STORM-3040: Improve scheduler performance
Github user revans2 commented on the issue: https://github.com/apache/storm/pull/2647 @danny0405 In my tests `TestResourceAwareScheduler.testLargeTopologiesCommon` went from about 7 mins to about 7 seconds. For `TestResourceAwareScheduler.testLargeTopologiesOnLargeClustersGras` I don't have a before value because I killed it after an hour. The after is about 7 seconds per topology, or about a min and a half. ---
[GitHub] storm issue #2647: STORM-3040: Improve scheduler performance
Github user danny0405 commented on the issue: https://github.com/apache/storm/pull/2647 @revans2 In total, this is a good promotion direction. So this promotion mainly focus on repetitive computation and some eagerly computed data structure, right? What is the average promotion percentage when applied this patch? ---
