[jira] [Created] (STORM-2085) Remove guava from storm-core pom.
Kyle Nusbaum created STORM-2085: --- Summary: Remove guava from storm-core pom. Key: STORM-2085 URL: https://issues.apache.org/jira/browse/STORM-2085 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Guava is not used by storm-core. Remove it as a dependency from the pom.xml -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-436) Clean up Clojure namespace declarations
[ https://issues.apache.org/jira/browse/STORM-436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-436. -- Resolution: Won't Fix > Clean up Clojure namespace declarations > --- > > Key: STORM-436 > URL: https://issues.apache.org/jira/browse/STORM-436 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Affects Versions: 0.9.2-incubating >Reporter: Daniel Compton >Priority: Minor > Labels: newbie > Time Spent: 20m > Remaining Estimate: 0h > > Some of the Clojure namespace declarations in the storm project are messy and > non-idiomatic. > https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/ui/core.clj#L18-L38 > is a good example of this. > There are a few things I'd like to improve: > 1. Coalesce multiple use/require/import's into a single use/require/import. > 2. Order imports use, require, import. > 3. Optionally, replacing use with some mix of > * [... :refer :all] > * Referring just the vars that are used > * Qualifying the namespace imports > Is there a reason why the namespaces were done in the way they have been? > What would be the preferred way to do 3? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-588) Executor-Level Rebalance Mechanism
[ https://issues.apache.org/jira/browse/STORM-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-588. -- Resolution: Won't Fix I'm going to close this and the corresponding pull request. This has been sitting for a long time, and I don't think this method fits with our current goals for storm. > Executor-Level Rebalance Mechanism > -- > > Key: STORM-588 > URL: https://issues.apache.org/jira/browse/STORM-588 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Affects Versions: 0.10.0, 0.9.3-rc2 >Reporter: troy ding >Assignee: troy ding > > I. The motivation > The current rebalance mechanism is implemented on the worker level. When > rebalance operation is triggered (e.g. by adding/removing a worker), storm > kills all the workers with different assignment. It means the rebalance > operation has to kill certain running workers and launches them according to > the new assignment. The advantage of the mechanism is the simplicity of the > implementation, but possibly incurs _huge_ overhead. Actually, the restarting > latency is usually more than one second, making the system almost impossible > to recover under high incoming data stream rate. No system administrator > dares to call rebalance, especially when the system is overloaded! To bring > back the real benefits of rebalancing operation, we believe it is important > to address the following problems: > *1. Resource wastage and additional initialization cost*: In most cases, the > changes on worker’s assignment (if not killed) only affect a small fraction > of running executors on it. Only part of them needs to be migrated or > created, while the remaining can keep running on the same worker. The current > implementation, however, forcefully restarts all the executors, and calls > unnecessary initializations (i.e. call Bolt.prepare() and Spout.prepare()) to > most of the running tasks. It not only wastes the computation resources of > unaffected executors, but also amplifies the initialization costs under > certain condition, e.g. index load in the bolt. > *2. Restarting workers causes avoidable in-memory data loss*: Currently, a > supervisor uses “kill -9” command to kill its correspondent worker. > Consequently, all the tasks on this worker have no chance to save the task > data. The running states of the workers, including important information when > resuming its duty, are simply lost, potentially causing unnecessary > recomputation on the states. > *3. JVM restart cost, long duration and lost of HotSpot optimizations*: > Restarting a JVM involves a long initialization procedure, and loses all the > runtime optimizations available for the application byte-code. As far as we > know, the HotSpot JVM is capable of detecting the performance-critical > sections in the code and dynamically translates the Java byte codes of these > hot spots into native machine code. In particular, tasks that are CPU-bound > can greatly benefit from this feature. If we directly kill the worker, all > the advantages of these features are lost. > II. Proposed solutions > 1. At the supervisor side: > The current supervisor implementation periodically calls the “sync-processes” > function to check whether a live worker should be killed: (1) the mapping > relationship between the worker and the topology has changed (e.g. this > worker is re-assigned to another topology or the serving topology is killed); > (2) the worker’s assignment has updated (e.g. the parallelism of some bolts > increases/decreases). > In order to reuse the worker’s JVM instance as much as possible, we propose > that we do not kill the workers mentioned in condition (2), but only kill > those that do not belong to the topology anymore (condition (1)). > 2. At the worker side: > Because of the reuse of the JVM instance, workers needs to periodically > synchronize its assigned executors. To achieve this, a new thread which is > similar to the existing “refresh-connections” is launched, to kill the > non-existing executors, and to start newly assigned ones. Note that, in > practice, the “refresh-connections“ threads already retrieves the assignment > information from the ZK, and this information can be shared with this new > thread, which reduce the load of the ZK. > Due to the change of the binging from the running executors to the worker, > re-routing tuple is also required. To fulfill this prepose, we need to > rewrite the following two functions, “transfer-local-fn” and “transfer-fn” > (note the rewrite is compulsive because these two functions are immutable in > the current implementation). > Another function needs careful modification is > “WorkerTopologyContext.getThisWorkerTasks()”, because the (defn- mk-grouper … >
[jira] [Resolved] (STORM-1954) Large Trident topologies can cause memory issues due to DefaultResourceDeclarer object reading config
[ https://issues.apache.org/jira/browse/STORM-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1954. - Resolution: Fixed Fix Version/s: 2.0.0 > Large Trident topologies can cause memory issues due to > DefaultResourceDeclarer object reading config > - > > Key: STORM-1954 > URL: https://issues.apache.org/jira/browse/STORM-1954 > Project: Apache Storm > Issue Type: Bug >Reporter: Kyle Nusbaum > Fix For: 2.0.0 > > > DefaultResourceDeclarer objects each read the full config on instantiation. > We've seen this causing slow startup and OOMs for large Trident topologies. > The config should be static in DefaultResourceDeclarer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1916) Add ability for worker-first classpath
[ https://issues.apache.org/jira/browse/STORM-1916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1916. - Resolution: Fixed Fix Version/s: 2.0.0 > Add ability for worker-first classpath > -- > > Key: STORM-1916 > URL: https://issues.apache.org/jira/browse/STORM-1916 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > Fix For: 2.0.0 > > > Debugging in multi-tenant environments can be difficult, and having the > ability to override the classpath can be helpful. We want a topology config > that specifies an overriding classpath for a given topology. This config is > disabled by default since users overriding the classpath is dangerous in a > multi-tenant environment. The config can be temporarily enabled on Nimbus to > allow submission of topologies with overridden classpaths. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1913) Additions and Improvements for Trident RAS API
[
https://issues.apache.org/jira/browse/STORM-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kyle Nusbaum resolved STORM-1913.
-
Resolution: Fixed
> Additions and Improvements for Trident RAS API
> --
>
> Key: STORM-1913
> URL: https://issues.apache.org/jira/browse/STORM-1913
> Project: Apache Storm
> Issue Type: Improvement
>Reporter: Kyle Nusbaum
>Assignee: Kyle Nusbaum
> Fix For: 2.0.0
>
>
> Trident's RAS API does not honor the following config values:
> {code}
> topology.component.resources.onheap.memory.mb
> topology.component.resources.offheap.memory.mb
> topology.component.cpu.pcore.percent
> {code}
> Trident does not receive the user's config as part of its builder API, so it
> does not know the value of these. Instead of altering the existing API (we
> want to remain backwards-compatible), add some new methods for dealing with
> this.
> There is also currently no way to set the master coord spouts' resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (STORM-1913) Additions and Improvements for Trident RAS API
[
https://issues.apache.org/jira/browse/STORM-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kyle Nusbaum updated STORM-1913:
Fix Version/s: 2.0.0
> Additions and Improvements for Trident RAS API
> --
>
> Key: STORM-1913
> URL: https://issues.apache.org/jira/browse/STORM-1913
> Project: Apache Storm
> Issue Type: Improvement
>Reporter: Kyle Nusbaum
>Assignee: Kyle Nusbaum
> Fix For: 2.0.0
>
>
> Trident's RAS API does not honor the following config values:
> {code}
> topology.component.resources.onheap.memory.mb
> topology.component.resources.offheap.memory.mb
> topology.component.cpu.pcore.percent
> {code}
> Trident does not receive the user's config as part of its builder API, so it
> does not know the value of these. Instead of altering the existing API (we
> want to remain backwards-compatible), add some new methods for dealing with
> this.
> There is also currently no way to set the master coord spouts' resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (STORM-1913) Additions and Improvements for Trident RAS API
[
https://issues.apache.org/jira/browse/STORM-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kyle Nusbaum closed STORM-1913.
---
Resolution: Fixed
> Additions and Improvements for Trident RAS API
> --
>
> Key: STORM-1913
> URL: https://issues.apache.org/jira/browse/STORM-1913
> Project: Apache Storm
> Issue Type: Improvement
>Reporter: Kyle Nusbaum
>Assignee: Kyle Nusbaum
>
> Trident's RAS API does not honor the following config values:
> {code}
> topology.component.resources.onheap.memory.mb
> topology.component.resources.offheap.memory.mb
> topology.component.cpu.pcore.percent
> {code}
> Trident does not receive the user's config as part of its builder API, so it
> does not know the value of these. Instead of altering the existing API (we
> want to remain backwards-compatible), add some new methods for dealing with
> this.
> There is also currently no way to set the master coord spouts' resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (STORM-1913) Additions and Improvements for Trident RAS API
[
https://issues.apache.org/jira/browse/STORM-1913?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kyle Nusbaum reopened STORM-1913:
-
> Additions and Improvements for Trident RAS API
> --
>
> Key: STORM-1913
> URL: https://issues.apache.org/jira/browse/STORM-1913
> Project: Apache Storm
> Issue Type: Improvement
>Reporter: Kyle Nusbaum
>Assignee: Kyle Nusbaum
> Fix For: 2.0.0
>
>
> Trident's RAS API does not honor the following config values:
> {code}
> topology.component.resources.onheap.memory.mb
> topology.component.resources.offheap.memory.mb
> topology.component.cpu.pcore.percent
> {code}
> Trident does not receive the user's config as part of its builder API, so it
> does not know the value of these. Instead of altering the existing API (we
> want to remain backwards-compatible), add some new methods for dealing with
> this.
> There is also currently no way to set the master coord spouts' resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (STORM-1949) Backpressure can cause spout to stop emitting and stall topology
[ https://issues.apache.org/jira/browse/STORM-1949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15371017#comment-15371017 ] Kyle Nusbaum commented on STORM-1949: - I'm okay with polling ZK for changes as a fix (for now) so we can keep backpressure on by default. There are obviously issues with the design of backpressure in general, but polling shouldn't make these issues significantly worse. I don't think the load on ZK will increase appreciably from a few nodes polling. The load problem is inherent to other parts of backpressure design. We could also turn it off by default, but it's nice to have a flow-control mechanism that works automatically without needing the user to perform acking. It might be some time until we have a better system in place. > Backpressure can cause spout to stop emitting and stall topology > > > Key: STORM-1949 > URL: https://issues.apache.org/jira/browse/STORM-1949 > Project: Apache Storm > Issue Type: Bug >Reporter: Roshan Naik > > Problem can be reproduced by this [Word count > topology|https://github.com/hortonworks/storm/blob/perftopos1.x/examples/storm-starter/src/jvm/org/apache/storm/starter/perf/FileReadWordCountTopo.java] > within a IDE. > I ran it with 1 spout instance, 2 splitter bolt instances, 2 counter bolt > instances. > The problem is more easily reproduced with WC topology as it causes an > explosion of tuples due to splitting a sentence tuple into word tuples. As > the bolts have to process more tuples than the spout is producing, spout > needs to operate slower. > The amount of time it takes for the topology to stall can vary.. but > typically under 10 mins. > *My theory:* I suspect there is a race condition in the way ZK is being > utilized to enable/disable back pressure. When congested (i.e pressure > exceeds high water mark), the bolt's worker records this congested situation > in ZK by creating a node. Once the congestion is reduced below the low water > mark, it deletes this node. > The spout's worker has setup a watch on the parent node, expecting a callback > whenever there is change in the child nodes. On receiving the callback the > spout's worker lists the parent node to check if there are 0 or more child > nodes it is essentially trying to figure out the nature of state change > in ZK to determine whether to throttle or not. Subsequently it setsup > another watch in ZK to keep an eye on future changes. > When there are multiple bolts, there can be rapid creation/deletion of these > ZK nodes. Between the time the worker receives a callback and sets up the > next watch.. many changes may have undergone in ZK which will go unnoticed by > the spout. > The condition that the bolts are no longer congested may not get noticed as a > result of this. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1954) Large Trident topologies can cause memory issues due to DefaultResourceDeclarer object reading config
[ https://issues.apache.org/jira/browse/STORM-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1954: Summary: Large Trident topologies can cause memory issues due to DefaultResourceDeclarer object reading config (was: Large Trident topologies can cause memory issues due to Node object reading config) > Large Trident topologies can cause memory issues due to > DefaultResourceDeclarer object reading config > - > > Key: STORM-1954 > URL: https://issues.apache.org/jira/browse/STORM-1954 > Project: Apache Storm > Issue Type: Bug >Reporter: Kyle Nusbaum > > DefaultResourceDeclarer objects each read the full config on instantiation. > We've seen this causing slow startup and OOMs for large Trident topologies. > The config should be static in DefaultResourceDeclarer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1954) Large Trident topologies can cause memory issues due to Node object reading config
[ https://issues.apache.org/jira/browse/STORM-1954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1954: Description: DefaultResourceDeclarer objects each read the full config on instantiation. We've seen this causing slow startup and OOMs for large Trident topologies. The config should be static in DefaultResourceDeclarer. was: Trident Node objects each read the full config on instantiation. We've seen this causing slow startup and OOMs for large Trident topologies. The config should be static in Node. > Large Trident topologies can cause memory issues due to Node object reading > config > -- > > Key: STORM-1954 > URL: https://issues.apache.org/jira/browse/STORM-1954 > Project: Apache Storm > Issue Type: Bug >Reporter: Kyle Nusbaum > > DefaultResourceDeclarer objects each read the full config on instantiation. > We've seen this causing slow startup and OOMs for large Trident topologies. > The config should be static in DefaultResourceDeclarer. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1954) Large Trident topologies can cause memory issues due to Node object reading config
Kyle Nusbaum created STORM-1954: --- Summary: Large Trident topologies can cause memory issues due to Node object reading config Key: STORM-1954 URL: https://issues.apache.org/jira/browse/STORM-1954 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum Trident Node objects each read the full config on instantiation. We've seen this causing slow startup and OOMs for large Trident topologies. The config should be static in Node. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1916) Add ability for worker-first classpath
Kyle Nusbaum created STORM-1916: --- Summary: Add ability for worker-first classpath Key: STORM-1916 URL: https://issues.apache.org/jira/browse/STORM-1916 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Debugging in multi-tenant environments can be difficult, and having the ability to override the classpath can be helpful. We want a topology config that specifies an overriding classpath for a given topology. This config is disabled by default since users overriding the classpath is dangerous in a multi-tenant environment. The config can be temporarily enabled on Nimbus to allow submission of topologies with overridden classpaths. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1913) Additions and Improvements for Trident RAS API
Kyle Nusbaum created STORM-1913:
---
Summary: Additions and Improvements for Trident RAS API
Key: STORM-1913
URL: https://issues.apache.org/jira/browse/STORM-1913
Project: Apache Storm
Issue Type: Improvement
Reporter: Kyle Nusbaum
Assignee: Kyle Nusbaum
Trident's RAS API does not honor the following config values:
{code}
topology.component.resources.onheap.memory.mb
topology.component.resources.offheap.memory.mb
topology.component.cpu.pcore.percent
{code}
Trident does not receive the user's config as part of its builder API, so it
does not know the value of these. Instead of altering the existing API (we want
to remain backwards-compatible), add some new methods for dealing with this.
There is also currently no way to set the master coord spouts' resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (STORM-1912) Additions and Improvements for Trident RAS API
Kyle Nusbaum created STORM-1912:
---
Summary: Additions and Improvements for Trident RAS API
Key: STORM-1912
URL: https://issues.apache.org/jira/browse/STORM-1912
Project: Apache Storm
Issue Type: Improvement
Reporter: Kyle Nusbaum
Assignee: Kyle Nusbaum
Trident's RAS API does not honor the following config values:
{code}
topology.component.resources.onheap.memory.mb
topology.component.resources.offheap.memory.mb
topology.component.cpu.pcore.percent
{code}
Trident does not receive the user's config as part of its builder API, so it
does not know the value of these. Instead of altering the existing API (we want
to remain backwards-compatible), add some new methods for dealing with this.
There is also currently no way to set the master coord spouts' resources.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (STORM-1764) Pacemaker is throwing some stack traces
Kyle Nusbaum created STORM-1764: --- Summary: Pacemaker is throwing some stack traces Key: STORM-1764 URL: https://issues.apache.org/jira/browse/STORM-1764 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Pacemaker throws a handful of stack traces when the occasional weird thing happens. This fixes an IndexOutOfBounds exception, a very strange IllegalStateException, and a NullPointerException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1611) port org.apache.storm.pacemaker.pacemaker to java
[ https://issues.apache.org/jira/browse/STORM-1611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1611. - Resolution: Fixed Fix Version/s: 2.0.0 > port org.apache.storm.pacemaker.pacemaker to java > - > > Key: STORM-1611 > URL: https://issues.apache.org/jira/browse/STORM-1611 > Project: Apache Storm > Issue Type: New Feature >Reporter: John Fang >Assignee: John Fang > Fix For: 2.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1673) log4j2/worker.xml refers old package of LoggerMetricsConsumer
[ https://issues.apache.org/jira/browse/STORM-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1673: Assignee: Jungtaek Lim (was: Kyle Nusbaum) > log4j2/worker.xml refers old package of LoggerMetricsConsumer > - > > Key: STORM-1673 > URL: https://issues.apache.org/jira/browse/STORM-1673 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Affects Versions: 1.0.0, 2.0.0 >Reporter: Jungtaek Lim >Assignee: Jungtaek Lim > > We changed package path from 'backtype.storm' to 'org.apache.storm'. Source > codes seem to moved properly, but missed log4j2 configuration, so metric log > is logged into worker log file, not metrics file. > It should be simple patch so I'd like to include this as 1.0.0. If we don't > want to include any bugfixes I'm OK to remove epic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1667) Log the IO Exception when deleting worker pid dir
[ https://issues.apache.org/jira/browse/STORM-1667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1667. - Resolution: Fixed > Log the IO Exception when deleting worker pid dir > - > > Key: STORM-1667 > URL: https://issues.apache.org/jira/browse/STORM-1667 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Affects Versions: 1.0.0 >Reporter: Zhuo Liu >Assignee: Zhuo Liu > Fix For: 1.0.0 > > > There is a race condition since shutdown-worker is called by both > sync-processes and synchronize-supervisor in supervisor. One pid directory > might have been already deleted when rmr-as-user tries to delete it. > Therefore, we should not let the FileNotFoundException get thrown and crash > the supervisor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (STORM-1673) log4j2/worker.xml refers old package of LoggerMetricsConsumer
[ https://issues.apache.org/jira/browse/STORM-1673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reassigned STORM-1673: --- Assignee: Kyle Nusbaum (was: Jungtaek Lim) > log4j2/worker.xml refers old package of LoggerMetricsConsumer > - > > Key: STORM-1673 > URL: https://issues.apache.org/jira/browse/STORM-1673 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Affects Versions: 1.0.0, 2.0.0 >Reporter: Jungtaek Lim >Assignee: Kyle Nusbaum > > We changed package path from 'backtype.storm' to 'org.apache.storm'. Source > codes seem to moved properly, but missed log4j2 configuration, so metric log > is logged into worker log file, not metrics file. > It should be simple patch so I'd like to include this as 1.0.0. If we don't > want to include any bugfixes I'm OK to remove epic. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (STORM-1581) Repair github links in the storm documentation
[ https://issues.apache.org/jira/browse/STORM-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reopened STORM-1581: - Assignee: Kyle Nusbaum (was: Abhishek Agarwal) > Repair github links in the storm documentation > -- > > Key: STORM-1581 > URL: https://issues.apache.org/jira/browse/STORM-1581 > Project: Apache Storm > Issue Type: Bug > Components: documentation >Affects Versions: 1.0.0 >Reporter: Abhishek Agarwal >Assignee: Kyle Nusbaum > Fix For: 1.0.0 > > > Java classes have been migrated to org.apache.storm and the github links are > now broken in the documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1581) Repair github links in the storm documentation
[ https://issues.apache.org/jira/browse/STORM-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1581. - Resolution: Fixed > Repair github links in the storm documentation > -- > > Key: STORM-1581 > URL: https://issues.apache.org/jira/browse/STORM-1581 > Project: Apache Storm > Issue Type: Bug > Components: documentation >Affects Versions: 1.0.0 >Reporter: Abhishek Agarwal >Assignee: Kyle Nusbaum > Fix For: 1.0.0 > > > Java classes have been migrated to org.apache.storm and the github links are > now broken in the documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1668) Flux silently fails while setting a non-existent property.
[ https://issues.apache.org/jira/browse/STORM-1668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1668. - Resolution: Fixed > Flux silently fails while setting a non-existent property. > -- > > Key: STORM-1668 > URL: https://issues.apache.org/jira/browse/STORM-1668 > Project: Apache Storm > Issue Type: Bug > Components: Flux >Affects Versions: 1.0.0, 2.0.0 >Reporter: Priyank Shah >Assignee: P. Taylor Goetz > Fix For: 1.0.0 > > > Currently, if a yaml file has a property with a name that does not exist on > the java topology component object then flux silently fails by logging a > message and does not throw an exception. This needs to be changed so that > flux throws the exception failing topology submission so that user can take > corrective action. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-1581) Repair github links in the storm documentation
[ https://issues.apache.org/jira/browse/STORM-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-1581. --- Resolution: Fixed Fix Version/s: 1.0.0 Fixed when [~revans2] moved the docs to SVN. > Repair github links in the storm documentation > -- > > Key: STORM-1581 > URL: https://issues.apache.org/jira/browse/STORM-1581 > Project: Apache Storm > Issue Type: Bug > Components: documentation >Affects Versions: 1.0.0 >Reporter: Abhishek Agarwal >Assignee: Abhishek Agarwal > Fix For: 1.0.0 > > > Java classes have been migrated to org.apache.storm and the github links are > now broken in the documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-1595) 'Fail' messages get stuck somewhere
[ https://issues.apache.org/jira/browse/STORM-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-1595. --- Resolution: Cannot Reproduce > 'Fail' messages get stuck somewhere > > > Key: STORM-1595 > URL: https://issues.apache.org/jira/browse/STORM-1595 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 1.0.0, 2.0.0 >Reporter: Kyle Nusbaum > Attachments: screenshot-1.png > > > 'Fail' acks seem to be getting stuck somewhere between the acker and the > spout. > After a long time - sometimes multiple minutes - the fails show up in the > spout. > I tested this on master and 1.x-branch and it occurs in both places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1595) 'Fail' messages get stuck somewhere
[ https://issues.apache.org/jira/browse/STORM-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15216581#comment-15216581 ] Kyle Nusbaum commented on STORM-1595: - [~ptgoetz] I'm not able to reproduce this anymore. Going to close it. > 'Fail' messages get stuck somewhere > > > Key: STORM-1595 > URL: https://issues.apache.org/jira/browse/STORM-1595 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 1.0.0, 2.0.0 >Reporter: Kyle Nusbaum > Attachments: screenshot-1.png > > > 'Fail' acks seem to be getting stuck somewhere between the acker and the > spout. > After a long time - sometimes multiple minutes - the fails show up in the > spout. > I tested this on master and 1.x-branch and it occurs in both places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-1497) Repo Branches Cleanup
[ https://issues.apache.org/jira/browse/STORM-1497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-1497. --- Resolution: Fixed > Repo Branches Cleanup > -- > > Key: STORM-1497 > URL: https://issues.apache.org/jira/browse/STORM-1497 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > > I've identified a bunch of branches that I think can go away: > Already merged: > STORM-633 > STORM-648 > STORM-651 > STORM-1153-V1 > better-trident-spouts > experimental > clojure1.3 > clojure1.4 > executors > gettuples > kryo-2.16 > mesos > new-resource-scheduler > nimbus-ha-branch > scheduler > security > transactional-spout > ui-url > Branches not merged, but I think we can delete: > STORM-1040 (harschach) > PR_736 (ptgoetz) > nimbus-ha (nathanmarz) > debug (do not want) (nathanmarz) > iso-scheduler (not merged but VERY old) (nathanmarz) > jarjar (not merged but VERY old) (nathanmarz) > statespout (not merged but VERY old) (nathanmarz) > std-redirect (not merged but VERY old) (nathanmarz) > superclojure (not merged but VERY old) (nathanmarz) > thunk (VERY old, don't want) (sritchie) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-1340) Use Travis-CI build matrix to improve test execution times
[ https://issues.apache.org/jira/browse/STORM-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-1340. --- Resolution: Won't Fix > Use Travis-CI build matrix to improve test execution times > -- > > Key: STORM-1340 > URL: https://issues.apache.org/jira/browse/STORM-1340 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > > Travis-CI provides a 'build-matrix' that we can use to parallelize the unit > tests of various components in storm. > It also provides a caching mechanism that can reduce the overhead of starting > up a test container. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1653) BYLAWS has disappeared
Kyle Nusbaum created STORM-1653: --- Summary: BYLAWS has disappeared Key: STORM-1653 URL: https://issues.apache.org/jira/browse/STORM-1653 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum When moving asf-site to svn, BYLAWS (and probably some other stuff) disappeared. Someone needs to compare the previous docs with what's on SVN now and fill in any missing spots. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-1616) Add RAS API for Trident
[ https://issues.apache.org/jira/browse/STORM-1616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-1616. --- Resolution: Fixed Fix Version/s: 1.0.0 > Add RAS API for Trident > --- > > Key: STORM-1616 > URL: https://issues.apache.org/jira/browse/STORM-1616 > Project: Apache Storm > Issue Type: Bug >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > Fix For: 1.0.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1595) 'Fail' messages get stuck somewhere
[ https://issues.apache.org/jira/browse/STORM-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1595: Attachment: (was: Screen Shot 2016-03-02 at 3.14.25 PM.png) > 'Fail' messages get stuck somewhere > > > Key: STORM-1595 > URL: https://issues.apache.org/jira/browse/STORM-1595 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 1.0.0, 2.0.0 >Reporter: Kyle Nusbaum > Attachments: screenshot-1.png > > > 'Fail' acks seem to be getting stuck somewhere between the acker and the > spout. > After a long time - sometimes multiple minutes - the fails show up in the > spout. > I tested this on master and 1.x-branch and it occurs in both places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1595) 'Fail' messages get stuck somewhere
[ https://issues.apache.org/jira/browse/STORM-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1595: Attachment: screenshot-1.png > 'Fail' messages get stuck somewhere > > > Key: STORM-1595 > URL: https://issues.apache.org/jira/browse/STORM-1595 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 1.0.0, 2.0.0 >Reporter: Kyle Nusbaum > Attachments: screenshot-1.png > > > 'Fail' acks seem to be getting stuck somewhere between the acker and the > spout. > After a long time - sometimes multiple minutes - the fails show up in the > spout. > I tested this on master and 1.x-branch and it occurs in both places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1595) 'Fail' messages get stuck somewhere
[ https://issues.apache.org/jira/browse/STORM-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1595: Attachment: Screen Shot 2016-03-02 at 3.14.25 PM.png > 'Fail' messages get stuck somewhere > > > Key: STORM-1595 > URL: https://issues.apache.org/jira/browse/STORM-1595 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 1.0.0, 2.0.0 >Reporter: Kyle Nusbaum > Attachments: Screen Shot 2016-03-02 at 3.14.25 PM.png > > > 'Fail' acks seem to be getting stuck somewhere between the acker and the > spout. > After a long time - sometimes multiple minutes - the fails show up in the > spout. > I tested this on master and 1.x-branch and it occurs in both places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1595) 'Fail' messages get stuck somewhere
Kyle Nusbaum created STORM-1595: --- Summary: 'Fail' messages get stuck somewhere Key: STORM-1595 URL: https://issues.apache.org/jira/browse/STORM-1595 Project: Apache Storm Issue Type: Bug Affects Versions: 1.0.0, 2.0.0 Reporter: Kyle Nusbaum 'Fail' acks seem to be getting stuck somewhere between the acker and the spout. After a long time - sometimes multiple minutes - the fails show up in the spout. I tested this on master and 1.x-branch and it occurs in both places. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1571) Improvment Kafka Spout Time Metric
[ https://issues.apache.org/jira/browse/STORM-1571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1571. - Resolution: Fixed Fix Version/s: 2.0.0 > Improvment Kafka Spout Time Metric > -- > > Key: STORM-1571 > URL: https://issues.apache.org/jira/browse/STORM-1571 > Project: Apache Storm > Issue Type: Improvement > Components: storm-kafka >Affects Versions: 0.9.3, 0.9.4, 0.9.5, 0.9.6 > Environment: Mac 10.11.1 JDK 1.8.0_40 >Reporter: darion yaphet >Assignee: darion yaphet >Priority: Minor > Fix For: 2.0.0 > > Attachments: 0001-Improvment-Kafka-Spout-Time-Metric.patch, > 0002-Update-time-interval-counting-on-TridentKafkaEmitter.patch > > > Use System.currentTimeMillis() to calculation time interval is better than > System.nanoTime() -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (STORM-1226) Port backtype.storm.util to java
[ https://issues.apache.org/jira/browse/STORM-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reassigned STORM-1226: --- Assignee: Kyle Nusbaum (was: Reza Farivar) > Port backtype.storm.util to java > > > Key: STORM-1226 > URL: https://issues.apache.org/jira/browse/STORM-1226 > Project: Apache Storm > Issue Type: New Feature > Components: storm-core >Reporter: Robert Joseph Evans >Assignee: Kyle Nusbaum > Labels: java-migration, jstorm-merger > Fix For: 2.0.0 > > > Port backtype.storm.util from clojure to java. In as many instances as > possible the same interface should be maintained, and calls to clojure > functions in the rest of the code should be replaces with calls to the > corresponding java code. > Some similar functions can be found at > https://github.com/apache/storm/blob/jstorm-import/jstorm-core/src/main/java/com/alibaba/jstorm/utils/JStormUtils.java > Although they are not identical. > For function callbacks we may need to evaluate adding in appropriate callback > interfaces instead. Please try to avoid using clojure internal java classes > unless necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1226) Port backtype.storm.util to java
[ https://issues.apache.org/jira/browse/STORM-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1226. - Resolution: Fixed Fix Version/s: 2.0.0 > Port backtype.storm.util to java > > > Key: STORM-1226 > URL: https://issues.apache.org/jira/browse/STORM-1226 > Project: Apache Storm > Issue Type: New Feature > Components: storm-core >Reporter: Robert Joseph Evans >Assignee: Kyle Nusbaum > Labels: java-migration, jstorm-merger > Fix For: 2.0.0 > > > Port backtype.storm.util from clojure to java. In as many instances as > possible the same interface should be maintained, and calls to clojure > functions in the rest of the code should be replaces with calls to the > corresponding java code. > Some similar functions can be found at > https://github.com/apache/storm/blob/jstorm-import/jstorm-core/src/main/java/com/alibaba/jstorm/utils/JStormUtils.java > Although they are not identical. > For function callbacks we may need to evaluate adding in appropriate callback > interfaces instead. Please try to avoid using clojure internal java classes > unless necessary. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1538) Exception being thrown after Utils conversion to java
[ https://issues.apache.org/jira/browse/STORM-1538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1538: Description: 1. TTransportException thrown when trying to visit topology page in UI: When trying to access the topology page on the storm ui, exceptions are thrown. Visiting a topology page displays a TTransportException. The cause can be seen in nimbus.log: java.lang.IllegalArgumentException: Don't know how to create ISeq from: clojure.core$_PLUS_ at clojure.lang.RT.seqFrom(RT.java:528) at clojure.lang.RT.seq(RT.java:509) at clojure.core$seq__4128.invoke(core.clj:137) at clojure.core.protocols$seq_reduce.invoke(protocols.clj:26) at clojure.core.protocols$fn__6500.invoke(protocols.clj:83) at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) at clojure.core$reduce.invoke(core.clj:6515) at org.apache.storm.stats$agg_pre_merge_topo_page_bolt.invoke(stats.clj:564) at org.apache.storm.stats$agg_topo_exec_stats_STAR_.invoke(stats.clj:744) at org.apache.storm.stats$fn__1257.invoke(stats.clj:792) at clojure.lang.MultiFn.invoke(MultiFn.java:243) at clojure.core$partial$fn__4529.invoke(core.clj:2501) at clojure.core.protocols$fn__6523.invoke(protocols.clj:167) at clojure.core.protocols$fn__6478$G__6473__6487.invoke(protocols.clj:19) at clojure.core.protocols$seq_reduce.invoke(protocols.clj:31) at clojure.core.protocols$fn__6506.invoke(protocols.clj:101) at clojure.core.protocols$fn__6452$G__6447__6465.invoke(protocols.clj:13) at clojure.core$reduce.invoke(core.clj:6519) at org.apache.storm.stats$aggregate_topo_stats.invoke(stats.clj:874) at org.apache.storm.stats$agg_topo_execs_stats.invoke(stats.clj:1037) at org.apache.storm.daemon.nimbus$fn__4817$exec_fn__2064__auto__$reify__4846.getTopologyPageInfo(nimbus.clj:2105) at org.apache.storm.generated.Nimbus$Processor$getTopologyPageInfo.getResult(Nimbus.java:3800) at org.apache.storm.generated.Nimbus$Processor$getTopologyPageInfo.getResult(Nimbus.java:3784) at org.apache.storm.thrift.ProcessFunction.process(ProcessFunction.java:39) at org.apache.storm.thrift.TBaseProcessor.process(TBaseProcessor.java:39) at org.apache.storm.security.auth.SimpleTransportPlugin$SimpleWrapProcessor.process(SimpleTransportPlugin.java:158) at org.apache.storm.thrift.server.AbstractNonblockingServer$FrameBuffer.invoke(AbstractNonblockingServer.java:518) at org.apache.storm.thrift.server.Invocation.run(Invocation.java:18) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) 2. Exception thrown in supervisor: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.String at org.apache.storm.daemon.supervisor$create_artifacts_link.invoke(supervisor.clj:1083) at org.apache.storm.daemon.supervisor$fn__7494.invoke(supervisor.clj:1180) at clojure.lang.MultiFn.invoke(MultiFn.java:251) at org.apache.storm.daemon.supervisor$get_valid_new_worker_ids$iter__7081__7085$fn__7086.invoke(supervisor.clj:393) at clojure.lang.LazySeq.sval(LazySeq.java:40) at clojure.lang.LazySeq.seq(LazySeq.java:49) at clojure.lang.RT.seq(RT.java:507) at clojure.core$seq__4128.invoke(core.clj:137) at clojure.core$dorun.invoke(core.clj:3009) at clojure.core$doall.invoke(core.clj:3025) at org.apache.storm.daemon.supervisor$get_valid_new_worker_ids.invoke(supervisor.clj:380) at org.apache.storm.daemon.supervisor$sync_processes.invoke(supervisor.clj:447) at clojure.core$partial$fn__4527.invoke(core.clj:2492) at org.apache.storm.event$event_manager$fn__6781.invoke(event.clj:40) at clojure.lang.AFn.run(AFn.java:22) at java.lang.Thread.run(Thread.java:745) 3. Exception in UI: java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Number at org.apache.storm.ui.core$worker_log_link.invoke(core.clj:150) at org.apache.storm.ui.core$get_error_json.invoke(core.clj:531) at org.apache.storm.ui.core$fn__11747.invoke(core.clj:553) at clojure.lang.MultiFn.invoke(MultiFn.java:238) at clojure.core$partial$fn__4529.invoke(core.clj:2500) at clojure.core$map$fn__4553.invoke(core.clj:2624) at clojure.lang.LazySeq.sval(LazySeq.java:40) at clojure.lang.LazySeq.seq(LazySeq.java:49) at clojure.lang.RT.seq(RT.java:507) at clojure.lang.SeqIterator.hasNext(SeqIterator.java:38) was: 1. TTransportException thrown when trying to visit topology page in UI: When trying to
[jira] [Created] (STORM-1502) Create per-version document scheme
Kyle Nusbaum created STORM-1502: --- Summary: Create per-version document scheme Key: STORM-1502 URL: https://issues.apache.org/jira/browse/STORM-1502 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum We want to modify our documentation scheme such that each version of storm has its own set of documentation published to the apache storm website. General consensus was that at least javadocs should be available per-version, but it is okay if parts of the process require manual intervention. The full conversation about it can be found here: http://mail-archives.apache.org/mod_mbox/storm-dev/201601.mbox/browser -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-1257) port backtype.storm.zookeeper to java
[ https://issues.apache.org/jira/browse/STORM-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15112905#comment-15112905 ] Kyle Nusbaum commented on STORM-1257: - Is this actively being worked on? If not I will pick this issue up. > port backtype.storm.zookeeper to java > - > > Key: STORM-1257 > URL: https://issues.apache.org/jira/browse/STORM-1257 > Project: Apache Storm > Issue Type: New Feature > Components: storm-core >Reporter: Robert Joseph Evans >Assignee: Basti Liu > Labels: java-migration, jstorm-merger > > A wrapper around zookeeper/curator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1497) Repo Branches Cleanup
Kyle Nusbaum created STORM-1497: --- Summary: Repo Branches Cleanup Key: STORM-1497 URL: https://issues.apache.org/jira/browse/STORM-1497 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum I've identified a bunch of branches that I think can go away: Already merged: STORM-633 STORM-648 STORM-651 STORM-1153-V1 better-trident-spouts experimental clojure1.3 clojure1.4 executors gettuples kryo-2.16 mesos new-resource-scheduler nimbus-ha-branch scheduler security transactional-spout ui-url Branches not merged, but I think we can delete: STORM-1040 (harschach) PR_736 (ptgoetz) nimbus-ha (nathanmarz) debug (do not want) (nathanmarz) iso-scheduler (not merged but VERY old) (nathanmarz) jarjar (not merged but VERY old) (nathanmarz) statespout (not merged but VERY old) (nathanmarz) std-redirect (not merged but VERY old) (nathanmarz) superclojure (not merged but VERY old) (nathanmarz) thunk (VERY old, don't want) (sritchie) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1467) Switch apache-rat plugin off by default, but enable for Travis-CI
Kyle Nusbaum created STORM-1467: --- Summary: Switch apache-rat plugin off by default, but enable for Travis-CI Key: STORM-1467 URL: https://issues.apache.org/jira/browse/STORM-1467 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum apache-rat is super annoying when developing. We should change it so that it doesn't run all the time, but can be manually triggered and runs in CI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (STORM-1466) Move the org.apache.thrift7 namespace to something correct/sensible
[ https://issues.apache.org/jira/browse/STORM-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reassigned STORM-1466: --- Assignee: Kyle Nusbaum > Move the org.apache.thrift7 namespace to something correct/sensible > --- > > Key: STORM-1466 > URL: https://issues.apache.org/jira/browse/STORM-1466 > Project: Apache Storm > Issue Type: Bug >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > > Thrift is still shaded to org.apache.thrift7 even though we are not using > Thrift 7. > We should also add something for temporary backwards compatibility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1466) Move the org.apache.thrift7 namespace to something correct/sensible
[ https://issues.apache.org/jira/browse/STORM-1466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1466: Description: Thrift is still shaded to org.apache.thrift7 even though we are not using Thrift 7. We should also add something for temporary backwards compatibility. was:Thrift is still shaded to org.apache.thrift7 even though we are not using Thrift 7. > Move the org.apache.thrift7 namespace to something correct/sensible > --- > > Key: STORM-1466 > URL: https://issues.apache.org/jira/browse/STORM-1466 > Project: Apache Storm > Issue Type: Bug >Reporter: Kyle Nusbaum > > Thrift is still shaded to org.apache.thrift7 even though we are not using > Thrift 7. > We should also add something for temporary backwards compatibility. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1466) Move the org.apache.thrift7 namespace to something correct/sensible
Kyle Nusbaum created STORM-1466: --- Summary: Move the org.apache.thrift7 namespace to something correct/sensible Key: STORM-1466 URL: https://issues.apache.org/jira/browse/STORM-1466 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum Thrift is still shaded to org.apache.thrift7 even though we are not using Thrift 7. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1432) Spurious failure in storm-kafka test
Kyle Nusbaum created STORM-1432: --- Summary: Spurious failure in storm-kafka test Key: STORM-1432 URL: https://issues.apache.org/jira/browse/STORM-1432 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum ExponentialBackoffMsgRetryManagerTest.testMaxBackoff() fails sometimes, I think due to too small times. It can miss the mark on slow machines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (STORM-1376) ZK Becoming deadlocked with zookeeper_state_factory
[ https://issues.apache.org/jira/browse/STORM-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reassigned STORM-1376: --- Assignee: Kyle Nusbaum (was: Sanket Reddy) > ZK Becoming deadlocked with zookeeper_state_factory > --- > > Key: STORM-1376 > URL: https://issues.apache.org/jira/browse/STORM-1376 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Affects Versions: 0.11.0 >Reporter: Daniel Schonfeld >Assignee: Kyle Nusbaum >Priority: Blocker > Fix For: 0.11.0 > > > Since the introduction of blobstore and pacemaker we've noticed that when > using nimbus with the new zookeeper_state_factory backing cluster state > module, some of our ZK nodes become unresponsive and show and increasing > amounts of outstanding requests (STAT 4-letter command). > Terminating storm supervisors and nimbus usually gets zookeeper to realize > after a few minutes those connections are dead and to become responsive > again. In some extreme cases we have to kill that ZK nodes and bring it back > up. > Our topologies ran across ~10 supervisor nodes with each having about > ~400-500 executors. > I mention the amount of executors cause I am not sure if someone made each > executor by mistake start sending heartbeats instead of each worker and that > might possibly be the reason for this slow down. > Final note. If someone can jot a few ideas of why this might be happening > i'd be more than happy to dig further in the storm code and submit a PR > myself. But I need some hint or direction of where to go with this... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1393) Sort out storm.log.dir configure and add documentation about logs
[ https://issues.apache.org/jira/browse/STORM-1393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1393. - Resolution: Fixed > Sort out storm.log.dir configure and add documentation about logs > - > > Key: STORM-1393 > URL: https://issues.apache.org/jira/browse/STORM-1393 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Affects Versions: 0.11.0 >Reporter: Zhuo Liu >Assignee: Zhuo Liu >Priority: Minor > Fix For: 0.11.0 > > > Currently, we have reorganized logs in STORM-901 and STORM-1387, it is > preferable for us to document the changes out for avoiding confusion to > users. > Also, the util/LOG-DIR and the way supervisor to get storm.log.dir in > worker-launch is inaccurate since it does not take the storm-conf into > account. We should fix it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1381) Client side topology submission hook.
[ https://issues.apache.org/jira/browse/STORM-1381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1381. - Resolution: Fixed > Client side topology submission hook. > - > > Key: STORM-1381 > URL: https://issues.apache.org/jira/browse/STORM-1381 > Project: Apache Storm > Issue Type: New Feature > Components: storm-core >Affects Versions: 0.11.0 >Reporter: Parth Brahmbhatt >Assignee: Parth Brahmbhatt >Priority: Trivial > Fix For: 0.11.0 > > > A client side hook is suppose to be invoked when a user submits the topology > using TopologySubmitter. We already have nimbus side hook for all the > topology actions however those are good if users don't want to actually > inspect the topology being submitted or the classes that makes up the > topology (spouts and bolts) as on nimbus side these classes are not available > in class path. > As a concrete example, in hortonworks we wanted to integrate storm with atlas > to provide complete lineage of data even when it passes through a storm > topology. Atlas needed to actually look inside the topology components (i.e. > kafka spout to figure out what topic the data is being pulled from, or hbase > bolt to figure out which cluster and what table data is being pushed into.) > to give a meaningful lineage. We originally proposed that they use the server > side hook but with that they had to download the user uploaded jar and add it > to the class path dynamically or spin a new jvn whose output will then be > read by the atlas integration hook. > The client side hook is suppose to make it easy when the topology itself > needs to be examined. We are using this in our internal repo for atlas > integration. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1395) Move JUnit dependency to top-level pom
[ https://issues.apache.org/jira/browse/STORM-1395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1395. - Resolution: Fixed Fix Version/s: 0.11.0 > Move JUnit dependency to top-level pom > -- > > Key: STORM-1395 > URL: https://issues.apache.org/jira/browse/STORM-1395 > Project: Apache Storm > Issue Type: Bug >Affects Versions: 0.11.0 >Reporter: P. Taylor Goetz >Assignee: P. Taylor Goetz > Fix For: 0.11.0 > > > The switch to profiles for running different types of tests requires that > certain submodules explicitly include the JUnit dependency. Moving it to the > top level pom will allow all submodules to inherit it. > Background: the build was failing in my environment because storm-redis did > not have the dependency. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1372) Update BlobStore Documentation - Follow up STORM-876
[ https://issues.apache.org/jira/browse/STORM-1372?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1372. - Resolution: Fixed Fix Version/s: 0.11.0 > Update BlobStore Documentation - Follow up STORM-876 > > > Key: STORM-1372 > URL: https://issues.apache.org/jira/browse/STORM-1372 > Project: Apache Storm > Issue Type: Story >Reporter: Sanket Reddy >Priority: Minor > Fix For: 0.11.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1376) ZK Becoming deadlocked with zookeeper_state_factory
[ https://issues.apache.org/jira/browse/STORM-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1376: Assignee: Sanket Reddy (was: Kyle Nusbaum) > ZK Becoming deadlocked with zookeeper_state_factory > --- > > Key: STORM-1376 > URL: https://issues.apache.org/jira/browse/STORM-1376 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Affects Versions: 0.11.0 >Reporter: Daniel Schonfeld >Assignee: Sanket Reddy >Priority: Blocker > Fix For: 0.11.0 > > > Since the introduction of blobstore and pacemaker we've noticed that when > using nimbus with the new zookeeper_state_factory backing cluster state > module, some of our ZK nodes become unresponsive and show and increasing > amounts of outstanding requests (STAT 4-letter command). > Terminating storm supervisors and nimbus usually gets zookeeper to realize > after a few minutes those connections are dead and to become responsive > again. In some extreme cases we have to kill that ZK nodes and bring it back > up. > Our topologies ran across ~10 supervisor nodes with each having about > ~400-500 executors. > I mention the amount of executors cause I am not sure if someone made each > executor by mistake start sending heartbeats instead of each worker and that > might possibly be the reason for this slow down. > Final note. If someone can jot a few ideas of why this might be happening > i'd be more than happy to dig further in the storm code and submit a PR > myself. But I need some hint or direction of where to go with this... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1376) ZK Becoming deadlocked with zookeeper_state_factory
[ https://issues.apache.org/jira/browse/STORM-1376?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1376. - Resolution: Fixed Fix Version/s: 0.11.0 > ZK Becoming deadlocked with zookeeper_state_factory > --- > > Key: STORM-1376 > URL: https://issues.apache.org/jira/browse/STORM-1376 > Project: Apache Storm > Issue Type: Bug > Components: storm-core >Affects Versions: 0.11.0 >Reporter: Daniel Schonfeld >Assignee: Sanket Reddy >Priority: Blocker > Fix For: 0.11.0 > > > Since the introduction of blobstore and pacemaker we've noticed that when > using nimbus with the new zookeeper_state_factory backing cluster state > module, some of our ZK nodes become unresponsive and show and increasing > amounts of outstanding requests (STAT 4-letter command). > Terminating storm supervisors and nimbus usually gets zookeeper to realize > after a few minutes those connections are dead and to become responsive > again. In some extreme cases we have to kill that ZK nodes and bring it back > up. > Our topologies ran across ~10 supervisor nodes with each having about > ~400-500 executors. > I mention the amount of executors cause I am not sure if someone made each > executor by mistake start sending heartbeats instead of each worker and that > might possibly be the reason for this slow down. > Final note. If someone can jot a few ideas of why this might be happening > i'd be more than happy to dig further in the storm code and submit a PR > myself. But I need some hint or direction of where to go with this... -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1388) Fix url and email links in README file
[ https://issues.apache.org/jira/browse/STORM-1388?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1388. - Resolution: Fixed Fix Version/s: 0.11.0 > Fix url and email links in README file > -- > > Key: STORM-1388 > URL: https://issues.apache.org/jira/browse/STORM-1388 > Project: Apache Storm > Issue Type: Bug > Components: documentation >Reporter: Xin Wang >Assignee: Xin Wang >Priority: Minor > Fix For: 0.11.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1397) Merge conflict from Pacemaker merge
Kyle Nusbaum created STORM-1397: --- Summary: Merge conflict from Pacemaker merge Key: STORM-1397 URL: https://issues.apache.org/jira/browse/STORM-1397 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum When Pacemaker was merged, some calls got missed in nimbus.clj, supervisor.clj, and worker.clj. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1397) Merge conflict from Pacemaker merge
[ https://issues.apache.org/jira/browse/STORM-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1397: Description: When Pacemaker was merged, some calls got missed in nimbus.clj, supervisor.clj, worker.clj and executor.clj. (was: When Pacemaker was merged, some calls got missed in nimbus.clj, supervisor.clj, and worker.clj.) > Merge conflict from Pacemaker merge > --- > > Key: STORM-1397 > URL: https://issues.apache.org/jira/browse/STORM-1397 > Project: Apache Storm > Issue Type: Bug >Reporter: Kyle Nusbaum > > When Pacemaker was merged, some calls got missed in nimbus.clj, > supervisor.clj, worker.clj and executor.clj. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1397) Merge conflict from Pacemaker merge
[ https://issues.apache.org/jira/browse/STORM-1397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1397: Description: When Pacemaker was merged, some calls got missed in nimbus.clj, supervisor.clj, worker.clj and executor.clj. This merge error prevents nimbus from spawning separate read/write zookeeper clients since the ClusterState ends up with daemonType == DaemonType.UNKNOWN. was:When Pacemaker was merged, some calls got missed in nimbus.clj, supervisor.clj, worker.clj and executor.clj. > Merge conflict from Pacemaker merge > --- > > Key: STORM-1397 > URL: https://issues.apache.org/jira/browse/STORM-1397 > Project: Apache Storm > Issue Type: Bug >Reporter: Kyle Nusbaum > > When Pacemaker was merged, some calls got missed in nimbus.clj, > supervisor.clj, worker.clj and executor.clj. > This merge error prevents nimbus from spawning separate read/write zookeeper > clients since the ClusterState ends up with daemonType == DaemonType.UNKNOWN. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1359) Change kryo links from google code to github
[ https://issues.apache.org/jira/browse/STORM-1359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1359. - Resolution: Fixed Fix Version/s: 0.11.0 > Change kryo links from google code to github > > > Key: STORM-1359 > URL: https://issues.apache.org/jira/browse/STORM-1359 > Project: Apache Storm > Issue Type: Bug > Components: documentation >Reporter: Xin Wang >Assignee: Xin Wang >Priority: Minor > Fix For: 0.11.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-1385) Divide by zero exception in stats.clj
[ https://issues.apache.org/jira/browse/STORM-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-1385. - Resolution: Fixed Fix Version/s: 0.11.0 > Divide by zero exception in stats.clj > - > > Key: STORM-1385 > URL: https://issues.apache.org/jira/browse/STORM-1385 > Project: Apache Storm > Issue Type: Bug >Reporter: Sanket Reddy >Assignee: Sanket Reddy >Priority: Minor > Fix For: 0.11.0 > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1340) Use Travis-CI build matrix to improve test execution times
Kyle Nusbaum created STORM-1340: --- Summary: Use Travis-CI build matrix to improve test execution times Key: STORM-1340 URL: https://issues.apache.org/jira/browse/STORM-1340 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1340) Use Travis-CI build matrix to improve test execution times
[ https://issues.apache.org/jira/browse/STORM-1340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1340: Description: Travis-CI provides a 'build-matrix' that we can use to parallelize the unit tests of various components in storm. It also provides a caching mechanism that can reduce the overhead of starting up a test container. > Use Travis-CI build matrix to improve test execution times > -- > > Key: STORM-1340 > URL: https://issues.apache.org/jira/browse/STORM-1340 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > > Travis-CI provides a 'build-matrix' that we can use to parallelize the unit > tests of various components in storm. > It also provides a caching mechanism that can reduce the overhead of starting > up a test container. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-885) Heartbeat Server (Pacemaker)
[ https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-885. Resolution: Fixed Fix Version/s: 0.11.0 > Heartbeat Server (Pacemaker) > > > Key: STORM-885 > URL: https://issues.apache.org/jira/browse/STORM-885 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Reporter: Robert Joseph Evans >Assignee: Kyle Nusbaum > Fix For: 0.11.0 > > > Large highly connected topologies and large clusters write a lot of data into > ZooKeeper. The heartbeats, that make up the majority of this data, do not > need to be persisted to disk. Pacemaker is intended to be a secure > replacement for storing the heartbeats without changing anything within the > heartbeats. In the future as more metrics are added in, we may want to look > into switching it over to look more like Heron, where a metrics server is > running for each node/topology. And can be used to aggregate/per-aggregate > them in a more scalable manor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (STORM-885) Heartbeat Server (Pacemaker)
[ https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011663#comment-15011663 ] Kyle Nusbaum edited comment on STORM-885 at 11/18/15 6:52 PM: -- [~LongdaFeng] I disagree with your assessment of this patch. Addressing each of your numbered points: 1. I'm not sure what you mean by this. Will you please clarify? 2. Pacemaker may become a bottleneck in the system, yes. But in our internal testing, Nimbus itself becomes a bottleneck before Pacemaker. Furthermore, it is *not* very hard to do extension. I have a prototype here for Pacemaker HA that only adds a small number of lines of code, which I am planning on pushing out in the relatively near future. 3. The logic looks fairly straight-forward to me, and the pacemaker server file itself is only 239 lines long, with nearly 100 of those being the license, imports, and statistics helpers. The client is only 125 lines long. Pacemaker is also an optional component, and it's ready now. I see the benefit of a 'TopologyMaster', but I don't see the benefit in blocking the merge of Pacemaker at this point. was (Author: knusbaum): [~LongdaFeng] I disagree with your assessment of this patch. 1. I'm not sure what you mean by this. Will you please clarify? 2. Pacemaker may become a bottleneck in the system, yes. But in our internal testing, Nimbus itself becomes a bottleneck before Pacemaker. Furthermore, it is *not* very hard to do extension. I have a prototype here for Pacemaker HA that only adds a small number of lines of code, which I am planning on pushing out in the relatively near future. 3. The logic looks fairly straight-forward to me, and the pacemaker server file itself is only 239 lines long, with nearly 100 of those being the license, imports, and statistics helpers. The client is only 125 lines long. Pacemaker is also an optional component, and it's ready now. I see the benefit of a 'TopologyMaster', but I don't see the benefit in blocking the merge of Pacemaker at this point. > Heartbeat Server (Pacemaker) > > > Key: STORM-885 > URL: https://issues.apache.org/jira/browse/STORM-885 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Reporter: Robert Joseph Evans >Assignee: Kyle Nusbaum > > Large highly connected topologies and large clusters write a lot of data into > ZooKeeper. The heartbeats, that make up the majority of this data, do not > need to be persisted to disk. Pacemaker is intended to be a secure > replacement for storing the heartbeats without changing anything within the > heartbeats. In the future as more metrics are added in, we may want to look > into switching it over to look more like Heron, where a metrics server is > running for each node/topology. And can be used to aggregate/per-aggregate > them in a more scalable manor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (STORM-885) Heartbeat Server (Pacemaker)
[ https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011663#comment-15011663 ] Kyle Nusbaum edited comment on STORM-885 at 11/18/15 7:05 PM: -- [~LongdaFeng] I disagree with your assessment of this patch. Addressing each of your numbered points: 1. I'm not sure what you mean by this. Will you please clarify? 2. Pacemaker may become a bottleneck in the system, yes. But in our internal testing, Nimbus itself becomes a bottleneck before Pacemaker. Furthermore, it is *not* very hard to do extension. I have a prototype here for Pacemaker HA that only adds a small number of lines of code, which I am planning on pushing out in the relatively near future. 3. The logic looks fairly straight-forward to me, and the pacemaker server file itself is only 239 lines long, with nearly 100 of those being the license, imports, and statistics helpers. The client is only 125 lines long. Pacemaker is also an optional component, and it's ready now. A TopologyMaster also does not reduce the total amount of data written to ZooKeeper, whereas Pacemaker does. was (Author: knusbaum): [~LongdaFeng] I disagree with your assessment of this patch. Addressing each of your numbered points: 1. I'm not sure what you mean by this. Will you please clarify? 2. Pacemaker may become a bottleneck in the system, yes. But in our internal testing, Nimbus itself becomes a bottleneck before Pacemaker. Furthermore, it is *not* very hard to do extension. I have a prototype here for Pacemaker HA that only adds a small number of lines of code, which I am planning on pushing out in the relatively near future. 3. The logic looks fairly straight-forward to me, and the pacemaker server file itself is only 239 lines long, with nearly 100 of those being the license, imports, and statistics helpers. The client is only 125 lines long. Pacemaker is also an optional component, and it's ready now. I see the benefit of a 'TopologyMaster', but I don't see the benefit in blocking the merge of Pacemaker at this point. > Heartbeat Server (Pacemaker) > > > Key: STORM-885 > URL: https://issues.apache.org/jira/browse/STORM-885 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Reporter: Robert Joseph Evans >Assignee: Kyle Nusbaum > > Large highly connected topologies and large clusters write a lot of data into > ZooKeeper. The heartbeats, that make up the majority of this data, do not > need to be persisted to disk. Pacemaker is intended to be a secure > replacement for storing the heartbeats without changing anything within the > heartbeats. In the future as more metrics are added in, we may want to look > into switching it over to look more like Heron, where a metrics server is > running for each node/topology. And can be used to aggregate/per-aggregate > them in a more scalable manor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (STORM-885) Heartbeat Server (Pacemaker)
[ https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011663#comment-15011663 ] Kyle Nusbaum edited comment on STORM-885 at 11/18/15 6:51 PM: -- [~LongdaFeng] I disagree with your assessment of this patch. 1. I'm not sure what you mean by this. Will you please clarify? 2. Pacemaker may become a bottleneck in the system, yes. But in our internal testing, Nimbus itself becomes a bottleneck before Pacemaker. Furthermore, it is *not* very hard to do extension. I have a prototype here for Pacemaker HA that only adds a small number of lines of code, which I am planning on pushing out in the relatively near future. 3. The logic looks fairly straight-forward to me, and the pacemaker server file itself is only 239 lines long, with nearly 100 of those being the license, imports, and statistics helpers. The client is only 125 lines long. Pacemaker is also an optional component, and it's ready now. I see the benefit of a 'TopologyMaster', but I don't see the benefit in blocking the merge of Pacemaker at this point. was (Author: knusbaum): I disagree with your assessment of this patch. 1. I'm not sure what you mean by this. Will you please clarify? 2. Pacemaker may become a bottleneck in the system, yes. But in our internal testing, Nimbus itself becomes a bottleneck before Pacemaker. Furthermore, it is *not* very hard to do extension. I have a prototype here for Pacemaker HA that only adds a small number of lines of code, which I am planning on pushing out in the relatively near future. 3. The logic looks fairly straight-forward to me, and the pacemaker server file itself is only 239 lines long, with nearly 100 of those being the license, imports, and statistics helpers. The client is only 125 lines long. Pacemaker is also an optional component, and it's ready now. I see the benefit of a 'TopologyMaster', but I don't see the benefit in blocking the merge of Pacemaker at this point. > Heartbeat Server (Pacemaker) > > > Key: STORM-885 > URL: https://issues.apache.org/jira/browse/STORM-885 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Reporter: Robert Joseph Evans >Assignee: Kyle Nusbaum > > Large highly connected topologies and large clusters write a lot of data into > ZooKeeper. The heartbeats, that make up the majority of this data, do not > need to be persisted to disk. Pacemaker is intended to be a secure > replacement for storing the heartbeats without changing anything within the > heartbeats. In the future as more metrics are added in, we may want to look > into switching it over to look more like Heron, where a metrics server is > running for each node/topology. And can be used to aggregate/per-aggregate > them in a more scalable manor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-885) Heartbeat Server (Pacemaker)
[ https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15011663#comment-15011663 ] Kyle Nusbaum commented on STORM-885: I disagree with your assessment of this patch. 1. I'm not sure what you mean by this. Will you please clarify? 2. Pacemaker may become a bottleneck in the system, yes. But in our internal testing, Nimbus itself becomes a bottleneck before Pacemaker. Furthermore, it is *not* very hard to do extension. I have a prototype here for Pacemaker HA that only adds a small number of lines of code, which I am planning on pushing out in the relatively near future. 3. The logic looks fairly straight-forward to me, and the pacemaker server file itself is only 239 lines long, with nearly 100 of those being the license, imports, and statistics helpers. The client is only 125 lines long. Pacemaker is also an optional component, and it's ready now. I see the benefit of a 'TopologyMaster', but I don't see the benefit in blocking the merge of Pacemaker at this point. > Heartbeat Server (Pacemaker) > > > Key: STORM-885 > URL: https://issues.apache.org/jira/browse/STORM-885 > Project: Apache Storm > Issue Type: Improvement > Components: storm-core >Reporter: Robert Joseph Evans >Assignee: Kyle Nusbaum > > Large highly connected topologies and large clusters write a lot of data into > ZooKeeper. The heartbeats, that make up the majority of this data, do not > need to be persisted to disk. Pacemaker is intended to be a secure > replacement for storing the heartbeats without changing anything within the > heartbeats. In the future as more metrics are added in, we may want to look > into switching it over to look more like Heron, where a metrics server is > running for each node/topology. And can be used to aggregate/per-aggregate > them in a more scalable manor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (STORM-1196) Upgrade to thrift 0.9.3
[ https://issues.apache.org/jira/browse/STORM-1196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reassigned STORM-1196: --- Assignee: Kyle Nusbaum > Upgrade to thrift 0.9.3 > --- > > Key: STORM-1196 > URL: https://issues.apache.org/jira/browse/STORM-1196 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum > > Thrift 0.9.3 has been released. > Upgrade allows us to get rid of the pesky date changes in all the thrift > files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-400) Thrift upgrade to 0.9.1
[ https://issues.apache.org/jira/browse/STORM-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-400. -- > Thrift upgrade to 0.9.1 > --- > > Key: STORM-400 > URL: https://issues.apache.org/jira/browse/STORM-400 > Project: Apache Storm > Issue Type: Dependency upgrade > Components: storm-core >Reporter: Kishor Patil >Assignee: Kishor Patil > Labels: master, thrift > Fix For: 0.10.0 > > > Upgrade Thrift to 0.9.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (STORM-400) Thrift upgrade to 0.9.1
[ https://issues.apache.org/jira/browse/STORM-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reopened STORM-400: > Thrift upgrade to 0.9.1 > --- > > Key: STORM-400 > URL: https://issues.apache.org/jira/browse/STORM-400 > Project: Apache Storm > Issue Type: Dependency upgrade > Components: storm-core >Reporter: Kishor Patil >Assignee: Kishor Patil > Labels: master, thrift > Fix For: 0.10.0 > > > Upgrade Thrift to 0.9.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (STORM-400) Thrift upgrade to 0.9.1
[ https://issues.apache.org/jira/browse/STORM-400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum resolved STORM-400. Resolution: Fixed > Thrift upgrade to 0.9.1 > --- > > Key: STORM-400 > URL: https://issues.apache.org/jira/browse/STORM-400 > Project: Apache Storm > Issue Type: Dependency upgrade > Components: storm-core >Reporter: Kishor Patil >Assignee: Kishor Patil > Labels: master, thrift > Fix For: 0.10.0 > > > Upgrade Thrift to 0.9.1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1197) Migrate Travis-CI to container-based builds.
Kyle Nusbaum created STORM-1197: --- Summary: Migrate Travis-CI to container-based builds. Key: STORM-1197 URL: https://issues.apache.org/jira/browse/STORM-1197 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum http://docs.travis-ci.com/user/migrating-from-legacy/?utm_source=legacy-notice_medium=banner_campaign=legacy-upgrade -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-1197) Migrate Travis-CI to container-based builds.
[ https://issues.apache.org/jira/browse/STORM-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-1197: Description: http://docs.travis-ci.com/user/migrating-from-legacy/ (was: http://docs.travis-ci.com/user/migrating-from-legacy/?utm_source=legacy-notice_medium=banner_campaign=legacy-upgrade) > Migrate Travis-CI to container-based builds. > > > Key: STORM-1197 > URL: https://issues.apache.org/jira/browse/STORM-1197 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum > > http://docs.travis-ci.com/user/migrating-from-legacy/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1196) Upgrade to thrift 0.9.3
Kyle Nusbaum created STORM-1196: --- Summary: Upgrade to thrift 0.9.3 Key: STORM-1196 URL: https://issues.apache.org/jira/browse/STORM-1196 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Thrift 0.9.3 has been released. Upgrade allows us to get rid of the pesky date changes in all the thrift files. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (STORM-1197) Migrate Travis-CI to container-based builds.
[ https://issues.apache.org/jira/browse/STORM-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-1197. --- Resolution: Invalid > Migrate Travis-CI to container-based builds. > > > Key: STORM-1197 > URL: https://issues.apache.org/jira/browse/STORM-1197 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum > > http://docs.travis-ci.com/user/migrating-from-legacy/ -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1124) 'schema' in ui core should be 'scheme'
Kyle Nusbaum created STORM-1124: --- Summary: 'schema' in ui core should be 'scheme' Key: STORM-1124 URL: https://issues.apache.org/jira/browse/STORM-1124 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-1059) Upgrade Storm to use Clojure 1.7.0
Kyle Nusbaum created STORM-1059: --- Summary: Upgrade Storm to use Clojure 1.7.0 Key: STORM-1059 URL: https://issues.apache.org/jira/browse/STORM-1059 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (STORM-1059) Upgrade Storm to use Clojure 1.7.0
[ https://issues.apache.org/jira/browse/STORM-1059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reassigned STORM-1059: --- Assignee: Kyle Nusbaum > Upgrade Storm to use Clojure 1.7.0 > -- > > Key: STORM-1059 > URL: https://issues.apache.org/jira/browse/STORM-1059 > Project: Apache Storm > Issue Type: Improvement >Reporter: Kyle Nusbaum >Assignee: Kyle Nusbaum >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-885) Heartbeat Server (Pacemaker)
[ https://issues.apache.org/jira/browse/STORM-885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14601537#comment-14601537 ] Kyle Nusbaum commented on STORM-885: [~kabhwan] Yes, pacemaker is mostly implemented. Planning on creating a pull request soon. Heartbeat Server (Pacemaker) Key: STORM-885 URL: https://issues.apache.org/jira/browse/STORM-885 Project: Apache Storm Issue Type: Improvement Reporter: Robert Joseph Evans Assignee: Kyle Nusbaum Large highly connected topologies and large clusters write a lot of data into ZooKeeper. The heartbeats, that make up the majority of this data, do not need to be persisted to disk. Pacemaker is intended to be a secure replacement for storing the heartbeats without changing anything within the heartbeats. In the future as more metrics are added in, we may want to look into switching it over to look more like Heron, where a metrics server is running for each node/topology. And can be used to aggregate/per-aggregate them in a more scalable manor. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (STORM-856) If nimbus goes down with a delayed kill topo cmd, it never comes up
[
https://issues.apache.org/jira/browse/STORM-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577902#comment-14577902
]
Kyle Nusbaum edited comment on STORM-856 at 6/8/15 9:57 PM:
Reproducable always on 0.10 branch (master). 0.9.x-branch is OK.
{code}
2015-06-08T16:55:17.162-0500 b.s.d.nimbus [INFO] Delaying event :remove for
secs for test-1-1433800457
2015-06-08T16:55:17.166-0500 b.s.d.nimbus [ERROR] Error on initialization of
server service-handler
java.lang.NullPointerException: null
at clojure.lang.Numbers.ops(Numbers.java:961) ~[clojure-1.6.0.jar:na]
at clojure.lang.Numbers.multiply(Numbers.java:146)
~[clojure-1.6.0.jar:na]
at clojure.lang.Numbers.multiply(Numbers.java:3659)
~[clojure-1.6.0.jar:na]
at backtype.storm.util$secs_to_millis_long.invoke(util.clj:243)
~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at backtype.storm.timer$schedule.doInvoke(timer.clj:92)
~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at clojure.lang.RestFn.invoke(RestFn.java:445) [clojure-1.6.0.jar:na]
at backtype.storm.daemon.nimbus$delay_event.invoke(nimbus.clj:242)
~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at
backtype.storm.daemon.nimbus$state_transitions$fn__8535.invoke(nimbus.clj:173)
~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at clojure.lang.AFn.applyToHelper(AFn.java:152) [clojure-1.6.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.6.0.jar:na]
at clojure.core$apply.invoke(core.clj:624) ~[clojure-1.6.0.jar:na]
at backtype.storm.daemon.nimbus$transition_BANG_.invoke(nimbus.clj:225)
~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at backtype.storm.daemon.nimbus$transition_BANG_.invoke(nimbus.clj:196)
~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at
backtype.storm.daemon.nimbus$fn__9152$exec_fn__1666__auto9153.invoke(nimbus.clj:1014)
~[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at clojure.lang.AFn.applyToHelper(AFn.java:156) [clojure-1.6.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.6.0.jar:na]
at clojure.core$apply.invoke(core.clj:624) ~[clojure-1.6.0.jar:na]
at
backtype.storm.daemon.nimbus$fn__9152$service_handler__9270.doInvoke(nimbus.clj:1006)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at clojure.lang.RestFn.invoke(RestFn.java:421) [clojure-1.6.0.jar:na]
at
backtype.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:1358)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at backtype.storm.daemon.nimbus$_launch.invoke(nimbus.clj:1389)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at backtype.storm.daemon.nimbus$_main.invoke(nimbus.clj:1411)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at clojure.lang.AFn.applyToHelper(AFn.java:152) [clojure-1.6.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.6.0.jar:na]
at backtype.storm.daemon.nimbus.main(Unknown Source)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
2015-06-08T16:55:17.170-0500 b.s.util [ERROR] Halting process: (Error on
initialization)
java.lang.RuntimeException: (Error on initialization)
at backtype.storm.util$exit_process_BANG_.doInvoke(util.clj:332)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at clojure.lang.RestFn.invoke(RestFn.java:423) [clojure-1.6.0.jar:na]
at
backtype.storm.daemon.nimbus$fn__9152$service_handler__9270.doInvoke(nimbus.clj:1006)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at clojure.lang.RestFn.invoke(RestFn.java:421) [clojure-1.6.0.jar:na]
at
backtype.storm.daemon.nimbus$launch_server_BANG_.invoke(nimbus.clj:1358)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at backtype.storm.daemon.nimbus$_launch.invoke(nimbus.clj:1389)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at backtype.storm.daemon.nimbus$_main.invoke(nimbus.clj:1411)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
at clojure.lang.AFn.applyToHelper(AFn.java:152) [clojure-1.6.0.jar:na]
at clojure.lang.AFn.applyTo(AFn.java:144) [clojure-1.6.0.jar:na]
at backtype.storm.daemon.nimbus.main(Unknown Source)
[storm-core-0.11.0-SNAPSHOT.jar:0.11.0-SNAPSHOT]
[1] + 21116 exit 13bin/storm nimbus
{code}
was (Author: knusbaum):
Reproducable always on 0.10 branch (master). 0.9.x-branch is OK.
If nimbus goes down with a delayed kill topo cmd, it never comes up
---
Key: STORM-856
URL: https://issues.apache.org/jira/browse/STORM-856
Project: Apache Storm
Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Derek Dagit
1. Describe observed behavior.
nimbus is in a crash loop
2. What is the expected behavior?
nimbus
[jira] [Comment Edited] (STORM-856) If nimbus goes down with a delayed kill topo cmd, it never comes up
[ https://issues.apache.org/jira/browse/STORM-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577902#comment-14577902 ] Kyle Nusbaum edited comment on STORM-856 at 6/8/15 9:47 PM: Reproducable always on 0.10 branch (master). 0.9.x-branch is OK. was (Author: knusbaum): Reproducable always on 0.10 branch. 0.9 branch is OK. If nimbus goes down with a delayed kill topo cmd, it never comes up --- Key: STORM-856 URL: https://issues.apache.org/jira/browse/STORM-856 Project: Apache Storm Issue Type: Bug Affects Versions: 0.10.0 Reporter: Derek Dagit 1. Describe observed behavior. nimbus is in a crash loop 2. What is the expected behavior? nimbus stays up 3. Outline the steps to reproduce the problem. launch a topo kill topo with `storm kill $topo` (no wait argument, so it defaults to 30s) Immediately restart nimbus -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-856) If nimbus goes down with a delayed kill topo cmd, it never comes up
[ https://issues.apache.org/jira/browse/STORM-856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14577902#comment-14577902 ] Kyle Nusbaum commented on STORM-856: Reproducable always on 0.10 branch. 0.9 branch is OK. If nimbus goes down with a delayed kill topo cmd, it never comes up --- Key: STORM-856 URL: https://issues.apache.org/jira/browse/STORM-856 Project: Apache Storm Issue Type: Bug Affects Versions: 0.10.0 Reporter: Derek Dagit 1. Describe observed behavior. nimbus is in a crash loop 2. What is the expected behavior? nimbus stays up 3. Outline the steps to reproduce the problem. launch a topo kill topo with `storm kill $topo` (no wait argument, so it defaults to 30s) Immediately restart nimbus -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-816) maven-gpg-plugin does not work with gpg 2.1
Kyle Nusbaum created STORM-816: -- Summary: maven-gpg-plugin does not work with gpg 2.1 Key: STORM-816 URL: https://issues.apache.org/jira/browse/STORM-816 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Priority: Minor GPG 2.1 changed some security things involving password entry, and it looks like our version of the plugin doesn't work with it. We should upgrade. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-801) Add Travis CI badge to README
Kyle Nusbaum created STORM-801: -- Summary: Add Travis CI badge to README Key: STORM-801 URL: https://issues.apache.org/jira/browse/STORM-801 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Priority: Trivial Add the Travis CI 'build status' badge to the readme for github. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-787) test-ns should announce test failures with 'BUILD FAILURE'
[ https://issues.apache.org/jira/browse/STORM-787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-787: --- Priority: Trivial (was: Major) test-ns should announce test failures with 'BUILD FAILURE' -- Key: STORM-787 URL: https://issues.apache.org/jira/browse/STORM-787 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Priority: Trivial Right now, test-ns.py exits with return code 0 and prints 'BUILD SUCCESS' even when tests fail. You have to look at the test tuple to tell if the tests actually passed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-787) test-ns should announce test failures with 'BUILD FAILURE'
Kyle Nusbaum created STORM-787: -- Summary: test-ns should announce test failures with 'BUILD FAILURE' Key: STORM-787 URL: https://issues.apache.org/jira/browse/STORM-787 Project: Apache Storm Issue Type: Bug Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Right now, test-ns.py exits with return code 0 and prints 'BUILD SUCCESS' even when tests fail. You have to look at the test tuple to tell if the tests actually passed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (STORM-710) bin/storm command list out all the classes in the output for a command
[ https://issues.apache.org/jira/browse/STORM-710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14371859#comment-14371859 ] Kyle Nusbaum commented on STORM-710: I would suggest the traditional -v flag. bin/storm command list out all the classes in the output for a command -- Key: STORM-710 URL: https://issues.apache.org/jira/browse/STORM-710 Project: Apache Storm Issue Type: Bug Reporter: Sriharsha Chintalapani Assignee: Oleg Ostashchuk Priority: Minor Labels: newbie when running bin/storm list command or any other command it outputs Running: /Library/Java/JavaVirtualMachines/jdk1.7.0_51.jdk/Contents/Home/bin/java -client -Dstorm.options= -Dstorm.home=/Users/schintalapani/build/apache-storm-0.10.0-SNAPSHOT -Dstorm.log.dir=/Users/schintalapani/build/apache-storm-0.10.0-SNAPSHOT/logs -Djava.library.path=/usr/local/lib:/opt/local/lib:/usr/lib -Dstorm.conf.file= -cp
[jira] [Closed] (STORM-625) Netty context never forgets about a Client.
[ https://issues.apache.org/jira/browse/STORM-625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum closed STORM-625. -- Resolution: Fixed Netty context never forgets about a Client. --- Key: STORM-625 URL: https://issues.apache.org/jira/browse/STORM-625 Project: Apache Storm Issue Type: Bug Reporter: Robert Joseph Evans Assignee: Robert Joseph Evans Labels: newbie The Netty Context (https://github.com/apache/storm/blob/master/storm-core/src/jvm/backtype/storm/messaging/netty/Context.java) keeps track of all connections so that it can close them when it is closed in term. But it never removes any of them from its list until it is terminated. If workers are rescheduled to new locations this can cause the old connections to be leaked. We should have the Connections remove themselves from the list when they are closed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (STORM-696) As a dev I would like to be able to run unit tests in a particular namespace.
Kyle Nusbaum created STORM-696: -- Summary: As a dev I would like to be able to run unit tests in a particular namespace. Key: STORM-696 URL: https://issues.apache.org/jira/browse/STORM-696 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Priority: Minor Right now, we have to run the full test suite through maven. If we want to test particular namespaces, we must do so from a REPL. Some quick way to launch unit tests for a namespace is desirable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (STORM-696) Single Namespace Test Launching
[ https://issues.apache.org/jira/browse/STORM-696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum updated STORM-696: --- Summary: Single Namespace Test Launching (was: As a dev I would like to be able to run unit tests in a particular namespace.) Single Namespace Test Launching --- Key: STORM-696 URL: https://issues.apache.org/jira/browse/STORM-696 Project: Apache Storm Issue Type: Improvement Reporter: Kyle Nusbaum Assignee: Kyle Nusbaum Priority: Minor Right now, we have to run the full test suite through maven. If we want to test particular namespaces, we must do so from a REPL. Some quick way to launch unit tests for a namespace is desirable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (STORM-441) Remove bootstrap macro from Clojure codebase
[ https://issues.apache.org/jira/browse/STORM-441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kyle Nusbaum reassigned STORM-441: -- Assignee: Kyle Nusbaum Remove bootstrap macro from Clojure codebase Key: STORM-441 URL: https://issues.apache.org/jira/browse/STORM-441 Project: Apache Storm Issue Type: Improvement Reporter: Dane Hammer Assignee: Kyle Nusbaum Priority: Trivial The bootstrap macro in backtype.storm.bootstrap is purely a convenience for importing/using/requiring a large number of dependencies, but it's not used for anything else. It removes those imports/uses/requires from the namespace form, making it harder to track down where a definition is coming from, which defeats some IDE tools. I propose removing it entirely, making the Clojure part of the codebase more readable and updated to current conventions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
