Re: [DISCUSS] Call for PMC Members and Contributors

also work on doing the mechanical work of a release >> (already knowing the quirks with asf infrastructure). >> >> Gruß >> Richard >> >> Am Mittwoch, dem 22.02.2023 um 18:02 -0500 schrieb P. Taylor Goetz: >>> Quick update: >>> >>&

[DISCUSS] Call for PMC Members and Contributors

Quick update: The ASF Board voted to accept both our report and the resolution to change PMC Chair. Board feedback on our report was positive. The next step is to expand the the PMC/Committers group. Official votes will necessarily be private, but I see no reason nominations can’t be public.

Re: [DISCUSSION] Apache Storm and moving to the Attic

er/PMC? -Taylor > On Feb 2, 2023, at 12:06 AM, P. Taylor Goetz wrote: > > IMO, there are tow scenarios: > > 1. Storm is in dead. No one uses it anymore, and no one needs security > patches, etc. > 2. Storm is in maintenance mode. While new features may not be added, there >

Re: [DISCUSSION] Apache Storm and moving to the Attic

IMO, there are tow scenarios: 1. Storm is in dead. No one uses it anymore, and no one needs security patches, etc. 2. Storm is in maintenance mode. While new features may not be added, there are enough contributors left tp at least address any security concerns. Wether we pursue the attic o

[ANNOUNCE] Apache Storm 2.0.0 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 2.0.0. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[CVE-2018-8008] Apache Storm arbitrary file write vulnerability

/06/04/storm113-released.html P. Taylor Goetz

[CVE-2018-1332] Apache Storm user impersonation vulnerability

: http://storm.apache.org/2018/06/04/storm122-released.html http://storm.apache.org/2018/06/04/storm113-released.html P. Taylor Goetz

[ANNOUNCE] Apache Storm 1.1.3 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.1.3. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[ANNOUNCE] Apache Storm 1.2.2 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.2.2. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[ANNOUNCE] Apache Storm 1.2.1 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.2.1. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[ANNOUNCE] Apache Storm 1.2.1 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.2.1. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[ANNOUNCE] Apache Storm 1.2.0 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.2.0. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[ANNOUNCE] Apache Storm 1.1.2 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.1.2. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[ANNOUNCE] Apache Storm 1.0.6 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.0.6. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[CVE-2017-9799] Apache Storm Possible Code Execution As A Different User

Severity: High Vendor: The Apache Software Foundation Versions Affected: Apache Storm 1.0.0, 1.0.1, 1.0.2, 1.0.3 Apache Storm 1.1.0 Description: It was found that under some situations and configurations of storm it is theoretically possible for the owner of a topology to trick the supervisor

[ANNOUNCE] Apache Storm 1.1.1 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.1.1. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[ANNOUNCE] Apache Storm 1.0.4 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.0.4. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

Re: Storm topology freezes and does not process tuples from Kafka

> Supervisor log at the time of freeze looks like below > > 2017-07-12 14:38:46.712 o.a.s.d.supervisor [INFO] > d8958816-5bc8-449e-94e3-87ddbb2c3d02 still hasn't started There are two situations where you would see those messages: When a topology is first deployed, and when a worker has died

Re: Decreasing value of Complete Latency in Storm UI

Over how long a period do you see the complete latency decreasing? Does it stabilize at some point? It’s typical for a topology to start out slow in terms of latency, then speed up as the worker JVMs “warm up.” The warm up period can last several minutes. The complete latency metric should

Re: How about Flink compatibility mode in Storm

Hi Alexandre, I’m not aware of any effort to port the Flink API to Storm. The closest thing to what you are looking for in Storm is a new “Streams” API that closely resembles the Java 8 Streams API, allows usage of lambdas, etc. That API will likely become available in the upcoming Storm 2.0

Re: github source code gone?

Apache infra is working on it. This is a github issue not ASF. The cononical ASF repos are unaffected, so our source repo is safe. -Taylor > On Apr 28, 2017, at 5:46 PM, Michael Moss wrote: > > https://github.com/apache/storm > > "This repository is empty." > > Am I

Re: [ANNOUNCE] Apache Storm 1.1.0 Released

llowing coordinates: > groupId: org.apache.storm > <http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.storm%22> > artifactId: storm-core > version: 1.0.2 > == > Shouldn't the version rather be "1.1.0" instead of "1.0.2" ? >

Re: Stateful topology hangs

What version of Storm are you using? And which Kafka spout (i.e. storm-kafka or storm-kafka-client)? -Taylor > On Feb 22, 2017, at 4:32 PM, Abhishek Raj wrote: > > Hello, I am using storm's state management feature in a topology. > The topology has a kafkaspout and a

Re: Storm (Core) versus Storm (with Trident)

on because I'm making a basic benchmarks and I have the > opposite result. The Storm Core (Kafka) is about 2x and tries more than > Trident (Kafka). > I still do not understand what's happening. > > > -- > Thomas Cristanis > > 2017-01-25 19:21 GMT-03:00 P. Ta

Re: Storm (Core) versus Storm (with Trident)

It comes down to a tradeoff between throughput and latency (as well as how and what you do in your topologies), and tuning parameters. In the benchmarking I’ve done, Storm Core has lower latency and lower throughput, Trident has higher throughput (~2x) and higher latency (~2-3x). But again, it

Re: Zero Worker Bug in Topology on 1.0.2

What does the Storm UI show? Are you out of slots? -Taylor > On Nov 21, 2016, at 6:28 PM, Joaquin Menchaca wrote: > > I am not sure what is causing this, but it seems that after 70 days of > running the cluster, it will not be able to run a topology with any workers >

Re: Explicit Topology Parallelism Shape

Yes, to do what you want you would need to implement a custom scheduler. More details can be found here: http://storm.apache.org/releases/1.0.2/Storm-Scheduler.html -Taylor > On Nov 7, 2016, at 11:31 AM, Arthur Maciejewicz

Re: Storm over both Ethernet and Infiniband on the same cluster

In your storm.yaml configuration on each supervisor machine, if you set “storm.local.hostname” to the Infiniband IP that should do what you want. -Taylor > On Nov 4, 2016, at 2:06 PM, Muhammad Haseeb Javed <11besemja...@seecs.edu.pk> > wrote: > > That is definitely a plausible solution but

Re: Messages are not being delivered fast enough

You can safely ignore that message, as it only relates to delivery of metrics information (i.e. not topology data). It has since been set to a DEBUG level message, but that change isn’t in an official release yet. What it means is that the handler got more than one metrics message when it was

Re: Is topology.tick.tuple.freq.secs a topology-level or component-level setting?

Hi Matt, It depends on where/how you set it: storm.yaml—> cluster wide topology conf. —> topology wide getComponentConfiguration —> component specific If you want to control it at the component level, leave it out of the storm.yaml and topology configuration. -Taylor > On Oct 31, 2016, at

Re: WorkerHook deserialization problem

I was able to verify this to be a bug in how worker hooks work in local mode. In trying to see if this affects distributed mode as well, a found a more serious issue that prevents workers from shutting down gracefully (an thus preventing shutdown hooks from running):

Re: When to use MemoryMapState while performing a persistentAggregate in Trident?

Storm has support for a Redis-backed map state: https://github.com/apache/storm/blob/master/external/storm-redis/src/main/java/org/apache/storm/redis/trident/state/RedisMapState.java

Re: java.lang.OutOfMemoryError: GC overhead limit exceeded in split bolt

That topology doesn’t use reliable delivery, so there is nothing to throttle that spout without the `sleep()`. So the spout will emit as fast as it can, which is faster than the bolts in the topology can process them. Try reducing the sleep time to something smaller like 5. -Taylor > On Oct

Re: Messages are not being delivered fast enough warning

You can safely ignore that message, as it only relates to delivery of metrics information (i.e. not topology data). It has since been set to a DEBUG level message, but that change isn’t in an official release yet. What it means is that the handler got more than one metrics message when it was

Re: Nimbus setting java.class.path to include javaagent

ress=8100,suspend= > y -Dmonitoring.agent.properties=/opt/monitoring_agent/monitoring.properties > -Dapplication.name=storm.test.lodgingamenity.nimbus" > > > supervisor.childopts: "-Xmx256m -DjmxRegistryPort=1098 -DjmxServerPort=1099 > -javaagent:/opt/monitoring_agent/bobbr

Re: Storm Bug? Cannot connect to cluster

ticular tag... > >> On Thu, Oct 6, 2016 at 1:43 PM, P. Taylor Goetz <ptgo...@gmail.com> wrote: >> This is a bug. The `storm list` command implementation doesn’t understand >> the “nimbus.seeds” configuration setting. >> >> Can you try the following? >

Re: Storm 1.0.2 - when does Storm schedule additional workers?

Hi Dominik, Storm will not (currently) allocate additional workers to a topology unless you tell it to using the rebalance command. It will start out with the number of workers you specify with either Config.setNumWorkers() or the topology.workers configuration key. The

Re: Storm Bug? Cannot connect to cluster

mmand.list > ... > 2264 [main] INFO b.s.thrift - Connecting to Nimbus at localhost:6627 as user: > ... > Caused by: java.net.ConnectException: Connection refused > ... > > On Thu, Oct 6, 2016 at 12:15 PM, P. Taylor Goetz <ptgo...@gmail.com > <mailto:ptgo...@gmail.com>>

Re: Nimbus setting java.class.path to include javaagent

enity.worker.%ID% > <http://dapplication.name%3dstorm.test.lodgingamenity.worker.%25id%25/>" > > > From: P. Taylor Goetz [mailto:ptgo...@gmail.com] > Sent: Tuesday, October 4, 2016 9:46 AM > To: user@storm.apache.org > Subject: Re: Nimbus setting java.class.path to in

Re: Nimbus setting java.class.path to include javaagent

Can you post your values for nimbus.childopts, supervisor.childopts, and worker.childopts? Also what version of Storm are you on? -Taylor > On Sep 28, 2016, at 6:06 PM, Brad Rhodes wrote: > > I have a situation where we have a javaagent jar file. Someone has added a >

Re: What determines the topology.acker.executors parameter value?

Hi Dominik, For the case you describe, I don’t think you’d need to increase the number of ackers. But you may want to experiment with increasing it to see if it makes a difference in performance. -Taylor > On Oct 3, 2016, at 5:09 PM, Dominik Safaric wrote: > > Hi

Re: StormMqtt, MqttPublishFunction does not work on SSL

Can you provide some more information about how you configure SSL and the contents of your keystore and truststore files? Off the top of my head, it sounds like a certificate issue. -Taylor > On Sep 30, 2016, at 1:27 AM, Ye Minkg wrote: > > I was trying to use StormMqtt

[ANNOUNCE] Apache Storm 0.10.2 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 0.10.2. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project

Re: How will storm replay the tuple tree?

Hi Cheney, Replays happen at the spout level. So if there is a failure at any point in the tuple tree (the tuple tree being the anchored emits, unanchored emits don’t count), the original spout tuple will be replayed. So the replayed tuple will traverse the topology again, including unanchored

Re: Netty Configurations in 1.0.2 still valid?

I would recommend removing that from your configuration and allow the defaults to take effect. The defaults are sane, and should only be overridden when necessary. Here are the current defaults as of 1.0.2: storm.messaging.netty.server_worker_threads: 1

Re: doc confusion - nimbus.seed config

Yes. > On Sep 7, 2016, at 6:33 PM, Joaquin Menchaca wrote: > > The docs say: > > nimbus.seeds: ["111.222.333.44"] > > Does this mean we can: > nimbus.seeds: ["192.168.51.5","192.168.51.6"] > > Where each seed refers to a nimbus server? > > -- > >

Re: Storm UI generated logviewer links problematic

You can override DNS by setting storm.local.hostname in the config yaml on each node. Use the public IP/hostname and that should fix your problem. -Taylor > On Sep 6, 2016, at 2:47 PM, Joaquin Menchaca wrote: > > This is a big issue to how storm ui generates the URLs

Re: [SURVEY] What version of Storm are you using?

+1 Any and all feedback is always welcome. -Taylor > On Aug 17, 2016, at 9:22 PM, Jungtaek Lim wrote: > > off-topic: > Andrew and Erik, please feel free to post the issue to dev@ whenever you meet > the issue from streamparse, storm-mesos, and etc. which might be cause of

[SURVEY] What version of Storm are you using?

On the Storm developer list, there are a number of discussions regarding ending support for various older versions of Storm. In order to make an informed decision I’d like to get an idea of what versions the user community is actively using. I’d like to ask the user community to answer the

[ANNOUNCE] Apache Storm 1.0.2 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.0.2. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

Re: Storm unique strengths

There are a few things to keep in mind when evaluating Heron and Storm: First is performance. Twitter benchmarked Heron against a very old, pre-Apache version of Storm (back when the transport layer was based on 0mq), so their claims of performance improvements over Storm are likely

Re: Storm 1.0.0 upgrade Serialization issue

I'm okay with a quick turnaround release for this fix. We've got two valid reports of it, and more will follow quickly as users continue to upgrade. -Taylor > On May 11, 2016, at 11:00 PM, Jungtaek Lim wrote: > > KB, > > Submitted pull request:

[ANNOUNCE] Apache Storm 1.0.1 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.0.1. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

[ANNOUNCE] Apache Storm 0.10.1 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 0.10.1. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project

Re: Spout Questions

nextTuple(), ack(), and fail() are all called by the same thread. nextTuple() should be fast, so you probably only want to emit one or a handful of tuples. Emitting a huge number of tuples in the nextTuple() method is what’s causing your problem. -Taylor > On May 2, 2016, at 9:08 AM, Adrien

Re: thread safe output collector

The bolt passes work to be completed to the > thread(s), the thread(s) communicate back to the bolt the result via a shared > queue. Add in the concept of tick tuples to ensure your bolt checks for > completed work on a regular basis? > > Is there a better way to do th

Re: thread safe output collector

che/storm/task/OutputCollector.html> > is thread-safe and can be called at any time." > > > > From the docs for 0.9.6: > http://storm.apache.org/releases/0.9.6/Concepts.html#bolts > <http://storm.apache.org/releases/0.9.6/Concepts.html#bolts> > > On Thu,

Re: thread safe output collector

IIRC there was discussion about making it thread safe, but I don't believe it was implemented. -Taylor > On Apr 28, 2016, at 3:52 AM, Julien Nioche > wrote: > > Hi Stephen > > I asked the same question in February but did not get a reply > >

[ANNOUNCE] Apache Storm 1.0 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 1.0.0. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

Re: Next version of storm?

1.0 should be released this month. We will likely update 0.10.x as well. 1.0 is a big release, and we want to make sure it's solid. -Taylor > On Mar 11, 2016, at 10:47 PM, Tech Id wrote: > > Hi, > > We need to have the fix for: > STORM-1207: Added flux support

Re: New Concurrent modification exception's after storm 0.10.0

Hi Stephen, Can you provide a stack trace that indicates where this is occurring? -Taylor > On Mar 2, 2016, at 1:49 PM, Stephen Powis wrote: > > Hey! > > Did anything change between storm 0.9.5 and 0.10.0 regarding > ConcurrentModificationExceptions and how they are

Re: version 1.0?

Hi Maciek, We’re getting close. Hopefully in the next 2-3 weeks. If you’re interested in tracking progress, look at this JIRA ticket: https://issues.apache.org/jira/browse/STORM-1491 We will likely release once all those issues have been

Re: Storm + HDFS

Assuming you have git and maven installed: git clone g...@github.com:apache/storm.git cd storm git checkout -b 1.x origin/1.x-branch mvn install -DskipTests That third step checks out the 1.x-branch branch which is the base for the upcoming 1.0 release. You can then include the storm-hdfs

Re: Acking of anchor tuple list decreases throughput?

Interesting conversation. The back pressure mechanism in 1.0 should help. Do you guys have environments that you could test that in? Better yet, do you have code to share? -Taylor > On Jan 30, 2016, at 9:05 PM, hokiege...@gmail.com wrote: > > Hey Kashyap, > > Excellent points, especially

Re: Custom implementation of ISpoutWaitStrategy

If the call to a spout's `nextTuple()` method does not emit anything, Storm will call the `emptyEmit()` method with the number of times `nextTuple()` has consecutively failed to emit anything (`streak` will be reset to 0 if the spout emits something). -Taylor > On Dec 22, 2015, at 6:03 AM,

Storm at Hadoop Summit Europe

Voting for sessions for Hadoop Summit Europe ends today. There are a number of Storm-related sessions that have been proposed. Below are the ones I found, but I didn’t do an exhaustive search. If there are others I missed, feel free to tack onto this thread. If you have a chance, cast a vote

Re: Fan in problem: virtually all time spent in network I/O wait

Hi John, I think it *may* make sense, but without more details like code/sample data, it is hard to say. Whenever you use a fields grouping, key distribution can come into play and affect scaling. -Taylor > On Dec 9, 2015, at 9:31 PM, John Yost wrote: > > Hi Everyone,

Re: Multithreading in Bolt vs more Bolts: tradeoffs?

I think it depends on what exactly you're doing in those bolts. In other words, we'd need more detail, like code or pseudo code. Otherwise we are guessing. -Taylor > On Nov 23, 2015, at 4:37 PM, John Yost wrote: > > Hi Everyone, > > I have a large fan out in my

Re: Compiling and executing storm

That will happen if you don’t have a GPG key setup, but the error is benign — the distribution archive will be created, it just won’t be signed. Or you can skip it as you’ve done. -Taylor > On Nov 13, 2015, at 2:50 PM, Rodrigo Valladares > wrote: > > That worked

[ANNOUNCE] Apache Storm 0.10.0 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 0.10.0. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project

[ANNOUNCE] Apache Storm 0.9.6 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 0.9.6. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

Re: Storm 0.9.6

Hi Praj, The 0.9.6 tag was created when the corresponding release candidate was created. That release candidate is not being voted upon on the dev@ mailing list and can be released once the requisite 3 +1 votes from committers have been cast. Once that happens, the artifacts will be released

Re: Storm vs Spark Streaming Tech Evaluation

Hi Satish, Great series of blog posts. Thanks for sharing! -Taylor > On Oct 19, 2015, at 4:24 AM, Satish Mittal wrote: > > Hi All, > > The data platform team at Inmobi recently performed an extensive evaluation > exercise in the process of finalizing the real-time

Re: Storm 0.10.0 release

That release was cancelled because it did not include some important bug fixes, so it has not been officially released. I will delete the tags so it does not appear to have been released. -Taylor > On Sep 23, 2015, at 6:17 PM, Prajwal Tuladhar wrote: > > And weirdly, that

Re: Tident: fork DAG without partitioning?

Hi Grant, I think the fluent API is tripping you up a bit, but what you are trying to do is possible. Stream rootStream = …; Stream streamA = rootStream.each(new Fields(…), filterA); streamA.localOrShuffle() .each(new Fields(…), eachA, new Fields(…)); Stream streamB =

Re: Disable_replay_timeouts

Yes, you can set the use the “topology.enable.message.timeouts” config property: topology.enable.message.timeouts: false -Taylor On Jul 27, 2015, at 9:05 AM, Ajay Chander ajay.che...@gmail.com wrote: Hi Everyone, Is there anyway to explicitly disable replay timeouts specific to the

Re: Does Storm Rebalance during scale down?

Yes, if you remove a supervisor node Nimbus will detect it and reassign the work to other supervisor nodes in the cluster. -Taylor On Jul 9, 2015, at 12:18 PM, Dillian Murphey crackshotm...@gmail.com wrote: If a node is removed, will Storm rebalance among the remaining nodes? I know in

Re: State in Storm

Hi Thilina, 1. With Storm’s Core API (spouts/bolts) if you are storing state in an in-memory structure such as a HashMap, that state will be lost in the event of a worker crash. It is up to you to provide the logic for persisting state and restoring it in the event of a failure; Storm will not

Re: HdfsBolt

There might be others here that are using Storm with CDH, but since Cloudera Manager is closed source/proprietary, you may well be better off asking the question on a Cloudera forum for specific details. That being said, the community here is likely willing to help, but you’ll need to get past

[CVE-2015-3188] Apache Storm remote code execution vulnerability

. Taylor Goetz signature.asc Description: Message signed with OpenPGP using GPGMail

Re: Using FLUX and multiple streams to the same bolt

--- spout-1 --FIELDS-- A A --SHUFFLE-- B A --SHUFFLE-- C C --SHUFFLE-- D B --SHUFFLE-- D -- Thanks again for reporting this, and helping out with beta testing. - Taylor On Jun 17, 2015, at 4:54 PM, P. Taylor Goetz ptgo...@gmail.com wrote: Hi Romeo

Re: Using FLUX and multiple streams to the same bolt

JIRA: https://issues.apache.org/jira/browse/STORM-873 On Jun 17, 2015, at 11:38 PM, P. Taylor Goetz ptgo...@gmail.com wrote: Romeo, I have a fix (see below). Should be included in the next release (beta or final). I will follow up with a JIRA ID for tracking. -- TOPOLOGY

Re: [ANNOUNCE] Apache Storm 0.10.0-beta Released

It is in the 0.10.0-beta release. It was originally committed to the 0.10.x branch, then applied to the 0.9.x branch. -Taylor On Jun 16, 2015, at 2:29 PM, Binh Nguyen Van binhn...@gmail.com wrote: Hi, This ticket also scheduled for 0.10.0 but I do not see it in the change log of this

[ANNOUNCE] Apache Storm 0.10.0-beta Released

The Apache Storm community is pleased to announce the release of Apache Storm version 0.10-0-beta. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project

Re: Applying a patch

The easiest way would be to build the v0.9.5 tag from git. That version includes that fix, among other stability improvements and should be released in the next few days. -Taylor On Jun 2, 2015, at 4:45 PM, Srividhya Shanmugam srividhyashanmu...@fico.com wrote: I am trying to apply a patch

Re: Global Count - Trident - Please help

Try moving “.parallelismHint(2)” to after the groupBy. With the current placement (before the groupBy) Storm is creating two instances of your spout, each outputting the same data set. -Taylor On May 29, 2015, at 11:09 AM, Ashish Soni ashish.us...@gmail.com wrote: HI All , I am trying

Re: Status of running storm on yarn (the yahoo project)

I also developed a prototype/proof-of-concept (read: duck tape and bailer twine) for running Storm on YARN. I took a slightly different approach than Yahoo’s storm-yarn and Slider which from a high level allow you to spin up a Storm cluster on top of YARN. In my PoC a topology is treated as a

Re: flux

the session termination. I must be missing something on my side. I'll look around to see if I can find anything. thank you Clay On Tue, May 26, 2015 at 11:53 AM, P. Taylor Goetz ptgo...@gmail.com wrote: I just tested with the following command (from the “flux-examples” directory): storm

Re: Trident persistentAggregate only working for the first batch!

Hi Josh, The trident kafka spout stores offsets in zookeeper as well. See: https://github.com/apache/storm/tree/master/external/storm-kafka -Taylor On May 1, 2015, at 5:00 AM, Josh jof...@gmail.com wrote: Looks like this was some kind of network issue... I ran the topology on another

Re: Storm Trident Topology -- ParallelismHint

In scenario “b”, set the parallelism for the spout before the `shuffle()` operation. Trident topologies compile down to regular spouts and bolts. Partitioning operations like `shuffle()`, etc. define the bolt boundaries and hence where parallelism hints take effect. -Taylor On Apr 29, 2015,

Re: Using Storm with IBM MQ Series

Hi Jeremy, I was the engineer who gave that webinar. And the statement you quoted is misleading (IIRC, that’s not the wording I used when the question was asked). Nathan is correct, the nextTuple(), ack() and fail() methods are all called from the same thread. At the moment I don’t have

Re: Using Storm with IBM MQ Series

://github.com/ptgoetz/storm-jms On Apr 1, 2015, at 11:55 AM, P. Taylor Goetz ptgo...@gmail.com wrote: Hi Jeremy, I was the engineer who gave that webinar. And the statement you quoted is misleading (IIRC, that’s not the wording I used when the question was asked). Nathan is correct, the nextTuple

Re: template driven topology

Hi Clay, I've working on something exactly along those lines: https://github.com/ptgoetz/flux It may or may not be what you had in mind, but any feedback is more than welcome. -Taylor On Mar 27, 2015, at 7:56 PM, clay teahouse clayteaho...@gmail.com wrote: Hi All, Is there anything

Re: template driven topology

teahouse clayteaho...@gmail.com wrote: Thank you, Taylor. This is very close to what I had in mind. Clay On Fri, Mar 27, 2015 at 7:10 PM, P. Taylor Goetz ptgo...@gmail.com wrote: Hi Clay, I've working on something exactly along those lines: https://github.com/ptgoetz/flux It may or may

[ANNOUNCE] Apache Storm 0.9.4 Released

The Apache Storm community is pleased to announce the release of Apache Storm version 0.9.4. Storm is a distributed, fault-tolerant, and high-performance realtime computation system that provides strong guarantees on the processing of data. You can read more about Storm on the project website:

Re: Need help with troubleshooting a worker not processing messages

More information about your topology would help, but.. I’ll assume you’re using a core API topology (spouts/bolts). On the kafka spout side, does the spout parallelism == the # of kafka partitions? (It should.) On the bolt side, are you using fields groupings at all, and if so, what does the

Re: How and where is TopologyContext built?

.//storm/storm-core/src/clj/backtype/storm/daemon/task.clj But the bigger question is what are you trying to do? A little more context would likely help us help you. -Taylor On Mar 19, 2015, at 7:02 PM, Ravali Kandur kandu...@umn.edu wrote: Hi, I am trying to use Storm for my

Re: kryo buffer underflow exception

Hi Luke, Can you elaborate on the steps necessary to reproduce the problem? There’s not much to go on here. -Taylor On Mar 19, 2015, at 4:40 PM, Luke Rohde rohde.l...@gmail.com wrote: How did you resolve it? This started being a problem after I added a third field to an output tuple, just

Re: real time warehouse loads

What you've read about trident's throughput is wrong. Of course it depends on what you actually do in your topology (it's possible to shoot yourself in the foot and kill performance with both the core and trident APIs), but trident can achieve nearly twice the throughput as a core api topology

Re: Newbie unable to run Storm on Windows

Trunk/master (i.e. 0.10.x) is currently in active development, and not yet released. One of the current focus areas is Windows support. If you have the time and resources, we'd encourage any feedback you could provide. Otherwise, I'd encourage you to use an official release. On Feb 9, 2015,

  1   2   >