Re: ShellBolt died

e it a try with Storm 0.10.0. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 3월 16일 (수) 오후 5:04, gzc <18810513...@126.com>님이 작성: > Hi, guys > > My ShellBolt died. > > 2016-03-15T14:02:03.028+0800 b.s.t.ShellBolt [ERROR] Halting process: > ShellBolt died. > java.lan

Re: Cannot start Storm UI 0.10.0

Hi Lorenzo, It seems a bug. I reproduced this to binary release of v0.10.0, while it's not reproduced to 1.0.0-SNAPSHOT. Could you please file an issue to http://issues.apache.org/jira/browse/STORM ? Thanks in advance. Jungtaek Lim (HeartSaVioR) 2016년 3월 24일 (목) 오전 12:52, Lorenzo Fundaró

Re: external/storm-elasticsearch - upgrade requested

Lakshmanan, We looked over elasticsearch-hadoop when we adopted storm-elasticsearch, and elasticsearch-hadoop has only basic feature at that time. (No trident implementation) Please refer https://github.com/apache/storm/pull/573 to track our discussion on this module. Thanks! Jungtaek Lim

Re: Large number of executors per worker = lots of time in LMAX layer -> multithread instead?

y and see usages of each core. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 3월 26일 (토) 오후 8:34, John Yost 님이 작성: > Hi Everyone, > > I have a very CPU-intensive bolt that requires a high (at least I think > it's high) number of executors per worker--in my case, 10. I am finding > t

Re: rest api upload endpoint

would be after 1.0.0. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 3월 25일 (금) 오후 6:55, Salimane Adjao Moustapha 님이 작성: > Hi, > > Currently looking at the 1.x branch, there is no upload endpoint for the > rest api. I've seen the endpoint at some point but it's no mor

Re: Problem when consuming storm metrics

Denis, Could you share your jstack output which contains locked information? And I'm not sure but you may hit STORM-1027 <https://issues.apache.org/jira/browse/STORM-1027> which is resolved at 0.10.0 and 0.9.6. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 3월 29일 (화) 오후 5:41, Denis DEBAR

Re: Storm Topology Hanging

hanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 6일 (수) 오후 4:34, Nitin Gupta 님이 작성: > Dear All, > > > > I am facing an issue where in one of my Topology hangs after 14-16 hours. > > > > From the Storm UI I can see the below statics: > > · Number of Records Emitted

Re: Upgrading to newest Storm version

, functionality, stable, usability so I recommend you to want for some days and try using it when releasing. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 11일 (월) 오후 11:20, Julián Bermejo Ferreiro | BEEVA < julian.berm...@beeva.com>님이 작성: > Hi! > > We have a 5 nodes production environment, run

Re: Topology error

There's breaking compatibility in Apache Storm 1.0.0 - base package name changed. You may want to modify your topology source code to reflect renamed package. Modified base package name is 'org.apache.storm'. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) On 2016년 4월 13일

Re: Backward compatibility of STORM 1.0.0

Thanks Abhishek for publishing missed thing to user list. Jungtaek Lim (HeartSaVioR) 2016년 4월 13일 (수) 오후 5:17, Abhishek Agarwal 님이 작성: > In the latest version, the class packages have been changed from > "backtype.storm" to "org.apache.storm" so the topology code com

Re: Storm Metrics Consumer Not Receiving Tuples

ed to 0.10.x, but asynchronous metrics consumer bolt <https://issues.apache.org/jira/browse/STORM-1698> is a simple patch so you can apply and build custom 0.10.0, and give it a try. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 14일 (목) 오후 11:06, Denis DEBARBIEUX 님이 작성: &

Re: How does one distribute database iteration across workers?

or Trident or maintain your own buffers. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 19일 (화) 오후 10:02, Jason Kusar 님이 작성: > Hi, > > I've done a similar thing before with the exception that I was reading > from Cassandra. The concept is the same though. Ass

Re: Monitoring Max Spout Pending

to set up. Please note that it may affect whole throughput (I guess it would be relatively small) so you may want to attach when only testing / debugging performance issue. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 20일 (수) 오전 9:41, Kevin Conaway 님이 작성: > In Storm 0.10, is there a way

Re: Monitoring Max Spout Pending

tuple latency. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 20일 (수) 오전 10:21, Kevin Conaway 님이 작성: > We are already sending our metrics to graphite but I don't see > __skipped-max-spout being logged. Was that added after 0.10? I don't even > see a reference to i

Re: Monitoring Max Spout Pending

ny issues related > to that here? > > > One thing you would like to check is, event handler on Spout is single > thread, which means that same thread calls nextTuple() and ack() and > fail(). > > I'm not following you here. What would I be checking here? > > On T

[DISCUSS] Would like to make collective intelligence about Metrics on Storm

Hi Storm users, I'm Jungtaek Lim, committer and PMC member of Apache Storm. If you subscribed dev@ mailing list, you may have seen that recently we're addressing the metrics feature on Apache Storm. For now, improvements are going forward based on current metrics feature. - Improve

Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm

orm provides only average latency and it's indeed based on sample rate. Do we feel OK with this? If not how much having also percentiles can help us? Thanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 20일 (수) 오전 10:55, Jungtaek Lim 님이 작성: > Hi Storm users, > > I'm Jungtaek Lim, commit

Re: How does one distribute database iteration across workers?

onsidered >> Trident, but didn't go ahead because it was not clear how I could implement >> multiple spouts in Trident, where each spout would iterate a certain number >> of rows of a database. Any idea how that could happen. >> >> On Tue, Apr 19, 2016 at 6:39 PM, Jung

Re: Newbie Questions for Cluster Dependencies

er, but there's some repository / blog post to refer on set up Storm with Docker. https://github.com/wurstmeister/storm-docker http://blog.baqend.com/post/142795871760/tutorial-deploying-apache-storm-on-docker-swarm Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 4월 23일 (토) 오후 1:22, Joaquin

Re: UI problem

Hi, Could you open developer tools from your browser and check any API calls are failing with UI page load? Jungtaek Lim (HeartSaVioR) 2016년 4월 29일 (금) 오후 2:47, Sai Dilip Reddy Kiralam < dkira...@aadhya-analytics.com>님이 작성: > > Hello, > > I Installed storm 0.1

Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm

omponent name, host, etc. are easy so I would like to see the needs before making the change. Do you think adding filter targets makes sense? Or we can just start with metrics name filter? Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 3일 (화) 오전 2:18, Harsha 님이 작성: > Jungtaek, >

Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm

opinions whether it's better to keep it simple, or give full flexibilities. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 3일 (화) 오후 2:47, Harsha 님이 작성: > Jungtaek, > I think filters that can support a regex gives more > felxibility. > Thanks, > Harsha > >

Re: How to you store database connections in a Spout or Bolt without serialization problems?

helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 4일 (수) 오후 6:48, Navin Ipe 님이 작성: > Hi, > > I know that if a MySQL database connection is instantiated in the > constructor of a Spout or Bolt, it won't work. It should be instantiated in > open() or prepare(). > > Pro

Re: How to you store database connections in a Spout or Bolt without serialization problems?

arsetConverter >> at >> java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184) >> ~[?:1.8.0_73]* >> >> I assume it is because of one of these which aren't getting serialized: >> >> >> *private Connection connRef; private Statement stmt; private >> ResultSet rs;* >> >> So if you can't declare them as class members because they don't get >> serialized, then how do you declare them so that the entire class will have >> access to it and I won't have to keep creating new connections for every >> query? >> >> I'm quite sure that declaring and initializing them in *prepare()* won't >> ensure that the rest of the class functions would be able to access it. >> >> >> -- >> >> Regards, >> >> Navin >> > > > > -- > Regards, > Navin > -- Name : Jungtaek Lim Blog : http://medium.com/@heartsavior Twitter : http://twitter.com/heartsavior LinkedIn : http://www.linkedin.com/in/heartsavior

Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm

t think nicer way for now and we still rely on metrics per task. Do you have some ideas or thought? Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 4일 (수) 오후 9:23, Kevin Conaway 님이 작성: > >For specific task, you can register your own metrics which resides per > task. > > Exactly, that

Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm

I believe eventually we will provide various information for latency. Stay tuned. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 6일 (금) 오후 10:29, Adam Meyerowitz (BLOOMBERG/ 731 LEX) < ameyerow...@bloomberg.net>님이 작성: > I recall seeing in another thread a discussion about monitoring metrics &

Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm

ly their frequencies independently. I guess B (or alternative way) should be address before addressing A. 2016년 5월 9일 (월) 오후 12:01, Jungtaek Lim 님이 작성: > Hi Adam, > > Thanks for the great input! Let me share my thought about two things. > > 1. There's metrics for disruptor queue so

Re: [DISCUSS] Would like to make collective intelligence about Metrics on Storm

Agreed. There could be latency SLAs for specific topology, then we >> would really want to see outliers and percentiles, too. Since providing >> them may affect performance we should address them with care. I believe >> eventually we will provide various information for latency. Stay

Re: Storm 1.0.0 upgrade Serialization issue

Samuel and KB, I think Storm 1.x hits the bug on commons-io (IO-368 <https://issues.apache.org/jira/browse/IO-368>). I'll file an issue. Thanks for reporting. Best Regards, Jungtaek Lim (HeartSaVioR) 2016년 5월 10일 (화) 오전 1:33, KB 님이 작성: > All, > > This problem persists in r

Re: Storm 1.0.0 upgrade Serialization issue

Filed: https://issues.apache.org/jira/browse/STORM-1773 KB, could you share sample topology which hits serialization issue? I would like to check whether patch helps resolving it or not. 2016년 5월 10일 (화) 오후 12:14, Jungtaek Lim 님이 작성: > Samuel and KB, > > I think Storm 1.x hits t

Re: Storm 1.0.0 upgrade Serialization issue

. Thanks in advance! Jungtaek Lim (HeartSaVioR) 2016년 5월 11일 (수) 오전 12:54, KB 님이 작성: > Thanks for this update. Actually our topology is built on multiple > wrappers on top of Storm. I'll try to create simple topology to reproduce > the problem. Meanwhile would it be possible to create a sn

Re: Storm 1.0.0 upgrade Serialization issue

n for next release immediately. Hopefully we can release next version within a month. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 11일 (수) 오후 11:34, KB 님이 작성: > Hi Jungtaek, > > Thanks for providing the snapshot build with this fix. I have verified the > fix and it is working fine. >

Re: Storm 1.0.0 upgrade Serialization issue

7;ve got two valid > reports of it, and more will follow quickly as users continue to upgrade. > > -Taylor > > On May 11, 2016, at 11:00 PM, Jungtaek Lim wrote: > > KB, > > Submitted pull request: https://github.com/apache/storm/pull/1412 for 1.x > > Since Storm 1.0.1

Re: Simple problem about the storm

le is sent from executor itself by registering recurring schedule to scheduler. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 12일 (목) 오후 3:37, fanxi...@travelsky.com 님이 작성: > Hi All, > > I have two questions. > > 1. > > I am puzzled about the latest storm

Re: tune up storm

Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 23일 (월) 오후 5:40, Julien Nioche 님이 작성: > Hi Joshua > > https://www.manning.com/books/storm-applied has a section on tuning which > you might find useful. > > Julien > > > On 23 May 2016 at 08:44, fanxi...@travelsky.com > wro

Re: Clarification on metrics sampling rate

consumer). While metrics provided to UI has time windows, metrics provided to metrics consumer resets their values every period. Hope this help. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 26일 (목) 오후 9:11, Thomas Cooper (PGR) 님이 작성: > Hi, > > > I'm a Computer Science PhD student wor

Re: Clarification on metrics sampling rate

ss? > > > Thanks, > > > Thomas Cooper > > PhD Student > Newcastle University, School of Computer Science > > > > -- > *From:* Jungtaek Lim > *Sent:* 26 May 2016 13:32 > *To:* user@storm.apache.org > *Subject:* Re: Clarifi

Re: Storm rebalancing problems

Hi Julián, Which version of Storm do you use? I remember some of Storm 0.9.x versions has some issues when workers are failing, so I'd like to know about it. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 26일 (목) 오후 5:53, Julián Bermejo Ferreiro | BEEVA < julian.berm...@beeva.com>님이 작

Re: StormTimelineMetricsReporter

Hi Duffie, StormTimelineMetricsReporter is created and maintained by Ambari side so you may want to make inquery to Ambari user group. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 5월 26일 (목) 오후 1:14, Duffie, John 님이 작성: > Can the StormTimelineMetricsReporter be used to export our custom metr

Re: Storm rebalancing problems

y emitted from Spout. If bolts can process all flowing tuples, it's a graceful restart. Same thing applies to kill, 'graceful stop' in this case. - Jungtaek Lim (HeartSaVioR) 2016년 5월 26일 (목) 오후 11:06, Julián Bermejo Ferreiro | BEEVA < julian.berm...@beeva.com>님이 작성: > Hi

Re: Storm unique strengths

Hi Leon, One thing to note is that we addressed some performance issues after benchmark is being done. I don't have environments for benchmark for the same, but worth to give it a try with recent release (1.0.1). Thanks, Jungtaek Lim (HeartSaVioR) 2016년 6월 1일 (수) 오후 6:43, 님이 작성: >

Re: Sending extra folder to all supervisor nodes with storm submit command

e this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 6월 3일 (금) 오전 4:49, anshu shukla 님이 작성: > Hello all, > > Is there any way to pass the folder to storm submit command so that all > the contents of it can be shared on all the supervisor nodes . > > Can we use lib folder ? As

Re: Topology gets stuck

Nitin, Could you check your configuration for message timeout seconds to see it's set to enough amount of time, and also check there're failing tuples at that time? Thanks, Jungtaek Lim (HeartSaVioR) 2016년 6월 9일 (목) 오후 11:05, Nitin Gupta 님이 작성: > Thanks Abhishek for the guidance.

Re: Basic questions about Strom

topology.testing.always.try.serialize" as debug purpose. Note that it affects performance so it should be disabled ("false" by default) for production environment. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 6월 11일 (토) 오전 3:27, Junguk Cho 님이 작성: > H

Re: Disable Backpressure in storm 1.0.1

Hi anshu, You can set topology.backpressure.enable to false to disable backpressure. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 6월 14일 (화) 오전 5:32, anshu shukla 님이 작성: > Hey all, > > With backpressure enabled it is quite difficult to identify/estimate the &

Re: Disable Backpressure in storm 1.0.1

entry in storm.yaml in place of default.yaml > > topology.backpressure.enable: false > > > On Tue, Jun 14, 2016 at 4:25 AM, Jungtaek Lim wrote: > >> Hi anshu, >> >> You can set topology.backpressure.enable to false to disable backpressure. >> >> Ho

Re: Basic questions about Strom

default stream, do all tuples include stream id > whenever they send? > > > 3. To be clear, if we set "false", storm does not use serialization for > inter-process and inter-node? > > Thanks in advance. > - Junguk > > > > > 2016-06-10 18:00 GMT-0

Re: Storm Redis TTL Control

control is not matched to the intention of these classes. And as we talked from initial pull request for storm-redis, 1 on 1 transition on Redis commands doesn't make sense, and we already provide Jedis instance so it would be easy to make own Bolt for flexibility. Thanks, Jungtaek Lim (HeartS

Re: Heartbeat timeout with Python spout

Could you share implementation of spout? In multi-lang user level functions shouldn't block, so heartbeat timeout will occur if your spout requests to Kafka and wait for response infinitely. - Jungtaek Lim (HeartSaVioR) 2016년 6월 23일 (목) 오후 7:25, cogumelosmaravilha 님이 작성: > Yes, bu

Re: Heartbeat timeout with Python spout

spout don't receive any message from ShellSpout thus no messages from multi-lang spout side, so no heartbeat is being marked. I'll file an issue and see if it's easy to fix. Jungtaek Lim (HeartSaVioR) 2016년 6월 23일 (목) 오후 10:59, cogumelosmaravilha 님이 작성: > Python Petrel Spou

Re: Heartbeat timeout with Python spout

FYI - Filed issue from myself here: https://issues.apache.org/jira/browse/STORM-1928 2016년 6월 23일 (목) 오후 11:33, Jungtaek Lim 님이 작성: > Do get_simple_consumer() blocks for receiving messages? If it is, can we > set timeout on this? > > Btw, I found edge-case from ShellSpout (no

Re: Can KafkaSpout emit tuple to specified stream on 0.9.5?

lines. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 6월 23일 (목) 오전 10:38, Chen Junfeng 님이 작성: > I want to emit tuples to a specified stream instead of default stream. > Then I found this one > https://github.com/apache/storm/pull/601/files#r53569108. > > > > It seems I only need to

Re: Worker process start time

Hi Rudraneel, What's your Storm version and how cluster is configured? Thanks, Jungtaek Lim (HeartSaVioR) 2016년 6월 24일 (금) 오전 9:37, Rudraneel chakraborty < rudraneel.chakrabo...@gmail.com>님이 작성: > Hi > > I am observing that my worker processes take a little over 15 seco

Re: Heartbeat timeout with Python spout

luate. Hope this helps, Thanks, Jungtaek Lim (HeartSaVioR) 2016년 6월 24일 (금) 오전 12:04, Jungtaek Lim 님이 작성: > FYI - Filed issue from myself here: > https://issues.apache.org/jira/browse/STORM-1928 > > 2016년 6월 23일 (목) 오후 11:33, Jungtaek Lim 님이 작성: > >> Do get_simple_consu

Re: unsubscribe

to unsubscribe send an email to user-subscr...@storm.apache.org and if you are on dev list send an email to dev-subscr...@storm.apache.org 2016년 6월 29일 (수) 오후 10:54, Patrick Nip 님이 작성: > >

Re: Can KafkaSpout emit tuple to specified stream on 0.9.5?

I mean 2.x is 2.0 (currently in progress) in master branch. - Jungtaek Lim (HeartSaVioR) On 2016년 6월 25일 (토) at 오전 4:18 Pablo Recabal wrote: > What do you mean by 2.x? > > Is that for Storm itself or only for the Kafka Spout? Couldn't find any > reference to it on ei

Re: Topology submision Bugs

rivate DNS when available. Jungtaek Lim (HeartSaVioR) 2016년 7월 5일 (화) 오후 6:08, Walid Aljoby 님이 작성: > Thank you.. > > nimbus.seeds: the IP address of machine running nimbus (which is the same > host through which the topology is submitted). > but the bugs still occur

Re: Topology submision Bugs

. > > Thank you > -- > Regards > > WA > -- > *From:* Jungtaek Lim > *To:* Walid Aljoby ; Harsha Chintalapani < > st...@harsha.io>; "user@storm.apache.org" > *Sent:* Tuesday, July 5, 2016 5:41 PM > > *Subject:* Re: Topo

Re: Topology submision Bugs

tion: Do I set storm.local.hostname to be the IP address of the > node that runs the Nimbus? and Where would be these setting, Is it for both > master and slave nodes? > > > Thank you > -- > Regards > WA > > > ------ > *From:* Jungtaek

Re: NoClassDefFoundError in cluster mode on some executors

I guess we need to have context to see what's behind on that. Sharing stack trace might help if you really don't mind. 2016년 7월 7일 (목) 오전 11:38, Abhishek Agarwal 님이 작성: > You check what is happening in the Class. There must be some static > initialization code which fails. Also note, that wheneve

Re: Is dynamic update of topologies possible?

Unfortunately no. You need to redeploy topology when you need to update content of topology. But, if your dynamic update is restricted to just replace resource files, using distributed cache on 1.0.0 makes it possible. Hope it helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 7월 8일 (금) 오전 6:30

Re: Is dynamic update of topologies possible?

running the storm jar command. So I could generate a Flux > config file based on user input from a UI and then invoke storm cli to > start the new topology and stop the old one. Does that sound feasible? > > > > > On 7 July 2016 at 22:56, Jungtaek Lim wrote: > >> Un

Re: Error in setBolt("cassandra", new CassandraWriterBolt());

rade your Storm cluster to 1.x to use storm-cassandra. Or you could port storm-cassandra to 0.10 manually which I think is not that hard but not recommended. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 7월 14일 (목) 오후 4:28, Anirudh Chhangani 님이 작성: > Hi, > > I am relatively new to storm and want

Re: Error in setBolt("cassandra", new CassandraWriterBolt());

Sorry forgot to paste: backtype.storm -> org.apache.storm 2016년 7월 14일 (목) 오후 5:41, Jungtaek Lim 님이 작성: > Hi Anirudh, > > Storm 1.x is not backward compatible with Storm 0.x since package name is > changed from "backtype.storm" to " across whole classes. > Sto

Re: Peformance Metrices

If you're not focused to comparison of streaming frameworks, check out this micro-benchmark result. http://hortonworks.com/blog/microbenchmarking-storm-1-0-performance/ Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 7월 21일 (목) 오전 10:22, Manu Zhang 님이 작성: > Hi Walid, >

Re: Is there a reason storm has a default of 4 ports?

That was presentation of Taylor's, and it's 1 worker per node "per topology". That suggestion is for reducing inter-JVM communications. So having multiple ports for multiple topology is OK. 2016년 7월 21일 (목) 오후 5:11, Sinnema, Remon 님이 작성: > I’m just curious, if the recommendation is to use 1 worke

Re: Suspicious numbers in Spout tuples emitted, failed, acked v/s max.pending tuples

evel, but it's planned to be included to Storm 1.1.0. https://issues.apache.org/jira/browse/STORM-1719 Hope this helps. Thanks, Jungtaek Lim (HeartSaVIoR) 2016년 7월 22일 (금) 오전 12:48, Girish Reddy 님이 작성: > Hello, > > I'm using storm-0.9.6 and have a topology with KafkaSpout and 3

Re: Suspicious numbers in Spout tuples emitted, failed, acked v/s max.pending tuples

or REST API. Are those numbers exposed >through NimbusClient Java API? (At least in Storm 1.0.1?) > > > Thank You!! > > > On Thu, Jul 21, 2016 at 9:02 AM, Jungtaek Lim wrote: > >> You can click 'Show topology visualization' with 'Show System Stats&#

Re: Supervisor is down after running topology for a while

Hi Chen, You might hit STORM-1934 <https://issues.apache.org/jira/browse/STORM-1934> which is fixed for 1.0.2. Since vote thread for 1.0.2 RC4 is open to dev@ list, we expect 1.0.2 to be released in several days unless we found any critical bugs from RC4. Thanks! Jungtaek Lim (Heart

Re: When max pending is too large, heartbeat times out.

participate the vote so that we can feel 1.0.2 as more stable. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 7월 30일 (토) 오전 4:43, Andrew Montalenti 님이 작성: > Hi Tim, > > You're actually hitting a Shell Spout death failure that we also > identified in production at Parse.ly using st

Re: Storm UI metrics data is cleared sometimes

vious values when first heartbeat from that worker is made This is a limitation of current metrics since we don't store it to any storages or centralized in-memory. We'd like to address it. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 7월 28일 (목) 오후 3:58, Chen Junfeng 님이 작성: > Hi buddies,

Re: Storm UI metrics data is cleared sometimes

trics consumer, so you still need to prepare external storage DB and relevant metrics consumer implementation. There's a popular pair on Graphite and verisign/storm-graphite, and upcoming Ambari 2.4 will have Storm metrics consumer for Ambari Metrics System. Thanks, Jungtaek Lim (HeartSaVioR)

Re: Example of worker level metric?

s for specific point, not accumulative things like counter. You can refer SystemBolt to how to implement gauge style metrics. Hope this helps. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 8월 5일 (금) 오후 11:07, Stephen Powis 님이 작성: > Basically in your bolts prepare() method you'd want to do somethi

Fwd: [Discussion] Dropping Java 7 support on master

I was mistaken the address of user mailing list. Forwarding this to user@. -- Forwarded message - From: Jungtaek Lim Date: 2016년 8월 12일 (금) 오후 6:38 Subject: Re: [Discussion] Dropping Java 7 support on master To: , I'm not sure we would like to continue discussion on

Re: Used Mem in RAS (storm 1.0.1 R-storm )

Worker has two -Xmx options, which one is for worker and another one is for log4j2. If you see used memory is not same as adding two, please show your case to take a look at. Thanks, Jungtaek Lim (HeartSaVioR) On Sunday, August 14, 2016, anshu shukla wrote: > What exactly does the entry

Re: [SURVEY] What version of Storm are you using?

Side note for users waiting stable version of 1.x: 1.0.2 is worth to try out since it's well tested, and fixes various critical bugs users met. - Jungtaek Lim (HeartSaVioR) 2016년 8월 18일 (목) 오전 6:58, Erik Weathers 님이 작성: > Groupon is using 0.9.6. > > Switching to 0.10.0 or 1.0+ wil

Re: [SURVEY] What version of Storm are you using?

his issue, "blocked by us dealing with mysterious Storm Worker > process heartbeat failures", do you have any topo's @ Groupon using > ShellSpout (multi-lang)? > > If so, the patch/resolution in STORM-1928 (released in 1.0.2, see JIRA for > details) may be helpful. > >

Re: Errors during testing when using TOPOLOGY_TESTING_ALWAYS_TRY_SERIALIZE

Hi Cory, It seems to be the issue we need to look at. Did you file an issue for Storm JIRA? If not would you please file one with affected (your) version? Thanks in advance! Jungtaek Lim (HeartSaVioR) 2016년 8월 18일 (목) 오전 9:14, Cory Kolbeck 님이 작성: > Hi folks, > > I'm fairly new

Re: State Management - Config Files in Jar, Outside Jar? Suggestions

e questions regarding above. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 8월 18일 (목) 오전 6:56, Joaquin Menchaca 님이 작성: > Hi. > > I was looking for ideas for state management of configuration for data > sources and tenants. We pull for customer databases, may have dev/prod/etc > environ

Re: Docker images for the latest version

Hi Jun Seong, Nice work! Thanks for sharing your repo. I'll try to use and do feedbacks if any. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 8월 22일 (월) 오전 1:45, 박준성 님이 작성: > Hi everyone, > > I've found there are a lot of Dockerfile relevant to Apache Storm but the > ver

Re: groupByKey and window aggregation in Trident

to be used as public API. You could still give it a try to wrap your Aggregator to GroupedAggregator and pass to window. https://github.com/apache/storm/blob/v1.0.2/storm-core/src/jvm/org/apache/storm/trident/operation/impl/GroupedAggregator.java Hope this helps. Thanks, Jungtaek Lim (HeartSaVi

Re: Storm nimbus election

We need to see Nimbus log to find out why Nimbus is down. Pasting or attaching Nimbus log would be better. And please explain your Storm version too. Jungtaek Lim (HeartSaVioR) 2016년 8월 26일 (금) 오후 1:16, pradeep s 님이 작성: > Hi , > While restarting nimbus and ui, i am g

Re: Storm nimbus election

As you suspected there seems to be ZK issue. Unfortunately I don't have experience to operate Storm cluster with EC2. You may want to check each Zookeeper instance is accessible from your Storm cluster. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 8월 28일 (일) 오후 5:27, pradeep s 님이 작성: > Hi

Re: cannot kill topology from dashboard

you state more than 2 hrs.) Thanks, Jungtaek Lim (HeartSaVioR) 2016년 8월 30일 (화) 오전 10:09, Manu Zhang 님이 작성: > Hi all, > > I have a topology running for more than 2 hours and then I cannot kill the > topology from dashboard. A dialog with "Error communicating with Nimbus"

Re: LogViewer Basic Question

ll request updating this doc. Please let me know if you would like to provide one. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 8월 31일 (수) 오전 8:40, Joaquin Menchaca 님이 작성: > The logviewer service should be run on each of the supervisors? Then > the ui service connects to the supervisors? >

Re: Errors during testing when using TOPOLOGY_TESTING_ALWAYS_TRY_SERIALIZE

Hi Cory, Pull request for that issue is available now. https://github.com/apache/storm/pull/1668 I expect this can be included to 1.0.3. If you don't mind could you test this on your environment? Thanks, Jungtaek Lim (HeartSaVioR) 2016년 8월 19일 (금) 오전 12:51, Jungtaek Lim 님이 작성: >

Re: Vagrant Scripts Updated for Storm 1.0.2

Great works Joaquin! Thanks for sharing. - Jungtaek Lim (HeartSaVioR) 2016년 9월 11일 (일) 오후 5:08, Joaquin Menchaca 님이 작성: > Hi all. > > So with the command *vagrant up*, you can bring up 1.0.2 cluster now. > Also, 0.10.1 and 0.9.7 are supported. Repo at: > https://github.com

Re: Storm 1.0.2 errors being reported coming from Nimbus instead of worker hosts

This is really odd. Please file an issue to JIRA if you can consistently reproduce this. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 9월 11일 (일) 오전 1:40, Stephen Powis 님이 작성: > But the topology gets deployed out across multiple nodes in the cluster, > the webui definitely shows various bolts

[SURVEY] Which external modules (connector) do you use actively?

ports Kafka.) In order to rank the priority to support, I'd like to see which external modules (connector) the user community are actively using. You can also enumerate not-released-yet modules: Druid, Kinesis, OpenTSDB as they're also candidates. Thanks in advance! Jungtaek Lim (Heart

Re: Getting total number of bolt tasks from Storm's REST API

ailed info. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 9월 16일 (금) 오후 2:31, Navin Ipe 님이 작성: > Help. Anyone? > > On Thu, Sep 15, 2016 at 3:47 PM, Navin Ipe < > navin@searchlighthealth.com> wrote: > >> Hi, >> >> I've been using Storm's REST

Re: Supervisor failed but workers continue to work

I guess you hit STORM-1934 <https://issues.apache.org/jira/browse/STORM-1934> which was resolved from Storm 1.0.2. Since Storm 1.0.2 includes many bugfixes, I strongly encourage you to upgrade your cluster to 1.0.2. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 9월 18일 (일) 오전 11:08

Re: Cannot submit topology in local mode on Storm 1.0.1

Hi Chen, Could you try running your topology with Storm 1.0.2 in local mode? Since it's in local mode you can easily try it out. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 9월 20일 (화) 오후 5:26, Chen Junfeng 님이 작성: > My topology runs well in cluster mode but throws exceptions when it is >

Re: Cannot submit topology in local mode on Storm 1.0.1

Unfortunately without the minimal code which can reproduce this I'm hard to dig more. I followed the path based on stack trace, and it would be IllegalArgumentException, but it should have message which describes why it is thrown. Do you find more stack trace informations on that? - Jungtae

Re: Metrics Instrumentation

nting-user-defined-metric-to-analysis-your.html Storm doesn't include native support of metrics consumers so you may want to find or create one. Btw, there will be some improvements on metrics features on upcoming 1.1.0 release. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 9월 22일 (목) 오후 12:49, Otis

Re: Running Apache Storm 2.0.0

. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 9월 23일 (금) 오후 3:09, Miyuru Dayarathna 님이 작성: > Hi, > > We want to try out Storm 2.0.0. We cloned the Storm 2.0.0-SNAPSHOT code > from https://github.com/apache/storm.git and built. But When we try to > start the supervisor we get the fo

Re: Problem in UI of Apache Storm 2.0.0 SNAPSHOT

Hi Eranga, Since Storm 2.0.0 ports many clojure files to Java, we definitely need to some iterations for QAing. Please file an issue what you observed. If you can provide pull request for that it would be much appreciated. Thanks! Jungtaek Lim (HeartSaVioR) On Thu, 29 Sep 2016 at 2:00 PM

Re: Error when running pacemaker.

jar - run pacemaker and nimbus with JDK 7 - see it reappears If it works perfectly, I guess that's the cause. If then please file an issue so that we can track it. Please let me know you'd like me to test this. Thanks, Jungtaek Lim (HeartSaVioR) 2016년 10월 6일 (목) 오후 4:03, Abhishek R

Re: How to iterate through mongodb under Storm + Trident

tency. Your query, schema of the table, index, and others matter. So experimenting with your query and your table schema, and your infra only give you to valid answer. - Jungtaek Lim (HeartSaVioR) 2016년 10월 7일 (금) 오후 2:14, Cheney Chen 님이 작성: > Wondering if anyone has any clues? > > On We

Re: Localcluster - ELK Bolt Example

The files which thrift generated are tricky. Please try `mvn clean install -DskipTests` to root of Storm directory, and back to IDEA and see it helps. - Jungtaek Lim (HeartSaVioR) 2016년 10월 9일 (일) 오후 6:09, Snabel Kabiya 님이 작성: > Hi Storm Users, > > > > My name is Snabel, I want

Re: Connection refused when trying to connect to nimbus

Hi Mostafa, What's your Storm version? - Jungtaek Lim (HeartSaVioR) 2016년 10월 10일 (월) 오후 6:59, Mostafa Gomaa 님이 작성: Hello All, I have a single node cluster running zookeeper and nimbus on a remote machine. Both zookeeper and nimbus are running and I can use the storm client s

  1   2   >