Re: HdfsSpout

It has now been committed. -roshan On 1/13/16, 4:17 PM, "Jungtaek Lim" wrote: >Hi Matthias, > >AFAIK, Roshan is working on providing HdfsSpout - >https://issues.apache.org/jira/browse/STORM-1199. >It's in reviewing so you can see the pull request and start reviewing, >too. >

Re: Performance Profiling - Storm core

>> >> Does the above work? You can also give the type hint for target tasks >> (^List target-tasks) >> >> It is not clear to me why this code path will be even invoked, since you >> are not using LoadAwareCustomStreamGrouping in your topology. &

Performance Profiling - Storm core

Attached is a screenshots of the performance profile for Storm core using a Speed of Light topology. Topology info: - 1 bolt instance, 1 spout instance, 1 worker. - ACKer count = 0 - Spout generates precomputes a random list of tuples, then keeps emitting them endlessly - Bolt just remits the

Re: Performance Profiling - Storm core

d what is the size of disruptor queue? Also If your >spout is slower compared to the bolts, this behavior is very much >expected. >Isn't it? > >On Wed, Feb 3, 2016 at 11:54 AM, Roshan Naik <ros...@hortonworks.com> >wrote: > >> Looks like the attachments were stripped off

Re: Performance Profiling - Storm core

/roshannaik/storm-benchmark-sol/blob/master/profiler/storm%20core%20-%20sol%20-%200%20acker/storm%20core%20-%20method%20stats.png -roshan From: Roshan Naik <ros...@hortonworks.com<mailto:ros...@hortonworks.com>> Reply-To: "dev@storm.apache.org<mailto:dev@storm.apache.org>&

Re: Problem about secure hdfs spout

Looks like some bug there .. Can you file a jira ? -roshan On 2/25/16, 7:39 PM, "ght230" wrote: >Hi: > >Recently I was trying secure hdfs spout about ³STORM-1199: HDFS Spout >Implementation³, but some error occured as following: > >"Login successful for user t...@example.com

Re: Any guidance around disruptor batch size ?

ize of the queue. >I have been talking to a few people in academia that are working on >auto-tuning of the batch size to optimize throughput for a given SLA, but >that is still a ways off. > - Bobby > >On Wednesday, February 10, 2016 8:59 PM, Roshan Naik ><ros...@horto

Any guidance around disruptor batch size ?

Wondering if there is any rule of thumb guidance around setting topology.disruptor.batch.size, for increasing throughput ? For example any correlation with topology.max.spout.pending, bolt executor count, spout count, etc ? -roshan

Re: Thought on complete latency

IMO, avoiding the time variation on machines makes total sense. But I feel that this is a tricky question. Couple more thoughts: 1) As per http://storm.apache.org/releases/current/Guaranteeing-message-processing.ht ml "Storm can detect when the tree of tuples is fully processed and can ack

Re: STORM JIRA being spammed - CANNOT comment on JIRA (as a result of the counter spam measures)

Myself too. Thanks. -Roshan On 5/12/16, 1:14 PM, "Bobby Evans" wrote: >Sorry about that you should be added as a contributor now. Is there >anyone else that I should add as a contributor? >Also we have not been maintaining the commiters ACL very well either. >Are

Re: [jira] [Commented] (STORM-1757) Apache Beam Runner for Storm

Never mind my prev comment… got confused by looking at BEAM-216. -roshan On 5/3/16, 4:55 PM, "Roshan Naik" <ros...@hortonworks.com> wrote: >Shouldn¹t we have a BEAM runner in Storm project instead of Storm runner >in Beam project ? >-roshan > >On 5/3/16,

Re: [jira] [Commented] (STORM-1757) Apache Beam Runner for Storm

Shouldn¹t we have a BEAM runner in Storm project instead of Storm runner in Beam project ? -roshan On 5/3/16, 1:15 PM, "Hugo Louro (JIRA)" wrote: > >[ >https://issues.apache.org/jira/browse/STORM-1757?page=com.atlassian.jira.p

Re: [VOTE] Release Apache Storm 0.10.1 (rc2)

This is missing HdfsSpout (STORM-1199) -roshan On 5/3/16, 3:39 PM, "Hugo Da Cruz Louro" wrote: >+1 (non binding) >- Found a very minor bug in the storm-solr test topology, for which there >is a very simple workaround and by NO means a blocker for the release. >-

Re: Request for reviewing

STORM-1910 -roshan On 7/14/16, 8:59 AM, "Jungtaek Lim" wrote: >We seems close to make all issues for 1.0.2 epic resolved soon. > >Epic link: https://issues.apache.org/jira/browse/STORM-1855 >All of open issues are in progress of reviewing. Most of them got +1 and >is

Re: [DISCUSS] Remove CHANGELOG file

Sounds like a good idea to me. -roshan On 2/23/17, 4:41 PM, "Jungtaek Lim" wrote: Hi devs, I guess we discussed about this before, but didn't move to actual work. I'd like to propose again removing CHANGELOG file in repository, and use JIRA issue's

Re: [Discuss] Storm hdfs spout improvements

On 2/13/17, 12:14 PM, "Sachin Pasalkar" wrote: >I have attached updated source code of HDFSSpout for more reference. I >have updated respective classes (not attached) Don¹t see any attachment. Answers are below. Better to do this discussion on a JIRA. On

Re: [CANCELED] [VOTE] Release Apache Storm 1.1.0 (RC2)

Looks like the formatting on my prev email got dropped. But observation is that storm-kafka-monitor and storm-druid are the biggest new contributors. -roshan On 3/1/17, 5:23 PM, "Roshan Naik" <ros...@hortonworks.com> wrote: Once all of the shaded examples are included the

Re: [CANCELED] [VOTE] Release Apache Storm 1.1.0 (RC2)

largely due to > shaded examples, etc. But it’s something to keep an eye on. Our binary > releases shouldn’t have to be that big. > > -Taylor > > On Mar 1, 2017, at 12:09 PM, Roshan Naik <ros...@hortonworks.com ros...@hortonworks.com>>

Re: Storm 1.x-branch won't build in IntelliJ anymore

clean install -DskipTests). Does workaround not work any more? Or do you see some spots which workaround doesn't work? - Jungtaek Lim (HeartSaVioR) On Wed, Mar 1, 2017 at 11:34 PM, Roshan Naik <ros...@hortonworks.com> wrote: > If we leave 1.x in this state

Storm 1.x-branch won't build in IntelliJ anymore

Lately (about a week and half maybe) it has not been possible to get the 1.x-branch to build inside IntelliJ. None of the modules are able to locate the LocalCluster class (which is a clojure class). Previously, every once in a while, I used to get this problem in the storm-starter module and

Re: [CANCELED] [VOTE] Release Apache Storm 1.1.0 (RC2)

nks for bringing these up Roshan. Feel free to file JIRA tickets for these issues and assign the “Release Apache Storm 1.1.0” epic so they can be tracked for this release. -Taylor > On Mar 1, 2017, at 9:27 AM, Roshan Naik <ros...@hortonworks.com> wrote: > >

Re: Performance improvements in 1.0.0

Here is one effort on quantification the improvements Š http://hortonworks.com/blog/microbenchmarking-storm-1-0-performance/ -roshan On 9/6/16, 10:17 AM, "S G" wrote: >Hi, > >On the http://storm.apache.org/2016/04/12/storm100-released.html page, I >see a notice

Re: [DISCUSS] Feature Branch for Apache Beam Runner

Good idea. Will the Beam API be implemented to run on top Storm Core primitives ? -roshan On 9/15/16, 2:00 PM, "P. Taylor Goetz" wrote: >I¹ve been tinkering with implementing an Apache Beam runner on top of >Storm and would like to open it up so others in the community can

Re: Details about the Storm Scheduler

On 8/22/16, 6:55 AM, "Bobby Evans" wrote: >Getting the CPU used for a worker is simple, but getting the CPU used for >individual components is not so simple/almost impossible for >multi-threaded bolts/spouts. The current scheduling assumes that the CPU >for all

Re: Details about the Storm Scheduler

AM To: "dev@storm.apache.org<mailto:dev@storm.apache.org>" <dev@storm.apache.org<mailto:dev@storm.apache.org>>, Bobby Evans <ev...@yahoo-inc.com<mailto:ev...@yahoo-inc.com>> Cc: Roshan Naik <ros...@hortonworks.com<mailto:ros...@hortonworks.com>>

Re: Storm benchmarks

http://hortonworks.com/blog/microbenchmarking-storm-1-0-performance/ -roshan On 11/4/16, 7:13 AM, "Paul Poulosky" wrote: >IEEE Xplore Document - Benchmarking Streaming Computation Engines: Storm, >Flink and Spark Streaming > > >| >| | >IEEE Xplore

Re: Too many threads per bolt instance (as seen with jconsole images)

at gets me 20 threads/bolt. And I am not creating any new threads inside my bolts. On Thu, Mar 16, 2017 at 6:12 PM, Roshan Naik <ros...@hortonworks.com> wrote: > I assume you mean ..3 thds per spout/bolt is on the high side ? > Currently there is an executor thd,

STORM-2423 for 1.1.0

All, Found a critical issue with JoinBolt that would like to get a fix into 1.1.0. https://issues.apache.org/jira/browse/STORM-2423 -roshan

Re: Too many threads per bolt instance (as seen with jconsole images)

Typically there are 3 threads per spout or bolt instance (unless the spout or bolt is spawning its own threads). There are the acker bolts, event loggers and system bolt running there too. Then there are several more per worker. See a summary of it in section 3.1 of this document

Re: Debugging on Storm Topologies

Submit your topology using LocalCluster in the topology’s main() method. Then you should be able to set breakpoints etc. -roshan On 3/14/17, 2:14 PM, "Vankadari, Chandra Sekhar (Contractor)" wrote: Hey, I want to set break points in my Storm

Re: Too many threads per bolt instance (as seen with jconsole images)

tal threads/node). Some of these would be essential for the worker's book-keeping but still it seems we can get a lot of performance boost if we can somehow reduce these threads and the associated context switching between them. On Thu, Mar 16, 2017 at 12:53 PM,

Re: [CANCELED] [VOTE] Release Apache Storm 1.1.0 (RC2)

> rare case. > > > > I would like to see the opposite approach, removing all connectors (or > > just keeping storm-kafka/storm-kafka-client and some more preferred > things) > > and its relevant examples from binary dist. > > > > What do you thi

Re: Storm 1.x-branch won't build in IntelliJ anymore

>address them at 2.1 or other minor versions. > >Any other opinions? > >- Jungtaek Lim (HeartSaVioR) > >2017년 3월 1일 (수) 오전 9:27, Roshan Naik <ros...@hortonworks.com>님이 작성: > >Lately (about a week and half maybe) it has not been possi

Re: [CANCELED] [VOTE] Release Apache Storm 1.1.0 (RC2)

Found these additional issues: 1- BUG: Even if topology.eventlogger.executors=0, the event_logger bolt is instantiated … previously observed to cause ~10% degradation in perf even with logging disabled. 2- Missing Jars: The storm-*-examples jars are missing in the binary distro (other than

Re: Few observations related to KafkaSpout implementation (1.1.0)

There is a set of simple topologies in storm-perf for such benchmarking… we have one there that measures perf of the old KafkaSpout : https://github.com/apache/storm/blob/1.x-branch/examples/storm-perf/src/main/java/org/apache/storm/perf/KafkaSpoutNullBoltTopo.java We don’t have one yet for the

Request for holding off commits to storm-client module

Storm Devs, My PR for STORM-2306 (messaging subsystem redesign) is almost ready. Not surprisingly, this PR brings extensive modifications in storm-client module and touches lightly on some others. Just finished manually rebasing my local changes (to ~90 files) from an old version of master

Re: Request for holding off commits to storm-client module

If you want to hold off commits just before you submit a pull request, and you expect to submit it in several days, I'm also OK to wait for that. I just would like to ensure that holding doesn't stay longer. Thanks, Jungtaek Lim (HeartSaVioR) 2017년 7월 24일 (월) 오전 7

Re: [DISCUSS] Code Style

- Personally not a big fan of _ for members either from the readability standpoint. Also much of the existing code doesn’t follow this. - Personally I find it is easier to write/read one-liner if/else without the curlys. Always wondered about the rationale behind that. Would suggest any such a

Recording - Storm & Kafka Meetup on April 20th 2017

Louro (Hortonworks) - [20m] – Rethinking the Storm 2.0 Worker - Roshan Naik (Hortonworks) - [57m] – Storm in Retail Context: Catalog data processing using Kafka, Storm & Microservices - Karthik Deivasigamani (WalMart Labs) - [1h: 54m:45sec] – Schema Regi

Re: [Discussion]: Storm Improvemement Proposal (SIP) to discuss changes

If I am looking at the Kafka site correctly, I see that Kafka has a total of 167 KIPs so far. So I assume that minor new features would not be parrt of the SIP ? Unlike Kafka, since Storm has a number of connectors (that keep growing), I am speculating the SIP process might get somewhat

Re: [DISCUSS] Storm 2.0 Roadmap

ics V2 sooner since > we > >> lost metrics even when doing normal operation like restarting worker, > >> rebalancing, and so on. Eventually we need to fight with dynamic > scaling, > >> and then metrics will be broken often. > >>

Re: [Discussion]: Storm Improvemement Proposal (SIP) to discuss changes

is really required for any enhancement which is not really big. On Sat, Jun 10, 2017 at 5:13 AM, Roshan Naik <ros...@hortonworks.com> wrote: If I am looking at the Kafka site correctly, I see that Kafka has a total of 167 KIPs so far. So I assume that min

Re: [DISCUSS] Decouple Storm core and connectors

il.com> wrote: I think we all agree that releasing connectors as part of a Storm release hinders the frequency of the release cycle for both Storm proper, as well as connectors. If that’s the case, then the question is how to proceed. -Taylor > On Jan 31, 2018,

Re: [DISCUSS] Decouple Storm core and connectors

One thought is to … - do a frequent separate release - *and also* include the latest stuff along with each Storm release. -roshan On 1/31/18, 10:43 AM, "generalbas@gmail.com on behalf of Stig Rohde Døssing" wrote: Hugo,

Re: [DISCUSS] Plan on Storm 2.0.0

Would be good to callout which of issues listed in Storm 2.0 epic are considered “blockers” vs “very desirable” vs “good to have” -roshan On 2/19/18, 3:29 PM, "Jungtaek Lim" wrote: Hi devs, We've just released Storm 1.2.1 (not officially announced but vote

Re: Question regarding Storm

Perhaps you can try some things depending on how feasible they it is for your case: 1- Try to design and deploy your topology to maximize communication within a worker process instead fo across worker processes. 2- Alternatively you can spin up multiple single worker instances of you toplo3-

Re: New Committer/PMC Member: Roshan Naik

; > > Congratulations! Roshan> > > >> > > > Ethan Li> > > >> > > > > On Apr 5, 2018, at 21:40, Jungtaek Lim <ka...@gmail.com> wrote:> > > > >> > > > > Congrats Roshan!> > > > >> > > >

Re: New Committer/PMC Member: Ethan Li

Congratulations Ethan.  Sent from Yahoo Mail for iPhone On Monday, April 16, 2018, 9:23 AM, Bobby Evans wrote: Please Join with me in welcoming Ethan Li as the newest Apache Storm committer and PMC member. Great work!

Re: [VOTE] Release Apache Storm 2.0.0 (rc3)

external/storm-autocreds  appears to have grown a lot. In 1.2.2:   93mb,  113 filesIn 2.0.0 :  173mb,  204 files. fyi: let me know if the formatting in my prev mail is messed up ... i can resend. -roshan On Sunday, October 21, 2018, 5:54:36 PM PDT, Roshan Naik wrote: The 2.0

Re: [VOTE] Release Apache Storm 2.0.0 (rc3)

The 2.0 compressed tar.gz is ~270 MB in comparison to 1.2.2 jar which stands at ~160MB. Seems like a big jump and wanted to make sure there is no unintended bloat occurring. Listing of files larger than 5MB in both (ordered by size):  23M  =>  apache-storm-1.2.2/lib/storm-core-1.2.2.jar 21M  => 

Re: Hadoop Contributor Meetup

I can do a talk on the rework of threading,  messaging and backpressure  for 2.0  Roshan Sent from Yahoo Mail for iPhone On Friday, September 21, 2018, 8:37 AM, Bobby Evans wrote: There is a Hadoop Contributor Meetup at Oath/Yahoo on Monday in Sunnyvale CA

Re: Storm 2.0.0 release?

;>>>> Thanks, >>>>> >>>>> Bobby >>>>> >>>>> On Thu, Dec 13, 2018 at 1:07 PM Stig Rohde Døssing < >>>> stigdoess...@gmail.com >>>>>> >>>>> wrote: >>>>> >>>

Re: [VOTE] Release Apache Storm 2.0.0 (rc4)

Was verifying the performance on 2.0 and noticed that the inter worker messaging has fallen off a cliff With ConstSpoutNullBoltTopo (producer batch size=1k, ackers=0, workers=2) it was ~3.2mill/sec down  and now its down to  ~1.6mill/sec. Something has impacted this recently. Will try to

Re: [VOTE] Release Apache Storm 2.0.0 (rc4)

mp, restart worker : OK   - log search : OK I don't think perf. degrade is a blocker for this release, since we are still providing far better performance then 1.x. As Roshan is planning to address this, that could be fixed in 2.0.x hopefully. Thanks, Jungtaek Lim (heartSaVioR) 2019년 1월 19일 (토) 오전 5:

Re: [NOTICE] Migration of storm repos to gitbox is done

Thanks Ethan.  Sent from Yahoo Mail for iPhone On Sunday, January 20, 2019, 7:33 PM, Ethan Li wrote: Hi Team, The migration to https://gitbox.apache.org/ is done (https://issues.apache.org/jira/browse/INFRA-17705 ). Thanks, - Ethan

Storm 2.0 blogs ?

Now that  2.0 has all the votes it needs to move forward, maybe a good time to think of some blogs to go with this long awaited release.   Some potential topics that come to mind are: 1- Overview of new features and major changes since 1.x2- Re-architecture (messaging, threading, back

Re: Storm 2.0 blogs ?

Like Taylor’s suggestion of collectively contributing small blurbs on features for the Release announcement. Thats the first thing people look on hearing a release announcement. The jira list is not very understandable at first glance. Based on suggestions so far and a quick scan of the jiras

Re: Storm 2.0.0 release?

Sounds like -  https://github.com/apache/storm/pull/2913 (STORM-3290) is merged -  https://github.com/apache/storm/pull/2908 (STORM-3276) is nearly complete and may need some small tweaks. -  https://github.com/apache/storm/pull/2907  (STORM-2990, STORM-3279) appears ready to be committed ?

Re: New Storm Committer and PMC Member

Congratulations Govind. Roshan Sent from Yahoo Mail for iPhone On Wednesday, January 9, 2019, 10:47 AM, Ethan Li wrote: Congratulations! Govind. Well deserved! Ethan > On Jan 9, 2019, at 12:40 PM, Hugo Louro wrote: > > Congratulations Govind. Very well deserved. Thank you for all your >

Re: Is the powered-by file in the Storm repository necessary?

Having it in the site is sufficient IMO as well.  On Saturday, September 15, 2018, 4:16:05 AM PDT, Stig Rohde Døssing wrote: Hi, We have a powered-by.md file in the Storm repository's /docs directory. There's also a powered-by file in the storm-site repo root, as well as in each

Re: Regarding releasing Apache Storm 2.0.0

Happy to see consensus in moving fwd with 2.0 soon.  I will try to get a minor patch (STORM-3205) within 24 hours ... as it seems like it has potential to deliver a decent perf boost and energy savings. One thing I am hoping we can address before releasing Storm 2 is... to fix the naming of the

Re: Storm 2.0 blogs ?

have dropped Druid support, so 13 should be only Kinesis. > > > > Which format should the blurbs be written in? (Markdown?) > > > > Den ons. 23. jan. 2019 kl. 08.57 skrev Roshan Naik > > : > > > >> Like Taylor’s suggestion of collectively contributing sm

Re: [VOTE] Release Apache Storm 2.0.0 RC5

 Will there be another RC or this one is good to continue with ? Sent from Yahoo Mail for iPhone On Wednesday, March 27, 2019, 9:41 AM, Derek Dagit wrote: * Downloaded source ZIP, `mvn clean install`, all passed * Verified signatures and checksums * Packaged my build and ran a single-tenant

Re: [RESULT] [VOTE] Release Apache Storm 1.2.3 (rc1)

Gita,    Sorry for the late response. Due to some impt issues being discovered just prior to the release was going to be published, the release was held off. Fixes are in and another RC should be out expected soon. RC typically takes about a week or two to get approved and published.-roshan

Re: [RESULT] [VOTE] Release Apache Storm 1.2.3 (rc1)

for the update. Any approximate timelines for the release? I see last release happened on May 17, 2018, any possibility of releasing 1.2.3 earlier if this is being blocked by 2.0 release? On Thu, Feb 28, 2019 at 4:29 AM Roshan Naik wrote: >  Gita,    Sorry for the late response. Due to some impt iss

Re: [VOTE] Release Apache Storm 2.0.0 (rc4)

Correct me if I am wrong, this seems to be a bug with a workaround but not an exploitable security hole. ? if this is not a security hole, and the workaround is realistic then we should go ahead with the current RC IMO.-roshan On Tuesday, January 29, 2019, 10:26:53 AM PST, Kishorkumar Patil

Re: [VOTE] Release Apache Storm 2.0.0 (rc4)

, 2019 at 8:22 AM Govind Menon wrote: > +1 for continuing with the RC > > On Tue, 29 Jan 2019 at 16:49, Arun Mahadevan wrote: > > > Agree with continuing with the RC if this is not a blocker. > > > > On Tue, 29 Jan 2019 at 12:29, Roshan Naik > > > wro

Re: [VOTE] Release Apache Storm 2.0.0 RC5

? On Wednesday, March 27, 2019, 4:39:33 PM PDT, Roshan Naik wrote:  Will there be another RC or this one is good to continue with ? Sent from Yahoo Mail for iPhone On Wednesday, March 27, 2019, 9:41 AM, Derek Dagit wrote: * Downloaded source ZIP, `mvn clean install`, all passed

Re: [RESULT] [VOTE] Release Apache Storm 1.2.3 (rc1)

Taylor,    Could you share your plans for the 1.2.3 rc  ?-roshan On Thursday, April 11, 2019, 10:20:50 PM PDT, gita gita wrote: Sorry for constant nagging, Any update on the release. On Thu, Mar 7, 2019 at 12:22 PM Roshan Naik wrote: > It appears more likely that both relea

Re: [Discuss] New release managers

Yes, having the release process documented will help a great deal to let anyone to step in as release manager.  Roshan  Sent from Yahoo Mail for iPhone

Re: Storm 2.0 Maven Artifacts

Does this mean that the artifacts were not been pushed to the mvn repo ? Or do we just need to wait for the artifacts to show up there .. and if so how long ?  -roshan On Monday, June 3, 2019, 9:33:44 AM PDT, Stig Rohde Døssing wrote:  No, I think you are right. It doesn't look like

Re: [VOTE] Release Apache Storm 1.2.3 (rc2)

Friendly reminder on this one. Roshan Sent from Yahoo Mail for iPhone On Saturday, June 15, 2019, 9:15 AM, P. Taylor Goetz wrote: This vote is now closed and passes with 3 binding +1 votes and 1 non-binding +1 vote. I will release the artifacts and create a release announcement. -Taylor

Re: Storm 2.0 blogs ?

Taylor,    Somewhere in the release notes it would be good to mention the following: - Compatibility: 2.0 is backward compatible with 1.x, but they need to recompile the topo jars.  - Dependency changes: Like Alexandre, others are likely to discover the hard way that storm-core is something

Re: [VOTE] Release Apache Storm 1.2.3 (rc2)

+1- verified binary distribution- setup local cluster - ran some perf topos with two workers and acking enabled- basic checking of the UI -roshan On Saturday, May 11, 2019, 1:21:26 PM EDT, Alexandre Vermeerbergen wrote: Oh and I forgot to mention that all my tests are made using

Re: [VOTE] Release Apache Storm 2.0.0 RC7

Yes, you need to rebuild your topology jars against 2.0.  If you have some settings to tweak perf with 1.x,  refer to https://github.com/apache/storm/blob/master/docs/Performance.md for the 2.x configs. On Tuesday, April 30, 2019, 2:41:49 PM PDT, Alexandre Vermeerbergen wrote:

Re: [RESULT] [VOTE] Release Apache Storm 2.0.0 RC7

Ooops... just missed it...  my +1 as well Just finished verifying the foll:- spun up local cluster with binary dist- checked single a 2 worker modes. with and without ackers in each mode- ran SimplifiedWordCountTopo- did some quick perf verifications with ConstSpoutNullBolt- basic checking of

Re: [Discuss] ARM CI for Storm

Thanks for volunteering.  Sent from Yahoo Mail for iPhone On Wednesday, June 26, 2019, 7:08 PM, Yikun Jiang wrote: Yes, we will definitely help to fix ARM test failures in PRs, and the OpenLab CI will tell us which PR perhaps has ARM compatible problem, if it's an easy fix problem, I think

Re: [DISCUSS] Create a checklist for testing new Storm release

+1  that does look like a nice list.  On Thursday, August 1, 2019, 09:30:24 AM PDT, Ethan Li wrote: Right. I found this http://www.apache.org/dev/release-distribution.html#sigs-and-sums In the new release, we should

Re: [ANNOUNCE] New Committer/PMC Member Aaron Gresch

Welcome Aaron, congratulations. On Saturday, August 10, 2019, 06:56:39 PM PDT, Kishorkumar Patil wrote: Congratulations Aaron! On Sat, Aug 10, 2019 at 8:14 PM Derek Dagit wrote: > Welcome, Aaron! > > On Sat, Aug 10, 2019 at 08:18:40AM -0500, Ethan Li wrote: > > > > Congratulations,

Steps for Updating Storm Website

Taylor,    Could you please share the steps for updating the Storm website?  -roshan

Re: Steps for Updating Storm Website

://github.com/apache/storm/blob/master/DEVELOPER.md#contribute-documentation . Essentially, it comes down to updating the github: https://git-wip-us.apache.org/repos/asf?p=storm-site.git;a=summary -Kishor On Mon, Sep 30, 2019 at 6:34 PM Roshan Naik wrote: > Taylor,Could you please share the st

Re: Location for 2.0 blogs

to the release posts, or maybe we could add another button next to news for blog posts? Den søn. 22. sep. 2019 kl. 12.32 skrev Roshan Naik : > I see that we have never published any blogs previouslyanywhere on the > Apache Storm website.So wondering where would be a good location to p

Location for 2.0 blogs

I see that we have never published any blogs previouslyanywhere on the Apache Storm website.So wondering where would be a good location to publish the (overdue) 2.0 perf blog that I am close to completing. I guess that might dictate the format (html / wiki) the final version needs to be in.

Recordings of Storm 2.0 talks at the Bay Area Hadoop Meetup last week.

1- Raising the performance bar for stream processing with Apache Storm 2.0 - Roshan Naik, (Uber) Discussion of performance improvements as well as the new 2.0 architecture for messaging subsystem, threading model & back pressure.  https://www.youtube.com/watch?v=QsPzAtZXIk4 2- Apache S

Storm 2.0 talks at Bay Area Hadoop meetup

If you are in the San Francisco Bay Area, please join us at the Bay Area Hadoop meetup tomorrow (Oct 29th 2019) featuring these talks on  Storm 2.0 : 1 - Raising the performance bar for stream processing with Apache Storm 2.0 - Roshan Naik (Uber),   The effort to rearchitect Storm's core

Re: Crashes when running storm 2.1 topologies on multiple workers

Ideally you want to use multiple instances of a bolt/spout over doing internal multithreading. if you must use internal threading, you will need to do your own synchronization as well. In general these distributed systems are designed to relieve users from doing their own MT and

[jira] [Updated] (STORM-1199) Create HDFS Spout

[ https://issues.apache.org/jira/browse/STORM-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated STORM-1199: --- Attachment: HDFSSpoutforStorm.pdf Uploading requirements and design notes document > Create H

[jira] [Updated] (STORM-1199) Create HDFS Spout

[ https://issues.apache.org/jira/browse/STORM-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated STORM-1199: --- Issue Type: New Feature (was: Bug) > Create HDFS Spout > - > >

[jira] [Created] (STORM-1199) Create HDFS Spout

Roshan Naik created STORM-1199: -- Summary: Create HDFS Spout Key: STORM-1199 URL: https://issues.apache.org/jira/browse/STORM-1199 Project: Apache Storm Issue Type: Bug Reporter

[jira] [Commented] (STORM-1351) Storm spouts and bolts need a way to communicate problems back to toplogy runner

[ https://issues.apache.org/jira/browse/STORM-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034553#comment-15034553 ] Roshan Naik commented on STORM-1351: The bolt will need backoff in case of certain error conditions

[jira] [Commented] (STORM-1351) Storm spouts and bolts need a way to communicate problems back to toplogy runner

[ https://issues.apache.org/jira/browse/STORM-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037197#comment-15037197 ] Roshan Naik commented on STORM-1351: My observation here is ... it is preferable to externalize

[jira] [Updated] (STORM-1199) Create HDFS Spout

[ https://issues.apache.org/jira/browse/STORM-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated STORM-1199: --- Attachment: hdfs-spout.1.patch Uploading initial implementation with a few UT for review. Includes

[jira] [Updated] (STORM-1351) Storm spouts and bolts need a way to communicate problems back to toplogy runner

[ https://issues.apache.org/jira/browse/STORM-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated STORM-1351: --- Description: A spout can be having a problem generating a tuple in nextTuple() because

[jira] [Commented] (STORM-1351) Storm spouts and bolts need a way to communicate problems back to toplogy runner

[ https://issues.apache.org/jira/browse/STORM-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15025659#comment-15025659 ] Roshan Naik commented on STORM-1351: Similar problem exists with Bolts too .. for example a Hive Bolt

[jira] [Assigned] (STORM-1351) Storm spouts and bolts need a way to communicate problems back to toplogy runner

[ https://issues.apache.org/jira/browse/STORM-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik reassigned STORM-1351: -- Assignee: Roshan Naik > Storm spouts and bolts need a way to communicate problems b

[jira] [Created] (STORM-1351) Storm spouts and bolts need a way to communicate problems back to toplogy runner

Roshan Naik created STORM-1351: -- Summary: Storm spouts and bolts need a way to communicate problems back to toplogy runner Key: STORM-1351 URL: https://issues.apache.org/jira/browse/STORM-1351 Project

[jira] [Commented] (STORM-1351) Storm spouts and bolts need a way to communicate problems back to toplogy runner

[ https://issues.apache.org/jira/browse/STORM-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15027632#comment-15027632 ] Roshan Naik commented on STORM-1351: I dont think there is a need for a different backoff logic

[jira] [Commented] (STORM-1351) Storm spouts and bolts need a way to communicate problems back to toplogy runner

[ https://issues.apache.org/jira/browse/STORM-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15032431#comment-15032431 ] Roshan Naik commented on STORM-1351: Thanks [~basti.lj] for the insight on CPU usage drop

[jira] [Assigned] (STORM-1199) Create HDFS Spout

[ https://issues.apache.org/jira/browse/STORM-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik reassigned STORM-1199: -- Assignee: Roshan Naik > Create HDFS Spout > - > >

[jira] [Updated] (STORM-1199) Create HDFS Spout

[ https://issues.apache.org/jira/browse/STORM-1199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Roshan Naik updated STORM-1199: --- Attachment: HDFSSpoutforStorm v2.pdf Uploading the v2 of the design and requirements doc

  1   2   >