I hate to give this answer, but I think it really depends on your application. If you're doing distributed machine learning or video compression or something that's CPU heavy, then it'll be CPU heavy. If you're doing pre-aggregation or rolling windows or other CPU-light analysis, you're more likely to be memory- or network- bound.
Or people might just be scaling horizontally across a lot of cheap worker nodes rather than fewer nodes with a lot of CPUs. :) -Cody On Thu, May 1, 2014 at 11:57 AM, Software Dev <[email protected]>wrote: > Seems like all of these setups involve a small number of CPU's??? Does > storm typically require more RAM than CPU.. ie which is usually the > bottleneck? > > On Wed, Apr 30, 2014 at 8:54 PM, Michael Rose <[email protected]> > wrote: > > In AWS, we're fans of c1.xlarges, m3.xlarges, and c3.2xlarges, but have > seen > > Storm successfully run on cheaper hardware. > > > > Our Nimbus server is usually bored on a m1.large. > > > > Michael Rose (@Xorlev) > > Senior Platform Engineer, FullContact > > [email protected] > > > > > > > > On Wed, Apr 30, 2014 at 9:48 PM, Cody A. Ray <[email protected]> > wrote: > >> > >> We use m1.larges in EC2 for both nimbus and supervisor machines (though > >> the m1 family have been deprecated in favor of m3). Our use case is to > do > >> some pre-aggregation before persisting the data in a store. (The main > >> bottleneck in this setup is the downstream datastore, but memory is the > >> primary constraint on the worker machines due to the in-memory cache > which > >> wraps the trident state.) > >> > >> For what its worth, Infochimps suggests c1.xlarge or m3.xlarge machines. > >> > >> Using the Amazon cloud machines as a reference, we like to use either > the > >> c1.xlarge machines (7GB ram, 8 cores, $424/month, giving the highest > >> CPU-performance-per-dollar) or the m3.xlargemachines (15 GB ram, 4 > cores, > >> $365/month, the best balance of CPU-per-dollar and RAM-per-dollar). You > >> shouldn’t use fewer than four worker machines in production, so if your > >> needs are modest feel free to downsize the hardware accordingly. > >> > >> Not sure what others would recommend. > >> > >> -Cody > >> > >> > >> On Wed, Apr 30, 2014 at 5:57 PM, Software Dev < > [email protected]> > >> wrote: > >>> > >>> What kind of specs are we looking at for > >>> > >>> 1) Nimbus > >>> 2) Workers > >>> > >>> Any recommendations? > >> > >> > >> > >> > >> -- > >> Cody A. Ray, LEED AP > >> [email protected] > >> 215.501.7891 > > > > > -- Cody A. Ray, LEED AP [email protected] 215.501.7891
