Hi, If you want one tip from my experience with sandstorm: don't use a thread pool per stage if you want to scale the number of stages.
A sandstorm thread manager has the responsibility to "run" the stages. It adapts to the load on stages by different mesurements (queue threshold, response time, etc...) The default technique in sandstorm to "balance" or "adapt", is to add threads, up to a max. The main flaw, is that this adjustement granularity is too coarse if you have small thread pools. (like if you had a max of 5 threads, you couldn't throttle very smoothly, so it will oscillate in the size controller) The first improvements is to add a thread to a pool only if you can borrow it from another pool, or basically, to not use a thread pool per stage, but just one global threadpool. (But then, you must ensure you have at least one thread per stage to avoid starving). I prefer thread management adjustment based on thread priority (for example, from 1 to 6, to hopefully preserve some CPU for "normal" priority 5 threads.) Note 1: I strongly recommend you plug-in your own "ThreadManagerIF" implementation. Note 2: Make sure you always rethrow InterruptedException! Note 3: You you allow a stage to block, you MUST have a blocking enqueue to the stage' sink, so that back pressure can be applied. Note 4: Tell the thread manager if a stage is about to block, so that it doesn't screw up the throttling routine (I use a similar pattern to doPriviledgeAction(), which I called TM.doBlockingAction(), which safely let the TM proxy the blocking call). Note 5: sandstorm FiniteQueue has many bugs. See the seda list archive. Good luck. Quartz12h. __________________________________ Do you Yahoo!? Yahoo! Hotjobs: Enter the "Signing Bonus" Sweepstakes http://hotjobs.sweepstakes.yahoo.com/signingbonus --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
