Thanks, Jon. I have already seen those documents. I don't like to do the performance tuning via trial and error, so I am looking for some facts to find the best value or at least a range of values for playing with parameters.
On Thu, Apr 20, 2017 at 10:52 PM, [email protected] <[email protected]> wrote: > There are some tuning suggestions peppered throughout the READMEs, but it > is somewhat specific to the environment. That said, we could probably do > better with documenting the performance tuning process. Sorry, I know it's > not really that helpful. > > https://github.com/apache/incubator-metron/tree/master/ > metron-platform/metron-indexing#notes-on-performance-tuning > https://github.com/apache/incubator-metron/tree/master/ > metron-platform/metron-parsers/#notes-on-performance-tuning > <https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-parsers/README.md#notes-on-performance-tuning> > https://github.com/apache/incubator-metron/tree/master/ > metron-platform/metron-enrichment/#notes-on-performance-tuning > <https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-enrichment/README.md#notes-on-performance-tuning> > > Jon > > On Thu, Apr 20, 2017 at 8:45 AM Ali Nazemian <[email protected]> > wrote: > >> Hi all, >> >> I was wondering what the best practice would be in terms of defining a >> right value for the number of workers and executors as well as right value >> for spout and bolt parallelisation? What about the number of partitions for >> "indexing", "enrichments" and device parsers Kafka topics? >> >> I have seen people using 1 worker per storm worker host and 1 executor >> per CPU core for a CPU intensive topologies and more than 1 executor per >> core for IO intensive ones. However, finding the right value is tightly >> bounded to the way of implementation, so general practice for Storm might >> be not completely correct for the Metron use case. >> >> Cheers, >> Ali >> > -- > > Jon > -- A.Nazemian
