There are some tuning suggestions peppered throughout the READMEs, but it is somewhat specific to the environment. That said, we could probably do better with documenting the performance tuning process. Sorry, I know it's not really that helpful.
https://github.com/apache/incubator-metron/tree/master/metron-platform/metron-indexing#notes-on-performance-tuning https://github.com/apache/incubator-metron/tree/master/metron-platform/metron-parsers/#notes-on-performance-tuning <https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-parsers/README.md#notes-on-performance-tuning> https://github.com/apache/incubator-metron/tree/master/metron-platform/metron-enrichment/#notes-on-performance-tuning <https://github.com/apache/incubator-metron/blob/master/metron-platform/metron-enrichment/README.md#notes-on-performance-tuning> Jon On Thu, Apr 20, 2017 at 8:45 AM Ali Nazemian <[email protected]> wrote: > Hi all, > > I was wondering what the best practice would be in terms of defining a > right value for the number of workers and executors as well as right value > for spout and bolt parallelisation? What about the number of partitions for > "indexing", "enrichments" and device parsers Kafka topics? > > I have seen people using 1 worker per storm worker host and 1 executor per > CPU core for a CPU intensive topologies and more than 1 executor per core > for IO intensive ones. However, finding the right value is tightly bounded > to the way of implementation, so general practice for Storm might be not > completely correct for the Metron use case. > > Cheers, > Ali > -- Jon
