Justin Leet created METRON-1657:
-----------------------------------
Summary: Parser aggregation in storm
Key: METRON-1657
URL: https://issues.apache.org/jira/browse/METRON-1657
Project: Metron
Issue Type: Bug
Reporter: Justin Leet
Assignee: Justin Leet
Currently our parsing solution requires one storm topology per sensor. It has
been complained that this may be wasteful of resources and that, rather than
one storm topology per sensor, it would be advantageous to have multiple
sensors in the same topology. The benefit to this is that it would require
fewer storm slots.
The issue with this is that whenever we've aggregated functionality like this
before, we've run into issues appropriately being able to scale storm (e.g.
batch vs random access indexing in the same topology). The main point in
addressing this is to recommend that parsers with similar velocities and
complexity are grouped together.
Particularly for a first cut, leave the configuration mostly as-is, while
allowing for comma separated lists of sensors in start_parser_topology.sh (e.g.
bro,yaf creates a aggregated parser consisting of those two).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
