Robert Joseph Evans created STORM-2889:
------------------------------------------
Summary: Add Metrics Query API (Thrift and REST)
Key: STORM-2889
URL: https://issues.apache.org/jira/browse/STORM-2889
Project: Apache Storm
Issue Type: Sub-task
Components: storm-server
Affects Versions: 2.0.0
Reporter: Robert Joseph Evans
Once we have a storage engine and even if it is just a few metrics we want to
add in an API to query the metrics.
It should be able to do simple aggregations (Min, Max, Mean, and Sum) across
different time ranges, with different roll up windows. It should also be able
to filter based off of things like topology id, component name, component id,
stream name, and host name. It is fine to have some of these required, such as
topology id, at the beginning.
It should be able to return results either all together or split up by the
different topology ids, component names, component ids, stream names, or host
names.
For Example the UI shows things like
SUM of {tuples_emitted, tuples_transferred, acked, failed} in the last {10
mins, 3 hours, 1 day, all time} for all spouts a.k.a a set of component names
that we got from the topology itself.
AVERAGE of {complete_latency} in the last {10 mins, 3 hours, 1 day, all time}
for a set of component names.
SUM of {tuples_emitted, tuples_transferred, acked, failed} in the last {10
mins, 3 hours, 1 day, all time} for all components separated by each component
name.
AVERAGE of {complete_latency} in the last {10 mins, 3 hours, 1 day, all time}
for all components separated by each component name.
And there are others too.
The API should be open to expansion for other types of filters, aggregates, and
even data types. The following are not things that should be implemented, but
are things that we should think about how we can leave the API open for
expansion alter on so we could support them in a clean way.
Specifically some filters I would like to see in the future include topology
name or even better a basic pattern we can use to match topology ids and the
owner of a topology.
I would also like to eventually support percentile sketches as a data type so
instead of just having an average of the 99th%ile collected by each component,
we get an much more accurate approximation of what the 99th%ile really is in
aggregate.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)