Robert Joseph Evans created STORM-2889:
------------------------------------------

             Summary: Add Metrics Query API (Thrift and REST)
                 Key: STORM-2889
                 URL: https://issues.apache.org/jira/browse/STORM-2889
             Project: Apache Storm
          Issue Type: Sub-task
          Components: storm-server
    Affects Versions: 2.0.0
            Reporter: Robert Joseph Evans


Once we have a storage engine and even if it is just a few metrics we want to 
add in an API to query the metrics.

It should be able to do simple aggregations (Min, Max, Mean, and Sum) across 
different time ranges, with different roll up windows.  It should also be able 
to filter based off of things like topology id, component name, component id, 
stream name, and host name.  It is fine to have some of these required, such as 
topology id, at the beginning.

It should be able to return results either all together or split up by the 
different topology ids, component names, component ids, stream names, or host 
names.

For Example the UI shows things like

SUM of {tuples_emitted, tuples_transferred, acked, failed}  in the last {10 
mins, 3 hours, 1 day, all time} for all spouts a.k.a a set of component names 
that we got from the topology itself.
AVERAGE of {complete_latency} in the last {10 mins, 3 hours, 1 day, all time} 
for a set of component names.

SUM of {tuples_emitted, tuples_transferred, acked, failed}  in the last {10 
mins, 3 hours, 1 day, all time} for all components separated by each component 
name.
AVERAGE of {complete_latency} in the last {10 mins, 3 hours, 1 day, all time} 
for all components separated by each component name.

And there are others too.



The API should be open to expansion for other types of filters, aggregates, and 
even data types.  The following are not things that should be implemented, but 
are things that we should think about how we can leave the API open for 
expansion alter on so we could support them in a clean way.

Specifically some filters I would like to see in the future include topology 
name or even better a basic pattern we can use to match topology ids and the 
owner of a topology.

I would also like to eventually support percentile sketches as a data type so 
instead of just having an average of the 99th%ile collected by each component, 
we get an much more accurate approximation of what the 99th%ile really is in 
aggregate.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Reply via email to