[
https://issues.apache.org/jira/browse/PHOENIX-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14214850#comment-14214850
]
Jan Fernando commented on PHOENIX-1452:
---------------------------------------
A couple of other metrics that I think would help with performance testing and
production monitoring would be:
1) Tracking # of spool files generated at a configurable interval. For example,
this would be great to graph so that if we don't expect to spool or cannot
spool, for compliance reasons, to monitor and alert if spooling was occurring.
2) Optionally logging query information on queries that generated spool files
to help analyze which queries were spooling.
3) Memory Management usage metrics - min, max, average percentage of max
memory allocated by the GlobalMemoryManager during a configurable interval.
This would give this ability to extrapolate out and determine how close to the
wind we are sailing in terms of needing to spool for our current workload and
understand how much data we are shipping back from HBase and whether it is
inline with expectations.
4) Histogram data of wait times of queued tasks - this would allow us to
understand how contention for threads in the thread pool changes with different
scenarios
> Add logging to allow easier monitoring of client-side Thread Pool and Queue
> usage
> ----------------------------------------------------------------------------------
>
> Key: PHOENIX-1452
> URL: https://issues.apache.org/jira/browse/PHOENIX-1452
> Project: Phoenix
> Issue Type: Improvement
> Affects Versions: 5.0.0, 4.2
> Reporter: Jan Fernando
>
> For performance testing and tuning of features that use Phoenix and for
> production monitoring it would be really helpful to easily be able to extract
> statistics about Phoenix's client-side Thread Pool and Queue Depth usage to
> help with tuning and being able to correlate the impact of tuning these 2
> parameters to query performance.
> For global per JVM logging one of the following would meet my needs, with a
> preference for #2:
> 1. A simple log line that that logs the data in ThreadPoolExecutor.toString()
> at a configurable interval
> 2. Exposing the ThreadPoolExecutor metrics in PhoenixRuntime or other global
> client exposed class and allow client to do their own logging.
> In addition to this it would also be really valuable to have a single log
> line per query that provides statistics about the level of parallelism i.e.
> number of parallel scans being executed. I don't full explain plan level of
> data but a good heuristic to be able to track over time how queries are
> utilizing the thread pool as data size grows etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
