GitHub user dasahcc opened a pull request:
https://github.com/apache/helix/pull/50
Monitors for Task framework
As more and more features added in Task Framework, monitoring workflows,
jobs and tasks take a vital part of stablizing Helix for long run. In present
Helix code, monitoring was implemented via Java MBean. Cluster status,
instances, instances per resource, resources, message queue, stage latency and
state transition stats have their own monitor beans. Similar for Task
Framework, there will be monitor beans for tracking task, job and workflow,
but a little different of the data that we are collecting. In following design
concept section, the explanation of design will be exposed. Metrics existed and
needed will be listed in section Metrics Content.
Overview
"Type" defined aggregation will be the key concept in monitoring metrics
design. The quantity of workflows, jobs and tasks is obviously large for daily
collection. It is better to have montioring for different types of workflows
and jobs since different users have their own requirements of monitoring.
Type Defined Aggregation
Helix will provides the metrics based on different types as user defined.
If user does not have any predefined workflow type, Helix will collect
"undefined" workflow as "DEFAULT" type. The metrics can be divided into three
categories: counters, gauge and tracking. Those three categories are monitored
as different purposes. These computational logics will be implemented in the
monitor class such as "WorkflowMonitor" and "JobMonitor". The data to be
monitored will described in following section, Metrics Content. In addition,
the metrics will be set in auto rebalancer, which is the place to mark number
changes.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dasahcc/helix helix-0.6.x
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/helix/pull/50.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #50
----
commit 36eeb673e59c3229c1aafca3bdfa4d8802ebab5c
Author: Junkai Xue <[email protected]>
Date: 2016-09-02T00:11:22Z
Monitors for Task framework
1. Add workflow and job monitor MBeans and implementations.
2. Add tests for MBean existing checking.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
