[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418704#comment-16418704
]
Shaofeng SHI commented on KYLIN-2722:
-
[~yaho] The patch author is you, the one who discussed is also you, but now you
change the Jira owner to "Wang Ken"? Can you give detail justification?
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Wang Ken
>Priority: Major
> Fix For: v2.3.0
>
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16212255#comment-16212255
]
liyang commented on KYLIN-2722:
---
Thanks Yanghong, I see where the reservoir coming from.
Commit merged.
https://github.com/apache/kylin/commit/2b32aa4ca91ccdb4a885c4c509ae173bea5658d5
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16206826#comment-16206826
]
Zhong Yanghong commented on KYLIN-2722:
---
bq. The term "active reservoir" sounds a little strange, because I don't see
inactive reservoir any where. The "active" is not meaningful without the
"inactive", just like there is no good without evil.
In codahale metrics, there's already a class named {{Reservoir}}, which is used
in {{Histogram}} for just holding events with different strategies. Here,
{{ActiveReservoir}} is not only for holding events, but also flushing events to
reporters actively.
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16205083#comment-16205083
]
liyang commented on KYLIN-2722:
---
Reviewing https://github.com/apache/kylin/pull/77, a few minor
questions/comments:
* The term "active reservoir" sounds a little strange, because I don't see
inactive reservoir any where. The "active" is not meaningful without the
"inactive", just like there is no good without evil.
* Also the term reservoir perhaps a little too big here? For me, it's just a
buffer holding events before flushing to reporter. I know people like fancy
word, but the scale is not fitting really. Be free to make a call. Naming shall
not prevent a nice feature to commit and we can always rename later.
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16134341#comment-16134341
]
liyang commented on KYLIN-2722:
---
Waiting on the refactor of dropwizard metrics
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.2.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16115750#comment-16115750
]
liyang commented on KYLIN-2722:
---
I see... Let's have an offline discussion about the high level design first and
update here. That will be more efficient.
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110664#comment-16110664
]
Zhong Yanghong commented on KYLIN-2722:
---
Hi [~liyang.g...@gmail.com], here the package is for kylin metrics rather than
for a new measure, like hllc.
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16095395#comment-16095395
]
liyang commented on KYLIN-2722:
---
Some quick comments on the patch.
* The package name could be better. It should align with existing measure
packages at least, like {{org.apache.kylin.measure.hllc}}. Maybe
{{org.apache.kylin.measure.reservoir}}?
* The config property has a naming convention [1]. Basically the property
namespace should map to java package. Thus it should be
{{kylin.measure.reservoir.active-default-class}} given above package name is
selected.
[1] http://kylin.apache.org/development/coding_naming_convention.html
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089611#comment-16089611
]
Zhong Yanghong commented on KYLIN-2722:
---
Hi [~Shaofengshi], I changed a bit of this patch.
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (KYLIN-2722) Introduce a new measure, called active reservoir, for actively pushing metrics to reporters
[
https://issues.apache.org/jira/browse/KYLIN-2722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16089374#comment-16089374
]
Zhong Yanghong commented on KYLIN-2722:
---
Hi [~Shaofengshi], thanks for your comments. I have modified it. Could you help
review on this?
> Introduce a new measure, called active reservoir, for actively pushing
> metrics to reporters
> ---
>
> Key: KYLIN-2722
> URL: https://issues.apache.org/jira/browse/KYLIN-2722
> Project: Kylin
> Issue Type: Sub-task
>Affects Versions: v2.0.0
>Reporter: Zhong Yanghong
>Assignee: Zhong Yanghong
> Attachments: APACHE-KYLIN-2722.patch
>
>
> For many existing metrics frameworks, they focus on maintaining metrics in
> memory independently for each instance. However, kylin server may consist of
> multiple instances. Thus we extend existing metrics framework by introducing
> *active reservoir* to actively push metrics to reporters which will report
> metrics of its instance to a unified storage.
> Here we introduced two *active reservoirs*. One is called
> {{BlockingReservoir}}, which will buffer the metrics. The other is called
> {{InstantReservoir}}, which owns no buffer and will directly push metrics to
> reporters.
> Generally, one *active reservoir* can push its metrics to multiple reporters
> and one reporter can only listen on one *active reservoir*.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
