[jira] [Created] (BEAM-2603) Add Meter in beam metrics

2017-07-12 Thread Cody (JIRA) 
Cody created BEAM-2603:
--

 Summary: Add Meter in beam metrics
 Key: BEAM-2603
 URL: https://issues.apache.org/jira/browse/BEAM-2603
 Project: Beam
  Issue Type: Sub-task
  Components: runner-core, sdk-java-core
Reporter: Cody
Assignee: Kenneth Knowles
 Fix For: 2.2.0


1. Add Meter interface and implementation
2. Add MeterData, MeterResult. Include MeterData in metric updates, and 
MeterResult in metric query results.
3. Add corresponding changes regarding MeterResult and MeterData.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (BEAM-2604) Delegate beam metrics to runners

2017-07-12 Thread Cody (JIRA) 
Cody created BEAM-2604:
--

 Summary: Delegate beam metrics to runners
 Key: BEAM-2604
 URL: https://issues.apache.org/jira/browse/BEAM-2604
 Project: Beam
  Issue Type: Sub-task
  Components: runner-flink, runner-spark
Reporter: Cody
Assignee: Aljoscha Krettek
 Fix For: 2.2.0


Delegate beam metrics to runners to avoid forwarding updates, i.e., extract 
updates from beam metrics and commit updates in runners.

For Flink/Spark runners, we can reference metrics within runner's metrics 
system in beam pipelines and update them directly without forwarding.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (BEAM-2602) Fully support dropwizard metrics in beam and runners

2017-07-11 Thread Cody (JIRA) 
Cody created BEAM-2602:
--

 Summary: Fully support dropwizard metrics in beam and runners
 Key: BEAM-2602
 URL: https://issues.apache.org/jira/browse/BEAM-2602
 Project: Beam
  Issue Type: Improvement
  Components: runner-core, runner-flink, runner-spark, sdk-java-core
Affects Versions: Not applicable
Reporter: Cody
Assignee: Kenneth Knowles
 Fix For: 2.2.0


As proposed at 
https://docs.google.com/document/d/1-35iyCIJ9P4EQONlakgXBFRGUYoOLanq2Uf2sw5EjJw/edit?usp=sharing
 , I'd like to add full support of dropwizard metrics by delegating beam 
metrics to runners.

The proposal involves a few subtasks, as far as I see, including:
1. add Meter interface in sdk-java-core and extend Distribution to support 
quantiles
2. add MeterData, extend DistributionData. Merge 
{Counter/Meter/Gauge/Distribution}Data instead of 
Counter/Meter/Gauge/Distribution at accumulators.
3. Runner changes over improved metrics. 

I will create subtasks later if there's no objection.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)