Re: Review Request 47595: Implement in ams collector batch insert operations to ams-hbase

2016-05-23 Thread Myroslav Papirkovskyy 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47595/#review134361
---


Ship it!




Ship It!

- Myroslav Papirkovskyy


On Травень 23, 2016, 6:11 після полудня, Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47595/
> ---
> 
> (Updated Травень 23, 2016, 6:11 після полудня)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-16766
> https://issues.apache.org/jira/browse/AMBARI-16766
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Check if there is performance boost after replacing insert operations with 
> bulk loading. 
> If yes, implement and provide switch property for enabling/disabling this 
> feature.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  83aa36b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricsCacheCommitterThread.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  52ab083 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  8e9a8d6 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  290a98a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  92e4dfc 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  e5758bf 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
>  8f369f7 
> 
> Diff: https://reviews.apache.org/r/47595/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 47595: Implement in ams collector batch insert operations to ams-hbase

2016-05-20 Thread Dmytro Sen 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47595/
---

(Updated Май 20, 2016, 11:47 д.п.)


Review request for Ambari, Aravindan Vijayan and Sid Wagle.


Bugs: AMBARI-16766
https://issues.apache.org/jira/browse/AMBARI-16766


Repository: ambari


Description
---

Check if there is performance boost after replacing insert operations with bulk 
loading. 
If yes, implement and provide switch property for enabling/disabling this 
feature.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
 974f951 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricsCacheCommitterThread.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 52ab083 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
 683e5d4 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
 290a98a 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
 06c71c5 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 a7fd9c8 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
 8f369f7 

Diff: https://reviews.apache.org/r/47595/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 47595: Implement in ams collector batch insert operations to ams-hbase

2016-05-20 Thread Dmytro Sen 


> On Май 20, 2016, 4:21 д.п., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml,
> >  line 557
> > 
> >
> > Any reason for chosing 3 seconds vs 5 ?

No, but it's better to have delay between receiving metrics and committing as 
small as possible. For small clusters it doesn't matter, for large - metrics 
will be committed more frequently than 3-5 seconds, right after the cache is 
filled up.


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47595/#review134093
---


On Май 19, 2016, 2:43 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47595/
> ---
> 
> (Updated Май 19, 2016, 2:43 п.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-16766
> https://issues.apache.org/jira/browse/AMBARI-16766
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Check if there is performance boost after replacing insert operations with 
> bulk loading. 
> If yes, implement and provide switch property for enabling/disabling this 
> feature.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  974f951 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricsCacheCommitter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  52ab083 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  683e5d4 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  290a98a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  06c71c5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  a7fd9c8 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
>  8f369f7 
> 
> Diff: https://reviews.apache.org/r/47595/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 47595: Implement in ams collector batch insert operations to ams-hbase

2016-05-19 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47595/#review134093
---




ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 (line 557)


Any reason for chosing 3 seconds vs 5 ?


- Sid Wagle


On May 19, 2016, 2:43 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47595/
> ---
> 
> (Updated May 19, 2016, 2:43 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-16766
> https://issues.apache.org/jira/browse/AMBARI-16766
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Check if there is performance boost after replacing insert operations with 
> bulk loading. 
> If yes, implement and provide switch property for enabling/disabling this 
> feature.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  974f951 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricsCacheCommitter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  52ab083 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  683e5d4 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  290a98a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  06c71c5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  a7fd9c8 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
>  8f369f7 
> 
> Diff: https://reviews.apache.org/r/47595/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 47595: Implement in ams collector batch insert operations to ams-hbase

2016-05-19 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47595/#review134092
---


Fix it, then Ship it!





ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 (line 205)


Lets rename to cacheComitInterval vs sending



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 (line 232)


again we should call this commit vs send.



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 (line 695)


Lets use >= instead, sicne not sure if size() is threadsafe



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
 (line 52)


Rename to evict or commit.



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 (line 557)


Units should be clear from name or description of property.


- Sid Wagle


On May 19, 2016, 2:43 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47595/
> ---
> 
> (Updated May 19, 2016, 2:43 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan and Sid Wagle.
> 
> 
> Bugs: AMBARI-16766
> https://issues.apache.org/jira/browse/AMBARI-16766
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Check if there is performance boost after replacing insert operations with 
> bulk loading. 
> If yes, implement and provide switch property for enabling/disabling this 
> feature.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
>  974f951 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricsCacheCommitter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
>  52ab083 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
>  683e5d4 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
>  290a98a 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
>  06c71c5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  a7fd9c8 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
>  8f369f7 
> 
> Diff: https://reviews.apache.org/r/47595/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 47595: Implement in ams collector batch insert operations to ams-hbase

2016-05-19 Thread Dmytro Sen 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47595/
---

Review request for Ambari, Aravindan Vijayan and Sid Wagle.


Bugs: AMBARI-16766
https://issues.apache.org/jira/browse/AMBARI-16766


Repository: ambari


Description
---

Check if there is performance boost after replacing insert operations with bulk 
loading. 
If yes, implement and provide switch property for enabling/disabling this 
feature.


Diffs
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
 974f951 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/MetricsCacheCommitter.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 52ab083 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/TimelineMetricConfiguration.java
 683e5d4 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
 290a98a 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/discovery/TestMetadataManager.java
 06c71c5 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 a7fd9c8 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/package/scripts/service_check.py
 8f369f7 

Diff: https://reviews.apache.org/r/47595/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen