[EC2] On demand monitoring and auto scaling
-------------------------------------------
Key: HBASE-2300
URL: https://issues.apache.org/jira/browse/HBASE-2300
Project: Hadoop HBase
Issue Type: Sub-task
Reporter: Andrew Purtell
Priority: Minor
Modify the scripts to install monitoring and auto-scaling rules to launch new
slaves on demand according to CPU, disk, or I/O (over)use.
http://docs.amazonwebservices.com/AmazonCloudWatch/latest/DeveloperGuide/
http://docs.amazonwebservices.com/AutoScaling/latest/DeveloperGuide/
CloudWatch metrics are provided as statistics ['Minimum', 'Maximum', 'Sum',
'Average', 'Samples'] in units of ['Seconds', 'Percent', 'Bytes', 'Bits',
'Count', 'Bytes/Second', 'Bits/Second', 'Count/Second']. Some useful metrics:
- NetworkIn
- NetworkOut
- CPUUtilization
- DiskWriteBytes
- DiskWriteOps
- DiskReadBytes
- DiskReadOps
We can be agnostic about statistics and scaling triggers and just pass them
through to EC2.
This would be especially easy if we port our scripts to Python like Hadoop has
on trunk and use boto: http://code.google.com/p/boto/.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
