Great stuff Andrew.
(I added an entry to the powered-by page. Please fix if I misrepresent).
St.Ack
Andrew Purtell wrote:
I just wanted to take this opportunity to report an HBase success story.
We are running Hadoop 0.18.1 and HBase 0.18.0.
Our application is a web crawling application with concurrent batch content
analysis of various kinds. All of the workflow components are implemented as
subclasses of TableMap and/or TableReduce. (So yes there will be some minor
refactoring necessary for 0.19...)
We are now at ~300 regions, most of them 512GB, hosted on a cluster of 25 nodes. We see a constant rate of 2500 requests/sec or greater, peaking periodically near 100K/sec when some of the batch scan tasks run. Since going into semi-production over last weekend there has been no downtime or service faults.
Feel free to add "Trend Micro Advanced Threats Research" to the powered by page.
- Andy
