Sounds great. +1. On Fri, Mar 29, 2013 at 3:49 PM, Aleksandr Shulman <[email protected]>wrote:
> Hi all, > > I'd like to propose a potential solution to testing HBase Rolling Upgrade > between minor versions of HBase. It would be great to get feedback. > > *Benefits:* > This is a tricky feature and it would be really strengthen our trust with > the product users if we can show that it is stable and correct (e.g. no > loss of data). The best way to do that is with a test framework that allows > for repeatability, transparency, and reporting. > > *Considerations:* > It would be great to see us all agree on what should be supported during a > rolling upgrade and work towards it using this test framework. > > To that end, I have a few goals in mind: > -Integration test should be able to trigger a rolling upgrade of an > existing cluster, regardless of how it is set up > -It should be extensible enough so that it can be extended to support > different ways of setting up HBase (tarballs, packages, parcels, or any > other formats that are relevant) > -It should be able to report results of various testing operations > -It should be easy to add new tests to the suite, particularly by those who > have only passing familiarity with rolling upgrade > -These tests should be first-class citizens WRT other tests > -Should standardize what the minimum set of services is for a rolling > upgrade (e.g. master, backup master, 2 regionservers, zk, etc.) > -Should support both secure and insecure rolling upgrade > > *Potential Solution:* > I propose using IntegrationTest to bind to an existing remote cluster. It > would be able to perform the rolling upgrade steps while running tests over > a cluster that is already in place. > > At a high level, we would first agree on and create a standard workflow for > upgrading a cluster. Then, we would work on using IntegrationTest to run > tests (e.g. insert data, do mapreduce over hbase jobs, etc.) while the > cluster is in flux. The challenge would be adding code that will handle > specific cluster setups (e.g. tarballs vs. packages vs. parcels), but we > can abstract it away. > > Please let me know what you think about this idea and if you'd like to work > together to accomplish this goal. > > -- > Best Regards, > > Aleks Shulman > 847.814.5804 > Cloudera > -- // Jonathan Hsieh (shay) // Software Engineer, Cloudera // [email protected]
