Any interest in setting up a consortium for a HBase testing facility? We can't make point contributions to ASF to go to a specific project, but it's not unreasonable to suggest a couple HBase dev shops could pitch in a few bucks a month for project administered EC2 account for spinning up hbase-it. (This would be minimum viable I think, EC2 is ok for functional and smoke testing at least.)
On Thu, Dec 20, 2012 at 3:52 PM, Andrew Purtell <[email protected]> wrote: > This test is obviously not helpful as is. We're not losing coverage by > moving it out to integration tests. We're gaining sanity though. > > > On Thu, Dec 20, 2012 at 3:49 PM, Jesse Yates <[email protected]>wrote: > >> On Thu, Dec 20, 2012 at 1:31 PM, Stack <[email protected]> wrote: >> >> > On Thu, Dec 20, 2012 at 12:01 PM, Jesse Yates <[email protected] >> > >wrote: >> > >> > > Not being intimately familiar with the replication tests, how well >> > covered >> > > on that stuff are we by the remaining unit tests? Also, since the >> > hbase-it >> > > tests still get run, are we really gaining anything in terms of CI >> > > reliability? >> > > >> > >> > The hbase-it tests are not run up on jenkins Jesse, not at the moment at >> > least. >> > >> > >> I'm just a bit worried that we will break things and not have it caught on >> check-in (the real arbiter of what's a valid patch), meaning we will break >> the branch without realizing. :-/ We need some way to ensure that the >> underlying code is covered. >> >> At the minimum it needs to be part of the release checklist that we run >> the >> replication IT test on two real clusters (assuming this is a black-box >> test >> and not messing with things too much). I don't expect needing more than >> small functional clusters (say max 5 nodes?) to test this adequately. The >> Jenkins machines don't seem sufficient for this, so my gut feel is that it >> will have to be a release item that the RM needs to verify works (either >> personally or by proxy). This could even apply to all the hbase-it, for >> releases going forward >> >> Ideally, we will also have some unit tests that subsume some of the tested >> functionality when/if we move them to a more infrequent tests, though its >> hard to say how possible/useful this would be to manage in practice. >> >> Short term, maybe we disable them and file a Jira to get them rock solid >> (or on -it and tested regularly somehow)? This goes back to the long >> standing discussion that we should disable flappers until they tell us >> something useful. >> >> Again don't know those tests very well, so these are just general >> platitudes :) >> >> >> > Agree to moving TestReplication and perhaps some of the mapreduce tests >> out >> > of unit test core. >> > >> > That means we should set up a regular run of hbase-it tests somewhere >> > though? >> > >> > St.Ack >> > >> > > > > -- > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet Hein > (via Tom White) > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
