>From Stack: ------- I think we can have it so everyone gets what they need; small/medium/large are done by surefire and then we have a new intergration test green field that is run by failsafe that can help us get tests for bigtop et al. to run.
------ I think the above proposal makes sense. Please share your comments. On Mon, Nov 28, 2011 at 12:23 PM, Ted Yu <[email protected]> wrote: > The following discussion is closely related to HBASE-4712. > > We should reach general consensus so that the execution of future test > strategy is smooth. > > On Mon, Nov 28, 2011 at 11:50 AM, Jesse Yates <[email protected]>wrote: > >> I was considering 'integration tests' as a separate concern from the >> large/medium/small _unit_ tests. >> >> That is, in fact, why the failsafe plugin was added (and is designed >> for). >> >> Currently, we have a lot of tests that fall in the realm of integration >> tests (testing integration between various pieces, rather than single >> functionality alone). The most obvious indicator that something is an >> integration test is when it is using the MiniHBaseCluster since that is >> really testing the integration between the new feature and the rest of the >> system. >> >> The surefire plugin should really be testing unit tests, and then those >> can be classified as small/medium/large. However, the large would be >> _extremely_ rare cases. >> >> If the community is completely moving to just using the annotations for >> denoting size of tests, rather than using the integrationTest/unit test >> stuff, thats fine. However, I still think that if we do that, we should >> have a separate designation for @IntegrationTest (but that is really >> already covered in the fact that they would be named differently and >> therefore run by the different plugin). The reason we need to split them >> out is to have that separate tier of testing - once they pass unit level, >> they can be tested at the integration level and then on the acceptance test >> level (though we don't have any of those yet). >> >> This probably means more reclassification of tests. >> >> Making this explicit is important so people know at what point in the >> process their patch breaks (as well as staging the testing being good >> practice in general). Therefore, the small/medium/large classifications are >> not complete. Further, we should should still be able to use the general >> maven testing (mvn test, mvn verify) to run all the tests as expected, >> rather than having to know which things extra commands to run. To that end, >> we need to integrate >> HBASE-4712<https://issues.apache.org/jira/browse/HBASE-4712>ASAP so its in >> the book and people know how to work with the current >> testing layout. >> >> I've been a bit behind on the testing discussions on dev@, so didn't get >> to put these thoughts in, but I think they can still be added on. This was >> also part of what Doug, Stack and I were kind of envisioning in the >> original email to the list. >> >> Thanks. >> >> -Jesse >> >> >> On Mon, Nov 28, 2011 at 11:29 AM, N Keywal <[email protected]> wrote: >> >>> Today's tests are: >>> - 416 small tests, executed in ~3 minutes >>> - 489 medium tests, executed in ~35 minutes (without parallelization) >>> - 280 large tests, executed in ~90 minutes (without parallelization) >>> >>> So may be you could put this new test as large? Will you want to run >>> this test for all central builds? Or do you see it as a specific tests that >>> will be executed only sometimes? >>> >>> >>> On Mon, Nov 28, 2011 at 8:24 PM, N Keywal <[email protected]> wrote: >>> >>>> So I guess it's likely to be filtered as it's "not large". >>>> >>>> >>>> On Mon, Nov 28, 2011 at 8:17 PM, Ted Yu <[email protected]> wrote: >>>> >>>>> N: >>>>> Currently IntegrationTestConstraint doesn't have @Category annotation. >>>>> >>>>> FYI >>>>> >>>>> >>>>> On Mon, Nov 28, 2011 at 11:08 AM, N Keywal <[email protected]> wrote: >>>>> >>>>>> Yes, I put the large test as integration tests (may be 3 weeks ago?). >>>>>> So if you mix naming pattern with categories, you can end up with no >>>>>> tests >>>>>> to do. >>>>>> >>>>>> It seems logical to me to put large tests in the failsafe module. If >>>>>> you want to use the failsafe tests for something else, I can remove then. >>>>>> >>>>>> see the doc in >>>>>> HBASE-4712<https://issues.apache.org/jira/browse/HBASE-4712>(last >>>>>> comment) and the patch >>>>>> HBASE-4847 <https://issues.apache.org/jira/browse/HBASE-4847> on how >>>>>> it designed to work today. >>>>>> >>>>>> Cheers, >>>>>> >>>>>> N. >>>>>> >>>>>> >>>>>> On Mon, Nov 28, 2011 at 8:03 PM, Ted Yu <[email protected]> wrote: >>>>>> >>>>>>> Copying N. >>>>>>> >>>>>>> On Mon, Nov 28, 2011 at 10:36 AM, Jesse Yates < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hmmm, that's odd. It was working last week, maybe it was the recent >>>>>>>> changes N put in. >>>>>>>> >>>>>>>> I'll look into it. >>>>>>>> >>>>>>>> Thanks! >>>>>>>> >>>>>>>> - Jesse Yates >>>>>>>> >>>>>>>> Sent from my iPhone. >>>>>>>> >>>>>>>> On Nov 28, 2011, at 9:51 AM, Ted Yu <[email protected]> wrote: >>>>>>>> >>>>>>>> > Hi, >>>>>>>> > Please categorize new tests for 4605. >>>>>>>> > >>>>>>>> > When I issued this command: >>>>>>>> > mvn failsafe:integration-test >>>>>>>> > >>>>>>>> > I got: >>>>>>>> > >>>>>>>> > Results : >>>>>>>> > >>>>>>>> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0 >>>>>>>> > >>>>>>>> > Do you have some idea ? >>>>>>>> > >>>>>>>> > Thanks >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >> >> >> -- >> ------------------- >> Jesse Yates >> 240-888-2200 >> @jesse_yates >> >> >
