> On Dec. 20, 2015, 12:18 a.m., Jonathan Hurley wrote: > > I think we need to drop the notion of fast vs non-fast. I know that Nate > > has been working on re-using the same persistence unit between tests (each > > test gets its own cluster, own unique hosts, etc) ... and this has > > dramatically increased performance. > > > > With that said, I think we can still categorize and label our tests. But I > > think we should take a difference approach. > > > > It's cool to group them logically like Alerts, Kerberos, ORM, Upgrade, etc. > > > > But we should also create groupings based on what the tests are doing. > > Something like: > > SmallTest - accesses only mocks, most used for direct unit tests of single > > functional aspects > > MediumTest - works with mocks and DB, sets up cluster and some data for a > > single unit of test > > LargeTest - an end-to-end style test (like testing an upgrade or downgrade) > > > > Even more, we can group them for quick runs from the command by mixing and > > matching above logical groupings. Consider: > > MAT (Minimum Acceptance Tests): Basic functionality that must pass (like > > cluster creation, adding services, pure ORM) > > SRG (Short Regression): A mixture of tests from above logical units > > (Alerts, Kerberos, Upgrade) which don't take too long > > LRG (Long Regression): A mixture of tests from above logical units (Alerts, > > Kerberos, Upgrade) which are known to take a long time > > Nate Cole wrote: > +1 to this.
Are not SmallTest/MediumTest/LargeTest something like unit/integration/functional tests? I'd say that a big part of our tests consists of over-detailed functional tests, that tend to test too much and that's why break on any implementation change like mock method invocation count. So I'd strongly +1 to clear separation of roles you are proposing. Behaviour Also, it would be great to have some commonly-used mock implementations like cluster with few hosts and services as a separate class, so we could skip defining something like createCluster() at every test suite. - Dmitro ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/41569/#review111389 ----------------------------------------------------------- On Dec. 19, 2015, 12:26 a.m., Alejandro Fernandez wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/41569/ > ----------------------------------------------------------- > > (Updated Dec. 19, 2015, 12:26 a.m.) > > > Review request for Ambari, Aravindan Vijayan, Dmytro Grinenko, Di Li, Dmitro > Lisnichenko, Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert > Levas, Robert Nettleton, Nahappan Somasundaram, Sumit Mohanty, Srimanth > Gunturi, Swapan Shridhar, Sid Wagle, and Yusaku Sako. > > > Bugs: AMBARI-14439 > https://issues.apache.org/jira/browse/AMBARI-14439 > > > Repository: ambari > > > Description > ------- > > Categorize the unit tests with wither SlowTest, FastTest, and some other > group based on the area. > This will allow us to run a test suite that matches a given group. > > mvn clean test -P ${profile_id} > mvn clean -Drat.ignoreErrors=true test -P FastTest > fast-tests.txt > mvn clean -Drat.ignoreErrors=true test -P NonFastTest > nonfast-tests.txt > > > Diffs > ----- > > ambari-server/pom.xml 56d9e44 > ambari-server/src/test/java/org/apache/ambari/server/AlertTest.java > PRE-CREATION > ambari-server/src/test/java/org/apache/ambari/server/AmbariUpgradeTest.java > PRE-CREATION > ambari-server/src/test/java/org/apache/ambari/server/BlueprintTest.java > PRE-CREATION > ambari-server/src/test/java/org/apache/ambari/server/FastTest.java > PRE-CREATION > ambari-server/src/test/java/org/apache/ambari/server/KerberosTest.java > PRE-CREATION > ambari-server/src/test/java/org/apache/ambari/server/SlowTest.java > PRE-CREATION > ambari-server/src/test/java/org/apache/ambari/server/StackUpgradeTest.java > PRE-CREATION > > ambari-server/src/test/java/org/apache/ambari/server/checks/ClientRetryPropertyCheckTest.java > 7b8239c > > ambari-server/src/test/java/org/apache/ambari/server/checks/ComponentsInstallationCheckTest.java > 450d74e > > ambari-server/src/test/java/org/apache/ambari/server/checks/ConfigurationMergeCheckTest.java > 68a0522 > > ambari-server/src/test/java/org/apache/ambari/server/checks/HiveDynamicServiceDiscoveryCheckTest.java > cdf13eb > > ambari-server/src/test/java/org/apache/ambari/server/checks/HiveMultipleMetastoreCheckTest.java > 16f383a > > ambari-server/src/test/java/org/apache/ambari/server/checks/HostMaintenanceModeCheckTest.java > 0e14376 > > ambari-server/src/test/java/org/apache/ambari/server/checks/HostsHeartbeatCheckTest.java > cc2c276 > > ambari-server/src/test/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheckTest.java > 9fcb319 > > ambari-server/src/test/java/org/apache/ambari/server/checks/HostsRepositoryVersionCheckTest.java > 4529554 > > ambari-server/src/test/java/org/apache/ambari/server/checks/InstallPackagesCheckTest.java > 080ca3a > > ambari-server/src/test/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheckTest.java > bfe0c3e > > ambari-server/src/test/java/org/apache/ambari/server/checks/RangerPasswordCheckTest.java > afa3789 > > ambari-server/src/test/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheckTest.java > e2617bf > > ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesMaintenanceModeCheckTest.java > a941b7a > > ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesMapReduceDistributedCacheCheckTest.java > 22f2b1b > > ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesNamenodeHighAvailabilityCheckTest.java > abe7abe > > ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesNamenodeTruncateCheckTest.java > 87d4167 > > ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesUpCheckTest.java > 88826a0 > > ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesYarnWorkPreservingCheckTest.java > 98cfb18 > > ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckOrderTest.java > 7d70311 > > ambari-server/src/test/java/org/apache/ambari/server/checks/YarnTimelineServerStatePreservingCheckTest.java > 7469bbc > > Diff: https://reviews.apache.org/r/41569/diff/ > > > Testing > ------- > > Verified it worked on the handful of tests I annotated. > > > Thanks, > > Alejandro Fernandez > >
