> On July 21, 2015, 12:05 p.m., Amareshwari Sriramadasu wrote: > > lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java, > > lines 709-711 > > <https://reviews.apache.org/r/36393/diff/23/?file=1017629#file1017629line709> > > > > Is removal of these tests required for this patch? > > > > If so, can we make they are tested by a different way?
Have parked a ticket to implement these test cases correctly. https://issues.apache.org/jira/browse/LENS-677 - Himanshu ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/36393/#review92397 ----------------------------------------------------------- On July 21, 2015, 9:48 a.m., Himanshu Gahlaut wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/36393/ > ----------------------------------------------------------- > > (Updated July 21, 2015, 9:48 a.m.) > > > Review request for lens. > > > Bugs: LENS-619 > https://issues.apache.org/jira/browse/LENS-619 > > > Repository: lens > > > Description > ------- > > Lens will always accept a new query from a user and put it in a scheduling > queue for processing. > > Next candidate query picked up from scheduling queue for processing will be > launched only if all query constraints evaluated on candidate query and > launched queries allows the candidate query to be launched, otherwise the > candidate query will be added to waiting queries. > > When any launched query is finished, a waiting query selector will select > waiting queries for rescheduling using a list of waiting queries selection > policy. Every waiting query selection policy will return a list of eligible > waiting queries to be rescheduled. Query Selector will calculate intersection > of multiple list of eligible waiting queries and add the result of > intersection to scheduling queue for reprocessing. > > At initialization, Query Constraints and Waiting Query Selection Policies > will be configured using configuration values. > > New Query Constraints and Waiting Query Selection Policies can be added at > runtime, without rebuilding and deploying lens module. Drivers should be > allowed to add more Query Constraints and Waiting Queries Selection policies. > Waiting Queries will be persisted across server restarts. > > Query Constraint 1: Allow a candidate query to be launched, only if, > cumulative query cost of all current queries launched by the user, who > submitted candidate query, is less than a cumulative query cost ceiling for > launching a new query. > Waiting Query Selection Policy 1: Select all waiting queries of the user > whose query has finished. > > Query Constraint 2: Allow a candidate query to be launched, only if, current > concurrent queries launched on the > selected driver are less than max concurrent queries allowed on the driver. > Waiting Query Selection Policy 2: Select all waiting queries of the driver > whose query has finished. > > > Diffs > ----- > > lens-api/src/main/java/org/apache/lens/api/Addable.java > 5a556918afcdb8967deee84472cd7863aeafc4fc > lens-api/src/main/java/org/apache/lens/api/query/LensQuery.java > f594133e8a75603e5bb0c913d62ea189248cfa05 > lens-api/src/main/java/org/apache/lens/api/query/QueryStatus.java > 539f0b0b0804257fae6b7ec0af8f2f715175d597 > lens-client/src/main/java/org/apache/lens/client/LensStatement.java > f7305fb288167cda088e263a62c77f264d779127 > lens-driver-hive/src/main/java/org/apache/lens/driver/hive/HiveDriver.java > 7c3e3e08bc7e6949588aa073665fd97596e1a1ae > lens-driver-hive/src/main/resources/hivedriver-default.xml > 300ce3fed7814d446b3aeb8688927c2dc8800f81 > lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriver.java > e3a879929ceb0dad054c71231bc4a70162ecd9ce > > lens-driver-jdbc/src/main/java/org/apache/lens/driver/jdbc/JDBCDriverConfConstants.java > ebcdaca7c7bab3a15feb10e69ef41238a8bae506 > lens-driver-jdbc/src/main/resources/jdbcdriver-default.xml > 99f6f205dfe6f1c16277ac6624106436df2b2b8f > lens-server-api/pom.xml b0b0fe017a239b24c3b9606b3be77a1722844e0f > > lens-server-api/src/main/java/org/apache/lens/server/api/LensConfConstants.java > 2b3a4800dc31e1c0787f2da9178456bec871a8a9 > > lens-server-api/src/main/java/org/apache/lens/server/api/common/ConfigBasedObjectCreationFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/driver/LensDriver.java > 52207d342ea9ea154ef6ebd1cf4266176039c561 > > lens-server-api/src/main/java/org/apache/lens/server/api/driver/MinQueryCostSelector.java > c3fe8eb142569c0b254f551c3cbd18502b6eac99 > > lens-server-api/src/main/java/org/apache/lens/server/api/metrics/MetricsService.java > 71128b7d35e91fadf38a9c3926ce46d42369ead4 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/AbstractQueryContext.java > 71a37f079f6b43a5cde689ce2c2e8fccb83eb78d > > lens-server-api/src/main/java/org/apache/lens/server/api/query/DriverSelectorQueryContext.java > 3e7de173a4e3083ca3ac5fa95fd7cd09e7f028bb > > lens-server-api/src/main/java/org/apache/lens/server/api/query/FinishedLensQuery.java > 6e7ab5dc1c78407a24e389c001ca3100563e3795 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryContext.java > d72727e7832d02ed2ea90379cd29c7a8e509a8f8 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/QueryExecutionService.java > 5d7163f112d7c7115ebf8f012ba397bfee63adbc > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicy.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/EstimatedImmutableQueryCollection.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/ImmutableQueryCollection.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/collect/WaitingQueriesSelectionPolicy.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraint.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintFactory.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/constraint/QueryLaunchingConstraint.java > PRE-CREATION > > lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/FactPartitionBasedQueryCost.java > 7fa5cdb55959a44b52a29884fd629c2479fdfb96 > > lens-server-api/src/main/java/org/apache/lens/server/api/query/cost/QueryCost.java > be7232134b59f7126edba569358389a9c26a5c74 > lens-server-api/src/main/java/org/apache/lens/server/api/util/LensUtil.java > f9262994debe0b943123d13c58a22c10b2015e80 > > lens-server-api/src/test/java/org/apache/lens/server/api/driver/MockDriver.java > dffcfd3567dd015e7c79773c5e8cd6669c2d9d95 > > lens-server-api/src/test/java/org/apache/lens/server/api/query/collect/DriverSpecificWaitingQueriesSelectionPolicyTest.java > PRE-CREATION > > lens-server-api/src/test/java/org/apache/lens/server/api/query/constraint/MaxConcurrentDriverQueriesConstraintTest.java > PRE-CREATION > > lens-server-api/src/test/java/org/apache/lens/server/api/query/cost/TestFactPartitionBasedQueryCost.java > 0ab0b0ec7c83c1eac8d61c8a30ecec5e9dc2e322 > lens-server/pom.xml 6ac30a52cc8c1c58bf8e92bb77df358db6e22a3e > > lens-server/src/main/java/org/apache/lens/server/metrics/MetricsServiceImpl.java > 39f9f5216acf6e1be7cca6a0851152a78ba66176 > lens-server/src/main/java/org/apache/lens/server/query/LensServerDAO.java > 190435036c5cf8f009d7fbd536ef1743b70c22d9 > > lens-server/src/main/java/org/apache/lens/server/query/QueryExecutionServiceImpl.java > 47d27d06272c0c26a93818fdee90cf978a650d10 > > lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollection.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/DefaultQueryCollection.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelector.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/MutableQueryCollection.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/QueryCollection.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeEstimatedQueryCollection.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/ThreadSafeQueryCollection.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicy.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyFactory.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/collect/WaitingQueriesSelector.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsChecker.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/QueryLaunchingConstraintsChecker.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraint.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactory.java > PRE-CREATION > > lens-server/src/main/java/org/apache/lens/server/util/FairPriorityBlockingQueue.java > PRE-CREATION > lens-server/src/main/java/org/apache/lens/server/util/UtilityMethods.java > 715347fab82b5d9d4ad673c45e72900b9a95fc81 > lens-server/src/main/resources/lensserver-default.xml > 2b540b828a976378bdfbd238e8427a22941fb7e8 > > lens-server/src/test/java/org/apache/lens/server/query/TestQueryService.java > 7e486aee1e995954bd25beac85a9fafed95ce20e > > lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultEstimatedQueryCollectionTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/DefaultQueryCollectionTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/IntersectingWaitingQueriesSelectorTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/QueryCollectUtil.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/ThreadSafetyTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/collect/UserSpecificWaitingQueriesSelectionPolicyTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/constraint/DefaultQueryLaunchingConstraintsCheckerTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintFactoryTest.java > PRE-CREATION > > lens-server/src/test/java/org/apache/lens/server/query/constraint/TotalQueryCostCeilingConstraintTest.java > PRE-CREATION > lens-server/src/test/resources/lens-site.xml > 3bcbf0e3e44d986a24c00cad84121be72596c64d > lens-server/src/test/resources/log4j.properties > e0f9daa62c7d88f5044803c5a0ce2ffb2ac9315b > src/site/apt/admin/config.apt c5bf70c612a8d9bc383d0031e5cb51338cbd1da8 > src/site/apt/admin/hivedriver-config.apt > 80a3028789348493a84e753965d32cc30690c503 > src/site/apt/admin/jdbcdriver-config.apt > b3d25f6751dd030a4ae818845a4a504ca41d7895 > src/site/apt/user/index.apt d48c7347314876989cf8de8f3e330dc972e3ec5d > > Diff: https://reviews.apache.org/r/36393/diff/ > > > Testing > ------- > > Patch rebased with master and all test cases passed: > > [INFO] > ------------------------------------------------------------------------ > [INFO] Reactor Summary: > [INFO] > [INFO] Lens Checkstyle Rules ............................. SUCCESS [3.312s] > [INFO] Lens .............................................. SUCCESS [4.667s] > [INFO] Lens API .......................................... SUCCESS [31.360s] > [INFO] Lens API for server and extensions ................ SUCCESS [33.701s] > [INFO] Lens Cube ......................................... SUCCESS [5:52.120s] > [INFO] Lens DB storage ................................... SUCCESS [27.322s] > [INFO] Lens Query Library ................................ SUCCESS [22.338s] > [INFO] Lens Hive Driver .................................. SUCCESS [3:44.595s] > [INFO] Lens Driver for JDBC .............................. SUCCESS [52.297s] > [INFO] Lens Server ....................................... SUCCESS > [11:28.682s] > [INFO] Lens client ....................................... SUCCESS [46.951s] > [INFO] Lens CLI .......................................... SUCCESS [3:54.445s] > [INFO] Lens Examples ..................................... SUCCESS [13.859s] > [INFO] Lens Distribution ................................. SUCCESS [13.041s] > [INFO] Lens ML Lib ....................................... SUCCESS [2:03.557s] > [INFO] Lens ML Ext Distribution .......................... SUCCESS [5.792s] > [INFO] Lens Regression ................................... SUCCESS [15.693s] > [INFO] > ------------------------------------------------------------------------ > [INFO] BUILD SUCCESS > [INFO] > ------------------------------------------------------------------------ > [INFO] Total time: 31:34.656s > [INFO] Finished at: Mon Jul 20 13:43:03 IST 2015 > [INFO] Final Memory: 123M/336M > [INFO] > ------------------------------------------------------------------------ > > > Thanks, > > Himanshu Gahlaut > >
