GSharayu commented on code in PR #9309:
URL: https://github.com/apache/pinot/pull/9309#discussion_r967315539
##########
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/assignment/segment/SegmentAssignment.java:
##########
@@ -31,20 +31,17 @@
/**
* Interface for segment assignment and table rebalance.
- * <p>
- * TODO: Add SegmentAssignmentStrategy interface and support custom segment
assignment strategy (e.g. cost based segment
- * assignment). SegmentAssignmentStrategy should not be coupled with
SegmentAssignment, and SegmentAssignment
- * should be able to choose the segment assignment strategy based on the
configuration.
*/
public interface SegmentAssignment {
/**
* Initializes the segment assignment.
- *
* @param helixManager Helix manager
* @param tableConfig Table config
+ * @param instancePartitionsMap Map from type (OFFLINE|CONSUMING|COMPLETED)
to instance partitions
*/
- void init(HelixManager helixManager, TableConfig tableConfig);
+ void init(HelixManager helixManager, TableConfig tableConfig,
+ Map<InstancePartitionsType, InstancePartitions> instancePartitionsMap);
Review Comment:
> ```
> 2022-09-08T23:47:33.2814091Z
org.apache.pinot.controller.api.exception.TableAlreadyExistsException: Table
config for testTable_OFFLINE already exists. If this is unexpected, try
deleting the table to remove all metadata associated with it.
> 2022-09-08T23:47:33.2815139Z at
org.apache.pinot.controller.helix.core.PinotHelixResourceManager.addTable(PinotHelixResourceManager.java:1412)
> 2022-09-08T23:47:33.2816081Z at
org.apache.pinot.controller.helix.core.rebalance.TableRebalancerClusterStatelessTest.testRebalanceWithTiers(TableRebalancerClusterStatelessTest.java:343)
> 2022-09-08T23:47:33.2816919Z at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 2022-09-08T23:47:33.2817485Z at
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 2022-09-08T23:47:33.2818120Z at
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 2022-09-08T23:47:33.2818640Z at
java.base/java.lang.reflect.Method.invoke(Method.java:566)
> 2022-09-08T23:47:33.2819129Z at
org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:108)
> 2022-09-08T23:47:33.2819624Z at
org.testng.internal.Invoker.invokeMethod(Invoker.java:661)
> 2022-09-08T23:47:33.2820029Z at
org.testng.internal.Invoker.invokeTestMethod(Invoker.java:869)
> 2022-09-08T23:47:33.2820469Z at
org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1193)
> 2022-09-08T23:47:33.2820970Z at
org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:126)
> 2022-09-08T23:47:33.2835876Z at
org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
> 2022-09-08T23:47:33.2836443Z at
org.testng.TestRunner.privateRun(TestRunner.java:744)
> 2022-09-08T23:47:33.2836810Z at
org.testng.TestRunner.run(TestRunner.java:602)
> 2022-09-08T23:47:33.2837169Z at
org.testng.SuiteRunner.runTest(SuiteRunner.java:380)
> 2022-09-08T23:47:33.2837552Z at
org.testng.SuiteRunner.runSequentially(SuiteRunner.java:375)
> 2022-09-08T23:47:33.2837963Z at
org.testng.SuiteRunner.privateRun(SuiteRunner.java:340)
> 2022-09-08T23:47:33.2838328Z at
org.testng.SuiteRunner.run(SuiteRunner.java:289)
> 2022-09-08T23:47:33.2838718Z at
org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
> 2022-09-08T23:47:33.2839140Z at
org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
> 2022-09-08T23:47:33.2839547Z at
org.testng.TestNG.runSuitesSequentially(TestNG.java:1301)
> 2022-09-08T23:47:33.2839920Z at
org.testng.TestNG.runSuitesLocally(TestNG.java:1226)
> 2022-09-08T23:47:33.2840271Z at
org.testng.TestNG.runSuites(TestNG.java:1144)
> 2022-09-08T23:47:33.2840588Z at
org.testng.TestNG.run(TestNG.java:1115)
> 2022-09-08T23:47:33.2841002Z at
org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:284)
> 2022-09-08T23:47:33.2841558Z at
org.apache.maven.surefire.testng.TestNGXmlTestSuite.execute(TestNGXmlTestSuite.java:75)
> 2022-09-08T23:47:33.2842129Z at
org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:119)
> 2022-09-08T23:47:33.2842887Z at
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:428)
> 2022-09-08T23:47:33.2843451Z at
org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:162)
> 2022-09-08T23:47:33.2843953Z at
org.apache.maven.surefire.booter.ForkedBooter.run(ForkedBooter.java:562)
> 2022-09-08T23:47:33.2844449Z at
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:548)
> 2022-09-08T23:47:33.2844718Z
> 2022-09-08T23:47:33.2845206Z [ERROR]
org.apache.pinot.controller.helix.core.PinotHelixResourceManagerStatelessTest.testUpdateBrokerResource
Time elapsed: 0.216 s
> 2022-09-08T23:47:33.2846308Z [ERROR]
org.apache.pinot.controller.helix.core.PinotHelixResourceManagerStatelessTest.testValidateDimTableTenantConfig
Time elapsed: 0.072 s
> 2022-09-08T23:47:33.2847330Z [ERROR]
org.apache.pinot.controller.helix.core.PinotHelixResourceManagerStatelessTest.testValidateTenantConfig
Time elapsed: 0.052 s
> 2022-09-08T23:47:33.2848188Z [ERROR]
org.apache.pinot.controller.helix.ControllerTenantStatelessTest.testBrokerTenant
Time elapsed: 0.953 s
> 2022-09-08T23:47:33.2848940Z [ERROR]
org.apache.pinot.controller.helix.ControllerTenantStatelessTest.testEmptyServerTenant
Time elapsed: 0.02 s
> 2022-09-08T23:47:33.2849714Z [ERROR]
org.apache.pinot.controller.helix.ControllerTenantStatelessTest.testServerTenant
Time elapsed: 0.388 s
> 2022-09-08T23:47:33.2850500Z [ERROR]
org.apache.pinot.controller.helix.PinotControllerModeStatelessTest.testDualModeController
Time elapsed: 9.822 s
> 2022-09-08T23:47:33.2851344Z [ERROR]
org.apache.pinot.controller.helix.PinotControllerModeStatelessTest.testHelixOnlyController
Time elapsed: 1.718 s
> 2022-09-08T23:47:33.2852184Z [ERROR]
org.apache.pinot.controller.helix.PinotControllerModeStatelessTest.testPinotOnlyController
Time elapsed: 5.248 s
> 2022-09-08T23:47:33.2853009Z [ERROR]
org.apache.pinot.controller.api.PinotBrokerRestletResourceStatelessTest.testGetBrokers
Time elapsed: 14.044 s
> 2022-09-08T23:47:33.2853853Z [ERROR]
org.apache.pinot.controller.api.PinotIngestionRestletResourceStatelessTest.testIngestEndpoint
Time elapsed: 0.492 s
> 2022-09-08T23:47:33.2855076Z [ERROR]
org.apache.pinot.controller.helix.core.minion.PinotTaskManagerStatelessTest.testDefaultPinotTaskManagerNoScheduler
Time elapsed: 3.905 s
> 2022-09-08T23:47:33.2856187Z [ERROR]
org.apache.pinot.controller.helix.core.minion.PinotTaskManagerStatelessTest.testPinotTaskManagerSchedulerWithRestart
Time elapsed: 8.857 s
> 2022-09-08T23:47:33.2857313Z [ERROR]
org.apache.pinot.controller.helix.core.minion.PinotTaskManagerStatelessTest.testPinotTaskManagerSchedulerWithUpdate
Time elapsed: 2.65 s
> 2022-09-08T23:47:33.2858407Z [ERROR]
org.apache.pinot.controller.validation.ValidationManagerStatelessTest.testRebuildBrokerResourceWhenBrokerAdded
Time elapsed: 0.04 s
> 2022-09-08T23:47:33.3510290Z [INFO]
> 2022-09-08T23:47:33.3511030Z [INFO] Results:
> 2022-09-08T23:47:33.3512144Z [INFO]
> 2022-09-08T23:47:33.3512504Z [ERROR] Failures:
> 2022-09-08T23:47:33.3513121Z [ERROR]
TableRebalancerClusterStatelessTest.testRebalance:210 expected [DONE] but found
[FAILED]
> 2022-09-08T23:47:33.3534306Z [ERROR]
TableRebalancerClusterStatelessTest.testRebalanceWithTiers:343 ยป
TableAlreadyExists
> ```
>
> It looks that rebalancing is failing.
Fixed this as well.I think this was not showing up because of other flaky
test. In the existing code, we set BalanceNumSegmentAssignmentStrategy as
default and update table config replication which did not match the assignment
strategy which caused this test to fail. Handled change in
strategyAssignmentFactory
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
