alirezazamani opened a new pull request #1142: URL: https://github.com/apache/helix/pull/1142
### Issues - [x] My PR addresses the following Helix issues and references them in the PR title: Fixes #957 Fixes #1141 ### Description - [x] Here are some details about my PR, including screenshots of any UI changes: In this PR, several scheduling parts have been changed which enforces the scheduler to respect maximum number of attempts for the tasks. Also, it has been observed that when a task being dropped and scheduled again, max number of attempts is not being respected. In this PR, further checks is added to not schedule the tasks again once we reach its maximum number of attempts. Note: Several of the tests have been changed. Specially the once with strict MaxNumberOfAttempts. These tests need to be changed because they are mostly related to the targeted jobs. Once we start the participants and targeted partition bounces between the participants, the tasks needs to be reassigned and number of attempts for the task will increase. We haven't noticed it before we haven't been respecting MaxNumberOfAttempts for the DROPPED tasks. In other word, previously controller increases the task number of attempts without actually respecting the fact that the task has reached maximum number of attempts. ### Tests - [x] The following tests are written for this issue: TestMaxNumberOfAttemptsMasterSwitch - [x] The following is the result of the "mvn test" command on the appropriate module: helix-core: ``` [INFO] Results: [INFO] [ERROR] Failures: [ERROR] TestJobQueueCleanUp.testJobQueueAutoCleanUp ยป ThreadTimeout Method org.testng.... [ERROR] TestClusterVerifier.testResourceSubset:225 expected:<false> but was:<true> [INFO] [ERROR] Tests run: 1150, Failures: 2, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:25 h [INFO] Finished at: 2020-07-07T12:19:10-07:00 [INFO] ------------------------------------------------------------------------ ``` The failed test succeeded when run individually. (TestJobQueueCleanUp has been stabilized in master branch) mvn test -Dtest="TestJobQueueCleanUp,TestClusterVerifier" ``` [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.827 s - in TestSuite [INFO] [INFO] Results: [INFO] [INFO] Tests run: 6, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 49.286 s [INFO] Finished at: 2020-07-07T13:03:07-07:00 [INFO] ------------------------------------------------------------------------ ``` helix-rest: ``` [INFO] Tests run: 159, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 41.184 s - in TestSuite [INFO] [INFO] Results: [INFO] [INFO] Tests run: 159, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 46.343 s [INFO] Finished at: 2020-07-07T13:14:33-07:00 [INFO] ------------------------------------------------------------------------ ``` ### Commits - [x] My commits all reference appropriate Apache Helix GitHub issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)": 1. Subject is separated from body by a blank line 1. Subject is limited to 50 characters (not including Jira issue reference) 1. Subject does not end with a period 1. Subject uses the imperative mood ("add", not "adding") 1. Body wraps at 72 characters 1. Body explains "what" and "why", not "how" ### Code Quality - [x] My diff has been formatted using helix-style.xml ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
