umustafi opened a new pull request, #3656: URL: https://github.com/apache/gobblin/pull/3656
Dear Gobblin maintainers, Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below! ### JIRA - [X] My PR addresses the following [Gobblin JIRA](https://issues.apache.org/jira/browse/GOBBLIN/) issues and references them in the PR title. For example, "[GOBBLIN-XXX] My Gobblin PR" - https://issues.apache.org/jira/browse/GOBBLIN-1797 ### Description - [X] Here are some details about my PR, including screenshots (if applicable): The [unschedule feature](https://jarvis.corp.linkedin.com/codesearch/result/?name=FlowConfigResourceLocalHandler.java&path=gobblin-elr%2Fgobblin-restli%2Fgobblin-flow-config-service%2Fgobblin-flow-config-service-server%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fgobblin%2Fservice&reponame=linkedin%2Fgobblin-elr#62) sets a schedule to run Jan 1st of 2050 so far in advance that it will "never run" but potentially there are over 100k of these flows so we are loading and scheduling many unnecessary flows increasing the startup time. On initialization we add a check that verifies the next run of the flow is within a certain time frame (100 days by default) and loads it into the scheduler if it is within that time frame. We choose that default value under the assumption that we will redeploy GaaS at least every 100 days and then if we approach a far out scheduled flow we will load it into the Scheduler. However, in most cases uses schedule flows for near future or immediately and those will al l be scheduled. This PR also renames metrics and adds helpful new ones. ### Tests - [X] My PR adds the following unit tests __OR__ does not need testing for this extremely good reason: Unit test checks the functionality for calculating the number of days until the next run for a cron expression and evaluates cases of far out scheduled flows, regularly occurring, and blank flows (adhoc ones). ### Commits - [X] My commits all reference JIRA issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "[How to write a good git commit message](http://chris.beams.io/posts/git-commit/)": 1. Subject is separated from body by a blank line 2. Subject is limited to 50 characters 3. Subject does not end with a period 4. Subject uses the imperative mood ("add", not "adding") 5. Body wraps at 72 characters 6. Body explains "what" and "why", not "how" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
