HanumathRao opened a new pull request #1677: DRILL-7068: Support memory adjustment framework for resource manageme… URL: https://github.com/apache/drill/pull/1677 …nt with Queues. This PR includes the following changes. a) Refactoring of the code in SimpleParallelizer to have a mechanism to adjust the memory for the minor fragments being generated. b) There is no change in the logic for generation of minor fragments and adjustment of the memory in the default case (i.e when no RM is selected). The existing logic of calculating the memory will be called during the adjustment phase. c) A new cost class called PrelCostEstimates is introduced to capture the cost estimates calculated during physical planning. These estimates are further used in the memory calculation in case when RM is selected. d) Consider this PR as introducing the framework for estimating and adjusting memory when RM is enabled. The memory estimation based on the cluster state will be further added over these changes. e) Currently only stubs are introduced for estimating the memory in case of Exchange operators. These needs to be incorporated with the Exchange batch sizing ongoing work. @amansinha100 @sohami Can you please review the changes.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
