kaknikhil opened a new pull request #533: URL: https://github.com/apache/madlib/pull/533
JIRA: MADLIB-1463 Previously, the calculation of `gpu_mem_fraction` assumed that num_segments = all_segments which is not always the case. The user can pass in a distribution rules table to input preprocessor and the data can be distributed to less segments than the total number of segments on the cluster. This commit replaces the get_segments_per_host function call with a new function `get_data_distribution_per_segment` which returns the actual distribution of the data instead of returning a list of all the segments. Using this, we can calculate the correct memory fraction. <!-- Thanks for sending a pull request! Here are some tips for you: 1. Refer to this link for contribution guidelines https://cwiki.apache.org/confluence/display/MADLIB/Contribution+Guidelines 2. Please Provide the Module Name, a JIRA Number and a short description about your changes. --> - [ ] Add the module name, JIRA# to PR/commit and description. - [ ] Add tests for the change. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
