szha opened a new pull request #11041: [WIP] gpu mem pool strategy URL: https://github.com/apache/incubator-mxnet/pull/11041 ## Description ## adjust GPU memory pool strategy ## Checklist ## ### Essentials ### Please feel free to remove inapplicable items for your PR. - [ ] Changes are complete (i.e. I finished coding on this PR) - [x] All changes have test coverage: - Unit tests are added for small changes to verify correctness (e.g. adding a new operator) - [x] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change ### Changes ### - [x] add knob for minimum memory pool chunk size - [x] add option (`MXNET_GPU_MEM_POOL_TYPE="Round"`) for using nearest power of 2 size for better memory reuse ## Comments ## - fixes #10453 when using MXNET_GPU_MEM_POOL_TYPE="Round". Before the change, memory size must be exact match to reuse the chunk in memory pool. For the workload in #10453, it required cudaMalloc for 55.45GB, whereas with the rounding the cudaMalloc call reduced to 1.32GB, during which the memory usage largely stayed the same. It also helped speed up workloads and improve stability with variations in size that cannot be hybridized yet.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
