andrei5055 opened a new pull request #18582:
URL: https://github.com/apache/incubator-mxnet/pull/18582
## Description ##
The functionality provided by `GPUPooledStorageManager` and
`GPUPooledRoundedStorageManager` classes was re-implemented. This allowed using
caching not only for the `GPU`, but also for the `CPU` and `CPU_Pinned`
memory.
## Checklist ##
### Essentials ###
- [x] Changes are complete (i.e. I finished coding on this PR)
- [x] All changes have test coverage:
- [x] Code is well-documented:
- [x] To the best of my knowledge, examples are either not affected by this
change, or have been fixed to be compatible with this change
### Changes ###
- [x] Class `PooledStorageManager` implemented as a template:
```cpp
template<typename BucketingStrategy, typename StoringMethod>
class PooledStorageManager : public StorageManager,
public BucketingStrategy, public StoringMethod { ... }
```
which
- allows using different methods/algorithms for bucketing and storing cached
data;
- eliminate duplication of code for different memory types: `gpu`, `cpu`,`
cpu_pinned`;
- [x] The default Storage Manager for CPU, CPU_Pinned memory will be
`Pooled` and not `Naive` as before.
- [x] Duplicated code related to the profiling and some parameter checks
(`handle.size` / `handle.dptr`) which was spread inside the different Storage
Manages now is located in corresponding methods of `StorageImpl` class.
- [x] New environment variables which define the bucketing parameters for
`CPU`, `CPU_Pinned` memory used with the Pooled Storage Managers are
implemented and described in `**docs/static_site/src/pages/api/faq/env_var.md**`
- [x] If needed, the user can
- set MXNET_USE_NAIVE_STORAGE_MANAGERS=1 and the Naive Storage Managers for
all types of memory
- use **MXNET_[XXX]_MEM_POOL_TYPE=Unpooled**, with XXX is one of {GPU, CPU,
CPU_PINNED} and start using Native storage Manager for that specific memory
type.
## Comments ##
Our experiments show a 2.2-2.3x improvement in training time when using
`CPU` /` CPU_Pinned` memory with new Pooled Storage Managers compared to their
usage with Naive (non-pooled) Storage Manager's
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
