[SYSTEMML-446] Minimize the chances of eviction for right indexing operation
Indexing is only supported on GPU if: 1. the input is of type matrix AND 2. the input is less than 2GB. The second condition is added for following reason: 1. Indexing is a purely memory-bound operation and doesnot benefit drastically from pushing down to GPU. 2. By forcing larger matrices to GPU (for example: training dataset), we run into risk of unnecessary evictions of parameters and the gradients. For single precision, there is additional overhead of converting training dataset to single precision every single time it is evicted. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/591a0f77 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/591a0f77 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/591a0f77 Branch: refs/heads/master Commit: 591a0f7754d85e0b9a170cb4385bc84d52e641e8 Parents: f040674 Author: Niketan Pansare <[email protected]> Authored: Wed Oct 25 21:10:21 2017 -0700 Committer: Niketan Pansare <[email protected]> Committed: Wed Oct 25 21:10:21 2017 -0700 ---------------------------------------------------------------------- src/main/java/org/apache/sysml/hops/IndexingOp.java | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/systemml/blob/591a0f77/src/main/java/org/apache/sysml/hops/IndexingOp.java ---------------------------------------------------------------------- diff --git a/src/main/java/org/apache/sysml/hops/IndexingOp.java b/src/main/java/org/apache/sysml/hops/IndexingOp.java index 5989c66..0b8509a 100644 --- a/src/main/java/org/apache/sysml/hops/IndexingOp.java +++ b/src/main/java/org/apache/sysml/hops/IndexingOp.java @@ -102,8 +102,15 @@ public class IndexingOp extends Hop return false; } else { - // only matrix indexing is supported on GPU - return (getDataType() == DataType.MATRIX); + // Indexing is only supported on GPU if: + // 1. the input is of type matrix AND + // 2. the input is less than 2GB. + // The second condition is added for following reason: + // 1. Indexing is a purely memory-bound operation and doesnot benefit drastically from pushing down to GPU. + // 2. By forcing larger matrices to GPU (for example: training dataset), we run into risk of unnecessary evictions of + // parameters and the gradients. For single precision, there is additional overhead of converting training dataset + // to single precision every single time it is evicted. + return (getDataType() == DataType.MATRIX) && getInputMemEstimate() < 2e+9; } }
