[MINOR] Improved autoencoder scripts (ordering row-shuffle/z-transform) This patch makes a minor performance improvement to the autoencoder script. So far, we first applied z-transform followed but a random row shuffling. Since the z-transform turns sparse datasets into dense, we now first perform the row shuffling, which makes this permutation matrix multiply significantly faster and can avoid unnecessary evictions.
Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/a51f8e81 Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/a51f8e81 Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/a51f8e81 Branch: refs/heads/master Commit: a51f8e8190281e4c32978d430c08c1b83083faa2 Parents: 1191dbf Author: Matthias Boehm <[email protected]> Authored: Sun Oct 22 21:01:37 2017 -0700 Committer: Matthias Boehm <[email protected]> Committed: Sun Oct 22 21:01:37 2017 -0700 ---------------------------------------------------------------------- scripts/staging/autoencoder-2layer.dml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/systemml/blob/a51f8e81/scripts/staging/autoencoder-2layer.dml ---------------------------------------------------------------------- diff --git a/scripts/staging/autoencoder-2layer.dml b/scripts/staging/autoencoder-2layer.dml index a17d86e..9eee8ae 100644 --- a/scripts/staging/autoencoder-2layer.dml +++ b/scripts/staging/autoencoder-2layer.dml @@ -150,15 +150,15 @@ max_epochs = $EPOCH n = nrow(X) m = ncol(X) +#randomly reordering rows +permut = table(seq(1,n,1), order(target=Rand(rows=n, cols=1, min=0, max=1, pdf="uniform"), by=1, index.return=TRUE), n, n) +X = permut %*% X + #z-transform, whitening operator is better means = colSums(X)/n stds = sqrt((colSums(X^2)/n - means*means)*n/(n-1)) + 1e-17 X = (X - means)/stds -#randomly reordering rows -permut = table(seq(1,n,1), order(target=Rand(rows=n, cols=1, min=0, max=1, pdf="uniform"), by=1, index.return=TRUE), n, n) -X = permut %*% X - W1 = sqrt(6)/sqrt(m + num_hidden1) * Rand(rows=num_hidden1, cols=m, min=-1, max=1, pdf="uniform") b1 = matrix(0, rows=num_hidden1, cols=1) W2 = sqrt(6)/sqrt(num_hidden1 + num_hidden2) * Rand(rows=num_hidden2, cols=num_hidden1, min=-1, max=1, pdf="uniform")
