[jira] [Updated] (SYSTEMML-618) Deep Learning DML Library
[
https://issues.apache.org/jira/browse/SYSTEMML-618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-618:
-
Description:
This issue tracks the creation of an experimental, layers-based library in pure
PyDML & DML that contains layer abstractions with simple forward/backward APIs
for affine, convolution (start with 2D), max-pooling, non-linearities (relu,
sigmoid, softmax, etc.), dropout, loss functions, other layers, optimizers, and
gradient checks.
*SystemML-NN*:
[https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
* Layers:
** Core:
*** Affine
*** Spatial Convolution
*** LSTM
*** Max Pooling
*** RNN
** Nonlinearities:
*** ReLU
*** Sigmoid
*** Softmax
*** Tanh
** Loss:
*** Cross-entropy loss
*** L1 loss
*** L2 loss
*** Log ("Logistic") loss
** Regularization:
*** Dropout
*** L1 reg
*** L2 reg
was:This issue tracks the creation of an experimental, layers-based library
in pure DML that contains layer abstractions with simple forward/backward APIs
for affine, convolution (start with 2D), max-pooling, non-linearities (relu,
sigmoid, softmax, etc.), dropout, loss functions, other layers, optimizers, and
gradient checks.
> Deep Learning DML Library
> -
>
> Key: SYSTEMML-618
> URL: https://issues.apache.org/jira/browse/SYSTEMML-618
> Project: SystemML
> Issue Type: New Feature
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>
> This issue tracks the creation of an experimental, layers-based library in
> pure PyDML & DML that contains layer abstractions with simple
> forward/backward APIs for affine, convolution (start with 2D), max-pooling,
> non-linearities (relu, sigmoid, softmax, etc.), dropout, loss functions,
> other layers, optimizers, and gradient checks.
> *SystemML-NN*:
> [https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
> * Layers:
> ** Core:
> *** Affine
> *** Spatial Convolution
> *** LSTM
> *** Max Pooling
> *** RNN
> ** Nonlinearities:
> *** ReLU
> *** Sigmoid
> *** Softmax
> *** Tanh
> ** Loss:
> *** Cross-entropy loss
> *** L1 loss
> *** L2 loss
> *** Log ("Logistic") loss
> ** Regularization:
> *** Dropout
> *** L1 reg
> *** L2 reg
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-652) Left-Indexing With Result of DML Function Changes Matrix Size
[
https://issues.apache.org/jira/browse/SYSTEMML-652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry closed SYSTEMML-652.
Assignee: Matthias Boehm
> Left-Indexing With Result of DML Function Changes Matrix Size
> -
>
> Key: SYSTEMML-652
> URL: https://issues.apache.org/jira/browse/SYSTEMML-652
> Project: SystemML
> Issue Type: Bug
> Components: Compiler
>Affects Versions: SystemML 0.9, SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Matthias Boehm
> Fix For: SystemML 0.10
>
>
> I've found a bug in which assigning the result of a DML function to a portion
> of a matrix with left-indexing results in the left-hand matrix being reduced
> in size dimensionally. This bug was encountered while working on the deep
> learning DML library, and the following simplified example aims to provide a
> simple, reproducible example.
> Given the following code,
> {code}
> N = 3
> M = 5
> forward = function(matrix[double] X) return (matrix[double] out) {
> out = 1 / (1 + exp(-X))
> }
> X = rand(rows=N, cols=M)
> X[,1:2] = forward(X[,1:2])
> print("X1: " + nrow(X) + "x" + ncol(X))
> if(1==1){}
> X = rand(rows=N, cols=M)
> temp = forward(X[,1:2])
> X[,1:2] = temp
> print("X2: " + nrow(X) + "x" + ncol(X))
> if(1==1){}
> print("")
> {code}
> , notice that {{X}} should always be a {{3x5}} matrix, as both cases are
> equivalent. However, in the first case, {{X}} is truncated to a {{3x2}}
> matrix:
> {code}
> X1: 3x2
> X2: 3x5
> {code}
> Note: The {{if(1==1){}}} statements are included because otherwise the print
> statements are executed out of order.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-679) Add `X.T` Transpose Attribute Method To PyDML
[
https://issues.apache.org/jira/browse/SYSTEMML-679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-679:
-
Issue Type: Improvement (was: Bug)
> Add `X.T` Transpose Attribute Method To PyDML
> -
>
> Key: SYSTEMML-679
> URL: https://issues.apache.org/jira/browse/SYSTEMML-679
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>
> Currently, PyDML allows one to transpose a matrix using {{X.transpose()}},
> which is valid in Python (NumPy) as well. However, it is much more common
> (for clear, understandable code) in Python/NumPy to simply use the transpose
> attribute, {{X.T}}.
> We should support the latter {{X.T}} syntax in PyDML.
> Example (PyDML):
> {code}
> X = matrix("[1 2 3; 4 5 6; 7 8 9; 10 11 12]")
> Z = X.transpose()
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-618) Deep Learning DML Library
[
https://issues.apache.org/jira/browse/SYSTEMML-618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278693#comment-15278693
]
Mike Dusenberry commented on SYSTEMML-618:
--
Updates:
* The library is currently written entirely in the DML (R-like) syntax. I am
about to start adding a PyDML version that will become the new main codebase.
I plan to maintain both versions in the same repo, with DML & PyDML versions
co-located in the repo.
* There is early support for (vanilla) RNNs and LSTMs. The plan is to expand
on both of these.
> Deep Learning DML Library
> -
>
> Key: SYSTEMML-618
> URL: https://issues.apache.org/jira/browse/SYSTEMML-618
> Project: SystemML
> Issue Type: New Feature
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>
> This issue tracks the creation of an experimental, layers-based library in
> pure PyDML & DML that contains layers with simple forward/backward APIs for
> affine, convolution (start with 2D), max-pooling, non-linearities (relu,
> sigmoid, softmax, etc.), dropout, loss functions, other layers, optimizers,
> and gradient checks.
> *SystemML-NN*:
> [https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
> _Current status:_
> * Layers:
> ** Core:
> *** Affine
> *** Spatial Convolution
> *** LSTM
> *** Max Pooling
> *** RNN
> ** Nonlinearities:
> *** ReLU
> *** Sigmoid
> *** Softmax
> *** Tanh
> ** Loss:
> *** Cross-entropy loss
> *** L1 loss
> *** L2 loss
> *** Log ("Logistic") loss
> ** Regularization:
> *** Dropout
> *** L1 reg
> *** L2 reg
> * Optimizers:
> ** Adagrad
> ** Adam
> ** RMSprop
> ** SGD
> ** SGD w/ Momentum
> ** SGD w/ Nesterov Momentum
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-618) Deep Learning DML Library
[
https://issues.apache.org/jira/browse/SYSTEMML-618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-618:
-
Description:
This issue tracks the creation of an experimental, layers-based library in pure
PyDML & DML that contains layers with simple forward/backward APIs for affine,
convolution (start with 2D), max-pooling, non-linearities (relu, sigmoid,
softmax, etc.), dropout, loss functions, other layers, optimizers, and gradient
checks.
*SystemML-NN*:
[https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
_Current status:_
* Layers:
** Core:
*** Affine
*** Spatial Convolution
*** LSTM
*** Max Pooling
*** RNN
** Nonlinearities:
*** ReLU
*** Sigmoid
*** Softmax
*** Tanh
** Loss:
*** Cross-entropy loss
*** L1 loss
*** L2 loss
*** Log ("Logistic") loss
** Regularization:
*** Dropout
*** L1 reg
*** L2 reg
* Optimizers:
** Adagrad
** Adam
** RMSprop
** SGD
** SGD w/ Momentum
** SGD w/ Nesterov Momentum
was:
This issue tracks the creation of an experimental, layers-based library in pure
PyDML & DML that contains layers with simple forward/backward APIs for affine,
convolution (start with 2D), max-pooling, non-linearities (relu, sigmoid,
softmax, etc.), dropout, loss functions, other layers, optimizers, and gradient
checks.
*SystemML-NN*:
[https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
* Layers:
** Core:
*** Affine
*** Spatial Convolution
*** LSTM
*** Max Pooling
*** RNN
** Nonlinearities:
*** ReLU
*** Sigmoid
*** Softmax
*** Tanh
** Loss:
*** Cross-entropy loss
*** L1 loss
*** L2 loss
*** Log ("Logistic") loss
** Regularization:
*** Dropout
*** L1 reg
*** L2 reg
* Optimizers:
** Adagrad
** Adam
** RMSprop
** SGD
** SGD w/ Momentum
** SGD w/ Nesterov Momentum
> Deep Learning DML Library
> -
>
> Key: SYSTEMML-618
> URL: https://issues.apache.org/jira/browse/SYSTEMML-618
> Project: SystemML
> Issue Type: New Feature
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>
> This issue tracks the creation of an experimental, layers-based library in
> pure PyDML & DML that contains layers with simple forward/backward APIs for
> affine, convolution (start with 2D), max-pooling, non-linearities (relu,
> sigmoid, softmax, etc.), dropout, loss functions, other layers, optimizers,
> and gradient checks.
> *SystemML-NN*:
> [https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
> _Current status:_
> * Layers:
> ** Core:
> *** Affine
> *** Spatial Convolution
> *** LSTM
> *** Max Pooling
> *** RNN
> ** Nonlinearities:
> *** ReLU
> *** Sigmoid
> *** Softmax
> *** Tanh
> ** Loss:
> *** Cross-entropy loss
> *** L1 loss
> *** L2 loss
> *** Log ("Logistic") loss
> ** Regularization:
> *** Dropout
> *** L1 reg
> *** L2 reg
> * Optimizers:
> ** Adagrad
> ** Adam
> ** RMSprop
> ** SGD
> ** SGD w/ Momentum
> ** SGD w/ Nesterov Momentum
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-676:
-
Component/s: PyDML
Parser
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e.,
> {{X\[:3,]}} should return all rows up to (but not including) 3. This means
> that the implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all
> rows starting at row 2 (0-based), which implies that the upper bound is equal
> to the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X1[:,:]}}, should also be valid, regardless if it
> is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-618) Deep Learning DML Library
[
https://issues.apache.org/jira/browse/SYSTEMML-618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-618:
-
Description:
This issue tracks the creation of an experimental, layers-based library in pure
PyDML & DML that contains layer abstractions with simple forward/backward APIs
for affine, convolution (start with 2D), max-pooling, non-linearities (relu,
sigmoid, softmax, etc.), dropout, loss functions, other layers, optimizers, and
gradient checks.
*SystemML-NN*:
[https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
* Layers:
** Core:
*** Affine
*** Spatial Convolution
*** LSTM
*** Max Pooling
*** RNN
** Nonlinearities:
*** ReLU
*** Sigmoid
*** Softmax
*** Tanh
** Loss:
*** Cross-entropy loss
*** L1 loss
*** L2 loss
*** Log ("Logistic") loss
** Regularization:
*** Dropout
*** L1 reg
*** L2 reg
* Optimizers:
** Adagrad
** Adam
** RMSprop
** SGD
** SGD w/ Momentum
** SGD w/ Nesterov Momentum
was:
This issue tracks the creation of an experimental, layers-based library in pure
PyDML & DML that contains layer abstractions with simple forward/backward APIs
for affine, convolution (start with 2D), max-pooling, non-linearities (relu,
sigmoid, softmax, etc.), dropout, loss functions, other layers, optimizers, and
gradient checks.
*SystemML-NN*:
[https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
* Layers:
** Core:
*** Affine
*** Spatial Convolution
*** LSTM
*** Max Pooling
*** RNN
** Nonlinearities:
*** ReLU
*** Sigmoid
*** Softmax
*** Tanh
** Loss:
*** Cross-entropy loss
*** L1 loss
*** L2 loss
*** Log ("Logistic") loss
** Regularization:
*** Dropout
*** L1 reg
*** L2 reg
> Deep Learning DML Library
> -
>
> Key: SYSTEMML-618
> URL: https://issues.apache.org/jira/browse/SYSTEMML-618
> Project: SystemML
> Issue Type: New Feature
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>
> This issue tracks the creation of an experimental, layers-based library in
> pure PyDML & DML that contains layer abstractions with simple
> forward/backward APIs for affine, convolution (start with 2D), max-pooling,
> non-linearities (relu, sigmoid, softmax, etc.), dropout, loss functions,
> other layers, optimizers, and gradient checks.
> *SystemML-NN*:
> [https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]
> * Layers:
> ** Core:
> *** Affine
> *** Spatial Convolution
> *** LSTM
> *** Max Pooling
> *** RNN
> ** Nonlinearities:
> *** ReLU
> *** Sigmoid
> *** Softmax
> *** Tanh
> ** Loss:
> *** Cross-entropy loss
> *** L1 loss
> *** L2 loss
> *** Log ("Logistic") loss
> ** Regularization:
> *** Dropout
> *** L1 reg
> *** L2 reg
> * Optimizers:
> ** Adagrad
> ** Adam
> ** RMSprop
> ** SGD
> ** SGD w/ Momentum
> ** SGD w/ Nesterov Momentum
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-676:
-
Description:
Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
However, a really useful piece of slicing syntax we are currently missing from
PyDML is the ability to have implicit lower or upper bounds, i.e., {{X\[:3,]}}
should return all rows up to (but not including) 3. This means that the
implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all rows
starting at row 2 (0-based), which implies that the upper bound is equal to the
number of rows of {{X}}.
We should update the grammar to allow these cases, row both row and column
slicing. More generally, {{X\[:,:]}}, should also be valid, regardless if it
is useful.
was:
Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
However, a really useful piece of slicing syntax we are currently missing from
PyDML is the ability to have implicit lower or upper bounds, i.e., {{X\[:3,]}}
should return all rows up to (but not including) 3. This means that the
implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all rows
starting at row 2 (0-based), which implies that the upper bound is equal to the
number of rows of {{X}}.
We should update the grammar to allow these cases, row both row and column
slicing. More generally, {{X1[:,:]}}, should also be valid, regardless if it
is useful.
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e.,
> {{X\[:3,]}} should return all rows up to (but not including) 3. This means
> that the implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all
> rows starting at row 2 (0-based), which implies that the upper bound is equal
> to the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X\[:,:]}}, should also be valid, regardless if it
> is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Attachment: time.txt
run.sh
perf.sh
perf-tf.py
perf-dml.dml
log.txt
Attached the files for the TensorFlow vs SystemML tests.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
>Priority: Critical
> Attachments: Im2colWrapper.java, log.txt, log.txt, perf-dml.dml,
> perf-tf.py, perf.sh, run.sh, systemml-nn.zip, time.txt
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/0.
> HOP DAGs recompile time: 0.268 sec.
> Functions recompiled: 129.
> Functions recompile time: 0.841 sec.
> ParFor loops optimized: 1.
> ParFor optimize time: 0.032 sec.
> ParFor initialize time: 0.015 sec.
> ParFor result merge time: 0.028 sec.
> ParFor total update in-place: 0/0/1559360
> Total JIT compile time: 14.235 sec.
> Total JVM GC count: 94.
> Total JVM GC time:0.366 sec.
> Heavy hitter
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Priority: Critical (was: Major)
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
>Priority: Critical
> Attachments: Im2colWrapper.java, log.txt, systemml-nn.zip
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/0.
> HOP DAGs recompile time: 0.268 sec.
> Functions recompiled: 129.
> Functions recompile time: 0.841 sec.
> ParFor loops optimized: 1.
> ParFor optimize time: 0.032 sec.
> ParFor initialize time: 0.015 sec.
> ParFor result merge time: 0.028 sec.
> ParFor total update in-place: 0/0/1559360
> Total JIT compile time: 14.235 sec.
> Total JVM GC count: 94.
> Total JVM GC time:0.366 sec.
> Heavy hitter instructions (name, time, count):
> -- 1) leftIndex 41.670 sec 1559360
> -- 2) forward 26.212 sec 1
> -- 3) im2col_t45 25.919 sec 8
> -- 4) im2col_t41 25.850 sec 8
> -- 5) im2col_t48 25.831 sec 8
> -- 6) im2col_t43
[jira] [Commented] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280532#comment-15280532
]
Mike Dusenberry commented on SYSTEMML-633:
--
Update: I ran a set of tests to compare *TensorFlow* with our *SystemML* engine
by using a forward convolution function. While the particular scenario is
convolution and deep learning, the reality is that it is just DML-bodied
functions and the engine being compared to TensorFlow's functions and engine.
*TensorFlow vs. SystemML*
---
* Machine
** Massive single node from a research cluster:
*** 2 Intel(R) Xeon(R) CPU E5-2620 v2 @ 2.10GHz (6 cores each, hyperthreaded to
24 total virtual cores)
*** 256GB RAM
* Tests:
** 4 setups of forward convolution for both SystemML & TensorFlow -- see
attached files {{perf-dml.dml}}, {{perf-tf.py}}.
* Results: (Engine, Iterations, Setup #, Time (sec))
** SystemML-NN,1,1,6.963174578
Tensorflow,1,1,2.409152860
SystemML-NN,1,2,80.491425876
Tensorflow,1,2,26.183729520
SystemML-NN,1,3,7.187359147
Tensorflow,1,3,2.656847511
SystemML-NN,1,4,83.597674899
Tensorflow,1,4,21.305138164
SystemML-NN,10,1,27.405661815
Tensorflow,10,1,2.096477385
SystemML-NN,10,2,338.673049859
Tensorflow,10,2,21.800133563
SystemML-NN,10,3,35.109882799
Tensorflow,10,3,2.028066097
SystemML-NN,10,4,396.854964030
Tensorflow,10,4,22.103966235
SystemML-NN,100,1,245.515412673
Tensorflow,100,1,2.511825601
SystemML-NN,100,2,3150.047737891
Tensorflow,100,2,26.059402039
SystemML-NN,100,3,326.527263634
Tensorflow,100,3,2.547926570
SystemML-NN,100,4,3821.133584436
Tensorflow,100,4,24.366318114
SystemML-NN,1000,1,2435.717824721
Tensorflow,1000,1,7.634583445
SystemML-NN,1000,2,30354.415483735
Tensorflow,1000,2,67.591256457
SystemML-NN,1000,3,3161.059507123
Tensorflow,1000,3,7.824721516
SystemML-NN,1000,4,36051.256351073
Tensorflow,1000,4,55.355726359
* Analysis:
** Rewrites, such as update-in-place, are not being applied to valid code if it
is located within a DML-bodied function.
** See log file: {{log.txt}}
*Overall*: The SystemML engine is currently *far behind* TensorFlow when
DML-bodied functions are used.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
>Priority: Critical
> Attachments: Im2colWrapper.java, log.txt, systemml-nn.zip
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}}
[jira] [Comment Edited] (SYSTEMML-590) Improve Namespace Handling for UDFs
[
https://issues.apache.org/jira/browse/SYSTEMML-590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281897#comment-15281897
]
Mike Dusenberry edited comment on SYSTEMML-590 at 5/12/16 6:41 PM:
---
This has been a lot of work, and UDFs/namespaces are much better now. Thanks,
[~gweidner]!
was (Author: mwdus...@us.ibm.com):
This has been a lot of work, and UDFs are much better now. Thanks, [~gweidner]!
> Improve Namespace Handling for UDFs
> ---
>
> Key: SYSTEMML-590
> URL: https://issues.apache.org/jira/browse/SYSTEMML-590
> Project: SystemML
> Issue Type: Improvement
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>
> Currently, if a UDF body involves calling another UDF, the default global
> namespace is assumed, unless a namespace is explicitly indicated. This
> becomes a problem when a file contains UDFs, and is then sourced from another
> script.
> Imagine a file {{funcs.dml}} as follows:
> {code}
> f = function(double x, int a) return (double ans) {
> x2 = g(x)
> ans = a * x2
> }
> g = function(double x) return (double ans) {
> ans = x * x
> }
> {code}
> Then, let's try to call {{f}}:
> {code}
> script = """
> source ("funcs.dml") as funcs
> ans = funcs::f(3, 1)
> print(ans)
> """
> ml.reset()
> ml.executeScript(script)
> {code}
> This results in an error since {{f}} is in the {{funcs}} namespace, but the
> call to {{g}} assumes {{g}} is still in the default namespace. Clearly, the
> user intends to the use the {{g}} that is located in the same file.
> Currently, we would need to adjust {{funcs.dml}} as follows to explicitly
> assume that {{f}} and {{g}} are in a {{funcs}} namespace:
> {code}
> f = function(double x, int a) return (double ans) {
> x2 = funcs::g(x)
> ans = a * x2
> }
> g = function(double x) return (double ans) {
> ans = x * x
> }
> {code}
> Instead, it would be better to simply first look for {{g}} in its parent's
> namespace. In this case, the "parent" would be the function {{f}}, and the
> namespace we have selected is {{funcs}}, although that choice would be left
> up to the end-user. Then, namespace assumptions would not be necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-590) Improve Namespace Handling for UDFs
[
https://issues.apache.org/jira/browse/SYSTEMML-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry resolved SYSTEMML-590.
--
Resolution: Fixed
Fix Version/s: SystemML 0.10
> Improve Namespace Handling for UDFs
> ---
>
> Key: SYSTEMML-590
> URL: https://issues.apache.org/jira/browse/SYSTEMML-590
> Project: SystemML
> Issue Type: Improvement
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
> Fix For: SystemML 0.10
>
>
> Currently, if a UDF body involves calling another UDF, the default global
> namespace is assumed, unless a namespace is explicitly indicated. This
> becomes a problem when a file contains UDFs, and is then sourced from another
> script.
> Imagine a file {{funcs.dml}} as follows:
> {code}
> f = function(double x, int a) return (double ans) {
> x2 = g(x)
> ans = a * x2
> }
> g = function(double x) return (double ans) {
> ans = x * x
> }
> {code}
> Then, let's try to call {{f}}:
> {code}
> script = """
> source ("funcs.dml") as funcs
> ans = funcs::f(3, 1)
> print(ans)
> """
> ml.reset()
> ml.executeScript(script)
> {code}
> This results in an error since {{f}} is in the {{funcs}} namespace, but the
> call to {{g}} assumes {{g}} is still in the default namespace. Clearly, the
> user intends to the use the {{g}} that is located in the same file.
> Currently, we would need to adjust {{funcs.dml}} as follows to explicitly
> assume that {{f}} and {{g}} are in a {{funcs}} namespace:
> {code}
> f = function(double x, int a) return (double ans) {
> x2 = funcs::g(x)
> ans = a * x2
> }
> g = function(double x) return (double ans) {
> ans = x * x
> }
> {code}
> Instead, it would be better to simply first look for {{g}} in its parent's
> namespace. In this case, the "parent" would be the function {{f}}, and the
> namespace we have selected is {{funcs}}, although that choice would be left
> up to the end-user. Then, namespace assumptions would not be necessary.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry closed SYSTEMML-654.
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
> Fix For: SystemML 0.10
>
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> {code}
> : org.apache.sysml.parser.LanguageException: Unsupported Parameters : ERROR:
> null -- line 6, column 0 -- Expecting matrix parameter for function MIN
> at
> org.apache.sysml.parser.Expression.raiseValidateError(Expression.java:421)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.checkMatrixParam(BuiltinFunctionExpression.java:1221)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.validateExpression(BuiltinFunctionExpression.java:314)
> at
> org.apache.sysml.parser.StatementBlock.validate(StatementBlock.java:598)
> at
> org.apache.sysml.parser.DMLTranslator.validateParseTree(DMLTranslator.java:136)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1325)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1227)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15281893#comment-15281893
]
Mike Dusenberry commented on SYSTEMML-654:
--
Merged in [commit 90f56da |
https://github.com/apache/incubator-systemml/commit/90f56da29d02d4a7b59d1c46a1b5b12993177233].
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
> Fix For: SystemML 0.10
>
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> {code}
> : org.apache.sysml.parser.LanguageException: Unsupported Parameters : ERROR:
> null -- line 6, column 0 -- Expecting matrix parameter for function MIN
> at
> org.apache.sysml.parser.Expression.raiseValidateError(Expression.java:421)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.checkMatrixParam(BuiltinFunctionExpression.java:1221)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.validateExpression(BuiltinFunctionExpression.java:314)
> at
> org.apache.sysml.parser.StatementBlock.validate(StatementBlock.java:598)
> at
> org.apache.sysml.parser.DMLTranslator.validateParseTree(DMLTranslator.java:136)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1325)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1227)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry resolved SYSTEMML-654.
--
Resolution: Fixed
Fix Version/s: SystemML 0.10
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
> Fix For: SystemML 0.10
>
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> {code}
> : org.apache.sysml.parser.LanguageException: Unsupported Parameters : ERROR:
> null -- line 6, column 0 -- Expecting matrix parameter for function MIN
> at
> org.apache.sysml.parser.Expression.raiseValidateError(Expression.java:421)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.checkMatrixParam(BuiltinFunctionExpression.java:1221)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.validateExpression(BuiltinFunctionExpression.java:314)
> at
> org.apache.sysml.parser.StatementBlock.validate(StatementBlock.java:598)
> at
> org.apache.sysml.parser.DMLTranslator.validateParseTree(DMLTranslator.java:136)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1325)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1227)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Attachment: systemml-nn-05.16.16.zip
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
>Priority: Critical
> Attachments: Im2colWrapper.java, log.txt, log.txt, perf-dml.dml,
> perf-tf.py, perf.sh, run.sh, systemml-nn-05.16.16.zip, systemml-nn.zip,
> time.txt
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/0.
> HOP DAGs recompile time: 0.268 sec.
> Functions recompiled: 129.
> Functions recompile time: 0.841 sec.
> ParFor loops optimized: 1.
> ParFor optimize time: 0.032 sec.
> ParFor initialize time: 0.015 sec.
> ParFor result merge time: 0.028 sec.
> ParFor total update in-place: 0/0/1559360
> Total JIT compile time: 14.235 sec.
> Total JVM GC count: 94.
> Total JVM GC time:0.366 sec.
> Heavy hitter instructions (name, time, count):
> -- 1) leftIndex 41.670 sec 1559360
> -- 2) forward 26.212 sec 1
> -- 3) im2col_t45 25.919
[jira] [Commented] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284875#comment-15284875
]
Mike Dusenberry commented on SYSTEMML-633:
--
[~mboehm7] Indeed, the zipped version of SystemML-NN was old and had hacks from
before we fixed the namespace issues. I uploaded the current version as
`systemml-nn-05.16.16.zip`, and now it should work correctly.
Thanks for looking into this!
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
>Priority: Critical
> Attachments: Im2colWrapper.java, log.txt, log.txt, perf-dml.dml,
> perf-tf.py, perf.sh, run.sh, systemml-nn-05.16.16.zip, systemml-nn.zip,
> time.txt
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/0.
> HOP DAGs recompile time: 0.268 sec.
> Functions recompiled: 129.
> Functions recompile time: 0.841 sec.
> ParFor loops optimized: 1.
> ParFor optimize time: 0.032 sec.
> ParFor initialize time: 0.015 sec.
> ParFor result merge time: 0.028 sec.
> ParFor total update in-place: 0/0/1559360
> Total JIT
[jira] [Comment Edited] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284875#comment-15284875
]
Mike Dusenberry edited comment on SYSTEMML-633 at 5/16/16 5:31 PM:
---
[~mboehm7] Indeed, the zipped version of SystemML-NN was old and had hacks from
before we fixed the namespace issues. I uploaded the current version as
{{systemml-nn-05.16.16.zip}}, and now it should work correctly.
Thanks for looking into this!
was (Author: mwdus...@us.ibm.com):
[~mboehm7] Indeed, the zipped version of SystemML-NN was old and had hacks from
before we fixed the namespace issues. I uploaded the current version as
`systemml-nn-05.16.16.zip`, and now it should work correctly.
Thanks for looking into this!
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
>Priority: Critical
> Attachments: Im2colWrapper.java, log.txt, log.txt, perf-dml.dml,
> perf-tf.py, perf.sh, run.sh, systemml-nn-05.16.16.zip, systemml-nn.zip,
> time.txt
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs
[jira] [Resolved] (SYSTEMML-670) PyDML `random.normal` Function Broken Due to Namespace Issue
[
https://issues.apache.org/jira/browse/SYSTEMML-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry resolved SYSTEMML-670.
--
Resolution: Fixed
Fix Version/s: SystemML 0.10
> PyDML `random.normal` Function Broken Due to Namespace Issue
>
>
> Key: SYSTEMML-670
> URL: https://issues.apache.org/jira/browse/SYSTEMML-670
> Project: SystemML
> Issue Type: Bug
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
> Fix For: SystemML 0.10
>
>
> In PyDML, the {{random.normal(...)}} built-in function is currently broken
> due to a namespace issue.
> Example:
> {code}
> N = 4
> D = 3
> X = random.normal(N, D, 1)
> print(sum(X))
> {code}
> {code}
> org.apache.sysml.parser.LanguageException: ERROR: null -- line 4, column 0 --
> function normal is undefined in namespace random
> at
> org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201)
> at
> org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:335)
> at
> org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:202)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1354)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1257)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1195)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285010#comment-15285010
]
Mike Dusenberry commented on SYSTEMML-676:
--
Merged in [commit 18bb8c5 |
https://github.com/apache/incubator-systemml/commit/18bb8c52cc10abd9679f5de3b6107a19ec79dc21].
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
> Fix For: SystemML 0.10
>
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e.,
> {{X\[:3,]}} should return all rows up to (but not including) 3. This means
> that the implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all
> rows starting at row 2 (0-based), which implies that the upper bound is equal
> to the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X\[:,:]}}, should also be valid, regardless if it
> is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry resolved SYSTEMML-676.
--
Resolution: Fixed
Fix Version/s: SystemML 0.10
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
> Fix For: SystemML 0.10
>
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e.,
> {{X\[:3,]}} should return all rows up to (but not including) 3. This means
> that the implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all
> rows starting at row 2 (0-based), which implies that the upper bound is equal
> to the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X\[:,:]}}, should also be valid, regardless if it
> is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-670) PyDML `random.normal` Function Broken Due to Namespace Issue
[
https://issues.apache.org/jira/browse/SYSTEMML-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry reassigned SYSTEMML-670:
Assignee: Mike Dusenberry
> PyDML `random.normal` Function Broken Due to Namespace Issue
>
>
> Key: SYSTEMML-670
> URL: https://issues.apache.org/jira/browse/SYSTEMML-670
> Project: SystemML
> Issue Type: Bug
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>
> In PyDML, the {{random.normal(...)}} built-in function is currently broken
> due to a namespace issue.
> Example:
> {code}
> N = 4
> D = 3
> X = random.normal(N, D, 1)
> print(sum(X))
> {code}
> {code}
> org.apache.sysml.parser.LanguageException: ERROR: null -- line 4, column 0 --
> function normal is undefined in namespace random
> at
> org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201)
> at
> org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:335)
> at
> org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:202)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1354)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1257)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1195)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-588) Improve UDFs
[ https://issues.apache.org/jira/browse/SYSTEMML-588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry reassigned SYSTEMML-588: Assignee: Mike Dusenberry (was: Glenn Weidner) > Improve UDFs > > > Key: SYSTEMML-588 > URL: https://issues.apache.org/jira/browse/SYSTEMML-588 > Project: SystemML > Issue Type: Epic >Affects Versions: SystemML 0.10 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry > > This epic aims to improve the state of user-defined functions (UDFs) in DML & > PyDML. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops in UDFs
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Summary: Improve Left-Indexing Performance with (Nested) Parfor Loops in
UDFs (was: Improve Left-Indexing Performance with (Nested) Parfor Loops)
> Improve Left-Indexing Performance with (Nested) Parfor Loops in UDFs
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
>Priority: Critical
> Attachments: Im2colWrapper.java, log.txt, log.txt, perf-dml.dml,
> perf-tf.py, perf.sh, run.sh, systemml-nn-05.16.16.zip, systemml-nn.zip,
> time.txt
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/0.
> HOP DAGs recompile time: 0.268 sec.
> Functions recompiled: 129.
> Functions recompile time: 0.841 sec.
> ParFor loops optimized: 1.
> ParFor optimize time: 0.032 sec.
> ParFor initialize time: 0.015 sec.
> ParFor result merge time: 0.028 sec.
> ParFor total update in-place: 0/0/1559360
> Total JIT compile time: 14.235 sec.
> Total JVM GC count: 94.
> Total JVM GC time:0.366 sec.
> Heavy hitter instructions
[jira] [Closed] (SYSTEMML-512) DML Script With UDFs Results In Out Of Memory Error As Compared to Without UDFs
[
https://issues.apache.org/jira/browse/SYSTEMML-512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry closed SYSTEMML-512.
> DML Script With UDFs Results In Out Of Memory Error As Compared to Without
> UDFs
> ---
>
> Key: SYSTEMML-512
> URL: https://issues.apache.org/jira/browse/SYSTEMML-512
> Project: SystemML
> Issue Type: Bug
>Reporter: Mike Dusenberry
>Assignee: Matthias Boehm
> Fix For: SystemML 0.10
>
> Attachments: test1.scala, test2.scala
>
>
> Currently, the following script for running a simple version of Poisson
> non-negative matrix factorization (PNMF) runs in linear time as desired:
> {code}
> # data & args
> X = read($X)
> X = X+1 # change product IDs to be 1-based, rather than 0-based
> V = table(X[,1], X[,2])
> V = V[1:$size,1:$size]
> max_iteration = as.integer($maxiter)
> rank = as.integer($rank)
> # run PNMF
> n = nrow(V)
> m = ncol(V)
> range = 0.01
> W = Rand(rows=n, cols=rank, min=0, max=range, pdf="uniform")
> H = Rand(rows=rank, cols=m, min=0, max=range, pdf="uniform")
> i=0
> while(i < max_iteration) {
> H = (H * (t(W) %*% (V/(W%*%H/t(colSums(W))
> W = (W * ((V/(W%*%H)) %*% t(H)))/t(rowSums(H))
> i = i + 1;
> }
> # compute negative log-likelihood
> negloglik_temp = -1 * (sum(V*log(W%*%H)) - as.scalar(colSums(W)%*%rowSums(H)))
> # write outputs
> negloglik = matrix(negloglik_temp, rows=1, cols=1)
> write(negloglik, $negloglikout)
> write(W, $Wout)
> write(H, $Hout)
> {code}
> However, a small refactoring of this same script to pull the core PNMF
> algorithm and the negative log-likelihood computation out into separate UDFs
> results in non-linear runtime and a Java out of memory heap error on the same
> dataset.
> {code}
> pnmf = function(matrix[double] V, integer max_iteration, integer rank) return
> (matrix[double] W, matrix[double] H) {
> n = nrow(V)
> m = ncol(V)
>
> range = 0.01
> W = Rand(rows=n, cols=rank, min=0, max=range, pdf="uniform")
> H = Rand(rows=rank, cols=m, min=0, max=range, pdf="uniform")
>
> i=0
> while(i < max_iteration) {
> H = (H * (t(W) %*% (V/(W%*%H/t(colSums(W))
> W = (W * ((V/(W%*%H)) %*% t(H)))/t(rowSums(H))
> i = i + 1;
> }
> }
> negloglikfunc = function(matrix[double] V, matrix[double] W, matrix[double]
> H) return (double negloglik) {
> negloglik = -1 * (sum(V*log(W%*%H)) - as.scalar(colSums(W)%*%rowSums(H)))
> }
> # data & args
> X = read($X)
> X = X+1 # change product IDs to be 1-based, rather than 0-based
> V = table(X[,1], X[,2])
> V = V[1:$size,1:$size]
> max_iteration = as.integer($maxiter)
> rank = as.integer($rank)
> # run PNMF and evaluate
> [W, H] = pnmf(V, max_iteration, rank)
> negloglik_temp = negloglikfunc(V, W, H)
> # write outputs
> negloglik = matrix(negloglik_temp, rows=1, cols=1)
> write(negloglik, $negloglikout)
> write(W, $Wout)
> write(H, $Hout)
> {code}
> The expectation would be that such modularization at the DML level should be
> allowed without any impact on performance.
> Details:
> - Data: Amazon product co-purchasing dataset from Stanford
> [http://snap.stanford.edu/data/amazon0601.html |
> http://snap.stanford.edu/data/amazon0601.html]
> - Execution mode: Spark {{MLContext}}, but should be applicable to
> command-line invocation as well.
> - Error message:
> {code}
> java.lang.OutOfMemoryError: Java heap space
> at
> org.apache.sysml.runtime.matrix.data.MatrixBlock.allocateDenseBlock(MatrixBlock.java:415)
> at
> org.apache.sysml.runtime.matrix.data.MatrixBlock.sparseToDense(MatrixBlock.java:1212)
> at
> org.apache.sysml.runtime.matrix.data.MatrixBlock.examSparsity(MatrixBlock.java:1103)
> at
> org.apache.sysml.runtime.instructions.cp.MatrixMatrixArithmeticCPInstruction.processInstruction(MatrixMatrixArithmeticCPInstruction.java:60)
> at
> org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:309)
> at
> org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:227)
> at
> org.apache.sysml.runtime.controlprogram.ProgramBlock.execute(ProgramBlock.java:169)
> at
> org.apache.sysml.runtime.controlprogram.WhileProgramBlock.execute(WhileProgramBlock.java:183)
> at
> org.apache.sysml.runtime.controlprogram.FunctionProgramBlock.execute(FunctionProgramBlock.java:115)
> at
> org.apache.sysml.runtime.instructions.cp.FunctionCallCPInstruction.processInstruction(FunctionCallCPInstruction.java:177)
> at
> org.apache.sysml.runtime.controlprogram.ProgramBlock.executeSingleInstruction(ProgramBlock.java:309)
> at
> org.apache.sysml.runtime.controlprogram.ProgramBlock.executeInstructions(ProgramBlock.java:227)
> at
>
[jira] [Commented] (SYSTEMML-670) PyDML `random.normal` Function Broken Due to Namespace Issue
[
https://issues.apache.org/jira/browse/SYSTEMML-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285055#comment-15285055
]
Mike Dusenberry commented on SYSTEMML-670:
--
Merged in [commit 66ea6f0 |
https://github.com/apache/incubator-systemml/commit/66ea6f0c6cf74d4d4413c80cc9b42a0b89f82b07].
> PyDML `random.normal` Function Broken Due to Namespace Issue
>
>
> Key: SYSTEMML-670
> URL: https://issues.apache.org/jira/browse/SYSTEMML-670
> Project: SystemML
> Issue Type: Bug
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>
> In PyDML, the {{random.normal(...)}} built-in function is currently broken
> due to a namespace issue.
> Example:
> {code}
> N = 4
> D = 3
> X = random.normal(N, D, 1)
> print(sum(X))
> {code}
> {code}
> org.apache.sysml.parser.LanguageException: ERROR: null -- line 4, column 0 --
> function normal is undefined in namespace random
> at
> org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201)
> at
> org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:335)
> at
> org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:202)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1354)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1257)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1195)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-670) PyDML `random.normal` Function Broken Due to Namespace Issue
[
https://issues.apache.org/jira/browse/SYSTEMML-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-670:
-
Assignee: Glenn Weidner (was: Mike Dusenberry)
> PyDML `random.normal` Function Broken Due to Namespace Issue
>
>
> Key: SYSTEMML-670
> URL: https://issues.apache.org/jira/browse/SYSTEMML-670
> Project: SystemML
> Issue Type: Bug
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>
> In PyDML, the {{random.normal(...)}} built-in function is currently broken
> due to a namespace issue.
> Example:
> {code}
> N = 4
> D = 3
> X = random.normal(N, D, 1)
> print(sum(X))
> {code}
> {code}
> org.apache.sysml.parser.LanguageException: ERROR: null -- line 4, column 0 --
> function normal is undefined in namespace random
> at
> org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201)
> at
> org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:335)
> at
> org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:202)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1354)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1257)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1195)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-670) PyDML `random.normal` Function Broken Due to Namespace Issue
[
https://issues.apache.org/jira/browse/SYSTEMML-670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15285052#comment-15285052
]
Mike Dusenberry commented on SYSTEMML-670:
--
[PR 156 | https://github.com/apache/incubator-systemml/pull/156] submitted by
[~gweidner].
> PyDML `random.normal` Function Broken Due to Namespace Issue
>
>
> Key: SYSTEMML-670
> URL: https://issues.apache.org/jira/browse/SYSTEMML-670
> Project: SystemML
> Issue Type: Bug
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>
> In PyDML, the {{random.normal(...)}} built-in function is currently broken
> due to a namespace issue.
> Example:
> {code}
> N = 4
> D = 3
> X = random.normal(N, D, 1)
> print(sum(X))
> {code}
> {code}
> org.apache.sysml.parser.LanguageException: ERROR: null -- line 4, column 0 --
> function normal is undefined in namespace random
> at
> org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201)
> at
> org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:335)
> at
> org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:202)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1354)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1257)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1195)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry closed SYSTEMML-676.
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
> Fix For: SystemML 0.10
>
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e.,
> {{X\[:3,]}} should return all rows up to (but not including) 3. This means
> that the implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all
> rows starting at row 2 (0-based), which implies that the upper bound is equal
> to the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X\[:,:]}}, should also be valid, regardless if it
> is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-670) PyDML `random.normal` Function Broken Due to Namespace Issue
[
https://issues.apache.org/jira/browse/SYSTEMML-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry closed SYSTEMML-670.
> PyDML `random.normal` Function Broken Due to Namespace Issue
>
>
> Key: SYSTEMML-670
> URL: https://issues.apache.org/jira/browse/SYSTEMML-670
> Project: SystemML
> Issue Type: Bug
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
> Fix For: SystemML 0.10
>
>
> In PyDML, the {{random.normal(...)}} built-in function is currently broken
> due to a namespace issue.
> Example:
> {code}
> N = 4
> D = 3
> X = random.normal(N, D, 1)
> print(sum(X))
> {code}
> {code}
> org.apache.sysml.parser.LanguageException: ERROR: null -- line 4, column 0 --
> function normal is undefined in namespace random
> at
> org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201)
> at
> org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:335)
> at
> org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:202)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1354)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1257)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1195)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-294) Print matrix capability
[
https://issues.apache.org/jira/browse/SYSTEMML-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry closed SYSTEMML-294.
> Print matrix capability
> ---
>
> Key: SYSTEMML-294
> URL: https://issues.apache.org/jira/browse/SYSTEMML-294
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Deron Eriksson
>Assignee: Nakul Jindal
> Fix For: SystemML 0.10
>
>
> The ability to output a matrix or a part of a matrix to standard output would
> be very useful to have while developing DML and PyDML scripts. By having the
> functionality as a built-in function, capabilities such as alignment of cell
> values could be performed. Such a built-in function could also optionally
> take parameters to limit the output to a certain subset of cells in the
> matrix. Such a function would be useful for small matrices. An error could be
> thrown if an attempt is made to use the function with a matrix that is too
> large.
> Here's an example of a printMatrix function in DML:
> {code}
> printMatrix = function(matrix[double] mat) {
> for (i in 1:nrow(mat)) {
> colVals = '| '
> for (j in 1:ncol(mat)) {
> n = mat[i,j]
> colVals = colVals + as.scalar(n) + ' | '
> }
> print(colVals)
> }
> }
> m = matrix("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here's an example of a printMatrix function in PyDML:
> {code}
> def printMatrix(mat:matrix[float]):
> for (i in 1:nrow(mat)):
> colVals = '| '
> for (j in 1:ncol(mat)):
> n = mat[i,j]
> colVals = colVals + scalar(n) + ' | '
> print(colVals)
> m = full("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here is the output:
> {code}
> | 1.0 | 2.0 | 3.0 |
> | 4.0 | 5.0 | 6.0 |
> | 7.0 | 8.0 | 9.0 |
> | 10.0 | 11.0 | 12.0 |
> {code}
> Notice how alignment readily becomes an issue if an attempt is made to do
> this with DML or PyDML.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-294) Print matrix capability
[
https://issues.apache.org/jira/browse/SYSTEMML-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry resolved SYSTEMML-294.
--
Resolution: Fixed
Fix Version/s: SystemML 0.10
> Print matrix capability
> ---
>
> Key: SYSTEMML-294
> URL: https://issues.apache.org/jira/browse/SYSTEMML-294
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Deron Eriksson
>Assignee: Nakul Jindal
> Fix For: SystemML 0.10
>
>
> The ability to output a matrix or a part of a matrix to standard output would
> be very useful to have while developing DML and PyDML scripts. By having the
> functionality as a built-in function, capabilities such as alignment of cell
> values could be performed. Such a built-in function could also optionally
> take parameters to limit the output to a certain subset of cells in the
> matrix. Such a function would be useful for small matrices. An error could be
> thrown if an attempt is made to use the function with a matrix that is too
> large.
> Here's an example of a printMatrix function in DML:
> {code}
> printMatrix = function(matrix[double] mat) {
> for (i in 1:nrow(mat)) {
> colVals = '| '
> for (j in 1:ncol(mat)) {
> n = mat[i,j]
> colVals = colVals + as.scalar(n) + ' | '
> }
> print(colVals)
> }
> }
> m = matrix("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here's an example of a printMatrix function in PyDML:
> {code}
> def printMatrix(mat:matrix[float]):
> for (i in 1:nrow(mat)):
> colVals = '| '
> for (j in 1:ncol(mat)):
> n = mat[i,j]
> colVals = colVals + scalar(n) + ' | '
> print(colVals)
> m = full("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here is the output:
> {code}
> | 1.0 | 2.0 | 3.0 |
> | 4.0 | 5.0 | 6.0 |
> | 7.0 | 8.0 | 9.0 |
> | 10.0 | 11.0 | 12.0 |
> {code}
> Notice how alignment readily becomes an issue if an attempt is made to do
> this with DML or PyDML.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-294) Print matrix capability
[
https://issues.apache.org/jira/browse/SYSTEMML-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282813#comment-15282813
]
Mike Dusenberry commented on SYSTEMML-294:
--
Thanks, [~nakul02]!
> Print matrix capability
> ---
>
> Key: SYSTEMML-294
> URL: https://issues.apache.org/jira/browse/SYSTEMML-294
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Deron Eriksson
>Assignee: Nakul Jindal
> Fix For: SystemML 0.10
>
>
> The ability to output a matrix or a part of a matrix to standard output would
> be very useful to have while developing DML and PyDML scripts. By having the
> functionality as a built-in function, capabilities such as alignment of cell
> values could be performed. Such a built-in function could also optionally
> take parameters to limit the output to a certain subset of cells in the
> matrix. Such a function would be useful for small matrices. An error could be
> thrown if an attempt is made to use the function with a matrix that is too
> large.
> Here's an example of a printMatrix function in DML:
> {code}
> printMatrix = function(matrix[double] mat) {
> for (i in 1:nrow(mat)) {
> colVals = '| '
> for (j in 1:ncol(mat)) {
> n = mat[i,j]
> colVals = colVals + as.scalar(n) + ' | '
> }
> print(colVals)
> }
> }
> m = matrix("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here's an example of a printMatrix function in PyDML:
> {code}
> def printMatrix(mat:matrix[float]):
> for (i in 1:nrow(mat)):
> colVals = '| '
> for (j in 1:ncol(mat)):
> n = mat[i,j]
> colVals = colVals + scalar(n) + ' | '
> print(colVals)
> m = full("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here is the output:
> {code}
> | 1.0 | 2.0 | 3.0 |
> | 4.0 | 5.0 | 6.0 |
> | 7.0 | 8.0 | 9.0 |
> | 10.0 | 11.0 | 12.0 |
> {code}
> Notice how alignment readily becomes an issue if an attempt is made to do
> this with DML or PyDML.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-294) Print matrix capability
[
https://issues.apache.org/jira/browse/SYSTEMML-294?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282812#comment-15282812
]
Mike Dusenberry commented on SYSTEMML-294:
--
Merged in [commit ea49e62 |
https://github.com/apache/incubator-systemml/commit/ea49e620ce846c41c8b9bb9b729986a9aa7fd4d5].
> Print matrix capability
> ---
>
> Key: SYSTEMML-294
> URL: https://issues.apache.org/jira/browse/SYSTEMML-294
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Deron Eriksson
>Assignee: Nakul Jindal
>
> The ability to output a matrix or a part of a matrix to standard output would
> be very useful to have while developing DML and PyDML scripts. By having the
> functionality as a built-in function, capabilities such as alignment of cell
> values could be performed. Such a built-in function could also optionally
> take parameters to limit the output to a certain subset of cells in the
> matrix. Such a function would be useful for small matrices. An error could be
> thrown if an attempt is made to use the function with a matrix that is too
> large.
> Here's an example of a printMatrix function in DML:
> {code}
> printMatrix = function(matrix[double] mat) {
> for (i in 1:nrow(mat)) {
> colVals = '| '
> for (j in 1:ncol(mat)) {
> n = mat[i,j]
> colVals = colVals + as.scalar(n) + ' | '
> }
> print(colVals)
> }
> }
> m = matrix("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here's an example of a printMatrix function in PyDML:
> {code}
> def printMatrix(mat:matrix[float]):
> for (i in 1:nrow(mat)):
> colVals = '| '
> for (j in 1:ncol(mat)):
> n = mat[i,j]
> colVals = colVals + scalar(n) + ' | '
> print(colVals)
> m = full("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here is the output:
> {code}
> | 1.0 | 2.0 | 3.0 |
> | 4.0 | 5.0 | 6.0 |
> | 7.0 | 8.0 | 9.0 |
> | 10.0 | 11.0 | 12.0 |
> {code}
> Notice how alignment readily becomes an issue if an attempt is made to do
> this with DML or PyDML.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282961#comment-15282961
]
Mike Dusenberry commented on SYSTEMML-676:
--
[PR 148 | https://github.com/apache/incubator-systemml/pull/148] submitted.
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e.,
> {{X\[:3,]}} should return all rows up to (but not including) 3. This means
> that the implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all
> rows starting at row 2 (0-based), which implies that the upper bound is equal
> to the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X\[:,:]}}, should also be valid, regardless if it
> is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-689) Add Negative Bounds to PyDML
[ https://issues.apache.org/jira/browse/SYSTEMML-689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-689: - Component/s: PyDML Parser > Add Negative Bounds to PyDML > > > Key: SYSTEMML-689 > URL: https://issues.apache.org/jira/browse/SYSTEMML-689 > Project: SystemML > Issue Type: New Feature > Components: Parser, PyDML >Reporter: Mike Dusenberry > > Python has a slicing functionality for having indexes that count backwards > from the bounds, i.e., X[:-1,] would index all rows except the final row. We > should add this to PyDML. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-690) Add reverse slicing to PyDML
[ https://issues.apache.org/jira/browse/SYSTEMML-690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-690: - Component/s: PyDML Parser > Add reverse slicing to PyDML > > > Key: SYSTEMML-690 > URL: https://issues.apache.org/jira/browse/SYSTEMML-690 > Project: SystemML > Issue Type: New Feature > Components: Parser, PyDML >Reporter: Mike Dusenberry > > In Python, the addition of a third "step" parameter in a slice will allow for > reverse slicing, i.e., X[3:0:-1,] will return row 3 (inclusive) back to row 0 > (exclusive) correctly reversed. We should add this to PyDML. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry reassigned SYSTEMML-676:
Assignee: Mike Dusenberry
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e.,
> {{X\[:3,]}} should return all rows up to (but not including) 3. This means
> that the implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all
> rows starting at row 2 (0-based), which implies that the upper bound is equal
> to the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X\[:,:]}}, should also be valid, regardless if it
> is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-296) Add elif (else if) to PyDML
[
https://issues.apache.org/jira/browse/SYSTEMML-296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry reassigned SYSTEMML-296:
Assignee: Mike Dusenberry
> Add elif (else if) to PyDML
> ---
>
> Key: SYSTEMML-296
> URL: https://issues.apache.org/jira/browse/SYSTEMML-296
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
>Priority: Minor
>
> "Else if" (elif) statements are useful since they avoid the need for nested
> if/else statements. Currently, DML has "else if" but PyDML does not have an
> equivalent "elif".
> As an example, here is DML containing "else if" statements:
> {code}
> x = 6
> if (x == 1) {
> print('A')
> } else if (x == 2) {
> print('B')
> } else if (x == 3) {
> print('C')
> } else if (x == 4) {
> print('D')
> } else if (x == 5) {
> print('E')
> } else {
> print('F')
> }
> {code}
> Here is the logical equivalent in PyDML using nested "if else" statements:
> {code}
> x = 6
> if (x == 1):
> print('A')
> else:
> if (x == 2):
> print('B')
> else:
> if (x == 3):
> print('C')
> else:
> if (x == 4):
> print('D')
> else:
> if (x == 5):
> print('E')
> else:
> print('F')
> {code}
> The nesting becomes especially challenging in a Python-like language as the
> number of nested "if else" statements increases due to the use of whitespace
> indentations to specify blocks of code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SYSTEMML-690) Add reverse slicing to PyDML
Mike Dusenberry created SYSTEMML-690: Summary: Add reverse slicing to PyDML Key: SYSTEMML-690 URL: https://issues.apache.org/jira/browse/SYSTEMML-690 Project: SystemML Issue Type: New Feature Reporter: Mike Dusenberry In Python, the addition of a third "step" parameter in a slice will allow for reverse slicing, i.e., X[3:0:-1,] will return row 3 (inclusive) back to row 0 (exclusive) correctly reversed. We should add this to PyDML. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-296) Add elif (else if) to PyDML
[
https://issues.apache.org/jira/browse/SYSTEMML-296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-296:
-
Assignee: Tatsuya Nishiyama (was: Mike Dusenberry)
> Add elif (else if) to PyDML
> ---
>
> Key: SYSTEMML-296
> URL: https://issues.apache.org/jira/browse/SYSTEMML-296
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Deron Eriksson
>Assignee: Tatsuya Nishiyama
>Priority: Minor
>
> "Else if" (elif) statements are useful since they avoid the need for nested
> if/else statements. Currently, DML has "else if" but PyDML does not have an
> equivalent "elif".
> As an example, here is DML containing "else if" statements:
> {code}
> x = 6
> if (x == 1) {
> print('A')
> } else if (x == 2) {
> print('B')
> } else if (x == 3) {
> print('C')
> } else if (x == 4) {
> print('D')
> } else if (x == 5) {
> print('E')
> } else {
> print('F')
> }
> {code}
> Here is the logical equivalent in PyDML using nested "if else" statements:
> {code}
> x = 6
> if (x == 1):
> print('A')
> else:
> if (x == 2):
> print('B')
> else:
> if (x == 3):
> print('C')
> else:
> if (x == 4):
> print('D')
> else:
> if (x == 5):
> print('E')
> else:
> print('F')
> {code}
> The nesting becomes especially challenging in a Python-like language as the
> number of nested "if else" statements increases due to the use of whitespace
> indentations to specify blocks of code.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-691) Audit & Cleanup Parser Layer
[ https://issues.apache.org/jira/browse/SYSTEMML-691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15282969#comment-15282969 ] Mike Dusenberry commented on SYSTEMML-691: -- fyi [~mboehm7] We can add subtasks to this epic when we're ready to start. > Audit & Cleanup Parser Layer > > > Key: SYSTEMML-691 > URL: https://issues.apache.org/jira/browse/SYSTEMML-691 > Project: SystemML > Issue Type: Epic > Components: Parser >Reporter: Mike Dusenberry > > This epic tracks a parser cleanup where we systematically go over the parser > and evaluate what really needs to be at the level of the syntactic parser. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (SYSTEMML-691) Audit & Cleanup Parser Layer
Mike Dusenberry created SYSTEMML-691: Summary: Audit & Cleanup Parser Layer Key: SYSTEMML-691 URL: https://issues.apache.org/jira/browse/SYSTEMML-691 Project: SystemML Issue Type: Epic Components: Parser Reporter: Mike Dusenberry This epic tracks a parser cleanup where we systematically go over the parser and evaluate what really needs to be at the level of the syntactic parser. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15280584#comment-15280584
]
Mike Dusenberry edited comment on SYSTEMML-633 at 5/11/16 6:30 PM:
---
Attached the files for the TensorFlow vs SystemML tests:
* {{log.txt}} // look at date for correct file
* {{perf-dml.dml}}
* {{perf-tf.py}}
* {{perf.sh}}
* {{run.sh}}
* {{time.txt}}
was (Author: mwdus...@us.ibm.com):
Attached the files for the TensorFlow vs SystemML tests.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
>Priority: Critical
> Attachments: Im2colWrapper.java, log.txt, log.txt, perf-dml.dml,
> perf-tf.py, perf.sh, run.sh, systemml-nn.zip, time.txt
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/0.
> HOP DAGs recompile time: 0.268 sec.
> Functions recompiled: 129.
> Functions recompile time: 0.841 sec.
> ParFor loops optimized: 1.
> ParFor optimize time: 0.032 sec.
> ParFor initialize time: 0.015 sec.
> ParFor result merge time: 0.028 sec.
>
[jira] [Commented] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264508#comment-15264508
]
Mike Dusenberry commented on SYSTEMML-654:
--
cc [~gweidner]
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-654:
-
Affects Version/s: SystemML 0.10
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SYSTEMML-654) DML Functions Should Override Builtin Functions
Mike Dusenberry created SYSTEMML-654:
Summary: DML Functions Should Override Builtin Functions
Key: SYSTEMML-654
URL: https://issues.apache.org/jira/browse/SYSTEMML-654
Project: SystemML
Issue Type: Sub-task
Reporter: Mike Dusenberry
Currently, if a user defines a DML-bodied function that has the same name as a
builtin function, an error will be returned. This occurs both if the function
is defined in the same file as it is being called (which could look like a
builtin function call, although the user does not wish it to be), or if the
function is defined in a separate file and called with a namespace notation.
As we grow the language with an increasing number of builtin functions, this is
not the desired behavior. Instead, any DML functions should override any
builtin functions.
Example 1:
{code}
min = function(int i) {
print("hi" + i)
}
tmp = min(1) # fail!
{code}
Example 2:
{code}
# util.dml
min = function(int i) {
print("hi" + i)
}
{code}
{code}
source("util.dml") as util
tmp = util::min(1) # fail!
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-654:
-
Assignee: Glenn Weidner
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> {code}
> : org.apache.sysml.parser.LanguageException: Unsupported Parameters : ERROR:
> null -- line 6, column 0 -- Expecting matrix parameter for function MIN
> at
> org.apache.sysml.parser.Expression.raiseValidateError(Expression.java:421)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.checkMatrixParam(BuiltinFunctionExpression.java:1221)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.validateExpression(BuiltinFunctionExpression.java:314)
> at
> org.apache.sysml.parser.StatementBlock.validate(StatementBlock.java:598)
> at
> org.apache.sysml.parser.DMLTranslator.validateParseTree(DMLTranslator.java:136)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1325)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1227)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15264843#comment-15264843
]
Mike Dusenberry commented on SYSTEMML-654:
--
Awesome, thanks!
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> {code}
> : org.apache.sysml.parser.LanguageException: Unsupported Parameters : ERROR:
> null -- line 6, column 0 -- Expecting matrix parameter for function MIN
> at
> org.apache.sysml.parser.Expression.raiseValidateError(Expression.java:421)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.checkMatrixParam(BuiltinFunctionExpression.java:1221)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.validateExpression(BuiltinFunctionExpression.java:314)
> at
> org.apache.sysml.parser.StatementBlock.validate(StatementBlock.java:598)
> at
> org.apache.sysml.parser.DMLTranslator.validateParseTree(DMLTranslator.java:136)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1325)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1227)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-638) Random Forest Predict Execution Fails
[
https://issues.apache.org/jira/browse/SYSTEMML-638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272746#comment-15272746
]
Mike Dusenberry commented on SYSTEMML-638:
--
cc [~farazmm], [~fmak...@us.ibm.com], [~prithvi_r_s], [~prithvianight] Have
you been able to look into this yet?
> Random Forest Predict Execution Fails
> -
>
> Key: SYSTEMML-638
> URL: https://issues.apache.org/jira/browse/SYSTEMML-638
> Project: SystemML
> Issue Type: Bug
>Affects Versions: SystemML 0.10
>Reporter: Stacey Ronaghan
>
> Issue executing the prediction for random forest algorithm on SystemML 0.10
> (incubating) via MLContext with Scala Spark on a cluster.
> Related to [SYSTEMML-597|https://issues.apache.org/jira/browse/SYSTEMML-597].
> X is the same input passed into execute for random-forest.dml (mentioned in
> [SYSTEMML-597|https://issues.apache.org/jira/browse/SYSTEMML-597]) and M is
> its output model.
> Code:
> {code}
> // Register inputs & outputs for prediction
> ml.reset()
> ml.registerInput("X", X)
> //ml.registerInput("Y", Y)
> ml.registerInput("M", M)
> ml.registerOutput("P")
> //ml.registerOutput("A")
> // Run the script
> //val nargs = Map("X" -> "", "Y" -> "", "M" -> "", "P" -> "", "A" -> "")
> val nargs = Map("X" -> "", "M" -> "", "P" -> "")
> val outputs =
> ml.execute("/home/biadmin/spark-enablement/installs/SystemML/algorithms/random-forest-predict.dml",
> nargs)
> val P = outputs.getDF(sqlContext, "P")
> //val A = outputs.getDF(sqlContext, "A")
> {code}
> Output:
> {code}
> import org.apache.sysml.api.MLContext ml: org.apache.sysml.api.MLContext =
> org.apache.sysml.api.MLContext@5649f7b4 nargs:
> scala.collection.immutable.Map[String,String] = Map(X -> "", M -> "", P ->
> "") org.apache.sysml.runtime.DMLRuntimeException:
> org.apache.sysml.runtime.DMLRuntimeException: ERROR: Runtime error in program
> block generated from statement block between lines 68 and 89 -- Error
> evaluating instruction:
> CP°groupedagg°target=_mVar60580°groups=_mVar60580°fn=count°k=40°_mVar60581·MATRIX·DOUBLE
> at org.apache.sysml.runtime.controlprogram.Program.execute(Program.java:152)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1365)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1225)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1173)
> at org.apache.sysml.api.MLContext.execute(MLContext.java:640) at
> org.apache.sysml.api.MLContext.execute(MLContext.java:675) at
> org.apache.sysml.api.MLContext.execute(MLContext.java:688) at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:41)
> at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:46)
> at $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:48) at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:50) at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:52) at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:54) at
> $iwC$$iwC$$iwC$$iwC$$iwC$$iwC.(:56) at
> $iwC$$iwC$$iwC$$iwC$$iwC.(:58) at
> $iwC$$iwC$$iwC$$iwC.(:60) at
> $iwC$$iwC$$iwC.(:62) at $iwC$$iwC.(:64) at
> $iwC.(:66) at (:68) at .(:72) at
> .() at .(:7) at .() at
> $print() at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
> Method) at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497) at
> org.apache.spark.repl.SparkIMain$ReadEvalPrint.call(SparkIMain.scala:1065) at
> org.apache.spark.repl.SparkIMain$Request.loadAndRun(SparkIMain.scala:1338) at
> org.apache.spark.repl.SparkIMain.loadAndRunReq$1(SparkIMain.scala:840) at
> org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:871) at
> org.apache.spark.repl.SparkIMain.interpret(SparkIMain.scala:819) at
> org.apache.zeppelin.spark.SparkInterpreter.interpretInput(SparkInterpreter.java:646)
> at
> org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:611)
> at
> org.apache.zeppelin.spark.SparkInterpreter.interpret(SparkInterpreter.java:604)
> at
> org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
> at
> org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
> at
> org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:292)
> at org.apache.zeppelin.scheduler.Job.run(Job.java:170) at
> org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:118) at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at
> java.util.concurrent.FutureTask.run(FutureTask.java:266) at
>
[jira] [Closed] (SYSTEMML-265) Update All Performance Tests to Use More Iterations
[ https://issues.apache.org/jira/browse/SYSTEMML-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry closed SYSTEMML-265. > Update All Performance Tests to Use More Iterations > --- > > Key: SYSTEMML-265 > URL: https://issues.apache.org/jira/browse/SYSTEMML-265 > Project: SystemML > Issue Type: Sub-task >Reporter: Mike Dusenberry >Assignee: Tatsuya Nishiyama > Labels: Starter > Fix For: SystemML 0.10 > > > Currently, several of the performance tests are hard-coded to use only 3 > iterations of an algorithm. This should be increased to at least 20, or > certain tests (k-means) do not run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-631) Improve handling of namespace conflicts
[ https://issues.apache.org/jira/browse/SYSTEMML-631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-631: - Priority: Major (was: Minor) > Improve handling of namespace conflicts > --- > > Key: SYSTEMML-631 > URL: https://issues.apache.org/jira/browse/SYSTEMML-631 > Project: SystemML > Issue Type: Sub-task > Components: Parser >Reporter: Glenn Weidner >Assignee: Glenn Weidner > Fix For: SystemML 0.10 > > > If one script sources two different scripts as same namespace, then qualified > function calls to first imported script fail with error indicating the > function is undefined in namespace. > An improvement would be to detect, report and/or resolve the namespace > conflict. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-617) Override default namespace for imported custom functions
[ https://issues.apache.org/jira/browse/SYSTEMML-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-617: - Priority: Major (was: Minor) > Override default namespace for imported custom functions > > > Key: SYSTEMML-617 > URL: https://issues.apache.org/jira/browse/SYSTEMML-617 > Project: SystemML > Issue Type: Sub-task > Components: Parser >Reporter: Glenn Weidner >Assignee: Glenn Weidner > Fix For: SystemML 0.10 > > > This sub-task targets a specific scenario described in SYSTEMML-590. > Example of error: > org.apache.sysml.api.DMLException: org.apache.sysml.parser.LanguageException: > ERROR: null -- line 0, column 0 -- function g is undefined in namespace > .defaultNS > at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:350) > at org.apache.sysml.api.DMLScript.main(DMLScript.java:197) > ... > Caused by: org.apache.sysml.parser.LanguageException: ERROR: null -- line 0, > column 0 -- function g is undefined in namespace .defaultNS > at > org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201) > at > org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:328) > at > org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:165) > at org.apache.sysml.api.DMLScript.execute(DMLScript.java:592) > at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:338) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-632) Improve handling of source statements in scripts that are also imported
[ https://issues.apache.org/jira/browse/SYSTEMML-632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-632: - Priority: Major (was: Minor) > Improve handling of source statements in scripts that are also imported > --- > > Key: SYSTEMML-632 > URL: https://issues.apache.org/jira/browse/SYSTEMML-632 > Project: SystemML > Issue Type: Sub-task > Components: Parser >Reporter: Glenn Weidner >Assignee: Glenn Weidner > Fix For: SystemML 0.10 > > > If script1 sources another script2 which also sources another script3, then > qualified functions calls in script2 to script3 fail with error indicating > the function is undefined in namespace. > An improvement would be to add processing such that dependent function calls > are successful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-265) Update All Performance Tests to Use More Iterations
[ https://issues.apache.org/jira/browse/SYSTEMML-265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry resolved SYSTEMML-265. -- Resolution: Fixed Fix Version/s: SystemML 0.10 > Update All Performance Tests to Use More Iterations > --- > > Key: SYSTEMML-265 > URL: https://issues.apache.org/jira/browse/SYSTEMML-265 > Project: SystemML > Issue Type: Sub-task >Reporter: Mike Dusenberry >Assignee: Tatsuya Nishiyama > Labels: Starter > Fix For: SystemML 0.10 > > > Currently, several of the performance tests are hard-coded to use only 3 > iterations of an algorithm. This should be increased to at least 20, or > certain tests (k-means) do not run. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-508) Extend "executeScript" In MLContext To Accept PyDML
[
https://issues.apache.org/jira/browse/SYSTEMML-508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15274497#comment-15274497
]
Mike Dusenberry commented on SYSTEMML-508:
--
[PR 139 | https://github.com/apache/incubator-systemml/pull/139] submitted.
Note, this is just a quick addition, as {{MLContext}} is receiving a major
overhaul soon.
> Extend "executeScript" In MLContext To Accept PyDML
> ---
>
> Key: SYSTEMML-508
> URL: https://issues.apache.org/jira/browse/SYSTEMML-508
> Project: SystemML
> Issue Type: Sub-task
>Reporter: Mike Dusenberry
>Priority: Minor
>
> When executing a script stored in a string with {{MLContext::executeScript}},
> PyDML is currently not supported. This task is to extend the
> {{executeScript}} API to accept PyDML, much like {{execute}} currently does.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-670) PyDML `random.normal` Function Broken Due to Namespace Issue
[
https://issues.apache.org/jira/browse/SYSTEMML-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-670:
-
Component/s: Parser
> PyDML `random.normal` Function Broken Due to Namespace Issue
>
>
> Key: SYSTEMML-670
> URL: https://issues.apache.org/jira/browse/SYSTEMML-670
> Project: SystemML
> Issue Type: Bug
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>
> In PyDML, the {{random.normal(...)}} built-in function is currently broken
> due to a namespace issue.
> Example:
> {code}
> N = 4
> D = 3
> X = random.normal(N, D, 1)
> print(sum(X))
> {code}
> {code}
> org.apache.sysml.parser.LanguageException: ERROR: null -- line 4, column 0 --
> function normal is undefined in namespace random
> at
> org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201)
> at
> org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:335)
> at
> org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:202)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1354)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1257)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1195)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-670) PyDML `random.normal` Function Broken Due to Namespace Issue
[
https://issues.apache.org/jira/browse/SYSTEMML-670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-670:
-
Affects Version/s: SystemML 0.10
> PyDML `random.normal` Function Broken Due to Namespace Issue
>
>
> Key: SYSTEMML-670
> URL: https://issues.apache.org/jira/browse/SYSTEMML-670
> Project: SystemML
> Issue Type: Bug
> Components: Parser
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>
> In PyDML, the {{random.normal(...)}} built-in function is currently broken
> due to a namespace issue.
> Example:
> {code}
> N = 4
> D = 3
> X = random.normal(N, D, 1)
> print(sum(X))
> {code}
> {code}
> org.apache.sysml.parser.LanguageException: ERROR: null -- line 4, column 0 --
> function normal is undefined in namespace random
> at
> org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201)
> at
> org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:335)
> at
> org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:202)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1354)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1257)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1195)
> at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-671) Improve PyDML `range(...)` Function
[
https://issues.apache.org/jira/browse/SYSTEMML-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-671:
-
Description:
Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}}, and
{{step}} arguments. In Python [1], {{range(...)}} comes in two forms: one form
only has a {{stop}} parameter, as in {{range(stop)}}, and the other form has
both the {{start}} and {{stop}} parameters, with an optional {{step}}
parameter. It is quite common to simply use an upper {{stop}} argument, i.e.
{{range(10)}}, to create a list from 0 up to, but not including, {{stop}}.
We should improve the PyDML {{range(...)}} function to be more inline with that
of Python, as described above.
**Note**: The upper bound {{stop}} should also be exclusive, as noted above,
i.e., {{range(4)}} should return a vector {{[0,1,2,3]}}, rather than
{{[0,1,2,3,4]}}.
**Note #2**: This should also be applicable when used with {{for}} expressions,
i.e.:
{code}
for i in range(12):
print(i)
{code}
was:
Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}}, and
{{step}} arguments. In Python [1], {{range(...)}} comes in two forms: one form
only has a {{stop}} parameter, as in {{range(stop)}}, and the other form has
both the {{start}} and {{stop}} parameters, with an optional {{step}}
parameter. It is quite common to simply use an upper {{stop}} argument, i.e.
{{range(10)}}, to create a list from 0 up to, but not including, {{stop}}.
We should improve the PyDML {{range(...)}} function to be more inline with that
of Python, as described above.
**Note**: This should also be applicable when used with {{for}} expressions,
i.e.:
{code}
for i in range(12):
print(i)
{code}
> Improve PyDML `range(...)` Function
> ---
>
> Key: SYSTEMML-671
> URL: https://issues.apache.org/jira/browse/SYSTEMML-671
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>
> Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}},
> and {{step}} arguments. In Python [1], {{range(...)}} comes in two forms:
> one form only has a {{stop}} parameter, as in {{range(stop)}}, and the other
> form has both the {{start}} and {{stop}} parameters, with an optional
> {{step}} parameter. It is quite common to simply use an upper {{stop}}
> argument, i.e. {{range(10)}}, to create a list from 0 up to, but not
> including, {{stop}}.
> We should improve the PyDML {{range(...)}} function to be more inline with
> that of Python, as described above.
> **Note**: The upper bound {{stop}} should also be exclusive, as noted above,
> i.e., {{range(4)}} should return a vector {{[0,1,2,3]}}, rather than
> {{[0,1,2,3,4]}}.
> **Note #2**: This should also be applicable when used with {{for}}
> expressions, i.e.:
> {code}
> for i in range(12):
> print(i)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-671) Improve PyDML `range(...)` Function
[
https://issues.apache.org/jira/browse/SYSTEMML-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-671:
-
Description:
Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}}, and
{{step}} arguments. In Python [1], {{range(...)}} comes in two forms: one form
only has a {{stop}} parameter, as in {{range(stop)}}, and the other form has
both the {{start}} and {{stop}} parameters, with an optional {{step}}
parameter. It is quite common to simply use an upper {{stop}} argument, i.e.
{{range(10)}}, to create a list from 0 up to, but not including, {{stop}}.
**Note**: The upper bound {{stop}} should also be exclusive, as noted above,
i.e., {{range(4)}} should return a vector {{[0,1,2,3]}}, rather than
{{[0,1,2,3,4]}}.
**Note #2**: This should also be applicable when used with {{for}} expressions,
i.e.:
{code}
for i in range(12):
print(i)
{code}
We should improve the PyDML {{range(...)}} function to be more inline with that
of Python, as described above.
was:
Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}}, and
{{step}} arguments. In Python [1], {{range(...)}} comes in two forms: one form
only has a {{stop}} parameter, as in {{range(stop)}}, and the other form has
both the {{start}} and {{stop}} parameters, with an optional {{step}}
parameter. It is quite common to simply use an upper {{stop}} argument, i.e.
{{range(10)}}, to create a list from 0 up to, but not including, {{stop}}.
We should improve the PyDML {{range(...)}} function to be more inline with that
of Python, as described above.
**Note**: The upper bound {{stop}} should also be exclusive, as noted above,
i.e., {{range(4)}} should return a vector {{[0,1,2,3]}}, rather than
{{[0,1,2,3,4]}}.
**Note #2**: This should also be applicable when used with {{for}} expressions,
i.e.:
{code}
for i in range(12):
print(i)
{code}
> Improve PyDML `range(...)` Function
> ---
>
> Key: SYSTEMML-671
> URL: https://issues.apache.org/jira/browse/SYSTEMML-671
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>
> Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}},
> and {{step}} arguments. In Python [1], {{range(...)}} comes in two forms:
> one form only has a {{stop}} parameter, as in {{range(stop)}}, and the other
> form has both the {{start}} and {{stop}} parameters, with an optional
> {{step}} parameter. It is quite common to simply use an upper {{stop}}
> argument, i.e. {{range(10)}}, to create a list from 0 up to, but not
> including, {{stop}}.
> **Note**: The upper bound {{stop}} should also be exclusive, as noted above,
> i.e., {{range(4)}} should return a vector {{[0,1,2,3]}}, rather than
> {{[0,1,2,3,4]}}.
> **Note #2**: This should also be applicable when used with {{for}}
> expressions, i.e.:
> {code}
> for i in range(12):
> print(i)
> {code}
> We should improve the PyDML {{range(...)}} function to be more inline with
> that of Python, as described above.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-672) Fix PyDML 1-Based Indexing Bug
[ https://issues.apache.org/jira/browse/SYSTEMML-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-672: - Summary: Fix PyDML 1-Based Indexing Bug (was: Fix PyDML 0-Based Indexing Bug) > Fix PyDML 1-Based Indexing Bug > -- > > Key: SYSTEMML-672 > URL: https://issues.apache.org/jira/browse/SYSTEMML-672 > Project: SystemML > Issue Type: Bug > Components: Parser, PyDML >Affects Versions: SystemML 0.9, SystemML 0.10 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry >Priority: Critical > > Currently, PyDML has a *bug* in which it uses 1-based indexing, as in R & > DML, rather than the correct 0-based indexing that Python uses. This simply > aims to fix this issue in the Parser, and then update the few PyDML test > scripts to use the correct 0-based indexing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-235) PyDML grammar hardening/consistency
[ https://issues.apache.org/jira/browse/SYSTEMML-235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-235: - Assignee: (was: Frederick Reiss) > PyDML grammar hardening/consistency > --- > > Key: SYSTEMML-235 > URL: https://issues.apache.org/jira/browse/SYSTEMML-235 > Project: SystemML > Issue Type: Task >Reporter: Matthias Boehm > Original Estimate: 40h > Remaining Estimate: 40h > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-294) Print matrix capability
[
https://issues.apache.org/jira/browse/SYSTEMML-294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry reassigned SYSTEMML-294:
Assignee: Mike Dusenberry
> Print matrix capability
> ---
>
> Key: SYSTEMML-294
> URL: https://issues.apache.org/jira/browse/SYSTEMML-294
> Project: SystemML
> Issue Type: Improvement
> Components: APIs
>Reporter: Deron Eriksson
>Assignee: Mike Dusenberry
>
> The ability to output a matrix or a part of a matrix to standard output would
> be very useful to have while developing DML and PyDML scripts. By having the
> functionality as a built-in function, capabilities such as alignment of cell
> values could be performed. Such a built-in function could also optionally
> take parameters to limit the output to a certain subset of cells in the
> matrix. Such a function would be useful for small matrices. An error could be
> thrown if an attempt is made to use the function with a matrix that is too
> large.
> Here's an example of a printMatrix function in DML:
> {code}
> printMatrix = function(matrix[double] mat) {
> for (i in 1:nrow(mat)) {
> colVals = '| '
> for (j in 1:ncol(mat)) {
> n = mat[i,j]
> colVals = colVals + as.scalar(n) + ' | '
> }
> print(colVals)
> }
> }
> m = matrix("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here's an example of a printMatrix function in PyDML:
> {code}
> def printMatrix(mat:matrix[float]):
> for (i in 1:nrow(mat)):
> colVals = '| '
> for (j in 1:ncol(mat)):
> n = mat[i,j]
> colVals = colVals + scalar(n) + ' | '
> print(colVals)
> m = full("1 2 3 4 5 6 7 8 9 10 11 12", rows=4, cols=3)
> z = printMatrix(m)
> {code}
> Here is the output:
> {code}
> | 1.0 | 2.0 | 3.0 |
> | 4.0 | 5.0 | 6.0 |
> | 7.0 | 8.0 | 9.0 |
> | 10.0 | 11.0 | 12.0 |
> {code}
> Notice how alignment readily becomes an issue if an attempt is made to do
> this with DML or PyDML.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-671) Improve PyDML `range(...)` Function
[
https://issues.apache.org/jira/browse/SYSTEMML-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-671:
-
Description:
Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}}, and
{{step}} arguments. In Python [1], {{range(...)}} comes in two forms: one form
only has a {{stop}} parameter, as in {{range(stop)}}, and the other form has
both the {{start}} and {{stop}} parameters, with an optional {{step}}
parameter. It is quite common to simply use an upper {{stop}} argument, i.e.
{{range(10)}}, to create a list from 0 up to, but not including, {{stop}}.
We should improve the PyDML {{range(...)}} function to be more inline with that
of Python, as described above.
**Note**: This should also be applicable when used with {{for}} expressions,
i.e.:
{code}
for i in range(12):
print(i)
{code}
was:
Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}}, and
{{step}} arguments. In Python [1], {{range(...)}} comes in two forms: one form
only has a {{stop}} parameter, as in {{range(stop)}}, and the other form has
both the {{start}} and {{stop}} parameters, with an optional {{step}}
parameter. It is quite common to simply use an upper {{stop}} argument, i.e.
{{range(10)}}, to create a list from 0 up to, but not including, {{stop}}.
We should improve the PyDML {{range(...)}} function to be more inline with that
of Python, as described above.
> Improve PyDML `range(...)` Function
> ---
>
> Key: SYSTEMML-671
> URL: https://issues.apache.org/jira/browse/SYSTEMML-671
> Project: SystemML
> Issue Type: Improvement
> Components: Parser, PyDML
>Reporter: Mike Dusenberry
>
> Currently, the PyDML {{range(...)}} function requires {{start}}, {{stop}},
> and {{step}} arguments. In Python [1], {{range(...)}} comes in two forms:
> one form only has a {{stop}} parameter, as in {{range(stop)}}, and the other
> form has both the {{start}} and {{stop}} parameters, with an optional
> {{step}} parameter. It is quite common to simply use an upper {{stop}}
> argument, i.e. {{range(10)}}, to create a list from 0 up to, but not
> including, {{stop}}.
> We should improve the PyDML {{range(...)}} function to be more inline with
> that of Python, as described above.
> **Note**: This should also be applicable when used with {{for}} expressions,
> i.e.:
> {code}
> for i in range(12):
> print(i)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (SYSTEMML-672) Fix PyDML 0-Based Indexing Bug
Mike Dusenberry created SYSTEMML-672: Summary: Fix PyDML 0-Based Indexing Bug Key: SYSTEMML-672 URL: https://issues.apache.org/jira/browse/SYSTEMML-672 Project: SystemML Issue Type: Bug Components: Parser, PyDML Affects Versions: SystemML 0.9, SystemML 0.10 Reporter: Mike Dusenberry Assignee: Mike Dusenberry Priority: Critical Currently, PyDML has a *bug* in which it uses 1-based indexing, as in R & DML, rather than the correct 0-based indexing that Python uses. This simply aims to fix this issue in the Parser, and then update the few PyDML test scripts to use the correct 0-based indexing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-669) Improve PyDML Language
[ https://issues.apache.org/jira/browse/SYSTEMML-669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-669: - Description: This epic aims to improve the PyDML language to be semantically correct with Python by improving current functions, adding new essential functions, fixing bugs, hardening the grammar, and adding documentation. (was: This epic aims to improve the PyDML language to be semantically correct with Python by improving current functions, adding new essential functions, fixing bugs, and hardening the grammar.) > Improve PyDML Language > -- > > Key: SYSTEMML-669 > URL: https://issues.apache.org/jira/browse/SYSTEMML-669 > Project: SystemML > Issue Type: Epic > Components: Parser, PyDML >Reporter: Mike Dusenberry > > This epic aims to improve the PyDML language to be semantically correct with > Python by improving current functions, adding new essential functions, fixing > bugs, hardening the grammar, and adding documentation. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-654:
-
Assignee: Glenn Weidner (was: Mike Dusenberry)
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Glenn Weidner
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> {code}
> : org.apache.sysml.parser.LanguageException: Unsupported Parameters : ERROR:
> null -- line 6, column 0 -- Expecting matrix parameter for function MIN
> at
> org.apache.sysml.parser.Expression.raiseValidateError(Expression.java:421)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.checkMatrixParam(BuiltinFunctionExpression.java:1221)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.validateExpression(BuiltinFunctionExpression.java:314)
> at
> org.apache.sysml.parser.StatementBlock.validate(StatementBlock.java:598)
> at
> org.apache.sysml.parser.DMLTranslator.validateParseTree(DMLTranslator.java:136)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1325)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1227)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Assigned] (SYSTEMML-654) DML Functions Should Override Builtin Functions
[
https://issues.apache.org/jira/browse/SYSTEMML-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry reassigned SYSTEMML-654:
Assignee: Mike Dusenberry (was: Glenn Weidner)
> DML Functions Should Override Builtin Functions
> ---
>
> Key: SYSTEMML-654
> URL: https://issues.apache.org/jira/browse/SYSTEMML-654
> Project: SystemML
> Issue Type: Sub-task
>Affects Versions: SystemML 0.10
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>
> Currently, if a user defines a DML-bodied function that has the same name as
> a builtin function, an error will be returned. This occurs both if the
> function is defined in the same file as it is being called (which could look
> like a builtin function call, although the user does not wish it to be), or
> if the function is defined in a separate file and called with a namespace
> notation. As we grow the language with an increasing number of builtin
> functions, this is not the desired behavior. Instead, any DML functions
> should override any builtin functions.
> Example 1:
> {code}
> min = function(int i) {
> print("hi" + i)
> }
> tmp = min(1) # fail!
> {code}
> {code}
> : org.apache.sysml.parser.LanguageException: Unsupported Parameters : ERROR:
> null -- line 6, column 0 -- Expecting matrix parameter for function MIN
> at
> org.apache.sysml.parser.Expression.raiseValidateError(Expression.java:421)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.checkMatrixParam(BuiltinFunctionExpression.java:1221)
> at
> org.apache.sysml.parser.BuiltinFunctionExpression.validateExpression(BuiltinFunctionExpression.java:314)
> at
> org.apache.sysml.parser.StatementBlock.validate(StatementBlock.java:598)
> at
> org.apache.sysml.parser.DMLTranslator.validateParseTree(DMLTranslator.java:136)
> at
> org.apache.sysml.api.MLContext.executeUsingSimplifiedCompilationChain(MLContext.java:1325)
> at
> org.apache.sysml.api.MLContext.compileAndExecuteScript(MLContext.java:1227)
> at org.apache.sysml.api.MLContext.executeScript(MLContext.java:1165)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:497)
> at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:231)
> at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:381)
> at py4j.Gateway.invoke(Gateway.java:259)
> at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:133)
> at py4j.commands.CallCommand.execute(CallCommand.java:79)
> at py4j.GatewayConnection.run(GatewayConnection.java:209)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> Example 2:
> {code}
> # util.dml
> min = function(int i) {
> print("hi" + i)
> }
> {code}
> {code}
> source("util.dml") as util
> tmp = util::min(1) # fail!
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Closed] (SYSTEMML-631) Improve handling of namespace conflicts
[ https://issues.apache.org/jira/browse/SYSTEMML-631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry closed SYSTEMML-631. > Improve handling of namespace conflicts > --- > > Key: SYSTEMML-631 > URL: https://issues.apache.org/jira/browse/SYSTEMML-631 > Project: SystemML > Issue Type: Sub-task > Components: Parser >Reporter: Glenn Weidner >Assignee: Glenn Weidner >Priority: Minor > Fix For: SystemML 0.10 > > > If one script sources two different scripts as same namespace, then qualified > function calls to first imported script fail with error indicating the > function is undefined in namespace. > An improvement would be to detect, report and/or resolve the namespace > conflict. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-672) Fix PyDML 1-Based Indexing Bug
[ https://issues.apache.org/jira/browse/SYSTEMML-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278528#comment-15278528 ] Mike Dusenberry commented on SYSTEMML-672: -- [PR 145 | https://github.com/apache/incubator-systemml/pull/145] submitted. > Fix PyDML 1-Based Indexing Bug > -- > > Key: SYSTEMML-672 > URL: https://issues.apache.org/jira/browse/SYSTEMML-672 > Project: SystemML > Issue Type: Bug > Components: Parser, PyDML >Affects Versions: SystemML 0.9, SystemML 0.10 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry >Priority: Critical > > Currently, PyDML has a *bug* in which it uses 1-based indexing, as in R & > DML, rather than the correct 0-based indexing that Python uses. This simply > aims to fix this issue in the Parser, and then update the few PyDML test > scripts to use the correct 0-based indexing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-676:
-
Description:
Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
However, a really useful piece of slicing syntax we are currently missing from
PyDML is the ability to have implicit lower or upper bounds, i.e., {{X\[:3,]}}
should return all rows up to (but not including) 3. This means that the
implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all rows
starting at row 2 (0-based), which implies that the upper bound is equal to the
number of rows of {{X}}.
We should update the grammar to allow these cases, row both row and column
slicing. More generally, {{X1[:,:]}}, should also be valid, regardless if it
is useful.
was:
Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
However, a really useful piece of slicing syntax we are currently missing from
PyDML is the ability to have implicit lower or upper bounds, i.e., {{X[ :3, ]}}
should return all rows up to (but not including) 3. This means that the
implicit lower bound is 0. Similarly, {{X[ 2:, ]}} should return all rows
starting at row 2 (0-based), which implies that the upper bound is equal to the
number of rows of {{X}}.
We should update the grammar to allow these cases, row both row and column
slicing. More generally, {{X[ :,: ]}}, should also be valid, regardless if it
is useful.
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e.,
> {{X\[:3,]}} should return all rows up to (but not including) 3. This means
> that the implicit lower bound is 0. Similarly, {{X\[2:,]}} should return all
> rows starting at row 2 (0-based), which implies that the upper bound is equal
> to the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X1[:,:]}}, should also be valid, regardless if it
> is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-676) Improve PyDML Slicing
[
https://issues.apache.org/jira/browse/SYSTEMML-676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278515#comment-15278515
]
Mike Dusenberry commented on SYSTEMML-676:
--
[~mboehm7]:
{quote}
just to provide some history here: at one point we supported implicit lower and
upper bounds, but removed it because (1) it is invalid R (1:3 is a shorthand
for sequence, which is undefined with missing lower/upper bounds), and (2)
because we cannot compile valid execution plans in case of unknown input size.
However, meanwhile (with dynamic recompilation), the latter problem can be
mitigated with additional runtime changes (e.g., for forced CP execution w/o
recompile).
{quote}
Great to see some of the history. W.r.t. (1), definitely understandable for
DML, but luckily we don't have to worry about it for PyDML, as {{a:b}} is only
used in the context of slicing. As for (2), is it possible to just inject,
say, {{nrow(X)}} at the parser level for the implicit slicing {{X\[2:,]}},
which would be equivalent to {{X\[2:nrow(X),]}}?
> Improve PyDML Slicing
> -
>
> Key: SYSTEMML-676
> URL: https://issues.apache.org/jira/browse/SYSTEMML-676
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>
> Currently, slicing correctly uses inclusive lower and exclusive upper bounds.
> However, a really useful piece of slicing syntax we are currently missing
> from PyDML is the ability to have implicit lower or upper bounds, i.e., {{X[
> :3, ]}} should return all rows up to (but not including) 3. This means that
> the implicit lower bound is 0. Similarly, {{X[ 2:, ]}} should return all rows
> starting at row 2 (0-based), which implies that the upper bound is equal to
> the number of rows of {{X}}.
> We should update the grammar to allow these cases, row both row and column
> slicing. More generally, {{X[ :,: ]}}, should also be valid, regardless if
> it is useful.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-672) Fix PyDML 1-Based Indexing Bug
[ https://issues.apache.org/jira/browse/SYSTEMML-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry resolved SYSTEMML-672. -- Resolution: Fixed Fix Version/s: SystemML 0.10 > Fix PyDML 1-Based Indexing Bug > -- > > Key: SYSTEMML-672 > URL: https://issues.apache.org/jira/browse/SYSTEMML-672 > Project: SystemML > Issue Type: Bug > Components: Parser, PyDML >Affects Versions: SystemML 0.9, SystemML 0.10 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry >Priority: Critical > Fix For: SystemML 0.10 > > > Currently, PyDML has a *bug* in which it uses 1-based indexing, as in R & > DML, rather than the correct 0-based indexing that Python uses. This simply > aims to fix this issue in the Parser, and then update the few PyDML test > scripts to use the correct 0-based indexing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-672) Fix PyDML 1-Based Indexing Bug
[ https://issues.apache.org/jira/browse/SYSTEMML-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278530#comment-15278530 ] Mike Dusenberry commented on SYSTEMML-672: -- Merged in [commit 59c1e55 | https://github.com/apache/incubator-systemml/commit/59c1e55408188a21cc83aadad1403b31572cb8f7]. > Fix PyDML 1-Based Indexing Bug > -- > > Key: SYSTEMML-672 > URL: https://issues.apache.org/jira/browse/SYSTEMML-672 > Project: SystemML > Issue Type: Bug > Components: Parser, PyDML >Affects Versions: SystemML 0.9, SystemML 0.10 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry >Priority: Critical > Fix For: SystemML 0.10 > > > Currently, PyDML has a *bug* in which it uses 1-based indexing, as in R & > DML, rather than the correct 0-based indexing that Python uses. This simply > aims to fix this issue in the Parser, and then update the few PyDML test > scripts to use the correct 0-based indexing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Closed] (SYSTEMML-672) Fix PyDML 1-Based Indexing Bug
[ https://issues.apache.org/jira/browse/SYSTEMML-672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry closed SYSTEMML-672. > Fix PyDML 1-Based Indexing Bug > -- > > Key: SYSTEMML-672 > URL: https://issues.apache.org/jira/browse/SYSTEMML-672 > Project: SystemML > Issue Type: Bug > Components: Parser, PyDML >Affects Versions: SystemML 0.9, SystemML 0.10 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry >Priority: Critical > Fix For: SystemML 0.10 > > > Currently, PyDML has a *bug* in which it uses 1-based indexing, as in R & > DML, rather than the correct 0-based indexing that Python uses. This simply > aims to fix this issue in the Parser, and then update the few PyDML test > scripts to use the correct 0-based indexing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-672) Fix PyDML 1-Based Indexing Bug
[ https://issues.apache.org/jira/browse/SYSTEMML-672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15278531#comment-15278531 ] Mike Dusenberry commented on SYSTEMML-672: -- [~luciano resende] Let's make sure this makes it into the 0.10 release as it was a major bug. > Fix PyDML 1-Based Indexing Bug > -- > > Key: SYSTEMML-672 > URL: https://issues.apache.org/jira/browse/SYSTEMML-672 > Project: SystemML > Issue Type: Bug > Components: Parser, PyDML >Affects Versions: SystemML 0.9, SystemML 0.10 >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry >Priority: Critical > Fix For: SystemML 0.10 > > > Currently, PyDML has a *bug* in which it uses 1-based indexing, as in R & > DML, rather than the correct 0-based indexing that Python uses. This simply > aims to fix this issue in the Parser, and then update the few PyDML test > scripts to use the correct 0-based indexing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-617) Override default namespace for imported custom functions
[ https://issues.apache.org/jira/browse/SYSTEMML-617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-617: - Issue Type: Sub-task (was: New Feature) Parent: SYSTEMML-590 > Override default namespace for imported custom functions > > > Key: SYSTEMML-617 > URL: https://issues.apache.org/jira/browse/SYSTEMML-617 > Project: SystemML > Issue Type: Sub-task > Components: Parser >Reporter: Glenn Weidner >Assignee: Glenn Weidner >Priority: Minor > Fix For: SystemML 0.10 > > > This sub-task targets a specific scenario described in SYSTEMML-590. > Example of error: > org.apache.sysml.api.DMLException: org.apache.sysml.parser.LanguageException: > ERROR: null -- line 0, column 0 -- function g is undefined in namespace > .defaultNS > at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:350) > at org.apache.sysml.api.DMLScript.main(DMLScript.java:197) > ... > Caused by: org.apache.sysml.parser.LanguageException: ERROR: null -- line 0, > column 0 -- function g is undefined in namespace .defaultNS > at > org.apache.sysml.parser.StatementBlock.isMergeableFunctionCallBlock(StatementBlock.java:201) > at > org.apache.sysml.parser.StatementBlock.mergeFunctionCalls(StatementBlock.java:328) > at > org.apache.sysml.parser.DMLTranslator.liveVariableAnalysis(DMLTranslator.java:165) > at org.apache.sysml.api.DMLScript.execute(DMLScript.java:592) > at org.apache.sysml.api.DMLScript.executeScript(DMLScript.java:338) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (SYSTEMML-495) Simplify SystemML Configuration Loading
[
https://issues.apache.org/jira/browse/SYSTEMML-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry resolved SYSTEMML-495.
--
Resolution: Fixed
> Simplify SystemML Configuration Loading
> ---
>
> Key: SYSTEMML-495
> URL: https://issues.apache.org/jira/browse/SYSTEMML-495
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>Priority: Minor
>
> Currently, SystemML always looks for a configuration file in a default
> location (in the current directory), and will always emit a warning if it is
> not found. Additionally, SystemML allows a user to supply a configuration
> file location. In this latter case, SystemML will still attempt to read the
> default configuration file, emitting a warning if it is not present (which
> causes confusion), as well as the user-supplied file. If both are loaded,
> the user-supplied file will be merged into the default configuration with an
> overwrite strategy.
> Always looking for a default file, even when a file is explicitly supplied,
> causes confusion, and may create unintended side effects. If a user
> explicitly supplies a configuration file, we should simply use that file,
> rather than attempting to also read an additional default file at the same
> time. In order to provide backwards compatibility, if the user does not
> supply a file, then we should fall back to the default file location. If
> that file is also not found, we should then fall back to the internal
> defaults.
> Additionally, in order to supply an additional configuration file, the
> {{-config=path/to/file}} flag has to be used, and it only works with the
> equals sign {{=}} present, which is unlike all other flags that use spaces.
> Instead, we should support {{-config path/to/file}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in the parfor
loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
in the
[{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
function used by {{conv::forward(...)}}, as well as in the
[{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
function used by {{conv::forward(...)}}.
* Test script (assuming the {{nn}} package is available):
** {{speed-633.dml}} {code}
source("nn/layers/conv.dml") as conv
source("nn/util.dml") as util
# Generate data
N = 64 # num examples
C = 30 # num channels
Hin = 28 # input height
Win = 28 # input width
F = 20 # num filters
Hf = 3 # filter height
Wf = 3 # filter width
stride = 1
pad = 1
X = rand(rows=N, cols=C*Hin*Win)
# Create layer
[W, b] = conv::init(F, C, Hf, Wf)
# Forward
[out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride, stride,
pad, pad)
print("Out: " + nrow(out) + "x" + ncol(out))
print("Hout: " + Hout)
print("Wout: " + Wout)
print("")
print(sum(out))
{code}
* Invocation:
** {{java -jar
$SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
speed-633.dml -stats -explain -exec singlenode}}
* Stats output (modified to output up to 100 instructions):
** {code}
...
Total elapsed time: 26.834 sec.
Total compilation time: 0.529 sec.
Total execution time: 26.304 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
Cache writes (WB, FS, HDFS): 3070724/0/0.
Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time: 0.268 sec.
Functions recompiled: 129.
Functions recompile time: 0.841 sec.
ParFor loops optimized: 1.
ParFor optimize time: 0.032 sec.
ParFor initialize time: 0.015 sec.
ParFor result merge time: 0.028 sec.
ParFor total update in-place: 0/0/1559360
Total JIT compile time: 14.235 sec.
Total JVM GC count: 94.
Total JVM GC time:0.366 sec.
Heavy hitter instructions (name, time, count):
-- 1) leftIndex 41.670 sec 1559360
-- 2) forward 26.212 sec 1
-- 3) im2col_t45 25.919 sec 8
-- 4) im2col_t41 25.850 sec 8
-- 5) im2col_t48 25.831 sec 8
-- 6) im2col_t43 25.752 sec 8
-- 7) im2col_t44 25.736 sec 8
-- 8) im2col_t42 25.695 sec 8
-- 9) im2col_t47 25.691 sec 8
-- 10) im2col_t46 25.519 sec 8
-- 11) rangeReIndex 13.392 sec 3012544
-- 12) rshape 8.197 sec 3064704
-- 13) rmvar 4.988 sec 20363504
-- 14) createvar 4.954 sec 7688965
-- 15) ncol 1.148 sec 3014529
-- 16) - 0.961 sec 3112834
-- 17) + 0.878 sec 3124617
-- 18) rand 0.839 sec 52228
-- 19) * 0.480 sec 1764229
-- 20) cpvar 0.366 sec 1607812
-- 21) ba+* 0.257 sec 64
-- 22) pad_image_t42 0.187 sec 8
-- 23) pad_image_t47 0.181 sec 8
-- 24) pad_image_t44 0.168 sec 8
-- 25) pad_image_t46 0.164
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in the parfor
loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
in the
[{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
function used by {{conv::forward(...)}}, as well as in the
[{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
function used by {{conv::forward(...)}}.
was:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/master/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}}
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in the parfor
loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
in the
[{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
function used by {{conv::forward(...)}}, as well as in the
[{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
function used by {{conv::forward(...)}}.
* Test script (assuming the {{nn}} package is available):
** {code}
source("nn/layers/conv.dml") as conv
source("nn/util.dml") as util
# Generate data
N = 64 # num examples
C = 30 # num channels
Hin = 28 # input height
Win = 28 # input width
F = 20 # num filters
Hf = 3 # filter height
Wf = 3 # filter width
stride = 1
pad = 1
X = rand(rows=N, cols=C*Hin*Win)
# Create layer
[W, b] = conv::init(F, C, Hf, Wf)
# Forward
[out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride, stride,
pad, pad)
print("Out: " + nrow(out) + "x" + ncol(out))
print("Hout: " + Hout)
print("Wout: " + Wout)
print("")
print(sum(out))
{code}
was:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in the parfor
loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
in the
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in the parfor
loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
in the
[{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
function used by {{conv::forward(...)}}, as well as in the
[{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
function used by {{conv::forward(...)}}.
* Test script (assuming the {{nn}} package is available):
** {{speed-633.dml}} {code}
source("nn/layers/conv.dml") as conv
source("nn/util.dml") as util
# Generate data
N = 64 # num examples
C = 30 # num channels
Hin = 28 # input height
Win = 28 # input width
F = 20 # num filters
Hf = 3 # filter height
Wf = 3 # filter width
stride = 1
pad = 1
X = rand(rows=N, cols=C*Hin*Win)
# Create layer
[W, b] = conv::init(F, C, Hf, Wf)
# Forward
[out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride, stride,
pad, pad)
print("Out: " + nrow(out) + "x" + ncol(out))
print("Hout: " + Hout)
print("Wout: " + Wout)
print("")
print(sum(out))
{code}
* Invocation:
** {{java -jar
$SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
speed-633.dml -stats -explain -exec singlenode}}
was:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in the parfor
loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
in the
[{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
function used by {{conv::forward(...)}}, as well as in the
[{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
function used by {{conv::forward(...)}}.
* Test script (assuming the {{nn}} package is available):
** {{speed-633.dml}} {code}
source("nn/layers/conv.dml") as conv
source("nn/util.dml") as util
# Generate data
N = 64 # num examples
C = 30 # num channels
Hin = 28 # input height
Win = 28 # input width
F = 20 # num filters
Hf = 3 # filter height
Wf = 3 # filter width
stride = 1
pad = 1
X = rand(rows=N, cols=C*Hin*Win)
# Create layer
[W, b] = conv::init(F, C, Hf, Wf)
# Forward
[out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride, stride,
pad, pad)
print("Out: " + nrow(out) + "x" + ncol(out))
print("Hout: " + Hout)
print("Wout: " + Wout)
print("")
print(sum(out))
{code}
* Invocation:
** {{java -jar
$SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
speed-633.dml -stats -explain -exec singlenode}}
* Stats output (modified to output up to 100 instructions):
** {code}
...
Total elapsed time: 26.834 sec.
Total compilation time: 0.529 sec.
Total execution time: 26.304 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
Cache writes (WB, FS, HDFS): 3070724/0/0.
Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time: 0.268 sec.
Functions recompiled: 129.
Functions recompile time: 0.841 sec.
ParFor loops optimized: 1.
ParFor optimize time: 0.032 sec.
ParFor initialize time: 0.015 sec.
ParFor result merge time: 0.028 sec.
ParFor total update in-place: 0/0/1559360
Total JIT compile time: 14.235 sec.
Total JVM GC count: 94.
Total JVM GC time:0.366 sec.
Heavy hitter instructions (name, time, count):
-- 1) leftIndex 41.670 sec 1559360
-- 2) forward 26.212 sec 1
-- 3) im2col_t45 25.919 sec 8
-- 4) im2col_t41 25.850 sec 8
-- 5) im2col_t48 25.831 sec 8
-- 6) im2col_t43 25.752 sec 8
-- 7) im2col_t44 25.736 sec 8
-- 8) im2col_t42 25.695 sec 8
-- 9) im2col_t47 25.691 sec 8
-- 10) im2col_t46 25.519 sec 8
-- 11) rangeReIndex 13.392 sec 3012544
-- 12) rshape 8.197 sec 3064704
-- 13) rmvar 4.988 sec 20363504
-- 14) createvar 4.954 sec 7688965
-- 15) ncol 1.148 sec 3014529
-- 16) - 0.961 sec 3112834
-- 17) + 0.878 sec 3124617
-- 18) rand 0.839 sec 52228
-- 19) * 0.480 sec 1764229
-- 20) cpvar 0.366 sec 1607812
-- 21) ba+* 0.257 sec 64
-- 22) pad_image_t42 0.187 sec 8
-- 23) pad_image_t47 0.181 sec 8
-- 24) pad_image_t44 0.168 sec 8
-- 25) pad_image_t46 0.164
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in the parfor
loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
in the
[{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
function used by {{conv::forward(...)}}, as well as in the
[{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
function used by {{conv::forward(...)}}.
* Test script (assuming the {{nn}} package is available):
** {{speed-633.dml}} {code}
source("nn/layers/conv.dml") as conv
source("nn/util.dml") as util
# Generate data
N = 64 # num examples
C = 30 # num channels
Hin = 28 # input height
Win = 28 # input width
F = 20 # num filters
Hf = 3 # filter height
Wf = 3 # filter width
stride = 1
pad = 1
X = rand(rows=N, cols=C*Hin*Win)
# Create layer
[W, b] = conv::init(F, C, Hf, Wf)
# Forward
[out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride, stride,
pad, pad)
print("Out: " + nrow(out) + "x" + ncol(out))
print("Hout: " + Hout)
print("Wout: " + Wout)
print("")
print(sum(out))
{code}
* Invocation:
** {{java -jar
$SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
speed-633.dml -stats -explain -exec singlenode}}
* Stats output (modified to output up to 100 instructions):
** {code}
...
Total elapsed time: 26.834 sec.
Total compilation time: 0.529 sec.
Total execution time: 26.304 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
Cache writes (WB, FS, HDFS): 3070724/0/0.
Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time: 0.268 sec.
Functions recompiled: 129.
Functions recompile time: 0.841 sec.
ParFor loops optimized: 1.
ParFor optimize time: 0.032 sec.
ParFor initialize time: 0.015 sec.
ParFor result merge time: 0.028 sec.
ParFor total update in-place: 0/0/1559360
Total JIT compile time: 14.235 sec.
Total JVM GC count: 94.
Total JVM GC time:0.366 sec.
Heavy hitter instructions (name, time, count):
-- 1) leftIndex 41.670 sec 1559360
-- 2) forward 26.212 sec 1
-- 3) im2col_t45 25.919 sec 8
-- 4) im2col_t41 25.850 sec 8
-- 5) im2col_t48 25.831 sec 8
-- 6) im2col_t43 25.752 sec 8
-- 7) im2col_t44 25.736 sec 8
-- 8) im2col_t42 25.695 sec 8
-- 9) im2col_t47 25.691 sec 8
-- 10) im2col_t46 25.519 sec 8
-- 11) rangeReIndex 13.392 sec 3012544
-- 12) rshape 8.197 sec 3064704
-- 13) rmvar 4.988 sec 20363504
-- 14) createvar 4.954 sec 7688965
-- 15) ncol 1.148 sec 3014529
-- 16) - 0.961 sec 3112834
-- 17) + 0.878 sec 3124617
-- 18) rand 0.839 sec 52228
-- 19) * 0.480 sec 1764229
-- 20) cpvar 0.366 sec 1607812
-- 21) ba+* 0.257 sec 64
-- 22) pad_image_t42 0.187 sec 8
-- 23) pad_image_t47 0.181 sec 8
-- 24) pad_image_t44 0.168 sec 8
-- 25) pad_image_t46 0.164
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
* Left-indexing has been identified as the bottleneck by a wide margin.
* Left-indexing is used in the main {{conv::forward(...)}} function in the
[last line in the parfor
loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
in the
[{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
function used by {{conv::forward(...)}}, as well as in the
[{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
function used by {{conv::forward(...)}}.
* Test script (assuming the {{nn}} package is available):
** {{speed-633.dml}} {code}
source("nn/layers/conv.dml") as conv
source("nn/util.dml") as util
# Generate data
N = 64 # num examples
C = 30 # num channels
Hin = 28 # input height
Win = 28 # input width
F = 20 # num filters
Hf = 3 # filter height
Wf = 3 # filter width
stride = 1
pad = 1
X = rand(rows=N, cols=C*Hin*Win)
# Create layer
[W, b] = conv::init(F, C, Hf, Wf)
# Forward
[out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride, stride,
pad, pad)
print("Out: " + nrow(out) + "x" + ncol(out))
print("Hout: " + Hout)
print("Wout: " + Wout)
print("")
print(sum(out))
{code}
* Invocation:
** {{java -jar
$SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
speed-633.dml -stats -explain -exec singlenode}}
* Stats output (modified to output up to 100 instructions):
** {code}
...
Total elapsed time: 26.834 sec.
Total compilation time: 0.529 sec.
Total execution time: 26.304 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
Cache writes (WB, FS, HDFS): 3070724/0/0.
Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time: 0.268 sec.
Functions recompiled: 129.
Functions recompile time: 0.841 sec.
ParFor loops optimized: 1.
ParFor optimize time: 0.032 sec.
ParFor initialize time: 0.015 sec.
ParFor result merge time: 0.028 sec.
ParFor total update in-place: 0/0/1559360
Total JIT compile time: 14.235 sec.
Total JVM GC count: 94.
Total JVM GC time:0.366 sec.
Heavy hitter instructions (name, time, count):
-- 1) leftIndex 41.670 sec 1559360
-- 2) forward 26.212 sec 1
-- 3) im2col_t45 25.919 sec 8
-- 4) im2col_t41 25.850 sec 8
-- 5) im2col_t48 25.831 sec 8
-- 6) im2col_t43 25.752 sec 8
-- 7) im2col_t44 25.736 sec 8
-- 8) im2col_t42 25.695 sec 8
-- 9) im2col_t47 25.691 sec 8
-- 10) im2col_t46 25.519 sec 8
-- 11) rangeReIndex 13.392 sec 3012544
-- 12) rshape 8.197 sec 3064704
-- 13) rmvar 4.988 sec 20363504
-- 14) createvar 4.954 sec 7688965
-- 15) ncol 1.148 sec 3014529
-- 16) - 0.961 sec 3112834
-- 17) + 0.878 sec 3124617
-- 18) rand 0.839 sec 52228
-- 19) * 0.480 sec 1764229
-- 20) cpvar 0.366 sec 1607812
-- 21) ba+* 0.257 sec 64
-- 22) pad_image_t42 0.187 sec 8
-- 23) pad_image_t47 0.181 sec 8
-- 24) pad_image_t44 0.168 sec 8
-- 25) pad_image_t46 0.164
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances, but the same issue is shared
across many areas of the library, particularly in the convolution and max
pooling layers.
Quick note on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/master/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image, extracts "patches" of the image
into columns of a matrix in a sliding fashion across the image with {{im2col}},
performs a matrix multiplication between the matrix of patch columns and the
weight matrix, and then saves the result into a matrix defined outside of the
parfor loop using left-indexing.
was:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances, but the same issue is shared
across many areas of the library, particularly in the convolution and max
pooling layers.
Quick note on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
({{conv::forward(...)}} in {{nn/layers/conv.dml}}) essentially accepts a matrix
{{X}} of images, a matrix of weights {{W}}, and several other parameters
corresponding to image sizes, filter sizes, etc. It then loops through the
images with a {{parfor}} loop, and for each image it pads the image, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{im2col}}, performs a matrix multiplication between the matrix of
patch columns and the weight matrix, and then saves the result into a matrix
defined outside of the parfor loop using left-indexing.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances, but the same issue is
> shared across many areas of the library, particularly in the convolution and
> max pooling layers.
> Quick note on setup for any of the below experiments. Please grab a copy of
> the above repo (particularly the {{nn}} directory), and run any experiments
> with the {{nn}} package available at the base directory of the experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/master/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image, extracts "patches" of the image
> into columns of a matrix in a sliding fashion across the image with
> {{im2col}}, performs a matrix multiplication between the matrix of patch
> columns and the weight matrix, and then saves the result into a matrix
> defined outside of the parfor loop using left-indexing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances with simplified examples, but
the same issue is shared across many areas of the library, particularly in the
convolution and max pooling layers, and is exaggerated in real use-cases.
*Quick note* on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/master/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image with {{util::pad_image}}, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{util::im2col}}, performs a matrix multiplication between the
matrix of patch columns and the weight matrix, and then saves the result into a
matrix defined outside of the parfor loop using left-indexing.
was:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances, but the same issue is shared
across many areas of the library, particularly in the convolution and max
pooling layers.
Quick note on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/master/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image, extracts "patches" of the image
into columns of a matrix in a sliding fashion across the image with {{im2col}},
performs a matrix multiplication between the matrix of patch columns and the
weight matrix, and then saves the result into a matrix defined outside of the
parfor loop using left-indexing.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/master/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix,
[jira] [Updated] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-
Attachment: systemml-nn.zip
Attaching a snapshot of the SystemML-NN library at [commit fd71c65 |
https://github.com/dusenberrymw/systemml-nn/tree/fd71c659f544e21a34ba3383b495e339c852cea0]
for historical purposes.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
> Attachments: Im2colWrapper.java, log.txt, systemml-nn.zip
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/0.
> HOP DAGs recompile time: 0.268 sec.
> Functions recompiled: 129.
> Functions recompile time: 0.841 sec.
> ParFor loops optimized: 1.
> ParFor optimize time: 0.032 sec.
> ParFor initialize time: 0.015 sec.
> ParFor result merge time: 0.028 sec.
> ParFor total update in-place: 0/0/1559360
> Total JIT compile time: 14.235 sec.
> Total JVM GC count: 94.
> Total JVM GC time:0.366 sec.
> Heavy hitter instructions (name, time, count):
> -- 1) leftIndex 41.670 sec 1559360
> -- 2)
[jira] [Commented] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246306#comment-15246306
]
Mike Dusenberry commented on SYSTEMML-633:
--
To continue on the thread of the original slow left-indexing, I'd like to
discuss more details of the briefly mentioned experiment with moving
{{util::im2col(...)}} to an external Java function.
I've attached the full {{Im2colWrapper.java}} class which can be placed in the
{{org.apache.sysml.udf.lib}} package and compile with the rest of SystemML.
Below is a snippet of the main portion of the code:
{code}
Matrix imgM = (Matrix) getFunctionInput(0);
double[][] img = imgM.getMatrixAsDoubleArray();
int C = (int) imgM.getNumRows();
int Hin =
Integer.parseInt(((Scalar)getFunctionInput(1)).getValue());
int Win =
Integer.parseInt(((Scalar)getFunctionInput(2)).getValue());
int Hf = Integer.parseInt(((Scalar)getFunctionInput(3)).getValue());
int Wf = Integer.parseInt(((Scalar)getFunctionInput(4)).getValue());
int strideh =
Integer.parseInt(((Scalar)getFunctionInput(5)).getValue());
int stridew = Integer.parseInt(((Scalar)
getFunctionInput(6)).getValue());
int Hout = (Hin - Hf) / strideh + 1;
int Wout = (Win - Wf) / stridew + 1;
double[][] img_cols = new double[C*Hf*Wf][Hout*Wout];
for (int c=0; c
[jira] [Commented] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246226#comment-15246226
]
Mike Dusenberry commented on SYSTEMML-633:
--
Thanks, [~mboehm7]! I will try these approaches to see what amount of speedup
can be achieved.
I'd like to break up this into separate threads though -- one focus can be on
exploring these vectorized solutions, while another focus should still be on
the original issue of slow left-indexing. I'll follow up on the latter in a
moment.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
> Attachments: log.txt
>
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances with simplified examples,
> but the same issue is shared across many areas of the library, particularly
> in the convolution and max pooling layers, and is exaggerated in real
> use-cases.
> *Quick note* on setup for any of the below experiments. Please grab a copy
> of the above repo (particularly the {{nn}} directory), and run any
> experiments with the {{nn}} package available at the base directory of the
> experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image with {{util::pad_image}}, extracts
> "patches" of the image into columns of a matrix in a sliding fashion across
> the image with {{util::im2col}}, performs a matrix multiplication between the
> matrix of patch columns and the weight matrix, and then saves the result into
> a matrix defined outside of the parfor loop using left-indexing.
> * Left-indexing has been identified as the bottleneck by a wide margin.
> * Left-indexing is used in the main {{conv::forward(...)}} function in the
> [last line in the parfor
> loop|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/layers/conv.dml#L61],
> in the
> [{{util::pad_image(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L196]
> function used by {{conv::forward(...)}}, as well as in the
> [{{util::im2col(...)}}|https://github.com/dusenberrymw/systemml-nn/blob/f6d3e077ae3c303eb8426b31329d3734e3483d5f/nn/util.dml#L96]
> function used by {{conv::forward(...)}}.
> * Test script (assuming the {{nn}} package is available):
> ** {{speed-633.dml}} {code}
> source("nn/layers/conv.dml") as conv
> source("nn/util.dml") as util
> # Generate data
> N = 64 # num examples
> C = 30 # num channels
> Hin = 28 # input height
> Win = 28 # input width
> F = 20 # num filters
> Hf = 3 # filter height
> Wf = 3 # filter width
> stride = 1
> pad = 1
> X = rand(rows=N, cols=C*Hin*Win)
> # Create layer
> [W, b] = conv::init(F, C, Hf, Wf)
> # Forward
> [out, Hout, Wout] = conv::forward(X, W, b, C, Hin, Win, Hf, Wf, stride,
> stride, pad, pad)
> print("Out: " + nrow(out) + "x" + ncol(out))
> print("Hout: " + Hout)
> print("Wout: " + Wout)
> print("")
> print(sum(out))
> {code}
> * Invocation:
> ** {{java -jar
> $SYSTEMML_HOME/target/systemml-0.10.0-incubating-SNAPSHOT-standalone.jar -f
> speed-633.dml -stats -explain -exec singlenode}}
> * Stats output (modified to output up to 100 instructions):
> ** {code}
> ...
> Total elapsed time: 26.834 sec.
> Total compilation time: 0.529 sec.
> Total execution time: 26.304 sec.
> Number of compiled MR Jobs: 0.
> Number of executed MR Jobs: 0.
> Cache hits (Mem, WB, FS, HDFS): 9196235/0/0/0.
> Cache writes (WB, FS, HDFS): 3070724/0/0.
> Cache times (ACQr/m, RLS, EXP): 1.474/1.120/26.998/0.000 sec.
> HOP DAGs recompiled (PRED, SB): 0/0.
> HOP DAGs recompile time: 0.268 sec.
> Functions recompiled: 129.
> Functions recompile time: 0.841 sec.
> ParFor loops optimized: 1.
> ParFor optimize time: 0.032 sec.
> ParFor initialize time: 0.015 sec.
> ParFor result merge time: 0.028 sec.
> ParFor total update in-place: 0/0/1559360
> Total JIT compile time: 14.235 sec.
> Total JVM GC count: 94.
> Total JVM
[jira] [Comment Edited] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246306#comment-15246306
]
Mike Dusenberry edited comment on SYSTEMML-633 at 4/18/16 6:51 PM:
---
To continue on the thread of the original slow left-indexing, I'd like to
discuss more details of the briefly mentioned experiment with moving
{{util::im2col(...)}} to an external Java function.
I've attached the full {{Im2colWrapper.java}} class which can be placed in the
{{org.apache.sysml.udf.lib}} package and compiled with the rest of SystemML.
Below is a snippet of the main portion of the code:
{code}
Matrix imgM = (Matrix) getFunctionInput(0);
double[][] img = imgM.getMatrixAsDoubleArray();
int C = (int) imgM.getNumRows();
int Hin =
Integer.parseInt(((Scalar)getFunctionInput(1)).getValue());
int Win =
Integer.parseInt(((Scalar)getFunctionInput(2)).getValue());
int Hf = Integer.parseInt(((Scalar)getFunctionInput(3)).getValue());
int Wf = Integer.parseInt(((Scalar)getFunctionInput(4)).getValue());
int strideh =
Integer.parseInt(((Scalar)getFunctionInput(5)).getValue());
int stridew = Integer.parseInt(((Scalar)
getFunctionInput(6)).getValue());
int Hout = (Hin - Hf) / strideh + 1;
int Wout = (Win - Wf) / stridew + 1;
double[][] img_cols = new double[C*Hf*Wf][Hout*Wout];
for (int c=0; c
[jira] [Comment Edited] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246306#comment-15246306
]
Mike Dusenberry edited comment on SYSTEMML-633 at 4/18/16 6:50 PM:
---
To continue on the thread of the original slow left-indexing, I'd like to
discuss more details of the briefly mentioned experiment with moving
{{util::im2col(...)}} to an external Java function.
I've attached the full {{Im2colWrapper.java}} class which can be placed in the
{{org.apache.sysml.udf.lib}} package and compiled with the rest of SystemML.
Below is a snippet of the main portion of the code:
{code}
Matrix imgM = (Matrix) getFunctionInput(0);
double[][] img = imgM.getMatrixAsDoubleArray();
int C = (int) imgM.getNumRows();
int Hin =
Integer.parseInt(((Scalar)getFunctionInput(1)).getValue());
int Win =
Integer.parseInt(((Scalar)getFunctionInput(2)).getValue());
int Hf = Integer.parseInt(((Scalar)getFunctionInput(3)).getValue());
int Wf = Integer.parseInt(((Scalar)getFunctionInput(4)).getValue());
int strideh =
Integer.parseInt(((Scalar)getFunctionInput(5)).getValue());
int stridew = Integer.parseInt(((Scalar)
getFunctionInput(6)).getValue());
int Hout = (Hin - Hf) / strideh + 1;
int Wout = (Win - Wf) / stridew + 1;
double[][] img_cols = new double[C*Hf*Wf][Hout*Wout];
for (int c=0; c
[jira] [Commented] (SYSTEMML-632) Improve handling of source statements in scripts that are also imported
[ https://issues.apache.org/jira/browse/SYSTEMML-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15246398#comment-15246398 ] Mike Dusenberry commented on SYSTEMML-632: -- Merged in [commit 53ac4ed | https://github.com/apache/incubator-systemml/commit/53ac4ed629d2863c58f01173e6a0665fde19a70f]. > Improve handling of source statements in scripts that are also imported > --- > > Key: SYSTEMML-632 > URL: https://issues.apache.org/jira/browse/SYSTEMML-632 > Project: SystemML > Issue Type: Sub-task > Components: Parser >Reporter: Glenn Weidner >Assignee: Glenn Weidner >Priority: Minor > Fix For: SystemML 0.10 > > > If script1 sources another script2 which also sources another script3, then > qualified functions calls in script2 to script3 fail with error indicating > the function is undefined in namespace. > An improvement would be to add processing such that dependent function calls > are successful. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-536) KNN DML script
[ https://issues.apache.org/jira/browse/SYSTEMML-536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15239924#comment-15239924 ] Mike Dusenberry commented on SYSTEMML-536: -- Merged in [commit 6aa4eed | https://github.com/apache/incubator-systemml/commit/6aa4eed525a4323a78e64d70af626e9fb8509048]. > KNN DML script > -- > > Key: SYSTEMML-536 > URL: https://issues.apache.org/jira/browse/SYSTEMML-536 > Project: SystemML > Issue Type: Sub-task > Components: Algorithms >Reporter: Tommy Yu >Assignee: Tommy Yu > > Work out KNN DML script include naive knn, k selection, feature selection, > feature importance -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (SYSTEMML-633) Improve Left-Indexing Performance with (Nested) Parfor Loops
[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15251013#comment-15251013
]
Mike Dusenberry commented on SYSTEMML-633:
--
For another experiment, I created a script that approximates the idea of the
{{conv::forward(...)}} function, particularly trying to isolate the matrix
multiplication and the left-indexing in the parfor loop, similar to the
previous experiment. In this case, I used no functions, and the
update-in-place was successfully applied to the parfor loop, thus removing
left-indexing as a bottleneck.
{code}
# Generate data
N = 64 # num examples
C = 30 # num channels
Hin = 28 # input height
Win = 28 # input width
F = 20 # num filters
Hf = 3 # filter height
Wf = 3 # filter width
stride = 1; strideh=stride; stridew=stride
pad = 0; padh=pad; padw=pad
max_iterations = 1
batch_size = 64
# Create layer
W = rand(rows=F, cols=C*Hf*Wf, pdf="normal") * sqrt(2.0/(C*Hf*Wf))
b = matrix(0, rows=F, cols=1)
Hout = as.integer((Hin + 2 * padh - Hf) / strideh + 1)
Wout = as.integer((Win + 2 * padw - Wf) / stridew + 1)
X = rand(rows=N, cols=C*Hf*Wf*Hout*Wout)
foo = 0
for (iter in 1:max_iterations) {
beg = ((iter-1) * batch_size) %% N + 1
end = min(N, beg + batch_size - 1)
X_batch = X[beg:end,]
# Create output volume
out = matrix(0, rows=N, cols=F*Hout*Wout)
# Forward
parfor (n in 1:nrow(X_batch), log=DEBUG) { # all examples
# Extract local image patches into columns with im2col, of shape (C*Hf*Wf,
Hout*Wout)
Xn_padded_cols = matrix(X_batch[n,], rows=C*Hf*Wf, cols=Hout*Wout)
# Convolve patches with filters
outn = W %*% Xn_padded_cols + b # shape (F, Hout*Wout)
out[n,] = matrix(outn, rows=1, cols=F*Hout*Wout) # reshape
}
foo = foo + min(0.1, sum(out))
}
print(foo)
{code}
{code}
16/04/20 17:25:02 INFO api.DMLScript: SystemML Statistics:
Total elapsed time: 0.993 sec.
Total compilation time: 0.278 sec.
Total execution time: 0.715 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 460/0/0/0.
Cache writes (WB, FS, HDFS):261/0/0.
Cache times (ACQr/m, RLS, EXP): 0.002/0.002/0.025/0.000 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time:0.000 sec.
ParFor loops optimized: 1.
ParFor optimize time: 0.017 sec.
ParFor initialize time: 0.014 sec.
ParFor result merge time: 0.021 sec.
ParFor total update in-place: 0/64/64
Total JIT compile time: 1.164 sec.
Total JVM GC count: 4.
Total JVM GC time: 0.023 sec.
Heavy hitter instructions (name, time, count):
-- 1) ba+*0.392 sec 64
-- 2) rand0.322 sec 4
-- 3) leftIndex 0.286 sec 64
-- 4) rangeReIndex0.234 sec 65
-- 5) rshape 0.031 sec 128
-- 6) uak+0.013 sec 1
-- 7) rmvar 0.010 sec 538
-- 8) ncol0.009 sec 129
-- 9) createvar 0.003 sec 326
-- 10) * 0.002 sec 5
{code}
Increasing the {{max_iterations}} to {{100}} and the number of examples {{N}}
to {{640}} continues this trend:
{code}
16/04/20 17:28:30 INFO api.DMLScript: SystemML Statistics:
Total elapsed time: 17.928 sec.
Total compilation time: 0.276 sec.
Total execution time: 17.652 sec.
Number of compiled MR Jobs: 0.
Number of executed MR Jobs: 0.
Cache hits (Mem, WB, FS, HDFS): 45901/0/0/0.
Cache writes (WB, FS, HDFS):25802/1/0.
Cache times (ACQr/m, RLS, EXP): 0.035/0.022/2.894/0.000 sec.
HOP DAGs recompiled (PRED, SB): 0/0.
HOP DAGs recompile time:0.019 sec.
ParFor loops optimized: 100.
ParFor optimize time: 0.210 sec.
ParFor initialize time: 0.120 sec.
ParFor result merge time: 1.090 sec.
ParFor total update in-place: 0/6400/6400
Total JIT compile time: 10.927 sec.
Total JVM GC count: 55.
Total JVM GC time: 1.356 sec.
Heavy hitter instructions (name, time, count):
-- 1) ba+*26.353 sec 6400
-- 2) rangeReIndex16.213 sec 6500
-- 3) leftIndex 10.236 sec 6400
-- 4) rand4.576 sec 103
-- 5) uak+0.509 sec 100
-- 6) rshape 0.210 sec 12800
-- 7) createvar 0.083 sec 32204
-- 8) rmvar 0.080 sec 52711
-- 9) ncol0.037 sec 12900
-- 10) cpvar 0.009 sec 6603
{code}
> Improve Left-Indexing Performance with (Nested) Parfor Loops
>
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor
>Reporter: Mike Dusenberry
> Attachments: Im2colWrapper.java, log.txt, systemml-nn.zip
>
>
> In the experimental deep learning
[jira] [Closed] (SYSTEMML-475) Add Implicit Conversion Between Scalar Values and 1x1 Matrices
[
https://issues.apache.org/jira/browse/SYSTEMML-475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry closed SYSTEMML-475.
Resolution: Later
> Add Implicit Conversion Between Scalar Values and 1x1 Matrices
> --
>
> Key: SYSTEMML-475
> URL: https://issues.apache.org/jira/browse/SYSTEMML-475
> Project: SystemML
> Issue Type: Improvement
> Components: Compiler, Parser
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>
> Currently, any DML functions that require scalars will not accept 1x1
> matrices. Matrix indexing always returns a matrix, so a statement such as
> {{A[1,1]}} from the following simple example will return a single-valued
> matrix, even though it logically represents a scalar. Thus, in order to use
> this value as a scalar, the {{as.scalar(...)}} function is required.
> {code}
> A = matrix(7, rows=10, cols=10)
>
> print( "A[1,1]: " + as.scalar(A[1,1])) // 7
> {code}
> In this scenario, the required usage of {{as.scalar(...)}} can cause
> confusion, and can be viewed as a violation of SystemML's goal of hiding the
> underlying data representation from the user.
> Additionally, many of the built-in functions, such as {{rowVars(...)}},
> accept matrices without other requirements of dimensionality, and thus will
> accept a 1x1 matrix, yet will not accept a scalar value. We should allow any
> such functions that accept 1x1 matrices to also accept scalar values.
> {code}
> a = 4
> print("Row variances of scalar are: " + rowVars(a)) // 0
> {code}
> In order to achieve this, we can add implicit casting between scalar values
> and 1x1 matrices within the parser during the validation stage as needed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (SYSTEMML-587) Improvements Triggered By Deep Learning Work
[ https://issues.apache.org/jira/browse/SYSTEMML-587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Dusenberry updated SYSTEMML-587: - Priority: Major (was: Minor) > Improvements Triggered By Deep Learning Work > > > Key: SYSTEMML-587 > URL: https://issues.apache.org/jira/browse/SYSTEMML-587 > Project: SystemML > Issue Type: Umbrella >Reporter: Mike Dusenberry >Assignee: Mike Dusenberry > > This convenience umbrella tracks all improvements triggered by the work on > deep learning (SYSTEMML-540), but not directly related to it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (SYSTEMML-495) Simplify SystemML Configuration Loading
[
https://issues.apache.org/jira/browse/SYSTEMML-495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-495:
-
Priority: Minor (was: Major)
> Simplify SystemML Configuration Loading
> ---
>
> Key: SYSTEMML-495
> URL: https://issues.apache.org/jira/browse/SYSTEMML-495
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>Priority: Minor
>
> Currently, SystemML always looks for a configuration file in a default
> location (in the current directory), and will always emit a warning if it is
> not found. Additionally, SystemML allows a user to supply a configuration
> file location. In this latter case, SystemML will still attempt to read the
> default configuration file, emitting a warning if it is not present (which
> causes confusion), as well as the user-supplied file. If both are loaded,
> the user-supplied file will be merged into the default configuration with an
> overwrite strategy.
> Always looking for a default file, even when a file is explicitly supplied,
> causes confusion, and may create unintended side effects. If a user
> explicitly supplies a configuration file, we should simply use that file,
> rather than attempting to also read an additional default file at the same
> time. In order to provide backwards compatibility, if the user does not
> supply a file, then we should fall back to the default file location. If
> that file is also not found, we should then fall back to the internal
> defaults.
> Additionally, in order to supply an additional configuration file, the
> {{-config=path/to/file}} flag has to be used, and it only works with the
> equals sign {{=}} present, which is unlike all other flags that use spaces.
> Instead, we should support {{-config path/to/file}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (SYSTEMML-495) Simplify SystemML Configuration Loading
[
https://issues.apache.org/jira/browse/SYSTEMML-495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15240104#comment-15240104
]
Mike Dusenberry commented on SYSTEMML-495:
--
Does anyone have anymore thoughts on this minor issue?
> Simplify SystemML Configuration Loading
> ---
>
> Key: SYSTEMML-495
> URL: https://issues.apache.org/jira/browse/SYSTEMML-495
> Project: SystemML
> Issue Type: Improvement
>Reporter: Mike Dusenberry
>Assignee: Mike Dusenberry
>Priority: Minor
>
> Currently, SystemML always looks for a configuration file in a default
> location (in the current directory), and will always emit a warning if it is
> not found. Additionally, SystemML allows a user to supply a configuration
> file location. In this latter case, SystemML will still attempt to read the
> default configuration file, emitting a warning if it is not present (which
> causes confusion), as well as the user-supplied file. If both are loaded,
> the user-supplied file will be merged into the default configuration with an
> overwrite strategy.
> Always looking for a default file, even when a file is explicitly supplied,
> causes confusion, and may create unintended side effects. If a user
> explicitly supplies a configuration file, we should simply use that file,
> rather than attempting to also read an additional default file at the same
> time. In order to provide backwards compatibility, if the user does not
> supply a file, then we should fall back to the default file location. If
> that file is also not found, we should then fall back to the internal
> defaults.
> Additionally, in order to supply an additional configuration file, the
> {{-config=path/to/file}} flag has to be used, and it only works with the
> equals sign {{=}} present, which is unlike all other flags that use spaces.
> Instead, we should support {{-config path/to/file}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
