[
https://issues.apache.org/jira/browse/SYSTEMML-633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-633:
-------------------------------------
Description:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances, but the same issue is shared
across many areas of the library, particularly in the convolution and max
pooling layers.
Quick note on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
([{{conv::forward(...)}} |
https://github.com/dusenberrymw/systemml-nn/blob/master/nn/layers/conv.dml#L8]
in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
matrix of weights {{W}}, and several other parameters corresponding to image
sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
loop, and for each image it pads the image, extracts "patches" of the image
into columns of a matrix in a sliding fashion across the image with {{im2col}},
performs a matrix multiplication between the matrix of patch columns and the
weight matrix, and then saves the result into a matrix defined outside of the
parfor loop using left-indexing.
was:
In the experimental deep learning DML library I've been building
([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
Here, I will highlight a few particular instances, but the same issue is shared
across many areas of the library, particularly in the convolution and max
pooling layers.
Quick note on setup for any of the below experiments. Please grab a copy of
the above repo (particularly the {{nn}} directory), and run any experiments
with the {{nn}} package available at the base directory of the experiment.
Scenario: *Convolution*
* In the library above, the forward pass of the convolution function
({{conv::forward(...)}} in {{nn/layers/conv.dml}}) essentially accepts a matrix
{{X}} of images, a matrix of weights {{W}}, and several other parameters
corresponding to image sizes, filter sizes, etc. It then loops through the
images with a {{parfor}} loop, and for each image it pads the image, extracts
"patches" of the image into columns of a matrix in a sliding fashion across the
image with {{im2col}}, performs a matrix multiplication between the matrix of
patch columns and the weight matrix, and then saves the result into a matrix
defined outside of the parfor loop using left-indexing.
> Improve Left-Indexing Performance with (Nested) Parfor Loops
> ------------------------------------------------------------
>
> Key: SYSTEMML-633
> URL: https://issues.apache.org/jira/browse/SYSTEMML-633
> Project: SystemML
> Issue Type: Improvement
> Reporter: Mike Dusenberry
>
> In the experimental deep learning DML library I've been building
> ([https://github.com/dusenberrymw/systemml-nn|https://github.com/dusenberrymw/systemml-nn]),
> I've experienced severe bottlenecks due to *left-indexing* in parfor loops.
> Here, I will highlight a few particular instances, but the same issue is
> shared across many areas of the library, particularly in the convolution and
> max pooling layers.
> Quick note on setup for any of the below experiments. Please grab a copy of
> the above repo (particularly the {{nn}} directory), and run any experiments
> with the {{nn}} package available at the base directory of the experiment.
> Scenario: *Convolution*
> * In the library above, the forward pass of the convolution function
> ([{{conv::forward(...)}} |
> https://github.com/dusenberrymw/systemml-nn/blob/master/nn/layers/conv.dml#L8]
> in {{nn/layers/conv.dml}}) essentially accepts a matrix {{X}} of images, a
> matrix of weights {{W}}, and several other parameters corresponding to image
> sizes, filter sizes, etc. It then loops through the images with a {{parfor}}
> loop, and for each image it pads the image, extracts "patches" of the image
> into columns of a matrix in a sliding fashion across the image with
> {{im2col}}, performs a matrix multiplication between the matrix of patch
> columns and the weight matrix, and then saves the result into a matrix
> defined outside of the parfor loop using left-indexing.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
