[
https://issues.apache.org/jira/browse/SYSTEMML-1821?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fei Hu updated SYSTEMML-1821:
-----------------------------
Description: For {{Remote_Spark}} mode, SystemML engine will export the
intermediate matrixes into HDFS, which may impair the performance of
distributed model training. Therefore the unnecessary intermediate matrixes
need to be avoided, and instead we can directly use the dimension index to
compute the final batch matrix. (was: SystemML engine will export the
intermediate matrixes into HDFS, which may impair the performance of
distributed model training. Therefore the unnecessary intermediate matrixes
need to be avoided, and instead we can directly use the dimension index to
compute the final batch matrix. )
> Improve the training process in mnist_lenet_distrib_sgd.dml
> -----------------------------------------------------------
>
> Key: SYSTEMML-1821
> URL: https://issues.apache.org/jira/browse/SYSTEMML-1821
> Project: SystemML
> Issue Type: Improvement
> Components: ParFor, Runtime
> Affects Versions: SystemML 1.0
> Reporter: Fei Hu
> Assignee: Fei Hu
>
> For {{Remote_Spark}} mode, SystemML engine will export the intermediate
> matrixes into HDFS, which may impair the performance of distributed model
> training. Therefore the unnecessary intermediate matrixes need to be avoided,
> and instead we can directly use the dimension index to compute the final
> batch matrix.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
