[2/4] incubator-systemml git commit: [SYSTEMML-1427] Fix issues of worst-case and parfor memory estimates

2017-03-21 Thread mboehm7
[SYSTEMML-1427] Fix issues of worst-case and parfor memory estimates This patch fixes various special cases of worst-case and parfor memory estimates, including (1) incomplete worst-case estimates, (2) parfor w/ spark execution type, (3) constrained parfor optimizer with unspecified degree of

[1/4] incubator-systemml git commit: [SYSTEMML-1425] Robustness parfor parameter handling (cmd args, quotes)

2017-03-21 Thread mboehm7
Repository: incubator-systemml Updated Branches: refs/heads/master 795a94e7b -> 39f75ca06 [SYSTEMML-1425] Robustness parfor parameter handling (cmd args, quotes) Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo Commit:

[3/4] incubator-systemml git commit: [SYSTEMML-1310] New parfor block partitioning (compiler/runtime/tests)

2017-03-21 Thread mboehm7
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/39f75ca0/src/test/scripts/functions/parfor/parfor_brdatapartitioning_neg.R -- diff --git a/src/test/scripts/functions/parfor/parfor_brdatapartitioning_neg.R

[2/3] incubator-systemml git commit: [SYSTEMML-1429] Performance parfor spark dp (handle replication factor)

2017-03-21 Thread mboehm7
[SYSTEMML-1429] Performance parfor spark dp (handle replication factor) So far, we simply used the default replication factor in the parfor spark data partitioning job, which writes individual sequence files per partition after grouping. However, the parfor optimizer already selects an