[SYSTEMML-1427] Fix issues of worst-case and parfor memory estimates
This patch fixes various special cases of worst-case and parfor memory
estimates, including (1) incomplete worst-case estimates, (2) parfor w/
spark execution type, (3) constrained parfor optimizer with unspecified
degree of
Repository: incubator-systemml
Updated Branches:
refs/heads/master 795a94e7b -> 39f75ca06
[SYSTEMML-1425] Robustness parfor parameter handling (cmd args, quotes)
Project: http://git-wip-us.apache.org/repos/asf/incubator-systemml/repo
Commit:
http://git-wip-us.apache.org/repos/asf/incubator-systemml/blob/39f75ca0/src/test/scripts/functions/parfor/parfor_brdatapartitioning_neg.R
--
diff --git a/src/test/scripts/functions/parfor/parfor_brdatapartitioning_neg.R
[SYSTEMML-1429] Performance parfor spark dp (handle replication factor)
So far, we simply used the default replication factor in the parfor
spark data partitioning job, which writes individual sequence files per
partition after grouping. However, the parfor optimizer already selects
an