GitHub user iyerr3 opened a pull request:
https://github.com/apache/incubator-madlib/pull/120
DT: Assign memory only for reachable nodes
JIRA: MADLIB-1057
TreeAccumulator assigns a matrix to track the statistics of rows
reaching the last layer of nodes. This matrix assumes a complete
tree and assigns memory for all nodes. As the tree gets deeper,
most of the nodes are unreachable, resulting in excessive wasted
memory. This commit reduces that waste by only assigning memory
for nodes that are reachable and accessing them through a lookup
table.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/iyerr3/incubator-madlib
feature/dt_reduce_memory
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-madlib/pull/120.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #120
----
commit b1cea55925ee1e3f6569d2d7aafac16e608c43b3
Author: Rahul Iyer <[email protected]>
Date: 2017-04-15T00:54:31Z
Initial commit for sparser stats matrices
commit a0875f23ff69f22462a227b500612965976e0358
Author: Rahul Iyer <[email protected]>
Date: 2017-04-18T20:38:04Z
Build lookup index vector
commit 67cb1b121a4829f4840f33f7cdc7eabe839ec343
Author: Rahul Iyer <[email protected]>
Date: 2017-04-19T00:39:24Z
Remove warnings
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
