[
https://issues.apache.org/jira/browse/SPARK-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425936#comment-15425936
]
DB Tsai commented on SPARK-17090:
---------------------------------
We could make it default to 0 which automatically figure out the best
aggregation depth based on the dimensions of features, and the number of
partition. Before that, we can make default 0 as 2 which is the current
behavior.
> Make tree aggregation level in linear/logistic regression configurable
> ----------------------------------------------------------------------
>
> Key: SPARK-17090
> URL: https://issues.apache.org/jira/browse/SPARK-17090
> Project: Spark
> Issue Type: Improvement
> Components: ML
> Reporter: Seth Hendrickson
> Priority: Minor
>
> Linear/logistic regression use treeAggregate with default aggregation depth
> for collecting coefficient gradient updates to the driver. For high
> dimensional problems, this can case OOM error on the driver. We should make
> it configurable, perhaps via an expert param, so that users can avoid this
> problem if their data has many features.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
