[
https://issues.apache.org/jira/browse/SPARK-3022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14096526#comment-14096526
]
Qiping Li commented on SPARK-3022:
----------------------------------
What's more, there's no need to store `feature2bins` array(which bin to put
labeled point for this feature) for every node, All nodes can reuse this every
if labeledpoint is valid on this node.This `feature2bins` array can be
precomputed before level-wise training.Each level can use this array.
> FindBinsForLevel in decision tree should call findBin only once for each
> feature
> --------------------------------------------------------------------------------
>
> Key: SPARK-3022
> URL: https://issues.apache.org/jira/browse/SPARK-3022
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 1.0.2
> Reporter: Qiping Li
> Original Estimate: 4h
> Remaining Estimate: 4h
>
> `findbinsForLevel` is applied to every `LabeledPoint` to find bins for all
> nodes at a given level. Given a specific `LabeledPoint` and a specific
> feature, the bin to put this labeled point should always be same.But in
> current implementation, `findBin` on a (labeledpoint, feature) pair is called
> for every node at a given level, which is a waste of computation. I proposed
> to call `findBin` only once and if a `LabeledPoint` is valid on a node, this
> result can be reused.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
