GitHub user emtl97 opened a pull request:
https://github.com/apache/spark/pull/3609
Update DecisionTree.scala
Hello,
Hope you are well. We've been using DecisionTree at Samsung and hope to
help in some small way. I was interested in setting the seed for the sampling
i.e. in line 988. We're in the process of creating tests for our code being
able to set the seed is helpful.
To that end, I think also the sample method here depends on a
PartitionwiseSampledRDD. In there the 'compute' method think uses a different
seed from the one that can be passed to the constructor of
PartitionSampledRDD, it uses split.seed (below). Well hope we can discuss
more! Thank you.. Best Wishes -Ed
override def compute(splitIn: Partition, context: TaskContext): Iterator[U]
= {
val split = splitIn.asInstanceOf[PartitionwiseSampledRDDPartition]
val thisSampler = sampler.clone
thisSampler.setSeed(split.seed)
thisSampler.sample(firstParent[T].iterator(split.prev, context))
}
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/emtl97/spark patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3609.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3609
----
commit b2f6926cc5d4e5020b6811f6952101d6882877e1
Author: Ed <[email protected]>
Date: 2014-12-04T21:11:41Z
Update DecisionTree.scala
Hello,
Hope you are well. We've been using DecisionTree at Samsung and hope to
help in some small way. I was interested in setting the seed for the sampling
i.e. in line 988. We're in the process of creating tests for our code being
able to set the seed is helpful.
To that end, I think also the sample method here depends on a
PartitionwiseSampledRDD. In there the 'compute' method think uses a different
seed from the one that can be passed to the constructor of
PartitionSampledRDD, it uses split.seed (below). Well hope we can discuss
more! Thank you.. Best Wishes -Ed
override def compute(splitIn: Partition, context: TaskContext): Iterator[U]
= {
val split = splitIn.asInstanceOf[PartitionwiseSampledRDDPartition]
val thisSampler = sampler.clone
thisSampler.setSeed(split.seed)
thisSampler.sample(firstParent[T].iterator(split.prev, context))
}
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
