Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/13176#discussion_r64273057
--- Diff: docs/ml-features.md ---
@@ -1092,14 +1097,11 @@ for more details on the API.
## QuantileDiscretizer
`QuantileDiscretizer` takes a column with continuous features and outputs
a column with binned
-categorical features.
-The bin ranges are chosen by taking a sample of the data and dividing it
into roughly equal parts.
-The lower and upper bin bounds will be `-Infinity` and `+Infinity`,
covering all real values.
-This attempts to find `numBuckets` partitions based on a sample of the
given input data, but it may
-find fewer depending on the data sample values.
-
-Note that the result may be different every time you run it, since the
sample strategy behind it is
-non-deterministic.
+categorical features. The number of bins is set by the `numBuckets`
parameter.
+The bin ranges are chosen using an approximate algorithm (see the
documentation for
[approxQuantile](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala)
--- End diff --
The link here should actually link to the generated HTML Scala API doc
(since I don't think there is a section for this function in the user guide).
Here's an
[example](https://github.com/apache/spark/pull/13176/files#diff-56b7fe109d2c152329452f7da634fb3fR1074)
from the example code links above.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
