Github user MLnick commented on a diff in the pull request:
https://github.com/apache/spark/pull/13176#discussion_r64273246
--- Diff: docs/ml-features.md ---
@@ -1092,14 +1097,11 @@ for more details on the API.
## QuantileDiscretizer
`QuantileDiscretizer` takes a column with continuous features and outputs
a column with binned
-categorical features.
-The bin ranges are chosen by taking a sample of the data and dividing it
into roughly equal parts.
-The lower and upper bin bounds will be `-Infinity` and `+Infinity`,
covering all real values.
-This attempts to find `numBuckets` partitions based on a sample of the
given input data, but it may
-find fewer depending on the data sample values.
-
-Note that the result may be different every time you run it, since the
sample strategy behind it is
-non-deterministic.
+categorical features. The number of bins is set by the `numBuckets`
parameter.
+The bin ranges are chosen using an approximate algorithm (see the
documentation for
[approxQuantile](https://github.com/apache/spark/blob/master/sql/core/src/main/scala/org/apache/spark/sql/DataFrameStatFunctions.scala)
+for a detailed description). The precision of the approximation can be
controlled with the
+`relativeError` parameter. When set to zero, exact quantiles are
calculated.
--- End diff --
We should add a note that computing exact quantiles can be very expensive.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
