Github user jkbradley commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20904#discussion_r180228711
  
    --- Diff: python/pyspark/ml/stat.py ---
    @@ -127,13 +113,86 @@ class Correlation(object):
         def corr(dataset, column, method="pearson"):
             """
             Compute the correlation matrix with specified method using dataset.
    +
    +        :param dataset:
    +          A Dataset or a DataFrame.
    +        :param column:
    +          The name of the column of vectors for which the correlation 
coefficient needs
    +          to be computed. This must be a column of the dataset, and it 
must contain
    +          Vector objects.
    +        :param method:
    +          String specifying the method to use for computing correlation.
    +          Supported: `pearson` (default), `spearman`.
    +        :return:
    +          A DataFrame that contains the correlation matrix of the column 
of vectors. This
    +          DataFrame contains a single row and a single column of name
    +          '$METHODNAME($COLUMN)'.
             """
             sc = SparkContext._active_spark_context
             javaCorrObj = _jvm().org.apache.spark.ml.stat.Correlation
             args = [_py2java(sc, arg) for arg in (dataset, column, method)]
             return _java2py(sc, javaCorrObj.corr(*args))
     
     
    +class KolmogorovSmirnovTest(object):
    +    """
    +    .. note:: Experimental
    +
    +    Conduct the two-sided Kolmogorov Smirnov (KS) test for data sampled 
from a continuous
    +    distribution.
    +
    +    By comparing the largest difference between the empirical cumulative
    +    distribution of the sample data and the theoretical distribution we 
can provide a test for the
    +    the null hypothesis that the sample data comes from that theoretical 
distribution.
    +
    +    >>> from pyspark.ml.stat import KolmogorovSmirnovTest
    --- End diff --
    
    Thanks for moving the method-specific documentation.  These doctests are 
method-specific too, though, so can you please move them as well?


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to