Repository: spark Updated Branches: refs/heads/master 2f6fd5256 -> 91a577d27
[SPARK-10249] [ML] [DOC] Add Python Code Example to StopWordsRemover User Guide jira: https://issues.apache.org/jira/browse/SPARK-10249 update user guide since python support added. Author: Yuhao Yang <hhb...@gmail.com> Closes #8620 from hhbyyh/swPyDocExample. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/91a577d2 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/91a577d2 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/91a577d2 Branch: refs/heads/master Commit: 91a577d2778ab5946f0c40cb80c89de24e3d10e8 Parents: 2f6fd52 Author: Yuhao Yang <hhb...@gmail.com> Authored: Tue Sep 8 22:33:23 2015 -0700 Committer: Xiangrui Meng <m...@databricks.com> Committed: Tue Sep 8 22:33:23 2015 -0700 ---------------------------------------------------------------------- docs/ml-features.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/91a577d2/docs/ml-features.md ---------------------------------------------------------------------- diff --git a/docs/ml-features.md b/docs/ml-features.md index 90654d1..58b31a5 100644 --- a/docs/ml-features.md +++ b/docs/ml-features.md @@ -512,6 +512,25 @@ DataFrame dataset = jsql.createDataFrame(rdd, schema); remover.transform(dataset).show(); {% endhighlight %} </div> + +<div data-lang="python" markdown="1"> +[`StopWordsRemover`](api/python/pyspark.ml.html#pyspark.ml.feature.StopWordsRemover) +takes an input column name, an output column name, a list of stop words, +and a boolean indicating if the matches should be case sensitive (false +by default). + +{% highlight python %} +from pyspark.ml.feature import StopWordsRemover + +sentenceData = sqlContext.createDataFrame([ + (0, ["I", "saw", "the", "red", "baloon"]), + (1, ["Mary", "had", "a", "little", "lamb"]) +], ["label", "raw"]) + +remover = StopWordsRemover(inputCol="raw", outputCol="filtered") +remover.transform(sentenceData).show(truncate=False) +{% endhighlight %} +</div> </div> ## $n$-gram --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
