Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/7244#discussion_r34114829
--- Diff: docs/ml-features.md ---
@@ -288,6 +288,88 @@ for words_label in wordsDataFrame.select("words",
"label").take(3):
</div>
+## $n$-gram
+
+An [n-gram](https://en.wikipedia.org/wiki/N-gram) is a sequence of $n$
tokens (typically words) for some integer $n$. The `NGram` class can be used to
transform input features into $n$-grams.
+
+`NGram` takes as input a sequence of strings (e.g. the output of a
[Tokenizer](api/scala/index.html#org.apache.spark.ml.feature.Tokenizer)). The
parameter `n` is used to determine the number of terms in each $n$-gram. The
output will consist of a sequence of $n$-grams where each $n$-gram is
represented by a space-delimited string of $n$ consecutive words. If the input
sequence contains fewer than `n` strings, no output is produced.
+
+<div class="codetabs">
+<div data-lang="scala" markdown="1">
+<div class="codetabs">
+
+<div data-lang="scala" markdown="1">
+
+[`NGram`](api/scala/index.html#org.apache.spark.ml.feature.NGram) takes an
input column name, an output column name, and an optional length parameter n
(n=2 by default).
+
+{% highlight scala %}
+import org.apache.spark.ml.feature.NGram
+
+val wordDataFrame = sqlContext.createDataFrame(Seq(
+ (0, Array("Hi", "I", "heard", "about", "Spark")),
+ (1, Array("I", "wish", "Java", "could", "use", "case", "classes")),
+ (2, Array("Logistic", "regression", "models", "are", "neat"))
+)).toDF("label", "words")
+
+val ngram = new NGram().setInputCol("words").setOutputCol("ngrams")
+val ngramDataFrame = ngram.transform(wordDataFrame)
+ngramDataFrame.select("ngrams", "label").take(3).foreach(println)
--- End diff --
If this looks weird, could you please modify it to be prettier?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
