Github user jkbradley commented on a diff in the pull request:
https://github.com/apache/spark/pull/7244#discussion_r34003748
--- Diff: docs/ml-features.md ---
@@ -288,6 +288,82 @@ for words_label in wordsDataFrame.select("words",
"label").take(3):
</div>
+## $n$-gram
+
+An [n-gram](https://en.wikipedia.org/wiki/N-gram) is a sequence of $n$
tokens (typically words) for some integer $n$. The
[NGram](api/scala/index.html#org.apache.spark.ml.feature.NGram) class can be
used to transform input features into $n$-grams.
+
+`NGram` takes as input a sequence of strings (e.g. the output of a
[Tokenizer](api/scala/index.html#org.apache.spark.ml.feature.Tokenizer)). The
parameter `n` is used to determine the number of terms in each $n$-gram. The
output will consist of a sequence of $n$-grams where each $n$-gram is
represented by a space-delimited string of $n$ consecutive words. If the input
sequence contains less than `n` strings, no output is produced.
+
+<div class="codetabs">
--- End diff --
To confirm, have you run the Scala & Python examples in the shells, and
tested the Java example in a test class? If needed, you can add the Java test
class as a unit test (modified not to print anything).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
