[
https://issues.apache.org/jira/browse/HIVE-1518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12898421#action_12898421
]
HBase Review Board commented on HIVE-1518:
------------------------------------------
Message from: "John Sichi" <[email protected]>
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
http://review.cloudera.org/r/644/
-----------------------------------------------------------
Review request for Hive Developers.
Summary
-------
review by JVS
This addresses bug HIVE-1518.
http://issues.apache.org/jira/browse/HIVE-1518
Diffs
-----
http://svn.apache.org/repos/asf/hadoop/hive/trunk/data/files/text-en.txt
985013
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
985013
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFContextNGrams.java
PRE-CREATION
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFnGrams.java
985013
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/NGramEstimator.java
PRE-CREATION
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/test/queries/clientpositive/udaf_context_ngrams.q
PRE-CREATION
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/test/queries/clientpositive/udaf_ngrams.q
985013
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out
985013
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/test/results/clientpositive/udaf_context_ngrams.q.out
PRE-CREATION
http://svn.apache.org/repos/asf/hadoop/hive/trunk/ql/src/test/results/clientpositive/udaf_ngrams.q.out
985013
Diff: http://review.cloudera.org/r/644/diff
Testing
-------
Thanks,
John
> context_ngrams() UDAF for estimating top-k contextual n-grams
> -------------------------------------------------------------
>
> Key: HIVE-1518
> URL: https://issues.apache.org/jira/browse/HIVE-1518
> Project: Hadoop Hive
> Issue Type: New Feature
> Components: Query Processor
> Affects Versions: 0.6.0
> Reporter: Mayank Lahiri
> Assignee: Mayank Lahiri
> Fix For: 0.7.0
>
> Attachments: HIVE-1518.1.patch, HIVE-1518.2.patch
>
>
> Create a new context_ngrams() function that generalizes the ngrams() UDAF to
> allow the user to specify context around n-grams. The analogy is
> "fill-in-the-blanks", and is best illustrated with an example:
> SELECT context_ngrams(sentences(tweets), array("i", "love", null), 300) FROM
> twitter;
> will estimate the top-300 words that follow the phrase "i love" in a database
> of tweets. The position of the null(s) specifies where to generate the n-gram
> from, and can be placed anywhere. For example:
> SELECT context_ngrams(sentences(tweets), array("i", "love", null, "but",
> "hate", null), 300) FROM twitter;
> will estimate the top-300 word-pairs that fill in the blanks specified by
> null.
> POSSIBLE USES:
> 1. Pre-computing search lookaheads
> 2. Sentiment analysis for products or entities -- e.g., querying with context
> = array("twitter", "is", null)
> 3. Navigation path analysis in URL databases
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
