[
https://issues.apache.org/jira/browse/MADLIB-933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15142116#comment-15142116
]
Frank McQuillan commented on MADLIB-933:
----------------------------------------
The following works OK with diff names than docid and words :
{code}
DROP TABLE IF EXISTS documents, documents_tf, documents_tf_vocabulary;
CREATE TABLE documents(docidd INTEGER, doc_contents TEXT);
INSERT INTO documents VALUES
(1, 'I like to eat broccoli and banana. I ate a banana and spinach smoothie for
breakfast.'),
(2, 'Chinchillas and kittens are cute.'),
(3, 'My sister adopted two kittens yesterday.'),
(4, 'Look at this cute hamster munching on a piece of broccoli.');
ALTER TABLE documents ADD COLUMN wordss TEXT[];
UPDATE documents SET wordss = regexp_split_to_array(lower(doc_contents),
E'[\\s+\\.]');
SELECT madlib.term_frequency('documents', 'docidd', 'wordss', 'documents_tf',
TRUE);
SELECT * FROM documents_tf_vocabulary;
{code}
Also the output vocabulary table is not a temp table anymore
> MADlib LDA term_frequency function bugs
> ---------------------------------------
>
> Key: MADLIB-933
> URL: https://issues.apache.org/jira/browse/MADLIB-933
> Project: Apache MADlib
> Issue Type: Bug
> Components: Module: Parallel Latent Dirichlet Allocation
> Reporter: Srivatsan
> Assignee: Rahul Iyer
> Fix For: v1.9
>
>
> 1. madlib.term_frequency() function
> (http://doc.madlib.net/latest/group__grp__text__utilities.html) takes the
> docid column and words columns as inputs, but this just fools us into
> thinking that we could name our columns as whatever we want, coz it complains
> if the columns are not actually named "docid" and "words"!
> 2. Secondly, it takes an output table as well as input (ex: documents_tf),
> but it creates a temp table for the vocabulary (therefore i can't specify a
> schema name like vatsan.documents_tf). This is annoying for two reasons
> a. The user can't immediately senses what's with the vocabulary table and why
> is it a temp table while the documents_tf table itself is not.
> b. If i have a real world dataset for LDA, my models are going to run for
> quite sometime. I may even terminate one session and run the LDA model in
> another session, this would mean the vocabulary temp table won't be available
> in the other session (or would have gotten dropped)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
