[
https://issues.apache.org/jira/browse/MAHOUT-962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13728062#comment-13728062
]
Liz Merkhofer commented on MAHOUT-962:
--------------------------------------
Thought this was broken in 0.8 because my dictionary file wasn't getting
smaller until I saw this Stack Overflow post - words are still saved so
vectors/dictionary are not shorter, but 0's are stored.
http://stackoverflow.com/questions/12537089/seq2sparse-seems-to-be-ignoring-the-value-of-my-x-parameter
In case it saves someone the troubleshooting!
> minDF and maxDFPercent filtering doesnt get applied when output weight is tf
> in SpareVecorsFromSequenceFile
> -----------------------------------------------------------------------------------------------------------
>
> Key: MAHOUT-962
> URL: https://issues.apache.org/jira/browse/MAHOUT-962
> Project: Mahout
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 0.6, 0.7, 0.8
> Reporter: John Conwell
> Assignee: Robin Anil
> Priority: Minor
> Labels: patch
> Fix For: 0.8
>
> Attachments: mahout_962.patch
>
>
> This is similar to the same reasoning behind the fix for MAHOUT-957. The
> desired output is term frequency vectors, but I want terms filtered by their
> min and max DF values. This might be valid in LDA, where tf vectors is
> desired for input, but filtering out the maxDFPercent is also useful.
> Currently minDF and maxDFPercent are only used when calculating tfidf, and
> the original tv vectors are not updated to represent the term filtering.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
