[
https://issues.apache.org/jira/browse/MAHOUT-344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12916579#action_12916579
]
Ted Dunning commented on MAHOUT-344:
------------------------------------
Sorry to chime in late,
{quote}
>Random - java.util.Random
Each mapper should get the exact copy of hash functions for constructing the
minhash signatures or else the chances of collision are quite less even for
highly similar items. that is the reason for hard-coding seed for Random(). The
reason for test failure is not that, it is the linear hash function that has a
higher false positive rate than murmur and polynomial hash functions. The
remedy is to concatenate more hashes for a key groups. For
testLinearMinHashMRJob() I changed number of hash functions to 20 and number of
key groups to 4 and now test passes successfully.
{quote}
This sounds like we should be using murmurHash instead. It is almost as fast a
j.u.Random and has MUCH better properties. This kind of change is fine to
defer to 0.5.
> Minhash based clustering
> -------------------------
>
> Key: MAHOUT-344
> URL: https://issues.apache.org/jira/browse/MAHOUT-344
> Project: Mahout
> Issue Type: Bug
> Components: Clustering
> Affects Versions: 0.3
> Reporter: Ankur
> Assignee: Ankur
> Fix For: 0.4
>
> Attachments: MAHOUT-344-v1.patch, MAHOUT-344-v2.patch,
> MAHOUT-344-v3.patch, MAHOUT-344-v4.patch, MAHOUT-344-v5.patch,
> MAHOUT-344-v6.patch, MAHOUT-344-v7.patch
>
>
> Minhash clustering performs probabilistic dimension reduction of high
> dimensional data. The essence of the technique is to hash each item using
> multiple independent hash functions such that the probability of collision of
> similar items is higher. Multiple such hash tables can then be constructed
> to answer near neighbor type of queries efficiently.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
