Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r82027114
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r82027003
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r82027065
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r82027088
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r82026834
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,334 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r82018962
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695300
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695352
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/RandomProjection.scala ---
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695361
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/RandomProjection.scala ---
@@ -0,0 +1,96 @@
+/*
+ * Licensed to the Apache Software
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695246
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695236
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695252
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695220
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695192
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695225
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695206
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695145
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695186
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695166
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695209
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695178
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695128
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695106
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695109
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695117
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695070
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695026
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695062
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81695079
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81694970
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,338 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on the issue:
https://github.com/apache/spark/pull/15148
@jkbradley I see. Thanks Joseph!
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81031684
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/LSH.scala ---
@@ -0,0 +1,322 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on the issue:
https://github.com/apache/spark/pull/15148
Hi @MLnick @jkbradley
Thanks for the code review. I made some changes based on your comments.
- I agree it's better to align the input types to vector in internal
implementation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000695
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/lsh/LSHTest.scala ---
@@ -0,0 +1,125 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000597
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/lsh/MinHash.scala ---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000578
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/lsh/MinHash.scala ---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000599
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/lsh/RandomProjection.scala ---
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000707
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/lsh/LSHTest.scala ---
@@ -0,0 +1,125 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000698
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/lsh/LSHTest.scala ---
@@ -0,0 +1,125 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000563
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,304 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000669
--- Diff:
mllib/src/test/scala/org/apache/spark/ml/feature/lsh/LSHTest.scala ---
@@ -0,0 +1,125 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000659
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/lsh/RandomProjection.scala ---
@@ -0,0 +1,88 @@
+/*
+ * Licensed to the Apache Software
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000584
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/lsh/MinHash.scala ---
@@ -0,0 +1,94 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000555
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,304 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000488
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,290 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000523
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,304 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000528
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,304 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000516
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,304 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000470
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,290 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000455
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,304 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r81000447
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,304 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on the issue:
https://github.com/apache/spark/pull/15148
Thanks @karlhigley All of your comments are very helpful. I made some
changes to make it work. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r80411490
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,290 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r80411374
--- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala ---
@@ -0,0 +1,290 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on the issue:
https://github.com/apache/spark/pull/15148
Hi @sethah
- My understanding is h(x) = floor((g1 dot x) / w) is one hash function, as
is in the wiki.
- In bulletpoint 6 of "Approach found on Wikipedia and here and here"
Github user Yunni commented on the issue:
https://github.com/apache/spark/pull/15148
Hi @sethah,
Thanks for the comments.
- I agree. I have moved `lsh` package to be under `feature`
- In "Similarity search in high dimensions via hashing", there is an
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79639164
--- Diff: mllib/src/main/scala/org/apache/spark/ml/lsh/LSH.scala ---
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Yunni commented on the issue:
https://github.com/apache/spark/pull/15148
Hi @sethah, I have updated the reference in the PR and scaladoc for LSH.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user Yunni commented on the issue:
https://github.com/apache/spark/pull/15148
Thanks very much for reviewing @viirya I made some changes based on your
comments. PTAL.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79505554
--- Diff: mllib/src/test/scala/org/apache/spark/ml/lsh/LSHTest.scala ---
@@ -0,0 +1,124 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79505544
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/lsh/RandomProjection.scala ---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79505534
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/lsh/RandomProjection.scala ---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79505523
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/lsh/RandomProjection.scala ---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79505486
--- Diff: mllib/src/main/scala/org/apache/spark/ml/lsh/LSH.scala ---
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79505439
--- Diff: mllib/src/main/scala/org/apache/spark/ml/lsh/LSH.scala ---
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79505085
--- Diff: mllib/src/main/scala/org/apache/spark/ml/lsh/LSH.scala ---
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79505015
--- Diff: mllib/src/main/scala/org/apache/spark/ml/lsh/LSH.scala ---
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
Github user Yunni commented on a diff in the pull request:
https://github.com/apache/spark/pull/15148#discussion_r79504728
--- Diff: mllib/src/main/scala/org/apache/spark/ml/lsh/LSH.scala ---
@@ -0,0 +1,270 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under
GitHub user Yunni opened a pull request:
https://github.com/apache/spark/pull/15148
Spark 5992 yunn lsh
## What changes were proposed in this pull request?
Implement Locality Sensitive Hashing along with approximate nearest
neighbors and approximate similarity join based
201 - 269 of 269 matches
Mail list logo
