Github user jkbradley commented on the issue:
https://github.com/apache/spark/pull/15148
@sethah
> What is the intended use of the output column generated by transform? As
an alternative set of features with decreased dimensionality?
I agree it's mainly for dimensionality reduction, though these LSH
functions are not ideal for that. (E.g., most people doing dimensionality
reduction would probably want to use random projections without bucketing.)
@karlhigley
I agree with your description of different dimensionalities and agree we
may just have to pick some terminology out of many choices. I'm fairly
ambivalent about what terminology we choose, though it would be great for it to
match whatever references we cite. (And maybe we do need another reference
cited for describing OR vs AND amplification and "dimensions.")
@Yunni
* Have you seen "HyperplaneProjection" used in literature?
* I'll respond about the hashDistance in
[https://github.com/apache/spark/pull/15800]
* Let's not implement both types of amplification just yet. Let's either:
* Fix the API so we can add them in the future, or
* Make LSH private for now so that we can change fix its API for 2.2.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
