[jira] [Commented] (SPARK-2335) k-Nearest Neighbor classification and regression for MLLib
[ https://issues.apache.org/jira/browse/SPARK-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14213849#comment-14213849 ] Kaushik Ranjan commented on SPARK-2335: --- Ha ha. "Shepherd" and I were working on this together. [~bgawalt] - if you could review the code and suggest changes(if any), I can take it forward > k-Nearest Neighbor classification and regression for MLLib > -- > > Key: SPARK-2335 > URL: https://issues.apache.org/jira/browse/SPARK-2335 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Brian Gawalt >Priority: Minor > Labels: features > > The k-Nearest Neighbor model for classification and regression problems is a > simple and intuitive approach, offering a straightforward path to creating > non-linear decision/estimation contours. It's downsides -- high variance > (sensitivity to the known training data set) and computational intensity for > estimating new point labels -- both play to Spark's big data strengths: lots > of data mitigates data concerns; lots of workers mitigate computational > latency. > We should include kNN models as options in MLLib. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Issue Comment Deleted] (SPARK-2335) k-Nearest Neighbor classification and regression for MLLib
[ https://issues.apache.org/jira/browse/SPARK-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaushik Ranjan updated SPARK-2335: -- Comment: was deleted (was: Hi [~bgawalt]. For evaluation of KNN-join, one needs to calculate z-scores of data-points within the dataset. Yu-ISHIKAWA has implemented the following https://gist.github.com/yu-iskw/37ae208c530f7018e048 Will it be justified to put up a NewFeature Issue to address z-scores? ) > k-Nearest Neighbor classification and regression for MLLib > -- > > Key: SPARK-2335 > URL: https://issues.apache.org/jira/browse/SPARK-2335 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Brian Gawalt >Priority: Minor > Labels: features > > The k-Nearest Neighbor model for classification and regression problems is a > simple and intuitive approach, offering a straightforward path to creating > non-linear decision/estimation contours. It's downsides -- high variance > (sensitivity to the known training data set) and computational intensity for > estimating new point labels -- both play to Spark's big data strengths: lots > of data mitigates data concerns; lots of workers mitigate computational > latency. > We should include kNN models as options in MLLib. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2335) k-Nearest Neighbor classification and regression for MLLib
[ https://issues.apache.org/jira/browse/SPARK-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14213380#comment-14213380 ] Kaushik Ranjan commented on SPARK-2335: --- Hi [~bgawalt], [~yu_ishikawa] PFA the code for knnJoin https://github.com/kaushikranjan/knnJoin It's in accordance to Algorithm1 from the paper cited below http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=5447837&tag=1 Regards, [~Kaushik619] [~Rusty] > k-Nearest Neighbor classification and regression for MLLib > -- > > Key: SPARK-2335 > URL: https://issues.apache.org/jira/browse/SPARK-2335 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Brian Gawalt >Priority: Minor > Labels: features > > The k-Nearest Neighbor model for classification and regression problems is a > simple and intuitive approach, offering a straightforward path to creating > non-linear decision/estimation contours. It's downsides -- high variance > (sensitivity to the known training data set) and computational intensity for > estimating new point labels -- both play to Spark's big data strengths: lots > of data mitigates data concerns; lots of workers mitigate computational > latency. > We should include kNN models as options in MLLib. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-2335) k-Nearest Neighbor classification and regression for MLLib
[ https://issues.apache.org/jira/browse/SPARK-2335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14186954#comment-14186954 ] Kaushik Ranjan commented on SPARK-2335: --- Hi [~bgawalt]. For evaluation of KNN-join, one needs to calculate z-scores of data-points within the dataset. Yu-ISHIKAWA has implemented the following https://gist.github.com/yu-iskw/37ae208c530f7018e048 Will it be justified to put up a NewFeature Issue to address z-scores? > k-Nearest Neighbor classification and regression for MLLib > -- > > Key: SPARK-2335 > URL: https://issues.apache.org/jira/browse/SPARK-2335 > Project: Spark > Issue Type: New Feature > Components: MLlib >Reporter: Brian Gawalt >Priority: Minor > Labels: features, newbie > > The k-Nearest Neighbor model for classification and regression problems is a > simple and intuitive approach, offering a straightforward path to creating > non-linear decision/estimation contours. It's downsides -- high variance > (sensitivity to the known training data set) and computational intensity for > estimating new point labels -- both play to Spark's big data strengths: lots > of data mitigates data concerns; lots of workers mitigate computational > latency. > We should include kNN models as options in MLLib. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
