Re: Confusion matrix for binary classification
Hi, In Spark 1.4, you may use DataFrame.stat.crosstab to generate the confusion matrix. This would be very simple if you are using the ML Pipelines Api, and are working with DataFrames. Best, Burak On Mon, Jun 22, 2015 at 4:21 AM, CD Athuraliya cdathural...@gmail.com wrote: Hi, I am looking for a way to get confusion matrix for binary classification. I was able to get confusion matrix for multiclass classification using this [1]. But I could not find a proper way to get confusion matrix in similar class available for binary classification [2]. Later I found this class [3] which corresponds to my requirement but I am not sure about the way I should use that class to get evaluation metrics for binary classification. e.g. Given the constructor BinaryConfusionMatrixImpl(BinaryLabelCounter count, BinaryLabelCounter totalCount), from where I can get this count and totalCount? Appreciate any help on this. [1] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/MulticlassMetrics.html#confusionMatrix() [2] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.html [3] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/binary/BinaryConfusionMatrixImpl.html Thanks -- *CD Athuraliya* Software Engineer WSO2, Inc. Mobile: +94 716288847 94716288847 LinkedIn http://lk.linkedin.com/in/cdathuraliya | Twitter https://twitter.com/cdathuraliya | Blog http://cdathuraliya.tumblr.com/
Confusion matrix for binary classification
Hi, I am looking for a way to get confusion matrix for binary classification. I was able to get confusion matrix for multiclass classification using this [1]. But I could not find a proper way to get confusion matrix in similar class available for binary classification [2]. Later I found this class [3] which corresponds to my requirement but I am not sure about the way I should use that class to get evaluation metrics for binary classification. e.g. Given the constructor BinaryConfusionMatrixImpl(BinaryLabelCounter count, BinaryLabelCounter totalCount), from where I can get this count and totalCount? Appreciate any help on this. [1] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/MulticlassMetrics.html#confusionMatrix() [2] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.html [3] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/binary/BinaryConfusionMatrixImpl.html Thanks -- *CD Athuraliya* Software Engineer WSO2, Inc. Mobile: +94 716288847 94716288847 LinkedIn http://lk.linkedin.com/in/cdathuraliya | Twitter https://twitter.com/cdathuraliya | Blog http://cdathuraliya.tumblr.com/
Re: Confusion matrix for binary classification
Hi Burak, Thanks for the response. I am using Spark version 1.3.0 through Java API. Regards, CD On Tue, Jun 23, 2015 at 5:11 AM, Burak Yavuz brk...@gmail.com wrote: Hi, In Spark 1.4, you may use DataFrame.stat.crosstab to generate the confusion matrix. This would be very simple if you are using the ML Pipelines Api, and are working with DataFrames. Best, Burak On Mon, Jun 22, 2015 at 4:21 AM, CD Athuraliya cdathural...@gmail.com wrote: Hi, I am looking for a way to get confusion matrix for binary classification. I was able to get confusion matrix for multiclass classification using this [1]. But I could not find a proper way to get confusion matrix in similar class available for binary classification [2]. Later I found this class [3] which corresponds to my requirement but I am not sure about the way I should use that class to get evaluation metrics for binary classification. e.g. Given the constructor BinaryConfusionMatrixImpl(BinaryLabelCounter count, BinaryLabelCounter totalCount), from where I can get this count and totalCount? Appreciate any help on this. [1] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/MulticlassMetrics.html#confusionMatrix() [2] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/BinaryClassificationMetrics.html [3] http://spark.apache.org/docs/1.3.0/api/java/org/apache/spark/mllib/evaluation/binary/BinaryConfusionMatrixImpl.html Thanks -- *CD Athuraliya* Software Engineer WSO2, Inc. Mobile: +94 716288847 94716288847 LinkedIn http://lk.linkedin.com/in/cdathuraliya | Twitter https://twitter.com/cdathuraliya | Blog http://cdathuraliya.tumblr.com/ -- *CD Athuraliya* Software Engineer WSO2, Inc. Mobile: +94 716288847 94716288847 LinkedIn http://lk.linkedin.com/in/cdathuraliya | Twitter https://twitter.com/cdathuraliya | Blog http://cdathuraliya.tumblr.com/