Thanks for summarizing Supun. Did we think about how we gonna create the cross-model comparisons view?
On Thu, Apr 30, 2015 at 8:33 AM, Supun Sethunga <[email protected]> wrote: > [-strategy@, +architecture@] > > On Thu, Apr 30, 2015 at 5:58 PM, Srinath Perera <[email protected]> wrote: > >> should go to arch@ >> >> On Thu, Apr 30, 2015 at 6:28 AM, Srinath Perera <[email protected]> wrote: >> >>> Thanks Supun!! this looks good. >>> >>> --Srinath >>> >>> On Thu, Apr 30, 2015 at 6:25 AM, Supun Sethunga <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> Following is the break down of the Model Summary illustrations that can >>>> be supported by ML at the moment. Initiating this thread to finalize on >>>> what we can support and what cannot, with the initial release. Blue colored >>>> ones are yet to implement. >>>> >>>> - Numerical Prediction >>>> - Standard Error [1] >>>> - Residual Plot [2] >>>> - Feature Importance (*Graph containing weights assigned to each >>>> of the feature in the model*) >>>> >>>> >>>> - Classification: >>>> - Binary >>>> - ROC [3] >>>> - AUC >>>> - Confusion Matrix (*Available on spark as a static metric. >>>> But if this was calculated manually, it can be made interactive, >>>> so that >>>> user can find the optimal threshold*) >>>> - Accuracy >>>> - Feature Importance >>>> - Multi-Class >>>> - Confusion Matrix (*Available on spark*) >>>> - Accuracy >>>> - Feature Importance >>>> >>>> >>>> - Clustering >>>> - Scatter plot with clustered points >>>> >>>> >>>> *Cross-comparing Models* >>>> >>>> As you can see, major limitation we have when cross comparing models >>>> within a project is, different categories have different summary >>>> statistics/plots, and hence we cannot compare two models in two categories. >>>> >>>> Following are the possibilities: >>>> >>>> - ROC can be used to compare Binary classification models. >>>> - Cobweb (a radar chart) can be used to compare Multi-Class >>>> classification models (This is the possible alternative for ROC in >>>> multi-class case. But the drawback is, the graph will be very unclear >>>> when >>>> there are excess amounts of features in the models). [4] [5] >>>> - Accuracy can be used to compare all classification models. >>>> >>>> Please add if I've missed anything. >>>> >>>> *Ref:* >>>> [1] http://onlinestatbook.com/2/regression/accuracy.html >>>> [2] http://stattrek.com/regression/residual-analysis.aspx >>>> [3] http://www.sciencedirect.com/science/article/pii/S016786550500303X >>>> [4] >>>> http://www.academia.edu/2519022/Visualization_and_analysis_of_classifiers_performance_in_multi-class_medical_data >>>> [5] >>>> http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.107.8450&rep=rep1&type=pdf >>>> >>>> >>>> Thanks, >>>> Supun >>>> >>>> -- >>>> *Supun Sethunga* >>>> Software Engineer >>>> WSO2, Inc. >>>> http://wso2.com/ >>>> lean | enterprise | middleware >>>> Mobile : +94 716546324 >>>> >>> >>> >>> >>> -- >>> ============================ >>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>> Site: http://people.apache.org/~hemapani/ >>> Photos: http://www.flickr.com/photos/hemapani/ >>> Phone: 0772360902 >>> >> >> >> >> -- >> ============================ >> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >> Site: http://people.apache.org/~hemapani/ >> Photos: http://www.flickr.com/photos/hemapani/ >> Phone: 0772360902 >> > > > > -- > *Supun Sethunga* > Software Engineer > WSO2, Inc. > http://wso2.com/ > lean | enterprise | middleware > Mobile : +94 716546324 > -- Thanks & regards, Nirmal Associate Technical Lead - Data Technologies Team, WSO2 Inc. Mobile: +94715779733 Blog: http://nirmalfdo.blogspot.com/
_______________________________________________ Architecture mailing list [email protected] https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
