Hi, Suranga,
> 1) The standard deviation in the precision of the two models (obtained using
> precision.std()) is always 0.0. I'm assuming that's a problem.
That’s weird. You are sure that “precision” has more than one value? E.g.,
>>> np.array([0.89]).std()
0.0
> 2) My boxplot is meant to display bars for the two models, but always
> displays only the first model (nn01)
Also here, your input array or list for the boxplot function may not be
formatted correctly. What you want is
two_models = [ 1Darray_of_model1_results, 1Darray_of_model2_results ]
plt.boxplot(two_models,
notch=False, # box instead of notch shape
sym='rs', # red squares for outliers
vert=True) # vertical box aligmnent
PS: If you are comparing specifically 2 neural network models, have you
considered McNemar’s test? E.g., see
https://github.com/rasbt/mlxtend/blob/master/docs/sources/user_guide/evaluate/mcnemar.ipynb
Best
Sebastian
> On Oct 30, 2016, at 3:24 PM, Suranga Kasthurirathne <[email protected]>
> wrote:
>
>
> Hi folks!
>
> I'm using scikit-learn to build two neural networks using 10% holdout, and
> compare their performance using precision. To compare statistical
> significance in the variance of precision, i'm using scikit's boxplots.
>
> My problem is twofold -
>
> 1) The standard deviation in the precision of the two models (obtained using
> precision.std()) is always 0.0. I'm assuming that's a problem.
> 2) My boxplot is meant to display bars for the two models, but always
> displays only the first model (nn01)
>
> My outcomes for this dataset is binary (0 or 1) since the models assume
> average=binary by default, is that a problem?
>
> For those who'd like to look, my source code can be seen at
> http://pastebin.com/yvE2T1Sw
>
> The code produces the following plot - which is of course only ONE of the
> bars that I need :(
>
>
> <Screen Shot 2016-10-30 at 12.17.22 PM.png>
>
>
> --
> Best Regards,
> Suranga
> _______________________________________________
> scikit-learn mailing list
> [email protected]
> https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________
scikit-learn mailing list
[email protected]
https://mail.python.org/mailman/listinfo/scikit-learn
