I was not adding gamma to grid search. I guess setting gamma=1/n_features may not be always a good choice…
Thanks, From: Michael Eickenberg [mailto:michael.eickenb...@gmail.com] Sent: Tuesday, October 21, 2014 11:17 AM To: scikit-learn-general@lists.sourceforge.net Subject: Re: [Scikit-learn-general] SVM with rbf kernel * not necessarily memory - also calculation complexity is O(n_samples x n_samples) On Tue, Oct 21, 2014 at 5:15 PM, Michael Eickenberg <michael.eickenb...@gmail.com<mailto:michael.eickenb...@gmail.com>> wrote: Dear Roberto, On Tue, Oct 21, 2014 at 4:27 PM, Pagliari, Roberto <rpagli...@appcomsci.com<mailto:rpagli...@appcomsci.com>> wrote: From the documentation: “The implementations is a based on libsvm. The fit time complexity is more than quadratic with the number of samples which makes it hard to scale to dataset with more than a couple of 10000 samples.” Does that mean that the results may also not be very accurate with that many samples (for example, numerical issues)? It essentially means a blow-up in memory. SVM with a kernel that is not linear is evaluated in dual (kernel-) space, which involves solving a quadratic programming problem of complexity around n_samples x n_samples. While one can try to use more or less effective tricks to keep the memory usage down (e.g. using only in some way "relevant" samples), the main tendency is still there. A high number of samples is usually a trigger for people to use linear models such as the SVM with a linear kernel or logistic regression, whos complexity is then a function of the dimension of the feature space. If you are concerned with numerical stability and results with LibLinear (LinearSVM) are satisfactory, then you should try using `sklearn.svm.SVC(kernel="linear")` and check if you obtain similar results. HTH, Michael Thank you, From: Pagliari, Roberto [mailto:rpagli...@appcomsci.com<mailto:rpagli...@appcomsci.com>] Sent: Tuesday, October 21, 2014 9:39 AM To: scikit-learn-general@lists.sourceforge.net<mailto:scikit-learn-general@lists.sourceforge.net> Subject: Re: [Scikit-learn-general] SVM with rbf kernel Hi, I was asking if having lot of features might be a problem, but it shouldn’t because with quantization it works just fine (default settings). I will try tuning gamma but, again, with quantization it seems to be working just fine. Thanks, From: Michael Eickenberg [mailto:michael.eickenb...@gmail.com] Sent: Tuesday, October 21, 2014 9:32 AM To: scikit-learn-general@lists.sourceforge.net<mailto:scikit-learn-general@lists.sourceforge.net> Subject: Re: [Scikit-learn-general] SVM with rbf kernel Dear Roberto, On Tue, Oct 21, 2014 at 2:58 PM, Pagliari, Roberto <rpagli...@appcomsci.com<mailto:rpagli...@appcomsci.com>> wrote: I sometimes get weird results with SVM and rbf kernel in terms of false positive/negative rates. I suspect there may be numerical issues going on, because I’m not seeing the same issues with linearSVC. The rbf kernel might simply be overfitting. What happens if you make gamma really large? Does anyone know if rbf is constrained in terms of number of dimensions? I am not sure I understand this question. Unfortunately I cannot share the data I am using. Thank you, ------------------------------------------------------------------------------ Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net<mailto:Scikit-learn-general@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general ------------------------------------------------------------------------------ Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://p.sf.net/sfu/Zoho _______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net<mailto:Scikit-learn-general@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
------------------------------------------------------------------------------
_______________________________________________ Scikit-learn-general mailing list Scikit-learn-general@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
