* not necessarily memory - also calculation complexity is O(n_samples x
n_samples)
On Tue, Oct 21, 2014 at 5:15 PM, Michael Eickenberg <
michael.eickenb...@gmail.com> wrote:
> Dear Roberto,
>
> On Tue, Oct 21, 2014 at 4:27 PM, Pagliari, Roberto <
> rpagli...@appcomsci.com> wrote:
>
>> From the documentation:
>>
>>
>>
>> *“The implementations is a based on libsvm. The fit time complexity is
>> more than quadratic with the number of samples which makes it hard to scale
>> to dataset with more than a couple of 10000 samples.”*
>>
>>
>>
>> Does that mean that the results may also not be very accurate with that
>> many samples (for example, numerical issues)?
>>
>>
>>
>
> It essentially means a blow-up in memory. SVM with a kernel that is not
> linear is evaluated in dual (kernel-) space, which involves solving a
> quadratic programming problem of complexity around n_samples x n_samples.
> While one can try to use more or less effective tricks to keep the memory
> usage down (e.g. using only in some way "relevant" samples), the main
> tendency is still there.
>
> A high number of samples is usually a trigger for people to use linear
> models such as the SVM with a linear kernel or logistic regression, whos
> complexity is then a function of the dimension of the feature space.
>
> If you are concerned with numerical stability and results with LibLinear
> (LinearSVM) are satisfactory, then you should try using
> `sklearn.svm.SVC(kernel="linear")` and check if you obtain similar results.
>
> HTH,
> Michael
>
>
>>
>>
>> Thank you,
>>
>>
>>
>> *From:* Pagliari, Roberto [mailto:rpagli...@appcomsci.com]
>> *Sent:* Tuesday, October 21, 2014 9:39 AM
>>
>> *To:* scikit-learn-general@lists.sourceforge.net
>> *Subject:* Re: [Scikit-learn-general] SVM with rbf kernel
>>
>>
>>
>> Hi,
>>
>> I was asking if having lot of features might be a problem, but it
>> shouldn’t because with quantization it works just fine (default settings).
>>
>>
>>
>> I will try tuning gamma but, again, with quantization it seems to be
>> working just fine.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> *From:* Michael Eickenberg [mailto:michael.eickenb...@gmail.com
>> <michael.eickenb...@gmail.com>]
>> *Sent:* Tuesday, October 21, 2014 9:32 AM
>> *To:* scikit-learn-general@lists.sourceforge.net
>> *Subject:* Re: [Scikit-learn-general] SVM with rbf kernel
>>
>>
>>
>> Dear Roberto,
>>
>>
>>
>> On Tue, Oct 21, 2014 at 2:58 PM, Pagliari, Roberto <
>> rpagli...@appcomsci.com> wrote:
>>
>> I sometimes get weird results with SVM and rbf kernel in terms of false
>> positive/negative rates.
>>
>>
>>
>> I suspect there may be numerical issues going on, because I’m not seeing
>> the same issues with linearSVC.
>>
>>
>>
>>
>>
>> The rbf kernel might simply be overfitting. What happens if you make
>> gamma really large?
>>
>>
>>
>> Does anyone know if rbf is constrained in terms of number of dimensions?
>>
>>
>>
>> I am not sure I understand this question.
>>
>>
>>
>>
>>
>> Unfortunately I cannot share the data I am using.
>>
>>
>>
>> Thank you,
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Comprehensive Server Monitoring with Site24x7.
>> Monitor 10 servers for $9/Month.
>> Get alerted through email, SMS, voice calls or mobile push notifications.
>> Take corrective actions from your mobile device.
>> http://p.sf.net/sfu/Zoho
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Comprehensive Server Monitoring with Site24x7.
>> Monitor 10 servers for $9/Month.
>> Get alerted through email, SMS, voice calls or mobile push notifications.
>> Take corrective actions from your mobile device.
>> http://p.sf.net/sfu/Zoho
>> _______________________________________________
>> Scikit-learn-general mailing list
>> Scikit-learn-general@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>>
>>
>
------------------------------------------------------------------------------
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
