Re: [R] Inconsistent results between caret+kernlab versions

2013-11-24 Thread Andrew Digby
Hi Max, Here's a bit more information regarding the 'memory not mapped' errors which occur in caret. 1. The segfault only occurs when knitting a Markdown file in RStudio. When the code is run 'normally' in R, everything's fine. 2. The error is very hard to replicate! It only occurs when the

Re: [R] Inconsistent results between caret+kernlab versions

2013-11-17 Thread Andrew Digby
Hi Max, Thanks very much for investigating and explaining that - your help and time is much appreciated. So as I understand it, using classProbs=F in trainControl() will give me the same accuracy results as before. However, I was relying on the class probabilities to return

Re: [R] Inconsistent results between caret+kernlab versions

2013-11-17 Thread Max Kuhn
Andrew, What I still don't quite understand is which accuracy values from train() I should trust: those using classProbs=T or classProbs=F? It depends on whether you need the class probabilities and class predictions to match (which they would if classProbs = TRUE). Another option is to use

Re: [R] Inconsistent results between caret+kernlab versions

2013-11-17 Thread Andrew Digby
OK, thanks. I haven't reported the memory map errors because I haven't been able to replicate them reliably: some times they occur, but some times don't, for the same code. I'll have another try, and will report if I can get more information. Thanks again. On 18/11/2013, at 14:42 , Max Kuhn

Re: [R] Inconsistent results between caret+kernlab versions

2013-11-15 Thread Max Kuhn
Or not! The issue with with kernlab. Background: SVM models do not naturally produce class probabilities. A secondary model (via Platt) is fit to the raw model output and a logistic function is used to translate the raw SVM output to probability-like numbers (i.e. sum to zero, between 0 and 1).

[R] Inconsistent results between caret+kernlab versions

2013-11-14 Thread Andrew Digby
I'm using caret to assess classifier performance (and it's great!). However, I've found that my results differ between R2.* and R3.* - reported accuracies are reduced dramatically. I suspect that a code change to kernlab ksvm may be responsible (see version 5.16-24 here: