I am trying to understand what is the difference between linear and
polynomial kernel:
linear: u'*v
polynomial: (gamma*u'*v + coef0)^degree
It would seem that polynomial kernel with gamma = 1; coef0 = 0 and degree
= 1
should be identical to linear kernel, however it gives me significantly
different results for very simple
data set, with linear kernel significantly outperforming polynomial
kernel.
*** mse, r2 = 0.5, 0.9 for linear
*** mse, r2 = 1.8, 0.1 for polynomial
What am I missing ?
Ryszard
P.S.
Here are my results:
# simple cross validation function
cv.svm <- function(formula, data, ntry = 3, kernel = "linear", scale =
FALSE, cross = 3,
gamma = 1/(dim(data)-1), degree = 3) {
mse <- 0; r2 <- 0
for (n in 1:ntry) {
svm.model <- svm(formula , data = data, scale = scale, kernel =
kernel,
cross = cross)
mse <- mse + svm.model$tot.MSE
r2 <- r2 + svm.model$scorrcoeff
}
mse <- mse/ntry; r2 <- r2/ntry; result <- c(mse, r2)
cat(sprintf("cv.svm> mse, r2 = %5.3f %5.3f\n", mse, r2))
return (result)
}
# define data set
x1 <- rnorm(9); x2 <- rnorm(9)
df <- data.frame(y = 2*x1 + x2, x1, x2)
# invoke cv.svm() for linear and polynomial kernels few times
> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1,
ntry = 32)
cv.svm> mse, r2 = 1.888 0.162
> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1,
ntry = 32)
cv.svm> mse, r2 = 1.867 0.146
> r <- cv.svm( y ~ ., df, kernel = "polynomial", gamma = 1, degree = 1,
ntry = 32)
cv.svm> mse, r2 = 1.818 0.105
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry =
32)
cv.svm> mse, r2 = 0.525 0.912
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry =
32)
cv.svm> mse, r2 = 0.537 0.878
> r <- cv.svm( y ~ ., df, kernel = "linear", gamma = 1, degree = 1, ntry =
32)
cv.svm> mse, r2 = 0.528 0.913
[[alternative HTML version deleted]]
______________________________________________
[EMAIL PROTECTED] mailing list
https://www.stat.math.ethz.ch/mailman/listinfo/r-help
