Hi Nacho, there is a huger number of potential problems here.
If your problem is not extremely structured (that is: sparse), then learning something from 750 samples in 1.3 million dimensions is absolutely hopeless. Learning theory tells you this. What type of dataset is this? For the polynommial kernel, it might well be that you observe artifacts. I don't understand why learning takes so long. The kernel matrix should be 750x750. That is, the 24h are spend computing the matrix I guess? In high dimensions, the kernel bandwidth is very different to smaller problems. You need to set it carefully. One useful heuristic is to use the median pairwise distance of your data (can use a subset for that), see for example here: https://gist.github.com/karlnapf/83fd6fe9ab5cdaa37982 In sparse, high-dimensional spaces, things might be different once again. Heiko 2016-01-21 21:59 GMT+00:00 nacho arroyo <[email protected]>: > Hi dear all, > > I recently tested MKLRegression with a dataset containing 750 vector > samples of 1.3million dense dimensions. It takes about 24hours for training > in our cluster. Nevertheless, The training uniquely seems to work for the > PolyKernel. For other kernels the machine does not learn, i.e. predicted > output for the test set is a constant (all predicted values are equal, near > to the mean of the real relationship values, see Figure attached for 50 > samples). > > What do you suggest me for tracking the problem? In this case, in order to > you reproduce the issue, I think it is needed you have my dataset (or > probably any other with similar characteristics). > > Thank you. > > -- > *Ignacio Arroyo-Fernández* >
