Do you have an implementation of a gaussian SVM for the same purpose? I am
following the same tutorial and I want to use a gaussian SVM for a
multiclass classification task.
On Monday, October 6, 2014 at 9:33:37 PM UTC+5:30, Arjun Jain wrote:
>
> Graham Taylor and I did something like the code bellow this long time ago,
> but use this code only for reference and do not expect any support from me
> for this code snippet, sorry! HTH
> -----------------------------------------------------
> import cPickle
> import gzip
> import os
> import sys
> import time
>
> import numpy as np
>
> import theano
> import theano.tensor as T
>
> class OVASVMLayer(object):
> """SVM-like layer
> """
> def __init__(self, input, n_in, n_out):
> """ Initialize the parameters of the logistic regression
>
> :type input: theano.tensor.TensorType
> :param input: symbolic variable that describes the input of the
> architecture (one minibatch)
>
> :type n_in: int
> :param n_in: number of input units, the dimension of the space in
> which the datapoints lie
>
> :type n_out: int
> :param n_out: number of output units, the dimension of the space in
> which the labels lie
>
> """
>
> # initialize with 0 the weights W as a matrix of shape (n_in,
> n_out)
> self.W = theano.shared(value=np.zeros((n_in, n_out),
>
> dtype=theano.config.floatX),
> name='W', borrow=True)
> # initialize the baises b as a vector of n_out 0s
> self.b = theano.shared(value=np.zeros((n_out,),
>
> dtype=theano.config.floatX),
> name='b', borrow=True)
>
> # parameters of the model
> self.params = [self.W, self.b]
>
> self.output = T.dot(input, self.W) + self.b
>
> self.y_pred = T.argmax(self.output, axis=1)
>
> def hinge(self, u):
> return T.maximum(0, 1 - u)
>
> def ova_svm_cost(self, y1):
> """ return the one-vs-all svm cost
> given ground-truth y in one-hot {-1, 1} form """
> y1_printed = theano.printing.Print('this is important')(T.max(y1))
> margin = y1 * self.output
> cost = self.hinge(margin).mean(axis=0).sum()
> return cost
>
>
> def errors(self, y):
> """ compute zero-one loss
> note, y is in integer form, not one-hot
> """
>
> # check if y has same dimension of y_pred
> if y.ndim != self.y_pred.ndim:
> raise TypeError('y should have the same shape as self.y_pred',
> ('y', target.type, 'y_pred', self.y_pred.type))
> # check if y is of the correct datatype
> if y.dtype.startswith('int'):
> # the T.neq operator returns a vector of 0s and 1s, where 1
> # represents a mistake in prediction
> return T.mean(T.neq(self.y_pred, y))
> else:
> raise NotImplementedError()
>
>
> def load_data(dataset):
> ''' Loads the dataset
>
> :type dataset: string
> :param dataset: the path to the dataset (here MNIST)
> '''
>
> #############
> # LOAD DATA #
> #############
>
> # Download the MNIST dataset if it is not present
> data_dir, data_file = os.path.split(dataset)
> if (not os.path.isfile(dataset)) and data_file == 'mnist.pkl.gz':
> import urllib
> origin = '
> http://www.iro.umontreal.ca/~lisa/deep/data/mnist/mnist.pkl.gz'
> print 'Downloading data from %s' % origin
> urllib.urlretrieve(origin, dataset)
>
> print '... loading data'
>
> # Load the dataset
> f = gzip.open(dataset, 'rb')
> train_set, valid_set, test_set = cPickle.load(f)
> f.close()
> #train_set, valid_set, test_set format: tuple(input, target)
> #input is an numpy.ndarray of 2 dimensions (a matrix)
> #witch row's correspond to an example. target is a
> #numpy.ndarray of 1 dimensions (vector)) that have the same length as
> #the number of rows in the input. It should give the target
> #target to the example with the same index in the input.
>
> def shared_dataset(data_xy, borrow=True):
> """ Function that loads the dataset into shared variables
>
> The reason we store our dataset in shared variables is to allow
> Theano to copy it into the GPU memory (when code is run on GPU).
> Since copying data into the GPU is slow, copying a minibatch
> everytime
> is needed (the default behaviour if the data is not in a shared
> variable) would lead to a large decrease in performance.
> """
> data_x, data_y = data_xy
> shared_x = theano.shared(np.asarray(data_x,
> dtype=theano.config.floatX),
> borrow=borrow)
> shared_y = theano.shared(np.asarray(data_y,
> dtype=theano.config.floatX),
> borrow=borrow)
>
> # one-hot encoded labels as {-1, 1}
> n_classes = len(np.unique(data_y)) # dangerous?
> y1 = -1 * np.ones((data_y.shape[0], n_classes))
> y1[np.arange(data_y.shape[0]), data_y] = 1
> shared_y1 = theano.shared(np.asarray(y1,
> dtype=theano.config.floatX),
> borrow=borrow)
>
> # When storing data on the GPU it has to be stored as floats
> # therefore we will store the labels as ``floatX`` as well
> # (``shared_y`` does exactly that). But during our computations
> # we need them as ints (we use labels as index, and if they are
> # floats it doesn't make sense) therefore instead of returning
> # ``shared_y`` we will have to cast it to int. This little hack
> # lets ous get around this issue
> return shared_x, T.cast(shared_y, 'int32'), T.cast(shared_y1,
> 'int32')
>
> test_set_x, test_set_y, test_set_y1 = shared_dataset(test_set)
> valid_set_x, valid_set_y, valid_set_y1 = shared_dataset(valid_set)
> train_set_x, train_set_y, train_set_y1 = shared_dataset(train_set)
>
> rval = [(train_set_x, train_set_y, train_set_y1),
> (valid_set_x, valid_set_y, valid_set_y1),
> (test_set_x, test_set_y, test_set_y1)]
> return rval
>
>
> On Mon, Oct 6, 2014 at 11:34 AM, Michael Segala <[email protected]
> <javascript:>> wrote:
>
>> Hi everyone,
>>
>> I am using the DBN tutorial to classify my own dataset.
>> Instead of using softmax regression I was wondering if anyone had a
>> theano based implementation of multiclass SVM?
>> I know this exists in pylearn2 as a wrapper to scikit-learn but I was
>> wondering how to do this in pure theano.
>> Is it possible to insert the scikit version directly into the DBN
>> tutorial?
>>
>> Thank you in advance for all your help.
>> Mike
>>
>> --
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "theano-users" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected] <javascript:>.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"theano-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
