Hi
On Mon, Jun 18, 2012 at 11:43 AM, Olivier Grisel
wrote:
> Err, no. The paper I mentioned is even newer:
>
> http://arxiv.org/abs/1206.1106
>
> Just to make it more explicit about the paper content, the title is:
> "No More Pesky Learning Rates" and it's a method for estimating the
> optimal l
Am 18.06.2012 11:43, schrieb Olivier Grisel:
> 2012/6/18 Andreas Mueller:
>> Hey David.
>> Olivier dug up this paper by LeCun's group:
>> http://users.ics.aalto.fi/kcho/papers/icml11.pdf
>> I think this might be quite interesting for the MLP.
> Err, no. The paper I mentioned is even newer:
>
>h
2012/6/18 Andreas Mueller :
> Hey David.
> Olivier dug up this paper by LeCun's group:
> http://users.ics.aalto.fi/kcho/papers/icml11.pdf
> I think this might be quite interesting for the MLP.
Err, no. The paper I mentioned is even newer:
http://arxiv.org/abs/1206.1106
Just to make it more exp
Hey David.
Olivier dug up this paper by LeCun's group:
http://users.ics.aalto.fi/kcho/papers/icml11.pdf
I think this might be quite interesting for the MLP.
It is probably also interesting for the linear SGD.
I'm surprised that they didn't compare against diagonal stochastic
Levenberg-Marquardt
On Fri, Jun 01, 2012 at 02:49:35AM +0200, David Marek wrote:
>I don't have much time these days because I have got exams in school.
Good luck!
>I have implemented a multi class cross entropy and soft max function and
>turned off some of the cython checks, the result is that the cython
Hi,
I don't have much time these days because I have got exams in school. I am
sorry I haven't informed you.
I have implemented a multi class cross entropy and soft max function and
turned off some of the cython checks, the result is that the cython
implementation is only slightly better, I guess
Hey David.
How is it going?
I haven't heard from you in a while.
Did you blog anything about your progress?
Cheers,
Andy
Am 16.05.2012 12:15, schrieb David Marek:
> On Tue, May 15, 2012 at 4:59 PM, David Warde-Farley
> wrote:
>> On Tue, May 15, 2012 at 12:12:34AM +0200, David Marek wrote:
>>> H
>>> * Shouldn't the backward step for computing delta_h be:
>>> delta_h[:] = np.dot(delta_o, weights_output.T) * hidden.doutput(x_hidden)
>>> where hidden.doutput is a derivation of the activation function for
>>> hidden layer?
>>
>> Offhand that sounds right. You can use Theano as a sanity
On 2012-05-16, at 6:31 AM, Andreas Mueller wrote:
> Btw, I am not sure theano is the best way to compute derivatives ;)
No? I would agree in the general case. However, in the case of MLPs and
backprop, it's a use case for which Theano has been designed and heavily
optimized. With it, it's very
On Tue, May 15, 2012 at 10:31 PM, Andreas Mueller
wrote:
> On the same topic: I'm not sure if we decided whether we want minibatch,
> batch and online learning.
> I have the feeling that it might be possible to do particular
> optimizations for online learning, and this
> is the algorithm that I f
Am 16.05.2012 12:29, schrieb David Marek:
> Hi
>
> Yes, I did. I am using gmail so I just quote one mail, didn't want to
> answer each mail separately when they are so similar. Sorry, I will
> try to be more specific in quoting.
>
Never mind, probably my mail program just acted up.
Btw, I am not su
Hi
Yes, I did. I am using gmail so I just quote one mail, didn't want to
answer each mail separately when they are so similar. Sorry, I will
try to be more specific in quoting.
David
16. 5. 2012 v 12:22, Andreas Mueller :
> Hi David.
> Did you also see this mail:
> http://permalink.gmane.org/gm
Hi David.
Did you also see this mail:
http://permalink.gmane.org/gmane.comp.python.scikit-learn/3071
For some reason it doesn't show up in my inbox and you didn't quote it.
So just making sure.
Cheers,
Andy
> Thank you David and Andreas for answering my questions. I will look at Theano.
On Tue, May 15, 2012 at 4:59 PM, David Warde-Farley
wrote:
> On Tue, May 15, 2012 at 12:12:34AM +0200, David Marek wrote:
>> Hi,
>>
>> I have worked on multilayer perceptron and I've got a basic
>> implementation working. You can see it at
>> https://github.com/davidmarek/scikit-learn/tree/gsoc_ml
2012/5/16 Mathieu Blondel :
>
>
> On Wed, May 16, 2012 at 5:31 AM, Andreas Mueller
> wrote:
>>
>>
>> The SequentialDataset was made for vector x vector operations. Depending
>> on whether we
>> do mini-batch or online learning, the MLP needs vector x matrix or
>> matrix x matrix operations.
>> In
2012/5/15 Mathieu Blondel :
>
>
> On Tue, May 15, 2012 at 11:59 PM, David Warde-Farley
> wrote:
>>
>>
>> I haven't had a look at these classes myself but I think working with raw
>> NumPy arrays is a better idea in terms of efficiency.
>
>
> Since it abstracts away the data representation, Sequent
On Wed, May 16, 2012 at 5:31 AM, Andreas Mueller
wrote:
>
> The SequentialDataset was made for vector x vector operations. Depending
> on whether we
> do mini-batch or online learning, the MLP needs vector x matrix or
> matrix x matrix operations.
> In particular matrix x matrix is probably not fe
On 05/15/2012 10:06 PM, David Warde-Farley wrote:
> On 2012-05-15, at 3:23 PM, Andreas Mueller wrote:
>
>> I am not sure if we want to support sparse data. I have no experience with
>> using MLPs on sparse data.
>> Could this be done efficiently? The weight vector would need to be
>> represented
On 2012-05-15, at 3:23 PM, Andreas Mueller wrote:
> I am not sure if we want to support sparse data. I have no experience with
> using MLPs on sparse data.
> Could this be done efficiently? The weight vector would need to be
> represented explicitly and densely, I guess.
>
> Any ideas?
People
On 05/15/2012 05:16 PM, Mathieu Blondel wrote:
On Tue, May 15, 2012 at 11:59 PM, David Warde-Farley
mailto:warde...@iro.umontreal.ca>> wrote:
I haven't had a look at these classes myself but I think working
with raw
NumPy arrays is a better idea in terms of efficiency.
Since i
On Wed, May 16, 2012 at 12:16:21AM +0900, Mathieu Blondel wrote:
> On Tue, May 15, 2012 at 11:59 PM, David Warde-Farley <
> warde...@iro.umontreal.ca> wrote:
>
> >
> > I haven't had a look at these classes myself but I think working with raw
> > NumPy arrays is a better idea in terms of efficiency
On Tue, May 15, 2012 at 11:59 PM, David Warde-Farley <
warde...@iro.umontreal.ca> wrote:
>
> I haven't had a look at these classes myself but I think working with raw
> NumPy arrays is a better idea in terms of efficiency.
>
Since it abstracts away the data representation, SequentialDataset is
us
On Tue, May 15, 2012 at 12:12:34AM +0200, David Marek wrote:
> Hi,
>
> I have worked on multilayer perceptron and I've got a basic
> implementation working. You can see it at
> https://github.com/davidmarek/scikit-learn/tree/gsoc_mlp The most
> important part is the sgd implementation, which can b
Hi David.
I'll have a look at your code later today.
Let me first answer your questions to my code
On 05/15/2012 12:12 AM, David Marek wrote:
> Hi,
>
> 2) I used Andreas' implementation as an inspiration and I am not sure
> I understand some parts of it:
> * Shouldn't the bias vector be initiali
Hi,
I have worked on multilayer perceptron and I've got a basic
implementation working. You can see it at
https://github.com/davidmarek/scikit-learn/tree/gsoc_mlp The most
important part is the sgd implementation, which can be found here
https://github.com/davidmarek/scikit-learn/blob/gsoc_mlp/skl
25 matches
Mail list logo
