Thanks for your reply.
I think these two delta_o have the same meaning. If you have "Pattern
Recognition and Machine Learning" by Bishop, you can find that Bishop use
exactly the second formula in the back propagation algorithm. I suspect
these two formulae lead to the same update iterations, but I can't see why
now.
What formula do you adopt in your implementation?
On Wed, Jun 6, 2012 at 6:53 PM, David Marek <h4wk...@gmail.com> wrote:
> Hi
>
> On Wed, Jun 6, 2012 at 10:38 AM, xinfan meng <mxf3...@gmail.com> wrote:
>
>> Hi, all. I post this question to the list, since it might be related to
>> the MLP being developed.
>>
>> I found two versions of the error function for output layer of MLP are
>> used in the literature.
>>
>>
>> 1. \delta_o = (y-a) f'(z)
>> http://ufldl.stanford.edu/wiki/index.php/Backpropagation_Algorithm
>> 2. \delta_o = (y-a) http://www.idsia.ch/NNcourse/backprop.html
>>
>> Given that they all use the same sigmoid activation function and loss
>> function, how can the error function be different? Also note that the error
>> functions will ultimately lead to different propagating errors in the
>> hidden layers.
>>
>>
> I just skimmed through them and there are few differencies between those
> two pages:
>
> * \delta_o doesn't mean the same in those pages. In the second one, it's
> just the derivative of the error function.
> * The second page doesn't use sigmoid as output function, look at the
> examples on next page and you'll see that y_o = a + f tanh(x) + g tanh(x).
> Derivative of this function is y. As can be seen in the matrix form \Delta
> W = \delta_l y_{l-1}
>
> I hope this answers your question. Sometimes it's possible to make the
> computations simpler, because the error function and output function are
> natural pairs, see
> http://www.willamette.edu/~gorr/classes/cs449/classify.html
>
> David
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Scikit-learn-general mailing list
> Scikit-learn-general@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
>
>
--
Best Wishes
--------------------------------------------
Meng Xinfan（蒙新泛）
Institute of Computational Linguistics
Department of Computer Science & Technology
School of Electronic Engineering & Computer Science
Peking University
Beijing, 100871
China
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Scikit-learn-general mailing list
Scikit-learn-general@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
