down votefavorite <http://stats.stackexchange.com/questions/223573/implementing-contractive-autoencoder-weight-decay#> i'm trying to implement the idea behind the contractive auto encoder and penalty the jacobian of output of a MLP wrt its input to get make it less sensitive to noise. there is a scalar coefficient befor the norm of jacobian term and i was wondering how small should i set it to be? if i set it to more than .001 it will saturate the cells and if i set the weight decay less than .9999 it will kill the weights. to clarify its not an autoencoder but an MLP and because its too big i cant just train it with noise i dont know where i'm going wrong? cause .001 seems too small. -- --- You received this message because you are subscribed to the Google Groups "theano-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
