The reason why most of the deep learning focus is on classification is because image classification and voice recognition is where all the research money and focus is for the large companies that are investing in machine learning, i.e. Google, Baidu, Facebook, Microsoft, etc.... Also a number of important competitions focus on image recognition. That doesn't mean that the same success, challenges, and tools don't exist for regression problems, it's just not a primary focus in the community right now. There are a number of interesting papers on solving regression problems and all the regularization techniques and network architectures that are useful but it really depends on your particular problem and how much data you have to work with. I can tell you from personal experience that neural networks can be very effective at solving these types of problems and fitting very complex functions but doing it correctly requires careful regularization and choosing the right network architecture. Also, if you use stochastic gradient descent or batch gradient descent, redundant inputs are not really a problem at all. Since these problems are so specific to each problem you're trying to solve I can't really give you more help other than this general advice. A more detailed discussion of all this is probably better suited to a private email exchange. As far as the most modern state of the art techniques go, many of them can be applied to regression problems as well even though the use examples shown are for classification.
On Saturday, January 30, 2016 at 7:46:06 AM UTC-7, [email protected] wrote: > > Thanks, that's pretty much my understanding. Scaling the inputs seems to > be important, too, from what I read. I'm also interested in a framework > that will trim off redundant inputs. > > I have run the mocha tutorial examples, and it looks very promising > because the structure is clear, and there are C++ and cuda backends. The > C++ backend, with openmp, gives me a good performance boost over the pure > Julia backend. However, I'm not so sure that it will allow for trimming > redundant inputs. Also, I have some ideas on how to restrict the net to > remove observationally equivalent configurations, which should aid in > training, and I don't think I could implement those ideas with mocha. > > From what I see, the focus of much recent work in neural nets seems to be > on classification and labeling of images, and regression examples using the > modern tools seem to be scarce. I'm wondering if that's because other tools > work better for regression, or simply because it's an old problem that is > considered to be well studied. I would like to see some examples of > regression nets that work well, using the modern tools, though, if there > are any out there. > > On Saturday, January 30, 2016 at 2:32:16 PM UTC+1, Jason Eckstein wrote: >> >> I've been using NN for regression and I've experimented with Mocha. I >> ended up coding my own network for speed purposes but in general you simply >> leave the final output of the neural network as a linear combination >> without applying an activation function. That way the output can represent >> a real number rather than compress it into a 0 to 1 or -1 to 1 range for >> classification. You can leave the rest of the network unchanged. >> >> On Saturday, January 30, 2016 at 3:45:27 AM UTC-7, [email protected] >> wrote: >>> >>> I'm interested in using neural networks (deep learning) for multivariate >>> multiple regression, with multiple real valued inputs and multiple real >>> valued outputs. At the moment, the mocha.jl package looks very promising, >>> but the examples seem to be all for classification problems. Does anyone >>> have examples of use of mocha (or other deep learning packages for Julia) >>> for regression problems? Or any tips for deep learning and regression? >>> >>
