2011/11/29 Alexandre Passos <[email protected]>: > On Tue, Nov 29, 2011 at 16:18, Vincent Dubourg > <[email protected]> wrote: >> @AlexP: What are you trying to do with this iterative construction? Are >> you trying to implement some optimization algorithm (like the efficient >> global optimizer by Jones etal [1])? If so, note that Jones' "expected >> improvement" starts being objective only as the dataset starts being "a >> bit" dense. Starting from one point only is definitely not a good idea >> (and adding points sequentially is not a so good idea either...). > > You're right on both counts. I just added one single point to test the > code, and found out it blew up, and it took me a while to realize that > the bug was that I was fitting on a single point.
If we would support incremental fitting in the GaussianProcess class it would be through a dedicated "partial_fit" method anyway: the contract of the fit method it to forget about any previous calls to fit and just converge to (one of) the optimal estimation(s) for the provided samples (and targets). partial_fit on the other hand does not start from scratch at every call nor does converge to the optimal model but just takes a step on the "best" direction based on the data at hand even though the precise definition of this "best direction" and the size of the step is implementation specific. Now back to you problem I think we should support fitting models with just one sample just for the sake of consistency / continuity even if theds is no practical application of fitting models with a single sample: fitting models with 2 samples would be almost as stupid as fitting a model with only one sample and there is no principled or natural, pre-determined threshold I know of that would give us the minimum number of samples to provide to an estimator. IMHO this is a bug. GaussianProcess and other scikit-learn estimators should accept to fit with singleton training sets and provide predictions that are mathematically consistent even if useless in practice. We could push this reasoning further and expect that models fitted with empty sets of samples should always predict the arbitrary value of int(0) for classifiers and float(0.0) for regressors but I would not argue for such Bayesian-prior-oriented integrist point of view :) -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel ------------------------------------------------------------------------------ All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d _______________________________________________ Scikit-learn-general mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/scikit-learn-general
