Hi Donald, Great to hear we are on the same page about this functionality.
I'm glad to contribute to the project. I'll follow the contribution guidelines in http://predictionio.incubator.apache.org/community/contribute-code/ and report back when I've made some progress. Best, Vijay On Tue, Sep 20, 2016 at 3:11 PM, Donald Szeto <don...@apache.org> wrote: > Hi Vijay, > > This is definitely an interesting idea and would be very useful for > production and debugging. In fact, if you look at the EngineInstances / > EvaluationInstances classes, some foundation is already in place and it > just desperately need a UI to expose it. Would it be something that you be > interested in contributing? > > Regards, > Donald > > On Mon, Sep 19, 2016 at 6:17 PM, Vijay Bhat <vijaysb...@gmail.com> wrote: > > > Hi all, > > > > I've been playing with PredictionIO recently and am impressed with its > > capabilities / ease of use. I like how the model serving engine provides > > clear visibility into the currently deployed ML model and its performance > > (latency, throughput). > > > > What I'm also interested in for some of the work I'm doing is tracking > the > > history of models were deployed to an engine. For example, in a > > classification model: > > > > - what algorithms and training parameters were used on each deploy. > > - historical latency and throughput, and how they changed with > retrained > > models (computational performance drift). > > - historical AUC (or other performance metric) to track model drift. > > > > Is this something on the Prediction IO roadmap, or something that others > > have expressed interest in? > > > > Thanks, > > Vijay > > >
