One more thought, I definitely am not opposed to making it distributed resource manager independent. I think what Otto is suggesting isn't a bad thread to pull on. Right now, MaaS is tied to Yarn inherently and it'd be nice to make that dependency pluggable. This would allow us to use other Lambda or Kubernetes or whatever for model deployment, which would be really neat.
On Tue, Apr 18, 2017 at 3:02 PM, Casey Stella <ceste...@gmail.com> wrote: > Regarding model performance, I've thought about that a bit. What I'd like > to see MaaS be able to do is provide an API that the models can communicate > through that will send events to kafka and provides a telemetry like any > other. Performance statistics, raw results for downstream analysis. We > have a system capable of analyzing telemetry data, it seems to me like MaaS > should use the dogfood in which it runs. I think we should build an API by > which the model can communicate with kafka. > > One of the things that I very much like about MaaS is that it's light on > the opinion about the language and library. I think that building the API > should be as simple as a log file that is monitored and the MaaS runner > will provide that proxy to kafka. I do think that it should be easier to > write models, but I think that should be solved through applications that > will turn model collateral into REST APIs if they conform to certain > standards (i.e. PMML, Spark MLLib serialized models, etc.) and allow users > the freedom to engage with MaaS via their own mechanism in the language of > their choice if their situation doesn't conform to our expectations. > > On Tue, Apr 18, 2017 at 2:50 PM, Simon Elliston Ball < > si...@simonellistonball.com> wrote: > >> Completely agree. It would be great to get more info from your experience. >> >> To an extent what we have at the moment is very much about isolating the >> implementation of a model from the deployment and discovery mechanism, and >> to my mind we should very much keep that to enable any kind of model to >> plugin. The other thing worth discussing here would be how we can wrap >> around a model while maintaining the lose coupling, to provide things like >> generalised performance metrics for the models. Any thoughts on that front >> anyone? >> >> >> > On 18 Apr 2017, at 11:45, Otto Fowler <ottobackwa...@gmail.com> wrote: >> > >> > I will have to go back to my notes. There was a day or so when I went >> through the code and was thinking of a couple of things, but that was a >> while ago. >> > >> > Off the top of my head, I would want something factored enough, or >> loosely coupled enough that it was not dependent on twill or maas or >> anything else. This >> > would not impose implementation on the services. This would have to >> revolve around a discovery/registration api and a rest interface contract. >> > >> > Does that make sense? >> > >> > >> > >> > >> > On April 18, 2017 at 14:34:28, Simon Elliston Ball ( >> si...@simonellistonball.com <mailto:si...@simonellistonball.com>) wrote: >> > >> >> Right, how about some sort of REST API? Or through Ambari? What would >> you say was the best way to start the service, and of course to submit >> model artefacts? >> >> >> >> Simon >> >> >> >>> On 18 Apr 2017, at 11:33, Otto Fowler <ottobackwa...@gmail.com >> <mailto:ottobackwa...@gmail.com>> wrote: >> >>> >> >>> In my mind, I didn’t want to deploy the service as a bash script or >> wrapped in one, if I recall correctly. >> >>> >> >>> >> >>> >> >>> On April 18, 2017 at 14:27:52, Simon Elliston Ball ( >> si...@simonellistonball.com <mailto:si...@simonellistonball.com>) wrote: >> >>> >> >>>> Any particular issues, or things that didn’t work Otto? >> >>>> >> >>>> Simon >> >>>> >> >>>> >> >>>> > On 18 Apr 2017, at 11:26, Otto Fowler <ottobackwa...@gmail.com >> <mailto:ottobackwa...@gmail.com>> wrote: >> >>>> > >> >>>> > I’ll try to take a look. There are a couple of things I wanted to >> do with MaaS but could not >> >>>> > figure out because of a couple of limitations. I’d like to see if >> twill offers more flexibility >> >>>> > >> >>>> > >> >>>> > >> >>>> > >> >>>> > On April 18, 2017 at 13:50:39, Nick Allen (n...@nickallen.org >> <mailto:n...@nickallen.org>) wrote: >> >>>> > >> >>>> > I ran across the Apache Twill [1] project recently whose goal is >> to reduce >> >>>> > the complexity of developing distributed applications that run on >> YARN. My >> >>>> > first thought is that it might offer additional capabilities and/or >> >>>> > simplify our current MaaS implementation. >> >>>> > >> >>>> > Here are a list of features provided by Twill that I think might >> be useful >> >>>> > for MaaS. >> >>>> > >> >>>> > - Service discovery >> >>>> > - Elastic scaling >> >>>> > - High Availability >> >>>> > - Placement policies - Which rack/host should the model run on? >> >>>> > - Security - Kerberos ticket refresh? >> >>>> > >> >>>> > Just wanted to float the thought in the community and see if >> anyone has >> >>>> > experience with Twill. I need to do some more research myself. >> >>>> > >> >>>> > [1] http://twill.apache.org/ <http://twill.apache.org/> >> > >
