Re: AI Actions as a first-class citizen in OpenWhisk
Dragos, Great wiki page! Since I've been playing with TensorFlow.js on OpenWhisk, I definitely think there's a sweet spot for running certain ML tasks using serverless platforms. The suggestions in the wiki page all make sense to me. Access to the GPU is one of the biggest barriers to using more complex models and operations (and even training longer-term). Nvida does have a Docker version that allows you to pass through GPUs ( https://github.com/NVIDIA/nvidia-docker). Another issue I found was the performance difference between warm and cold activations when using customised run images. This could have been resolved it either there was a pre-warmed Node.js image with TF-JS libraries or we have a mechanism to create action packages larger than 48MB. This might include creating actions from packages at an external HTTP address or object storage URI. On 13 August 2018 at 22:17, Dragos Dascalita Haut wrote: > Once you've experienced FaaS, you don't wanna go back. This has been my > experience with AI and FaaS. > > > In particular, running AI inferences in FaaS proved to be a great match: > > - Each function processes one request at a time. A model usually takes 1 > data input and produces 1 data output. > > - Enough code to fit into a function. An AI action loads a model, runs the > inference, and returns the result. > > - In addition, FaaS provides a model to scale to 0 and scale to millions > with the traffic. > > > With OpenWhisk I think we're very close to make AI Actions a first-class > citizen for developers, and I've created a wiki to explore what it would > take to get there [1]. Coincidently James Thomas also published today his > experience with Tensorflow and OpenWhisk [2] > > > I'm interested in your thoughts, and see if there's enough interest in our > community to make this a reality. > > > Feel free to contribute to the wiki with edits, comments, anything you'd > wanna add. > > > [1] - https://cwiki.apache.org/confluence/display/OPENWHISK/AI+Actions > > [2] - https://medium.com/openwhisk/serverless-machine-learning- > with-tensorflow-js-4aa24494a9b4 > > > Thanks, > > dragos > -- Regards, James Thomas
Re: AI Actions as a first-class citizen in OpenWhisk
Second that, Dragos Get Outlook for iOS<https://aka.ms/o0ukef> From: Dragos Dascalita Haut Sent: Monday, August 13, 2018 2:17:20 PM To: dev@openwhisk.apache.org Subject: AI Actions as a first-class citizen in OpenWhisk Once you've experienced FaaS, you don't wanna go back. This has been my experience with AI and FaaS. In particular, running AI inferences in FaaS proved to be a great match: - Each function processes one request at a time. A model usually takes 1 data input and produces 1 data output. - Enough code to fit into a function. An AI action loads a model, runs the inference, and returns the result. - In addition, FaaS provides a model to scale to 0 and scale to millions with the traffic. With OpenWhisk I think we're very close to make AI Actions a first-class citizen for developers, and I've created a wiki to explore what it would take to get there [1]. Coincidently James Thomas also published today his experience with Tensorflow and OpenWhisk [2] I'm interested in your thoughts, and see if there's enough interest in our community to make this a reality. Feel free to contribute to the wiki with edits, comments, anything you'd wanna add. [1] - https://cwiki.apache.org/confluence/display/OPENWHISK/AI+Actions [2] - https://medium.com/openwhisk/serverless-machine-learning-with-tensorflow-js-4aa24494a9b4 Thanks, dragos
AI Actions as a first-class citizen in OpenWhisk
Once you've experienced FaaS, you don't wanna go back. This has been my experience with AI and FaaS. In particular, running AI inferences in FaaS proved to be a great match: - Each function processes one request at a time. A model usually takes 1 data input and produces 1 data output. - Enough code to fit into a function. An AI action loads a model, runs the inference, and returns the result. - In addition, FaaS provides a model to scale to 0 and scale to millions with the traffic. With OpenWhisk I think we're very close to make AI Actions a first-class citizen for developers, and I've created a wiki to explore what it would take to get there [1]. Coincidently James Thomas also published today his experience with Tensorflow and OpenWhisk [2] I'm interested in your thoughts, and see if there's enough interest in our community to make this a reality. Feel free to contribute to the wiki with edits, comments, anything you'd wanna add. [1] - https://cwiki.apache.org/confluence/display/OPENWHISK/AI+Actions [2] - https://medium.com/openwhisk/serverless-machine-learning-with-tensorflow-js-4aa24494a9b4 Thanks, dragos
