Acknowledging that dev cycles, performance tests, or other spiky traffic may not be initially captured, I would put my $$$ on teaching OW how to learn the traffic pattern and let it pre-warm based on that. There are existing algorithms that could be used for prediction, if we just have the features/data captured. This is a beautiful area, yet unchartered territory so far ?
A related aspect is "how many VMs should an OW operator keep up". I intuitively relate both problems to the learning capacity of a system to adapt to the load, in order to perform both: 1. tear-down or warm-up action containers 2. shut-down or start-up new VMs to run actions On Thu, May 31, 2018 at 10:36 AM Michele Sciabarra <mich...@sciabarra.com> wrote: > The recent work introduced also support to use docker image as a compiler > (and it is what I normally do). > > I "compile" the binary offliine and I send it as a single binary, that is > started at init time. > > However while for Go and Swift it is "easy", I do not see how we can > "precompile" a nodejs application. > I checked around and it looks like NodeJS does a Just In Time Compilatoin, > I have not found a way to cache the compilation. > > So this is where Go applications may have an edge... > > -- > Michele Sciabarra > mich...@sciabarra.com > > On Thu, May 31, 2018, at 1:34 PM, Nick Mitchell wrote: > > for nodejs at least: the cost of a few requires of common packages can > > easily get you up to the 150-200ms range (e.g. request is a big hitter; > and > > this is all on top of the cost of starting a container!). perhaps, for > > nodejs at least, there are only a few options, ultimately: user pays more > > for idle resources; provider pays more for idle stem cells; or users > take a > > very hard line on the modules they import. > > > > switching to other (compiled) runtimes might help, e.g. with the recent > > work on precompiled go and swift actions? we'd still be left with the > > container start times, but at least this is something we can control, > e.g. > > by requiring users to pay more for access to a larger prewarmed pool? > > > > nick > > > > > > On Thu, May 31, 2018 at 7:22 AM, James Thomas <jthomas...@gmail.com> > wrote: > > > > > One of most frequent complaints[1][2][3] I hear from developers using > > > serverless platforms is coping with cold-start latency when dealing > with > > > sudden bursts of traffic. > > > > > > Developers often ask for a feature where they can set the number of > warm > > > containers kept in the cache for a function. This would allow them to > keep > > > a higher number of warm containers for applications with bursty traffic > > > and/or upgrade the cached number prior to an anticpated burst of > traffic > > > arriving. This would be exposed by the managed platforms as a chargable > > > feature. > > > > > > Is this something we could support on OpenWhisk? Ignoring the > complexity > > > and feasibility of any solution, from a developer POV I can image > having an > > > action annotation `max-warm` which would set the maximum number of warm > > > containers allowed in the cache. > > > > > > Tyson is currently working on concurrent activation processing, which > is > > > one approach to reducing cold-start delays[4]. However, there are some > > > downsides to concurrent activations, like no runtime isolation for > request > > > processing, which might make this feature inappropraite for some users. > > > > > > [1] > > > https://www.reddit.com/r/aws/comments/6w1hip/how_many_ > > > successive_lambda_invocations_will_use_a/ > > > [2] > > > https://twitter.com/search?f=tweets&vertical=default&q=%20% > > > 23AWSWishlist%20warm&src=typd > > > [3] > > > https://theburningmonk.com/2018/01/im-afraid-youre- > > > thinking-about-aws-lambda-cold-starts-all-wrong/ > > > [4] - https://github.com/apache/incubator-openwhisk/pull/2795 > > > > > > -- > > > Regards, > > > James Thomas > > > >
