2022-08-05 11:41:56 UTC - Areej: Hi all , I am deploy OW on k8s and I have two node for invoker when I invoke 3 actions ( action2 invoked from inside action1 and action3 invoke from inside action2 ) all these invocation are scheduled on node1 ( invoker1 ) .. why ? it is not using invoker2 on node 2 ? if I invoke these actions several times the invoker1 is too loaded and paused . my question how can I balance the invokeation between the available invokers if I have such actions . I have create another action4 and it is invoked on invoker2 ( just to test that invoker2 is working) https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1659699716764249?thread_ts=1659699716.764249&cid=C3TPCAQG1 ---- 2022-08-05 14:05:52 UTC - Areej: Hi all , I am deploy OW on k8s , I have created web action how can I access this action using the url form the browser ( on different server ) https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1659708352109439 ---- 2022-08-05 17:29:34 UTC - Brendan Doyle: @Jiang PengCheng Would be great to get the action versioning feature in finally now that we've gotten mostly through the scheduler. Not sure if it needs to be added on to because I think it was made before mongodb support was introduced. Happy to help with anything. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1659720574733579?thread_ts=1659720574.733579&cid=C3TPCAQG1 ---- 2022-08-05 21:12:46 UTC - Brendan Doyle: Are you using the new scheduler? You're running into the major flaw of the original openwhisk implementation with hash based load balancing. The idea was to keep the critical path as lightweight as possible so pushing activations to invokers would hash to a specific invoker to maximize container reuse and minimize latency to get.
Works well when you have a few actions and a huge amount of invokers. However when the number of actions you have increases, you're more likely to have actions hash to the same invoker which defeats the purpose of load balancing. You get into a state of constant container recreations which is twice as bad as a cold start since you have to first remove a container then perform the cold start. The new scheduler was developed by naver to solve this problem. It flips the entire activation processing on its head moving from push to pull based activation retrieval to the invoker where the function containers will request work when it has capacity to the scheduler service so that you're getting optimal container reuse and proper load balancing. Their work is extremely impressive, it's a major innovation in serverless resource usage imo. Combine that with turning on intra container concurrency for nodejs so your function container can process multiple activations at once and now you have an eventing worker model similar to knative and reduce cold starts even further. Aws Lambda still doesn't even support execution concurrency within a container. Hopefully we can find time to iron out the kinks in this feature (requires your deployment to have external logging due to concurrency to stdout) and bring it to other major languages than just nodejs. I've verified it does work on the new scheduler as is. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1659733966294449?thread_ts=1659699716.764249&cid=C3TPCAQG1 ---- 2022-08-05 21:15:42 UTC - Brendan Doyle: On a semi related topic, I feel that interest in the project has grown quite a bit again with active users since the new scheduler was introduced. Would anyone be interested in bringing back the technical interchange to discuss the future of the project? (Maybe a discussion to be had on the dev list, but figured I'd start here) https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1659734142130029 ---- 2022-08-05 21:32:37 UTC - Matt Welke: > However when the number of actions you have increases, you're more likely to have actions hash to the same invoker which defeats the purpose of load balancing. You get into a state of constant container recreations which is twice as bad as a cold start since you have to first remove a container then perform the cold start. Maybe this is why my Python action deployed to OW on IBM Cloud sometimes takes 30s to cold start. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1659735157630219?thread_ts=1659735157.630219&cid=C3TPCAQG1 ---- 2022-08-05 21:33:05 UTC - Matt Welke: Might be a high # of actions relative to invokers situation in that big multitenant environment. I wonder if they use the old scheduler. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1659735185156009 ----
