2020-03-04 02:23:45 UTC - Dominic Kim: There can be two options. One is (which is taken by many operators with K8S here) having another set of OW and do the blue-green deployment. This is the simplest way but it would require more(doubled) resources.
The other is rolling upgrade one by one that we here are taking. I think currently OW does not support this without downtime. So you need some changes. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583288625123900?thread_ts=1583279584.123800&cid=C3TPCAQG1 ---- 2020-03-04 02:25:09 UTC - Dominic Kim: 1. You need to make sure the health information should not be compatible with each other between an old version and a new version. 2. You need to disable components. So we need to make sure no more request is coming until the component is handling all existing request. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583288709124100?thread_ts=1583279584.123800&cid=C3TPCAQG1 ---- 2020-03-04 02:25:43 UTC - Dominic Kim: For number 1, I think you can try with another set of Kafka. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583288743124300?thread_ts=1583279584.123800&cid=C3TPCAQG1 ---- 2020-03-04 02:26:58 UTC - Dominic Kim: For number 2, it's easy to disable a controller. Just remove it from nginx. But for an invoker, you can try to stop HealthProducer. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583288818124500?thread_ts=1583279584.123800&cid=C3TPCAQG1 ---- 2020-03-04 02:27:46 UTC - Dominic Kim: Seems this would be a good feature to contribute. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583288866124700?thread_ts=1583279584.123800&cid=C3TPCAQG1 ---- 2020-03-04 03:57:53 UTC - Jin Choi: @Rodric Rabbah Hello. I need your help. I had incubator-openwhisk on serving so far and I just upgraded it with official openwhisk without the incubator tag. But I wanted to keep the subjects and whisks from couchdb so that didn't upgrade my couchdb and just let OW point to the exising couchdb. I found that the view for subjects was modified and I had it updated. Python 2 and 3 actions from the past works well on this new OW system. But this java actions doesn't work very well. ```$ wsk action invoke -r helloworld error: Unable to invoke action 'helloworld': Resource by this name exists but is not in this collection. (code ff3c73976f9371cddfa53cb703edbe40)``` https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583294273130900 ---- 2020-03-04 03:59:48 UTC - Jin Choi: I suspect that the document format for java action must have modified since the incubator era. What is the best move for this conformace error situation? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583294388133200 ---- 2020-03-04 04:03:30 UTC - Rodric Rabbah: the java kind was renamed to java:8 from just java iirc, that may be the issue thankyou : Jin Choi https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583294610133800 ---- 2020-03-04 04:16:31 UTC - Jin Choi: @Rodric Rabbah Is there any other renamings? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583295391135400 ---- 2020-03-04 04:19:52 UTC - Rodric Rabbah: dont think so https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583295592135600 ---- 2020-03-04 04:37:52 UTC - Jin Choi: @Rodric Rabbah Thanks rr https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583296672136000 ---- 2020-03-04 04:39:54 UTC - Jin Choi: @Rodric Rabbah one last thing, I have hundreds of java actions with the issue. Do you have a tool for this kind of field update for CouchDB? Or, do you think I'd better start making some batch script? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583296794138200?thread_ts=1583296794.138200&cid=C3TPCAQG1 ---- 2020-03-04 04:50:38 UTC - Rodric Rabbah: you might want to change the runtime manifest to set the java kind back to `java` +1 : Jin Choi https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583297438138400?thread_ts=1583296794.138200&cid=C3TPCAQG1 ---- 2020-03-04 04:51:06 UTC - Rodric Rabbah: @Sven Lange-Last might have a suggestion https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583297466138600?thread_ts=1583296794.138200&cid=C3TPCAQG1 ---- 2020-03-04 04:51:14 UTC - Rodric Rabbah: i dont think the project has a migration script to migrate the db heavy_check_mark : Jin Choi https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583297474138800?thread_ts=1583296794.138200&cid=C3TPCAQG1 ---- 2020-03-04 07:11:23 UTC - Ali Shahidi: @Rodric Rabbah Could you help me with this? I really appreciate it. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583305883139200?thread_ts=1582966242.092000&cid=C3TPCAQG1 ---- 2020-03-04 09:34:04 UTC - Justin Halsall: Hi all, today is the OpenWhisk Tech Interchange call in 5 and a half hours. Does anyone has anything they'd like to present today? Could be a contribution, tooling, bug or issue you're working on for OpenWhisk or something else OpenWhisk related you'd like to share. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583314444141700 ---- 2020-03-04 11:18:20 UTC - Rob Allen: I wish my client call didn’t keep clashing https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583320700143100?thread_ts=1583320700.143100&cid=C3TPCAQG1 ---- 2020-03-04 12:14:29 UTC - Rodric Rabbah: That’s your openwhisk key. Do you know how to find this? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583324069144000?thread_ts=1582966242.092000&cid=C3TPCAQG1 ---- 2020-03-04 13:09:05 UTC - Justin Halsall: is your Zoom acting up @Rob Allen? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583327345144500?thread_ts=1583320700.143100&cid=C3TPCAQG1 ---- 2020-03-04 13:11:48 UTC - Rob Allen: No. I mean that I have a call with a client at the same time again as all my US clients like to have calls in the morning https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583327508144700?thread_ts=1583320700.143100&cid=C3TPCAQG1 ---- 2020-03-04 13:14:07 UTC - Justin Halsall: Ah right! Sorry dyslexia turned your clashing call into a crashing call https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583327647144900?thread_ts=1583320700.143100&cid=C3TPCAQG1 ---- 2020-03-04 13:28:53 UTC - Rob Allen: That would be my experience with Skype for Business :joy: https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583328533145200?thread_ts=1583320700.143100&cid=C3TPCAQG1 ---- 2020-03-04 13:47:06 UTC - Justin Halsall: Somehow I'm not surprised :smile: https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583329626145400?thread_ts=1583320700.143100&cid=C3TPCAQG1 ---- 2020-03-04 14:47:40 UTC - Michele Sciabarra: @Justin Halsall do not forget to warn before starting here and post the link as I believe the majority of people watch here and join when they see a message here +1 : Justin Halsall https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583333260146400?thread_ts=1583333260.146400&cid=C3TPCAQG1 ---- 2020-03-04 14:49:29 UTC - Justin Halsall: Good point Michele! https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583333369146600?thread_ts=1583333260.146400&cid=C3TPCAQG1 ---- 2020-03-04 14:50:12 UTC - Justin Halsall: STARTING IN 10 MINUTES: Apache OpenWhisk project Tech. Interchange meeting: <https://zoom.us/my/asfopenwhisk> https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583333412147100 ---- 2020-03-04 15:00:33 UTC - Justin Halsall: @here STARTING NOW: Apache OpenWhisk project Tech. Interchange meeting: <https://zoom.us/my/asfopenwhisk> https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334033147500 ---- 2020-03-04 15:06:54 UTC - Tom Barber: So I offloaded CouchDB to an EC2 node and after a little fiddling openwhisk came to life but the kafkaprovider fails to authenticate against it https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334414148600 ---- 2020-03-04 15:07:22 UTC - Tom Barber: is that some known bug? have I done something wrong? or entirely unheard of Tom, what on earth are you talking about? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334442149300 ---- 2020-03-04 15:08:59 UTC - Rodric Rabbah: Tom, what on earth are you talking about? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334539149500 ---- 2020-03-04 15:09:08 UTC - Tom Barber: excellent https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334548149700 ---- 2020-03-04 15:09:26 UTC - Tom Barber: seems like Kafka isn't sending across any credentials as its 401ing https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334566150200 ---- 2020-03-04 15:10:39 UTC - Rodric Rabbah: By kafka provider you mean feed provider? And if I understood, it’s not able to communicate with the couch service? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334639152100 ---- 2020-03-04 15:11:02 UTC - Tom Barber: ```owdev-kafkaprovider-7b8c56777c-sddkr 0/1 CrashLoopBackOff 5 6m51s``` https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334662152400 ---- 2020-03-04 15:11:04 UTC - Tom Barber: whatever that is https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334664152600 ---- 2020-03-04 15:11:20 UTC - Tom Barber: ```requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: http://<>:5984/_session``` https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334680152900 ---- 2020-03-04 15:12:34 UTC - Rodric Rabbah: Looks like couch host is missing? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334754153300 ---- 2020-03-04 15:12:55 UTC - Tom Barber: sorry I snipped the ip +1 : Rodric Rabbah https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334775153600 ---- 2020-03-04 15:13:09 UTC - Tom Barber: if I visit the url in a browser and login it works fine https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334789153900 ---- 2020-03-04 15:16:24 UTC - Tom Barber: ironically, I spun one up because occasionally couchdb in EKS craps out and falls over with some storage issue, but then investigating further I found that the docs about the lack of default storagevolume support in EKS are stale and there is support, so I've sent a PR to remove that stuff, so I'll comment this back out for now and hopefully the couchdb in the pod wont fall over https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334984155700 ---- 2020-03-04 15:16:36 UTC - Tom Barber: in fact I might be mean and kill it to test it https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583334996156000 ---- 2020-03-04 15:18:32 UTC - Tom Barber: Other helm related question, if I tweak the chart to support multiple release names so you can run > 1 OW cluster per Kubernetes cluster, I assume thats worthwhile and there isn't some reason its hardcoded to owdev? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583335112157100 ---- 2020-03-04 15:43:59 UTC - Dave Grove: The chart is _supposed_ to allow you to pick any release name you want. It is also supposed to allow you to deploy multiple instances to the same cluster, although I don’t test the later feature very often. If support for either has rotted, PRs to fix them would be great. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583336639159600 ---- 2020-03-04 15:45:26 UTC - Tom Barber: yeah if you change the name @Dave Grove it can't find some config maps because they're deployed using the release name, but the chart looks up ow-dev https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583336726160300 ---- 2020-03-04 15:45:37 UTC - Tom Barber: that was the first issue, I dunno how often that occurs though https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583336737160600 ---- 2020-03-04 15:47:31 UTC - Dave Grove: that’s a bug. Maybe I should change the travis scripts to deploy with a random release name each time to try to stop it from happening again. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583336851161800 ---- 2020-03-04 15:49:26 UTC - Tom Barber: oh actually looking at upstream, I might be lying https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583336966162900 ---- 2020-03-04 15:49:33 UTC - Dave Grove: not seeing that in the charts in git though…is it a local bug? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583336973163400 ---- 2020-03-04 15:50:08 UTC - Tom Barber: yeah, I'll try and figure out why I had to map some volume stuff around I've forgotten the reason for that, looks like my fault ignore me https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583337008164000 ---- 2020-03-04 15:51:17 UTC - Dave Grove: for the deploying multiple instances, one thing that will probably need tweaking is the node affinity stuff. If you cared about affinity, you’d need to override the default labels for the roles. Not sure if that works; i’ve never tested it https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583337077165800 ---- 2020-03-04 15:52:29 UTC - Tom Barber: I know the affinity stuff does have an effect because if you launch more invokers than nodes it'll complain about it, but, I guess if you have multiple releases in different namespaces the affinity stuff goes out the window https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583337149167900 ---- 2020-03-04 15:53:26 UTC - Will Plusnick: I want to apologize for my poor job of contextualizating what I was proposing at the Tech Interchange Call that led to a chaotic discussion on the call this morning. I'm going to give some thought on how to more clearly outline our intent and scope for the changes, and shoot off a post to the dev list in a few hours. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583337206168500 ---- 2020-03-04 15:54:19 UTC - Tom Barber: @Dave Grove did you see the bit about the EKS PVC's? I believe that documentation to be stale, but if you know otherwise feel free to kick it back https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583337259169300 ---- 2020-03-04 15:56:11 UTC - Dave Grove: I’d believe its stale. most of that documentation came from me playing around on AWS more than a year ago. Thanks for the PR with the update https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583337371170000 ---- 2020-03-04 15:59:19 UTC - Tom Barber: yeah, I think most of it is fine. There's some weirdness with SSL certificates on the load balancer that I we currently have to work around and I'll figure out the proper resolution for in the next couple of weeks now we've split our prod/dev/test environments up so I can break stuff more https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583337559171100 ---- 2020-03-04 15:59:29 UTC - Tom Barber: the rest seems pretty correct though https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583337569171400 ---- 2020-03-04 19:16:14 UTC - Kostis Kaffes: Hey @Rodric Rabbah , is the load balancer implemented as described in the comments <https://github.com/apache/openwhisk/blob/master/core/controller/src/main/scala/org/apache/openwhisk/core/loadBalancer/ShardingContainerPoolBalancer.scala|here>? From tests I am running it seems that once a server is full, activations are directed to a different invoker much faster than what the 10-sec health-check interval could explain. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583349374174100?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 19:18:55 UTC - Markus Thömmes: fullness isn't determined via the health check. The Loadbalancer counts in memory https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583349535174300?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 19:21:10 UTC - Kostis Kaffes: Thanks! For the “toy” busy spin functions I use, cpu rather than memory should be the bottleneck. https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583349670174500?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 19:21:51 UTC - Kostis Kaffes: Is there a parameter that determines the maximum number of concurrent activations per invoker? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583349711174700?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 19:22:42 UTC - Markus Thömmes: see here: <https://github.com/apache/openwhisk/blob/05ed4e18eb53f5630735ffbe28726dce5a3481b5/core/controller/src/main/scala/org/apache/openwhisk/core/loadBalancer/ShardingContainerPoolBalancer.scala#L91-L97> https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583349762174900?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 19:25:07 UTC - Kostis Kaffes: Got it! The user-memory is defined here (<https://github.com/apache/openwhisk/blob/05ed4e18eb53f5630735ffbe28726dce5a3481b5/core/invoker/src/main/resources/application.conf#L60>) to be by default 1GB, right? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583349907175200?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 19:28:24 UTC - Markus Thömmes: yep https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583350104175500?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 19:31:38 UTC - Kostis Kaffes: Great, thanks for the pointer. Btw, if I remember correctly, there is a parameter that determines the minimum memory attributed to each function. Could you point me to that? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583350298175700?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 21:58:43 UTC - Rodric Rabbah: can you clarify? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583359123175900?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 21:58:49 UTC - Rodric Rabbah: theres a class called MemoryLimit https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583359129176100?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 21:58:59 UTC - Rodric Rabbah: but i dont understand what youre asking for https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583359139176300?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 22:06:23 UTC - Kostis Kaffes: That’s what I was looking for. Even if an action uses less than 128MB of memory (the default minimum limit), it will acquire Invoker slots corresponding to 128MB, right? https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583359583176500?thread_ts=1583349374.174100&cid=C3TPCAQG1 ---- 2020-03-04 22:06:58 UTC - Rodric Rabbah: yes the slot is reserved based on the declared limit thankyou : Kostis Kaffes https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1583359618176700?thread_ts=1583349374.174100&cid=C3TPCAQG1 ----