Re: Looking to contribute; which areas need some love?

2019-07-19 Thread Matt Sicker 
That does sound like an interesting thing to do. I've been looking for
an excuse to practice more Kubernetes things, so this could be rather
practical there.

On Fri, 19 Jul 2019 at 13:21, Michele Sciabarra  wrote:
>
> In my opinion, given your experience with Kubernetes, a long needed feature 
> is to add support for clustered CouchDb in the helm chart. It is the only 
> missing pieces to allow for a true ‘production’ CouchDb. Kafka is already 
> clustered, CouchDb is not. David recommend to use an external one but it is 
> not always an option when you have an internal deployment for example.
>
> --
>  Michele Sciabarra
>  mich...@sciabarra.com
>
>
>
> - Original message -
> From: Matt Sicker 
> To: dev@openwhisk.apache.org
> Subject: Looking to contribute; which areas need some love?
> Date: Friday, July 19, 2019 7:10 PM
>
> Hey all, I've been working on some other personal project the past few
> weekends, but I've been itching to explore this project more and make
> some code contributions. I see the main openwhisk repo is Scala/Akka
> which is tech that I've used before. However, I'm also interested in
> hearing which areas feel neglected right now and could use some
> attention.
>
> As a bonus, this can also start an idea of adding entries to
> https://helpwanted.apache.org/ to recruit more contributors. I see
> there's already some GSoC projects going on, so this is an additional
> resource that can be useful here.
>
> --
> Matt Sicker 
>


-- 
Matt Sicker

Re: Looking to contribute; which areas need some love?

2019-07-19 Thread Michele Sciabarra 
In my opinion, given your experience with Kubernetes, a long needed feature is 
to add support for clustered CouchDb in the helm chart. It is the only missing 
pieces to allow for a true ‘production’ CouchDb. Kafka is already clustered, 
CouchDb is not. David recommend to use an external one but it is not always an 
option when you have an internal deployment for example.

-- 
 Michele Sciabarra
 mich...@sciabarra.com



- Original message -
From: Matt Sicker 
To: dev@openwhisk.apache.org
Subject: Looking to contribute; which areas need some love?
Date: Friday, July 19, 2019 7:10 PM

Hey all, I've been working on some other personal project the past few
weekends, but I've been itching to explore this project more and make
some code contributions. I see the main openwhisk repo is Scala/Akka
which is tech that I've used before. However, I'm also interested in
hearing which areas feel neglected right now and could use some
attention.

As a bonus, this can also start an idea of adding entries to
https://helpwanted.apache.org/ to recruit more contributors. I see
there's already some GSoC projects going on, so this is an additional
resource that can be useful here.

-- 
Matt Sicker

Looking to contribute; which areas need some love?

2019-07-19 Thread Matt Sicker 
Hey all, I've been working on some other personal project the past few
weekends, but I've been itching to explore this project more and make
some code contributions. I see the main openwhisk repo is Scala/Akka
which is tech that I've used before. However, I'm also interested in
hearing which areas feel neglected right now and could use some
attention.

As a bonus, this can also start an idea of adding entries to
https://helpwanted.apache.org/ to recruit more contributors. I see
there's already some GSoC projects going on, so this is an additional
resource that can be useful here.

-- 
Matt Sicker

Re: OpenWhisk invoker overloads - "Rescheduling Run message"

2019-07-19 Thread Tyson Norris 
   The PR you referenced above contains a lot of 
other changes. It does not only improve this particular area but also 
includes a lot of other changes - in particular, it adds a different way 
of managing containers. Due to the PR's size and complexity, it's very 
hard to understand and review... Would you be able to split this PR up 
into smaller changes?

Indeed, I will split the PR ASAP. There are several changes there that are 
candidates for this, I think, so will try to create PR for each.
Thanks
Tyson

Re: OpenWhisk invoker overloads - "Rescheduling Run message"

2019-07-19 Thread Sven Lange-Last 
Hello Tyson,

regarding your feedback:

> Related to the "Rescheduling Run message", one problem we have 
> encountered in these cases is that the invoker becomes unstable due 
> ( I think) to a tight message loop, since the message that couldn't 
> run is immediately resent to the pool to be run, which fails again, 
> etc. We saw CPU getting pegged, and invoker eventually would crash.
> I have a PR related to cluster managed resources where, among other 
> things, this message looping is removed:
> https://urldefense.proofpoint.com/v2/url?
> 
u=https-3A__github.com_apache_incubator-2Dopenwhisk_pull_4326_files-23diff-2D726b36b3ab8c7cff0b93dead84311839L198&d=DwIGaQ&c=jf_iaSHvJObTbx-
> 
siA1ZOg&r=Q324lzlz3X6vUQUlgmuIdvLXO6nnIRzq6I6LyOBKHBs&m=yqwkeUxYxei_G_X3fWA0cYYm47ekuejeO6sRUKUwUos&s=KEJSKEJwE-
> zaTlnh8fovCFY4vY_uWmAQsgDsTkfueRI&e= 
> 
> Instead of resending the message to the pool immediately, it just 
> waits in the runbuffer, and the runbuffer is processed in reaction 
> to any potential change in resources: NeedWork, ContainerRemoved, 
> etc. This may add delay to any buffered message(s), but seems to 
> avoid the catastrophic crash in our systems. 

>From my point of view, your proposal on changing processing of rescheduled 
Run messages makes sense. The PR you referenced above contains a lot of 
other changes. It does not only improve this particular area but also 
includes a lot of other changes - in particular, it adds a different way 
of managing containers. Due to the PR's size and complexity, it's very 
hard to understand and review... Would you be able to split this PR up 
into smaller changes?


Mit freundlichen Grüßen / Regards,

Sven Lange-Last
Senior Software Engineer
IBM Cloud Functions
Apache OpenWhisk


E-mail: sven.lange-l...@de.ibm.com
Find me on:  


Schoenaicher Str. 220
Boeblingen, 71032
Germany




IBM Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, 
HRB 243294


Tyson Norris  wrote on 2019/07/08 17:52:14:

> From: Tyson Norris 
> To: "dev@openwhisk.apache.org" 
> Date: 2019/07/08 18:01
> Subject: [EXTERNAL] Re:  Re: OpenWhisk invoker overloads - 
> "Rescheduling Run message"
> 
> Related to the "Rescheduling Run message", one problem we have 
> encountered in these cases is that the invoker becomes unstable due 
> ( I think) to a tight message loop, since the message that couldn't 
> run is immediately resent to the pool to be run, which fails again, 
> etc. We saw CPU getting pegged, and invoker eventually would crash.
> I have a PR related to cluster managed resources where, among other 
> things, this message looping is removed:
> https://urldefense.proofpoint.com/v2/url?
> 
u=https-3A__github.com_apache_incubator-2Dopenwhisk_pull_4326_files-23diff-2D726b36b3ab8c7cff0b93dead84311839L198&d=DwIGaQ&c=jf_iaSHvJObTbx-
> 
siA1ZOg&r=Q324lzlz3X6vUQUlgmuIdvLXO6nnIRzq6I6LyOBKHBs&m=yqwkeUxYxei_G_X3fWA0cYYm47ekuejeO6sRUKUwUos&s=KEJSKEJwE-
> zaTlnh8fovCFY4vY_uWmAQsgDsTkfueRI&e= 
> 
> Instead of resending the message to the pool immediately, it just 
> waits in the runbuffer, and the runbuffer is processed in reaction 
> to any potential change in resources: NeedWork, ContainerRemoved, 
> etc. This may add delay to any buffered message(s), but seems to 
> avoid the catastrophic crash in our systems. 
> 
> Thanks
> Tyson
> 
> On 7/5/19, 1:16 AM, "Sven Lange-Last"  
wrote:
> 
> Hello Dominic,
> 
> thanks for your detailed response.
> 
> I guess your understanding is right - just this small correction:
> 
> > So the main issue here is there are too many "Rescheduling 
> Run" messages 
> in invokers?
> 
> It's not the main issue to see these log entries in the invoker. 
This is 
> just the indication that something is going wrong in the invoker - 
more 
> activations are waiting to be processed than the ContainerPool can 
> currently serve.
> 
> Actually, there are different reasons why "Rescheduling Run message" 
log 
> entries can show up in the invoker:
> 
> 1. Controllers send too many activations to an invoker.
> 
> 2. In the invoker, the container pool sends a Run message to a 
container 
> proxy but the container proxy fails to process it properly and hands 
it 
> back to the container pool. Examples: a Run message arrives while 
the 
> proxy is already removing the container; if concurrency>1, the proxy 

> buffers Run messages and returns them in failure situations.
> 
> Although I'm not 100% sure, I see more indications for reason 1 in 
our 
> logs than for reason 2.
> 
> Regarding hypothesis "#controllers * getInvokerSlot(invoker user 
memory 
> size) > invoker user memory size": I can rule out this hypothesis in 
our 
> environments. We have "#controllers * getInvokerSlot(invoker user 
memory 
> size) = invoker user memory size". I provided PR [1

Re: devtools and make-quickstart

2019-07-19 Thread Dominic Kim 
For me, what Chetan suggested makes sense and sounds very attractive.
We are observing a lot of questions and novices suffering from setting up
OpenWhisk.

The standalone jar would be a great alternative for the quick start if they
have no preference for fundamental tech(compose, k8s, etc).

Best regards
Dominic


2019년 7월 19일 (금) 오후 2:53, Chetan Mehrotra 님이 작성:

> For switching to standalone jar we would first need to publish it
> (#4525). Once we make it available for download somewhere we can then
> guide users to download it from there
>
> > I also like the standalone JAR. Should we consider adding to that the API
> Management features made available through OW GW, or for that we should
> keep the docker-compose route
>
> If we would like to simplify the Api GW usage specially with
> standalone case then we can possibly try adding some support for it
> within standalone flow itself
>
> 1. Package the route management action zips in jar
> 2. Upon startup (say with -g flag) it would install the routes
> 3. It would then also spin up required containers like api gw, redis,
> minio?
>
> So end user may just need to run with `-g` flag to get the gateway
> functionality working.
>
> Chetan Mehrotra
>
> On Sat, Jul 13, 2019 at 12:40 AM Rodric Rabbah  wrote:
> >
> > > I also like the standalone JAR. Should we consider adding to that the
> API
> > Management features made available through OW GW, or for that we should
> > keep the docker-compose route ?
> >
> > I did get the API gateway to run (in the container) with standalone
> > openwhisk. I ran the ansible playbook (wskdev apigw) to get it up and
> > going. It doesn't work with sls (serverless framework because of the DNS
> > resolution issue I've posted about in the past) but will work with wsk.
> >
> > -r
>

[slack-digest] [2019-07-18] #general

2019-07-19 Thread OpenWhisk Team Slack 
2019-07-18 02:07:54 UTC - chris: Guys, i have a question the script `wsk -i 
package list /whisk.system`   why the package list is empty ?  the openwhisk is 
installed by docker-compose `make quick-start`
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563415674026500

2019-07-18 02:12:50 UTC - Rodric Rabbah: hi Chris
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563415970026700

2019-07-18 02:13:08 UTC - Rodric Rabbah: dont think the packages are installed 
by default - try `make add-catalog` first
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563415988027200?thread_ts=1563415988.027200&cid=C3TPCAQG1

2019-07-18 02:13:47 UTC - Rodric Rabbah: by web api do you mean a web action
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563416027027400?thread_ts=1563381349.022500&cid=C3TPCAQG1

2019-07-18 02:16:15 UTC - chris: @Rodric Rabbah oh my god Love u so much 
:heart:  i just found it in the gihub 

These are not included by default with the devtools make quick-start command.

If you want to install these packages, run the following make command.

`make add-catalog`
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563416175028400?thread_ts=1563415988.027200&cid=C3TPCAQG1

2019-07-18 02:20:27 UTC - Rodric Rabbah: :+1:
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563416427028600?thread_ts=1563415988.027200&cid=C3TPCAQG1

2019-07-18 10:21:40 UTC - Satwik Kolhe: Please correct me if I am wrong -

I continuously had issues with apigateway --
```
==> /var/log/api-gateway/management.log <==
2019/07/18 09:36:30 [error] 38#0: *40 lua entry thread aborted: runtime error: 
/etc/api-gateway/scripts/lua/management/lib/tenants.lua:130: attempt to compare 
nil with number
stack traceback:
coroutine 0:
/etc/api-gateway/scripts/lua/management/lib/tenants.lua: in function 
'getTenantAPIs'
/etc/api-gateway/scripts/lua/management/routes/apis.lua:62: in function 
'getAPIs'
/etc/api-gateway/scripts/lua/management/routes/apis.lua:41: in function 
'requestHandler'
access_by_lua(management_apis.conf:36):5: in function 
, client: 10.44.0.0, server: 
management_gw, request: "GET 
/v2/23bc46b1-71f6-4ed5-8c54-816aa4f8c502/apis?limit=undefined&skip=undefined&basePath=%2F
 HTTP/1.1", host: "10.110.90.93:9000"
```

TO which I ended up checking the openwhisk/apigateway image tag - which is..
```
root@ow-master-1:~# kl get pods/slsow-apigateway-7b7879b7bb-vr64v -o yaml -n 
slsow | grep image
image: openwhisk/apigateway:0.10.0-incubating
imagePullPolicy: IfNotPresent
image: openwhisk/apigateway:0.10.0-incubating
imageID: 

```

And possible fixes for the issue on 

 - Where I found that this issue 
() is 
already reported and fixed and merged into master.
The build process is failing for master (which is also seen in TravisCI) - and 
these changes are not in openshisk/apigateway:0.10.0-incubating.

I have made necessary changes in my running image of apigateway! How can this 
be solved permanently?

apache/incubator-openwhisk-apigateway --branch 0.10.0-incubating >> is 
not available on github!
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563445300042300?thread_ts=1563445300.042300&cid=C3TPCAQG1

2019-07-18 10:27:54 UTC - Jiang PengCheng: sorry, but I didn't get what you 
want, it seems that you need to get the code of `0.10.0-incubating` and make 
some changes on it? then there is a tag `0.10.0-incubating` on github
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563445674042700?thread_ts=1563445300.042300&cid=C3TPCAQG1

2019-07-18 10:36:52 UTC - Satwik Kolhe: Hey @Jiang PengCheng,

The image that is used (openwhisk/apigateway:0.10.0-incubating) does not have 
the necessary changes that are mentioned here 
() -  
which are merged in master branch.

I tried to build the apigateway image from this master branch, but it failed - 
same is signified from TravisCI logs.

Because of all this I had to make temporary changes in my deployment of 
openwhisk, where I updated the changes mentioned here 
()

How can we make this permanent?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563446212042900?thread_ts=1563445300.042300&cid=C3TPCAQG1

2019-07-18 10:56:54 UTC - Satwik Kolhe: @Jiang PengCheng Got it - Thanks, I 
mixed branch and tags, will pull 0.10.0-incubating and make necessary changes 
to this stable version of apigateway
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1563447414043100?thread_ts=1563445300.042300&cid=C3TPCAQG1

2019-07-18 11:53:17 UTC - Michael Schmidt: I actually found this was just a 
mistake by my k8s

[slack-digest] [2019-07-18] #random

2019-07-19 Thread OpenWhisk Team Slack 
2019-07-18 06:41:32 UTC - Rodric Rabbah: From the Apache board meeting on 
Wednesday:

```
The following resolutions were passed unanimously:

 Establish the Apache OpenWhisk Project (Dave Grove, VP)
```

Congrats to the entire community. 
tada : Jiang PengCheng, Roberto Santiago, chetanm, James Thomas, Satwik Kolhe, 
Neeraj Mangal, Bruce Adams, yihuaf, Robert Felker
yay : mmarth, James Thomas, Bruce Adams
https://openwhisk-team.slack.com/archives/C3UDXSFA6/p1563432092039300

2019-07-18 06:50:05 UTC - Roberto Santiago: This is so awesome!
https://openwhisk-team.slack.com/archives/C3UDXSFA6/p1563432605039800

2019-07-18 07:03:18 UTC - Markus Thömmes: Amazing, great job to all who worked 
very hard to get all the paperwork together :slightly_smiling_face:
https://openwhisk-team.slack.com/archives/C3UDXSFA6/p1563433398040500

2019-07-18 08:35:37 UTC - James Thomas: woop.
https://openwhisk-team.slack.com/archives/C3UDXSFA6/p1563438937041400

2019-07-18 10:24:20 UTC - Michele Sciabarra: 
100 : Satwik Kolhe, James Thomas, Robert Felker
slightly_smiling_face : mmarth, James Thomas
heart : chetanm
https://openwhisk-team.slack.com/archives/C3UDXSFA6/p1563445460041600?thread_ts=1563445460.041600&cid=C3UDXSFA6

2019-07-18 11:13:37 UTC - Rodric Rabbah: That’s awesome. Is that on the ASF 
website?!
https://openwhisk-team.slack.com/archives/C3UDXSFA6/p1563448417042500?thread_ts=1563445460.041600&cid=C3UDXSFA6

2019-07-18 12:05:39 UTC - Michele Sciabarra: no no
https://openwhisk-team.slack.com/archives/C3UDXSFA6/p1563451539042900?thread_ts=1563445460.041600&cid=C3UDXSFA6

2019-07-18 12:05:45 UTC - Michele Sciabarra: I made it
https://openwhisk-team.slack.com/archives/C3UDXSFA6/p1563451545043100?thread_ts=1563445460.041600&cid=C3UDXSFA6