Re: enhancing scanCode to respect gitignore files

[Rodric Rabbah]

I opened a PR
https://github.com/apache/incubator-openwhisk-utilities/pull/57 to bundle
the relevant parts of the "pathspec" library. This avoids pip install in
all downstream clients.

The pathspec library is Mozilla Public License 2.0 [1]. My reading of the
Mozilla License, their FAQ [2] and the Apache 3rd Party License Policy [3]
leads me to conclude the bundling is acceptable.

[1] https://www.mozilla.org/en-US/MPL/2.0/
[2] https://www.mozilla.org/en-US/MPL/2.0/FAQ
[3] https://apache.org/legal/resolved.html

-r

On Fri, Mar 29, 2019 at 8:55 PM Rodric Rabbah  wrote:

> >  Am I understanding correctly that we'd need to go change the
> .travis.yaml files for pretty much every openwhisk repo to do the `pip
> install pathspec` as part of its install phase?
>
> That's right. I could mitigate this by having scancode itself install the
> module (via) pip if necessary.
> Alternatively, I could bundle the relevant parts of library into scancode.
> It's Mozilla Licensed and not very big (
> https://github.com/cpburnz/python-path-specification).
>
> -r
>
>
>

Re: enhancing scanCode to respect gitignore files

[Rodric Rabbah]

>  Am I understanding correctly that we'd need to go change the
.travis.yaml files for pretty much every openwhisk repo to do the `pip
install pathspec` as part of its install phase?

That's right. I could mitigate this by having scancode itself install the
module (via) pip if necessary.
Alternatively, I could bundle the relevant parts of library into scancode.
It's Mozilla Licensed and not very big (
https://github.com/cpburnz/python-path-specification).

-r

Re: [VOTE] release Apache OpenWhisk Package Alarm, Package Cloudant, and Package Kafka 2.0.0 (incubating)

[Matt Sicker]

+1 to the full checklist. Neat that there's an easy to use python
script for helping validate.

On Tue, 26 Mar 2019 at 07:48, David P Grove  wrote:
>
>
>
> The Apache OpenWhisk Community has voted to make the first Apache release
> of the OpenWhisk "Package Alarm", "Package Cloudant" and "Package Kafka"
> components.
>
> The OpenWhisk dev list voting thread is here:
> https://lists.apache.org/thread.html/e0fb73d8ca0d0c584650f4f5b8b02054f5c99d64895f3e5ace352234@%3Cdev.openwhisk.apache.org%3E
>
> We request that IPMC Members please review and vote on this incubator
> release as described below.
>
> These three Apache OpenWhisk packages provide basic event-based programming
> capabilities to an OpenWhisk deployment.
>
> This release comprises of source code distribution only. There are three
> components (git repos) included in this release of the OpenWhisk Packages
> Group. All release artifacts were built by PR#251 in the openwhisk-release
> repo from the following Git commit IDs:
> * openwhisk-package-alarms: dd6ab4d81c8893436e7242170d57951c089d5185
> * openwhisk-package-cloudant: ce5dac9cb0204fb2f78195d9e3f9364236613a31
> * openwhisk-package-kafka: 49820dd24170f24a37c02fae6ba7ec06e190423f
>
> openwhisk-package-alarms (C158D76 B5D9A53A AC10AC42 4CD5060C 7A6467D5
> E9624519 117E9C8C AB171590 CCC3505D 5C7E630D C41769B4 44F85F03 BBA40A6A
> E04D2264 79FFA78B B3C5BFB8)
> src.tar.gz:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-alarms-2.0.0-incubating-sources.tar.gz
> sha512 checksum:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-alarms-2.0.0-incubating-sources.tar.gz
> .sha512
> signature:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-alarms-2.0.0-incubating-sources.tar.gz
> .asc
>
> openwhisk-package-cloudant (380E1BA8 EC3C72A8 09B10DCF 03BC1D2D 4BFA7E4F
> 892995EE 39D26BBE 758AFD5D FA680E81 54A9C726 A0D386C5 B447C947 70D26385
> 2A8D07BF 5848318A 42A3BBDD)
> src.tar.gz:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-cloudant-2.0.0-incubating-sources.tar.gz
> sha512 checksum:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-cloudant-2.0.0-incubating-sources.tar.gz
> .sha512
> signature:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-cloudant-2.0.0-incubating-sources.tar.gz
> .asc
>
> openwhisk-package-kafka (303B8286 AA9B945D 1A740FCB 0DE908D5 A287FAF5
> C20A9DF9 4BBD2F16 C488DF4B 3E82A75B 9D071B1A E1CCDB54 7B623A36 5AC88E5D
> 9994539B 00ECEA5D A066D156)
> src.tar.gz:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-kafka-2.0.0-incubating-sources.tar.gz
> sha512 checksum:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-kafka-2.0.0-incubating-sources.tar.gz
> .sha512
> signature:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/apache-openwhisk-2.0.0-incubating-rc2/openwhisk-package-kafka-2.0.0-incubating-sources.tar.gz
> .asc
>
> All release artifacts were signed with key 72AF0CC22C4CF320.
> KEYS file is available here:
> https://dist.apache.org/repos/dist/dev/incubator/openwhisk/KEYS
>
> How to verify the artifacts can be found at:
> https://cwiki.apache.org/confluence/display/OPENWHISK/How+to+verify+the
> +release+checklist+and+vote+on+OpenWhisk+modules+under+Apache
>
> Please vote on releasing version 2.0.0-incubating of Apache OpenWhisk
> Package Alarm, Package Cloudant, and Package Kafka as described above.
>
> The vote will be open for at least 72 hours.
> [ ] +1 Release Apache OpenWhisk Package Alarm, Package Cloudant, and
> Package Kafka 2.0.0-incubating
> [ ] +0 no opinion
> [ ] -1 Do not release and the reason
>
> Checklist for reference:
> [ ] Download links are valid.
> [ ] Checksums and PGP signatures are valid.
> [ ] DISCLAIMER is included.
> [ ] Source code artifacts have correct names matching the current release.
> [ ] LICENSE and NOTICE files are correct for each OpenWhisk repository.
> [ ] All files have license headers if necessary.
> [ ] No compiled archives bundled in source archive.
>
> regards,
>
> --dave



-- 
Matt Sicker 

Re: Current status of Jenkins setup for OpenWhisk

[Vincent S Hou]

Hi OpenWhiskers,

I have already made the Jenkins pipeline working.
I resolved the issue about the constantly failing test case by enabling the 
memory limit of docker on each openwhisk VM.
Three VMs are able to run three instances of OpenWhisk deployment and test at 
the same time.

Normally the result will turn out passed for our code located in master branch 
of OpenWhisk. The pipeline is healthy now.
If the pipeline build fails, it could be due to the error of the PR. However, 
report the issues you find, if you believe it is caused by mistake of the 
pipeline.

Thank you.
 
Best wishes.
Vincent Hou (侯胜博)

Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM Cloud

Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com,
Phone: +1(919)254-7182
Address: 4205 S Miami Blvd (Cornwallis Drive), Durham, NC 27703, United States

-Vincent S Hou/Raleigh/IBM wrote: -
To: dev@openwhisk.apache.org
From: Vincent S Hou/Raleigh/IBM
Date: 03/26/2019 11:26AM
Subject: Re: Current status of Jenkins setup for OpenWhisk

Hi folks,

I have improved the way how we utilize the three VMs for Jenkins pipeline.
Instead of using openwhisk1 as the edge service node, running all the services 
except invoker, we can also schedule openwhisk2 and openwhisk3 to run the edge 
service as well.

How to implement the distributed model to deploy openwhisk?

The deployment topology is orchestrated as below:
If openwhisk1 runs as the edge service, openwhisk2 runs invoker0 and openwhisk3 
runs invoker1.
If openwhisk2 runs as the edge service, openwhisk3 runs invoker0 and openwhisk1 
runs invoker1.
If openwhisk3 runs as the edge service, openwhisk1 runs invoker0 and openwhisk2 
runs invoker1.
 
Even three of these deployments are able to run at the same time, instead of 
one deployment at a time.
For safety reasons, I also added resource lock to each node in Jenkinsfile, so 
that even if the job is scheduled to a busy node, it will wait there till the 
previous job finishes.

I am now looking at the test to stop one of the invokers and run the designated 
tests. If you have any idea on how to do that, welcome to chime in.
Thank you.

Best wishes.
Vincent Hou (侯胜博)

Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM Cloud

Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com,
Phone: +1(919)254-7182
Address: 4205 S Miami Blvd (Cornwallis Drive), Durham, NC 27703, United States

-Vincent S Hou/Raleigh/IBM wrote: -
To: dev@openwhisk.apache.org
From: Vincent S Hou/Raleigh/IBM
Date: 03/12/2019 10:59AM
Subject: Current status of Jenkins setup for OpenWhisk

Hi OpenWhiskers,

I have come back from my holiday last week, and officially start to work this 
week. During the past few days, I have made some progress in terms of getting 
the access to openwhisk VMs and Apache Jenkins.

Current state:
My apache ID has granted root access to the VMs:
openwhisk-vm1-he-de.apache.org
openwhisk-vm2-he-de.apache.org
openwhisk-vm3-he-de.apache.org

I am now able to use the command ssh openwhisk-vm1-he-de.apache.org -l  to access the VM via terminal.

My Apache ID has been added into the hudson-jobadmin group: 
https://whimsy.apache.org/roster/group/hudson-jobadmin, which means I have 
earned the admin access to Apache Jenkins environment at 
https://builds.apache.org/. I should be able to create build or pipeline.

I have granted the user "jenkins" root access within three of these VMs.

Work items to do:
Next, I will configured the ssh access among three of these VMs, since I plan 
to use one VM as the master, launching all the services except invoker, and the 
other two as designated nodes to launch invoker service.

Travis CI has already tested the case, when all the openwhisk services are 
installed on one server. I plan to use Jenkins to test openwhisk installed in a 
distributed way.

Welcome to chime in and add your comments, if you have any idea about how to 
use these three VMs for OpenWhisk.

Thank you.

Best wishes.
Vincent Hou (侯胜博)

Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM Cloud

Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com,
Phone: +1(919)254-7182
Address: 4205 S Miami Blvd (Cornwallis Drive), Durham, NC 27703, United States

Re: release verification help

[James Thomas]

On Tue, 26 Mar 2019 at 17:21, James Thomas  wrote:
>
> On Mon, 25 Mar 2019 at 17:40, Carlos Santana  wrote:
> >
> > Using the website I would think it doesn’t count as verifying the artifacts.
> >
> > Just playing devils advocate here.
> >
> > Going to a website and selecting something from a Dropbox and see green 
> > checks don’t know if the useful for the person doing the voting. Maybe 
> > taking the > code from the website or bash scripts and running locally 
> > seems more real.
>
> The tool is open-source and I'd encourage others to review the source
> code to understand what it does and check for issues. If you are
> concerned about using an external service, you can host the tool
> yourself.
>
> --
> Regards,
> James Thomas



-- 
Regards,
James Thomas

Re: Execute OpenWhisk action via AWS Lambda

[Matt Sicker]

This sounds pretty neat. Could it lead to OpenWhisk being like the
Kubernetes of serverless? :)

On Fri, 29 Mar 2019 at 02:34, Bertrand Delacretaz
 wrote:
>
> Hi Chetan,
>
> On Fri, Mar 29, 2019 at 7:09 AM Chetan Mehrotra  wrote:
> > ...We recently did a POC around executing actions deployed on OpenWhisk
> > via AWS Lambda...
>
> I think you forgot a link to https://en.wikipedia.org/wiki/Inception ;-)
>
> > ...3. Invoker can delegate execution to Lambda based on some policy...
>
> Out of curiosity, what kind of policies do you envision?
>
> Annotate the functions to indicate that they should run on Lambda,
> decide based on the current load of other compute providers, these
> kinds of things?
>
> -Bertrand



-- 
Matt Sicker 

Updating aka and scala deps

[Rodric Rabbah]

Chetan has updated the following library dependencies
Akka modules - 2.5.12 -> 2.5.21
Akka HTTP - 10.1.1 -> 10.1.8
Scala - 2.12.7 - 2.12.8
In https://github.com/apache/incubator-openwhisk/pull/4316

These are required for #4315 and #4217. The PR is ready to merge now. Will wait 
24hrs before merging by silent assent if there are no new comments.

-r

Re: Execute OpenWhisk action via AWS Lambda

[Bertrand Delacretaz]

Hi Chetan,

On Fri, Mar 29, 2019 at 7:09 AM Chetan Mehrotra  wrote:
> ...We recently did a POC around executing actions deployed on OpenWhisk
> via AWS Lambda...

I think you forgot a link to https://en.wikipedia.org/wiki/Inception ;-)

> ...3. Invoker can delegate execution to Lambda based on some policy...

Out of curiosity, what kind of policies do you envision?

Annotate the functions to indicate that they should run on Lambda,
decide based on the current load of other compute providers, these
kinds of things?

-Bertrand

[slack-digest] [2019-03-28] #kubernetes

[OpenWhisk Team Slack]

2019-03-28 17:38:06 UTC - Roberto Previtera: bye
https://openwhisk-team.slack.com/archives/C4J3R7JFL/p1553794686000200

2019-03-28 17:38:27 UTC - Roberto Previtera: I will contact you here in the 
slack channel with a few questions on the status of the projects.
https://openwhisk-team.slack.com/archives/C4J3R7JFL/p1553794707000700

2019-03-28 17:38:28 UTC - Roberto Previtera: Bye
https://openwhisk-team.slack.com/archives/C4J3R7JFL/p1553794708000900

[slack-digest] [2019-03-28] #general

[OpenWhisk Team Slack]

2019-03-28 04:51:46 UTC - Xue Shuai: Hi, everyone, do i have to redeploy the 
entire openwhisk with openwhisk.yml every time I change parameters, eg. 
invoker_user_memory, or just redeploy invoker with invoker.yml? Thank you.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553748706001900?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 04:52:30 UTC - chetanm: You can run `wskdev invoker`
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553748750002000?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 04:52:39 UTC - chetanm: That would just redeploy invoker
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553748759002200?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 04:55:25 UTC - Xue Shuai: @chetanm Thanks for your relpy, You mean 
that I can change the parameters `invoker_user_memory`, just redeploy invoker?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553748925002400?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 04:57:47 UTC - chetanm: Yup
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553749067002600?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 05:06:00 UTC - Xue Shuai: Got it, thank you. By the way, i used 
ansible to redeoloy invoker and passed the env variable with -e before , like 
`ansible-playbook -i environments/local invoker.yml -e skip_pull_runtimes=True 
-e  invoker_user_memory="2048m" ` . Is there any documnet for the tool 
`wskdev`. I only found a 
[readme]()
 mentioned it,  I dont down how to change the env variable this way.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553749560002800?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 05:16:51 UTC - chetanm: Its documented as `redo` here at 

https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553750211003100?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 05:17:14 UTC - chetanm: Opened 
 to update the docs
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553750234003300?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 05:18:04 UTC - chetanm: I typically update the 
`ansible/environment/local/group_vars/all` and add any ansible variable there
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553750284003500?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 05:18:38 UTC - chetanm: May be there is a better way as I often need 
to shelve my changes when refreshing the git repo
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553750318003700?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 05:25:48 UTC - Xue Shuai: Your support is greatly appreciated.:heart:
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553750748003900?thread_ts=1553748706.001900=C3TPCAQG1

2019-03-28 10:29:17 UTC - Barry: Hi, My controller log says "No invokers 
available" after I restart the invoker container, but invoker keeps receive 
"invokerHealthTestAction" action and the results are "ok"

Controller log
[ShardingContainerPoolBalancer] [marker:controller_kafka_start:32283281]
[ShardingContainerPoolBalancer] posted to invoker1[0][308] 
[marker:controller_kafka_finish:32283283:2]

Invoker log
[DockerContainer] initialization result: ok
[DockerContainer] running result: ok
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553768957005400

2019-03-28 10:42:30 UTC - Dominic Kim: You can try run /invokers API against 
the controller.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553769750006000

2019-03-28 10:42:56 UTC - Dominic Kim: Is the invoker offline or unhealthy?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553769776006700

2019-03-28 10:55:14 UTC - Barry: Hi, I already terminated the cluster, I 
remember the invoker is unhealthy
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553770514007300

2019-03-28 10:56:30 UTC - Barry: What is the condition of a invoker change to 
healthy ?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553770590009200

2019-03-28 10:56:32 UTC - Dominic Kim: Invoker is supposed to be Healthy after 
health action is successfully invoked 7 times.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553770592009300

2019-03-28 10:57:13 UTC - Barry: by default is 7 times?
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553770633009600

2019-03-28 10:57:48 UTC - Barry: I seems checked more than 7 times
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553770668010200

2019-03-28 10:58:37 UTC - Dominic Kim: AFAIK, if Invoker does not become 
healthy the controller will send health action.
https://openwhisk-team.slack.com/archives/C3TPCAQG1/p1553770717011500

2019-03-28 11:01:17 UTC - Barry: [2019-03-28T10:58:01.164Z] [INFO] 
[#tid_sid_invokerHealth] 

Execute OpenWhisk action via AWS Lambda

[Chetan Mehrotra]

Hi Team,

We recently did a POC around executing actions deployed on OpenWhisk
via AWS Lambda. Some of the key objectives of this POC were

1. Supports most of OpenWhisk features
2. Developers use std OpenWhisk tooling to create functions
3. Invoker can delegate execution to Lambda based on some policy
4. Execution via Lambda is an implementation detail not surfaced to
end developers

Some of the benefits we are trying to target here are

1. Tap into cloud provider platforms for scaling out
2. Bring compute near to content
3. Retain the rich programming model provided by OpenWhisk

PR #4412 [1] captures the current state of the POC which enables
exexecuting basic NodeJS actions via lambda integration.

At high level there are following major parts

1. A custom runtime layer for Lambda which mimics the OpenWhisk protocol when
running in Lambda
2. A LambdaBuilder service which listens for changes in `whisks`
collection and transforms them to lambda function
3. A LambdaContainerFactory which routes the activations to Lambda invocations
4. (TBD) CloudWatchLogProvider which would collect logs from CloudWatch

More details are provided in the PR [1]. At this stage its a poc but
it opens up a possible way to target hybrid deployments for OpenWhisk
setups in various cloud providers. That would enable us to leverage
the rich development flow and constructs that OpenWhisk provides and
use the underlying cloud providers scalable compute resources for
actual execution processing for some cases.

[1] https://github.com/apache/incubator-openwhisk/pull/4412