Welcome new committer: Neeraj Mangal

[Dragos Dascalita Haut]

The Project Management Committee (PMC) for Apache OpenWhisk
has invited Neeraj to become a committer and we are pleased
to announce that he has accepted.

Please join me in welcoming Neeraj to his new role on the project !

Neeraj has been contributing to the Kube deployment, the CLI, and the main 
project.

Welcome new committer: Dan McWeeny

[Dragos Dascalita Haut]

It is my pleasure to share that the OpenWhisk PPMC
has elected Dan McWeeny as a Committer,
based on his ongoing and valuable contributions to the project,
the most recent ones being around moving to Java 11, and encrypting default 
params.

Dan has accepted the invitation.

Please join me in welcoming Dan to his new role on the project !

Being a committer enables easier contribution to the project
since there is no need to go via the patch submission process.
This should enable better productivity.
Being a PMC member enables assistance with the management
and to guide the direction of the project.


dragos

Welcome new committer: Cosmin Stanciu

[Dragos Dascalita Haut]

It is my pleasure to share that the OpenWhisk PPMC
has elected Cosmin Stanciu as a Committer,
based on his ongoing and valuable contributions to the project,
especially around user events, monitoring and kubernetes deployment.

Cosmin has accepted the invitation.

Please join me in welcoming Cosmin to his new role on the project !

Being a committer enables easier contribution to the project
since there is no need to go via the patch submission process.
This should enable better productivity.
Being a PMC member enables assistance with the management
and to guide the direction of the project.


dragos

Re: Reminder: Tech Interchange meeting tomorrow, Wed Aug 15th

[Dragos Dascalita Haut]

Unfortunately I'm in a flight during the meeting and I won't be able to attend.


RE AI Actions I agree with Rodric: the time might have been a bit short. I'm 
incorporating the feedback I'm receiving into the wiki page and hopefully it 
will be in a better shape for our next meeting. If anyone has more thoughts pls 
let me know or update the wiki, or add comments directly.



From: Rodric Rabbah 
Sent: Tuesday, August 14, 2018 6:23:24 AM
To: dev@openwhisk.apache.org
Subject: Re: Reminder: Tech Interchange meeting tomorrow, Wed Aug 15th

Looks like a nice agenda. Thanks Ben for hosting this one. I can’t make it 
unfortunately but will catch the replay.

Dragos’ AI actions might be another although maybe the runway too short.

-r

> On Aug 14, 2018, at 9:09 AM, Ben Browning  wrote:
>
> Greetings!
>
> Our next tech interchange call is Wednesday, August 15th at 11am US
> Eastern - that's tomorrow! Use the attached .ics file if you'd like to
> add a reminder to your calendar.
>
> Call details:
> Web Meeting: Tech Interchange (bi-weekly):
> - Day-Time: Wednesdays, 11AM EDT (Eastern US), 5PM CEST (Central Europe),
> 3PM UTC, 11PM CST (Beijing)
> - Zoom: 
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fzoom.us%2Fmy%2Fasfopenwhiskdata=02%7C01%7Cddascal%40adobe.com%7C0e2a5f620c55405e73a708d601e92409%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636698498170347002sdata=U3zigzoOAhbhy0%2BjmdfNSJ%2BJtzHlAx%2BUiYF8hGazHp0%3Dreserved=0
>
> Based on recent mailing list and Slack discussions, here are some
> proposed discussion topics. If you'd like to speak about one of these
> or have another topic, please email or message me on Slack before our
> meeting. I'll send out an updated agenda shortly before the meeting
> tomorrow.
>
> * 0.9.0 release update (Vincent?)
> * website update (Matt & Priti?)
> * knative update (Ben or Markus)
> * system env vars in user containers (Vadim, Markus, Rodric, Chetan,
> Carlos, Tyson, Dragos?)
> * pluggable API gateways (Henry, Rodric, Dragos?)
> * AI actions (Dragos?)
> * BDD function and performance tests (Rahul, Martin, Markus?)
> * Recap of recent notable changes (?)
> * Anything else - let me know!
>
>
> Thanks!
>
> Ben
> 

Re: logging baby step -- worth pursuing?

[Dragos Dascalita Haut]

"...we should be able to fully
process the logs offline and in a streaming manner and get the needed
activation id injected into every logline..."


+1 IIRC for concurrent activations Tyson Norris and Dan McWeeney were going 
down this path as well. Having this natively supported by all OpenWhisk 
runtimes can only make things easier.


From: David P Grove 
Sent: Tuesday, August 14, 2018 2:29:12 PM
To: dev@openwhisk.apache.org
Subject: logging baby step -- worth pursuing?



Even if we think structured logging is the right eventual goal, it could
take a while to get there (especially since it is changing functionality
users may have grown accustomed to).

However, for non-concurrent, non-blackbox runtimes we could make a small,
not-user visible change, that could enable fully offline and streaming log
processing.  We already generate an end-of-log sentinel to stdout/stderr
for these runtimes.  If we also generated a start-of-log sentinel to
stdout/stderr that included the activation id, we should be able to fully
process the logs offline and in a streaming manner and get the needed
activation id injected into every logline.

Is this worth pursuing?   I'm motivated to get log processing out of the
Invoker/ContainerRouter so we can push ahead with some of the scheduler
redesignwithout tackling logging, I don't think we'll be able to assess
the true scalability potential of the new scheduling architectures.

--dave

Re: Proposal on a future architecture of OpenWhisk

[Dragos Dascalita Haut]

Markus, I appreciate the enhancements you mentioned in the wiki, and I'm very 
much inline with the ideas you brought in there.



"...having the ContainerManager be a cluster singleton..."

I was just in process to reply with the same idea :)

In addition, I was thinking we can leverage Akka Distributed Data [1] to keep 
all ContainerRouter actors eventually consistent. When creating a new 
container, the ContainerManager can write with a consistency "WriteAll"; it 
would be a little slower but it would improve consistency.


The "edge-case" isn't clear to me b/c I'm coming from the assumption that it 
doesn't matter which ContainerRouter handles the next request, given that all 
actors have the same data. Maybe you can help me understand better the 
edge-case ?


Re Knative approach, can you expand why the execution layer/data plane would be 
replaced entirely by Knative serving ? I think knative serving handles very 
well some cases like API requests, but it's not designed to guarantee 
concurrency restrictions like "1 request at a time per container" - something 
that AI Actions need.


Thanks,

dragos


[1] - https://doc.akka.io/docs/akka/2.5/distributed-data.html



From: David P Grove 
Sent: Tuesday, August 14, 2018 2:15:13 PM
To: dev@openwhisk.apache.org
Subject: Re: Proposal on a future architecture of OpenWhisk




"Markus Thömmes"  wrote on 08/14/2018 10:06:49
AM:
>
> I just published a revision on the initial proposal I made. I still owe a
> lot of sequence diagrams for the container distribution, sorry for taking
> so long on that, I'm working on it.
>
> I did include a clear seperation of concerns into the proposal, where
> user-facing abstractions and the execution (loadbalacing, scaling) of
> functions are loosely coupled. That enables us to exchange the execution
> system while not changing anything in the Controllers at all (to an
> extent). The interface to talk to the execution layer is HTTP.
>

Nice writeup!

For me, the part of the design I'm wondering about is the separation of the
ContainerManager and the ContainerRouter and having the ContainerManager by
a cluster singleton. With Kubernetes blinders on, it seems more natural to
me to fuse the ContainerManager into each of the ContainerRouter instances
(since there is very little to the ContainerManager except (a) talking to
Kubernetes and (b) keeping track of which Containers it has handed out to
which ContainerRouters -- a task which is eliminated if we fuse them).

The main challenge is dealing with your "edge case" where the optimal
number of containers to create to execute a function is less than the
number of ContainerRouters.  I suspect this is actually an important case
to handle well for large-scale deployments of OpenWhisk.  Having 20ish
ContainerRouters on a large cluster seems plausible, and then we'd expect a
long tail of functions where the optimal number of container instances is
less than 20.

I wonder if we can partially mitigate this problem by doing some amount of
smart routing in the Controller.  For example, the first level of routing
could be based on the kind of the action (nodejs:6, python, etc).  That
could then vector to per-runtime ContainerRouters which dynamically
auto-scale based on load.  Since there doesn't have to be a fixed division
of actual execution resources to each ContainerRouter this could work.  It
also lets easily stemcells for multiple runtimes without worrying about
wasting too many resources.

How do you want to deal with design alternatives?  Should I be adding to
the wiki page?  Doing something else?

--dave

AI Actions as a first-class citizen in OpenWhisk

[Dragos Dascalita Haut]

Once you've experienced FaaS, you don't wanna go back. This has been my 
experience with AI and FaaS.


In particular, running AI inferences in FaaS proved to be a great match:

- Each function processes one request at a time. A model usually takes 1 data 
input and produces 1 data output.

- Enough code to fit into a function. An AI action loads a model, runs the 
inference, and returns the result.

- In addition, FaaS provides a model to scale to 0 and scale to millions with 
the traffic.


With OpenWhisk I think we're very close to make AI Actions a first-class 
citizen for developers, and I've created a wiki to explore what it would take 
to get there [1].  Coincidently James Thomas also published today his 
experience with Tensorflow and OpenWhisk [2]


I'm interested in your thoughts, and see if there's enough interest in our 
community to make this a reality.


Feel free to contribute to the wiki with edits, comments, anything you'd wanna 
add.


[1] - https://cwiki.apache.org/confluence/display/OPENWHISK/AI+Actions

[2] - 
https://medium.com/openwhisk/serverless-machine-learning-with-tensorflow-js-4aa24494a9b4


Thanks,

dragos

Re: [RESULT][VOTE] Release Apache OpenWhisk 0.9.0-incubating rc2: main OpenWhisk module

[Dragos Dascalita Haut]

"...Please count my vote as well (binding)..."

Same for me.


From: Matt Rutkowski 
Sent: Tuesday, July 10, 2018 4:11:23 PM
To: dev@openwhisk.apache.org
Subject: Re: [RESULT][VOTE] Release Apache OpenWhisk 0.9.0-incubating rc2: main 
OpenWhisk module


Please count my vote as well (binding)
-mr

On 2018/07/10 20:35:14, "Vincent S Hou"  wrote:
> Dear IPMC members,
>
> This vote has passed with 6 +1 votes (bindings), 2 +1 votes (non-binding) and 
> no 0 or -1 votes.
>
> +1 (binding), David Grove
> +1 (binding), James Dubee
> +1 (binding), James Thomas
> +1 (binding), Rodric Rabbah
> +1 (binding), Vincent Hou
> +1 (binding), Ying Chun Guo
>
>
> +1 (non-binding), Justin Halsall
> +1 (non-binding), Priti Desai
>
> Please vote on the OpenWhisk main module to be released as a candidate for 
> OpenWhisk 0.9.0. Thank you very much.
>
> We are currently using the tool called 
> openwhisk-release(https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk-releasedata=02%7C01%7Cddascal%40adobe.com%7C3cc28d07d7f14826c89308d5e6ba75e2%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668610870355699sdata=5pwzAsk6Yc9Pu1h7m99whpi4%2BrWyng0mnAuAkIYIPUs%3Dreserved=0)
>  to release all the modules of OpenWhisk. The instruction for release 
> managers can be found at: 
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk-release%2Fblob%2Fmaster%2Fdocs%2Frelease_instructions.mddata=02%7C01%7Cddascal%40adobe.com%7C3cc28d07d7f14826c89308d5e6ba75e2%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668610870355699sdata=%2F5rxIqkFMyq%2BdhnHL6MsMj%2FcAnUF6WO7n8yAGrPFTsI%3Dreserved=0.
>
> This tool supports both manual and automated modes to package the source 
> code, sign the artifacts and upload the artifacts into Apache SVN 
> repositories.
>
> Best wishes.
> Vincent Hou (侯胜博)
>
> Advisory Software Engineer, OpenWhisk Contributor, Open Technology, IBM Cloud
>
> Notes ID: Vincent S Hou/Raleigh/IBM, E-mail: s...@us.ibm.com,
> Phone: +1(919)254-7182
> Address: 4205 S Miami Blvd (Cornwallis Drive), Durham, NC 27703, United States
>
> -Vincent S Hou/Raleigh/IBM wrote: -
> To: dev@openwhisk.apache.org
> From: Vincent S Hou/Raleigh/IBM
> Date: 07/03/2018 02:55PM
> Subject: [VOTE] Release Apache OpenWhisk 0.9.0-incubating rc2: main OpenWhisk 
> module
>
> Hi everyone,
>
> This is to call for a vote for the release of Apache OpenWhisk 
> 0.9.0-incubating rc2: main OpenWhisk module.
>
> We have resolved all the issues regarding the dependency's license, gradle 
> wrapper, some documentation issues, etc, based on we discussed during last 
> voting mail thread for rc1.
>
> This release comprises of source code distribution only. There is one module 
> within this release. The artifact was built from the following Git commit ID:
> * openwhisk: b1476b9, add until for all ansible retries (#3806)
>
> The source code artifacts can be found at:
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fapache-openwhisk-0.9.0-incubating-rc2%2Fopenwhisk-0.9.0-incubating-sources.tar.gzdata=02%7C01%7Cddascal%40adobe.com%7C3cc28d07d7f14826c89308d5e6ba75e2%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668610870355699sdata=ApPSUBF4Pn4S810megnJehGCTWujGnJJCMzabe53FaM%3Dreserved=0
>
> The SHA-512 checksum for each artifact can be found via:
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fapache-openwhisk-0.9.0-incubating-rc2%2Fopenwhisk-0.9.0-incubating-sources.tar.gz.sha512data=02%7C01%7Cddascal%40adobe.com%7C3cc28d07d7f14826c89308d5e6ba75e2%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668610870355699sdata=H9nGPnzZ%2BfNW%2Bj6PgtARielw238dQd5VQ3dfvOKKF3s%3Dreserved=0
>
> The signature of this artifact can be found via:
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fapache-openwhisk-0.9.0-incubating-rc2%2Fopenwhisk-0.9.0-incubating-sources.tar.gz.ascdata=02%7C01%7Cddascal%40adobe.com%7C3cc28d07d7f14826c89308d5e6ba75e2%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668610870365722sdata=P8DvOZimahaRm6498Nv1yoB9IOm2oPKHmdGcMdabhcA%3Dreserved=0
>
> KEYS file is available here:
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2FKEYSdata=02%7C01%7Cddascal%40adobe.com%7C3cc28d07d7f14826c89308d5e6ba75e2%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668610870365722sdata=6%2FXt5EUl168%2B%2FAkn7pfP9gH5OvpQlwDXjQfNooFt0FY%3Dreserved=0
>
> The documentation can be found via:
> 

Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating rc2: main OpenWhisk module

[Dragos Dascalita Haut]

I vote +1 for Release as Apache OpenWhisk 0.9.0-incubating


Checklist for reference:

[X] Download links are valid.
[X] Checksums and PGP signatures are valid.
[X] Source code artifacts have correct names matching the current release.
[X] LICENSE and NOTICE files are correct for each OpenWhisk repo.
[X] All files have license headers if necessary.
[X] No compiled archives bundled in source archive.


dragos

From: Priti Desai 
Sent: Tuesday, July 10, 2018 12:30:26 PM
To: dev@openwhisk.apache.org
Subject: Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating rc2: main 
OpenWhisk module

I vote +1 for Release as Apache OpenWhisk 0.9.0-incubating



Checklist for reference:

[X] Download links are valid.

[O] Checksums and PGP signatures are valid.

[X] Source code artifacts have correct names matching the current release.

[X] LICENSE and NOTICE files are correct for each OpenWhisk repo.

[X] All files have license headers if necessary.

[O] No compiled archives bundled in source archive.



Cheers

Priti









From:   Rodric Rabbah 

To: dev@openwhisk.apache.org

Date:   07/10/2018 08:34 AM

Subject:Re: [VOTE] Release Apache OpenWhisk 0.9.0-incubating rc2:

main OpenWhisk module







I vote +1 for Release as Apache OpenWhisk 0.9.0-incubating



Checklist for reference:

[X] Download links are valid.

[X] Checksums and PGP signatures are valid.

[X] Source code artifacts have correct names matching the current release.

[X] LICENSE and NOTICE files are correct for each OpenWhisk repo.

[X] All files have license headers if necessary.

[X] No compiled archives bundled in source archive.



On Tue, Jul 10, 2018 at 10:55 AM, Ying Chun Guo 

wrote:



> I vote +1 for Release as Apache OpenWhisk 0.9.0-incubating

>

> Checklist for reference:

> [X] Download links are valid.

> [O] Checksums and PGP signatures are valid.

> [X] Source code artifacts have correct names matching the current

release.

> [X] LICENSE and NOTICE files are correct for each OpenWhisk repo.

> [X] All files have license headers if necessary.

> [X] No compiled archives bundled in source archive.

>

>

> Best regards

> Ying Chun Guo (Daisy)

>

>

> -"Vincent S Hou"  wrote: -

> To: dev@openwhisk.apache.org

> From: "Vincent S Hou" 

> Date: 07/04/2018 02:55AM

> Subject: [VOTE] Release Apache OpenWhisk 0.9.0-incubating rc2: main

> OpenWhisk module

>

> Hi everyone,

>

> This is to call for a vote for the release of Apache OpenWhisk

> 0.9.0-incubating rc2: main OpenWhisk module.

>

> We have resolved all the issues regarding the dependency's license,

gradle

> wrapper, some documentation issues, etc, based on we discussed during

last

> voting mail thread for rc1.

>

> This release comprises of source code distribution only. There is one

> module within this release. The artifact was built from the following

Git

> commit ID:

> * openwhisk: b1476b9, add until for all ansible retries (#3806)

>

> The source code artifacts can be found at:

>

https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fdata=02%7C01%7Cddascal%40adobe.com%7C08b5dd94b1a24e6aa96508d5e69b9c9f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668478378653959sdata=%2BElcewu5j3%2BrIG0au4YHbfNz50WRIV%2FaFL%2BUTZDYbKY%3Dreserved=0



> apache-openwhisk-0.9.0-incubating-rc2/openwhisk-0.9.

> 0-incubating-sources.tar.gz

>

> The MD5 checksum for each artifact can be found via:

>

https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fdata=02%7C01%7Cddascal%40adobe.com%7C08b5dd94b1a24e6aa96508d5e69b9c9f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668478378653959sdata=%2BElcewu5j3%2BrIG0au4YHbfNz50WRIV%2FaFL%2BUTZDYbKY%3Dreserved=0



> apache-openwhisk-0.9.0-incubating-rc2/openwhisk-0.9.

> 0-incubating-sources.tar.gz.md5

>

> The SHA-512 checksum for each artifact can be found via:

>

https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fdata=02%7C01%7Cddascal%40adobe.com%7C08b5dd94b1a24e6aa96508d5e69b9c9f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668478378653959sdata=%2BElcewu5j3%2BrIG0au4YHbfNz50WRIV%2FaFL%2BUTZDYbKY%3Dreserved=0



> apache-openwhisk-0.9.0-incubating-rc2/openwhisk-0.9.

> 0-incubating-sources.tar.gz.sha512

>

> The signature of this artifact can be found via:

>

https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fincubator%2Fopenwhisk%2Fdata=02%7C01%7Cddascal%40adobe.com%7C08b5dd94b1a24e6aa96508d5e69b9c9f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636668478378653959sdata=%2BElcewu5j3%2BrIG0au4YHbfNz50WRIV%2FaFL%2BUTZDYbKY%3Dreserved=0



> apache-openwhisk-0.9.0-incubating-rc2/openwhisk-0.9.

> 0-incubating-sources.tar.gz.asc

>

> KEYS file is available here:

>

Re: Invoker HA on Mesos

[Dragos Dascalita Haut]

"... Interacting with the Kubernetes API server to do a labeling operation 
takes around 10ms, so we
couldn't do this on a truly hot path"


I was gonna make a similar comment for this "data plane" part, imagining that a 
performance optimized LB would be interested to have this visibility in memory, 
w/o relying on other components for each single invocation.


From: David P Grove 
Sent: Friday, March 30, 2018 9:58:05 AM
To: dev@openwhisk.apache.org
Subject: Re: Invoker HA on Mesos


Tyson Norris  wrote on 03/27/2018 06:25:59 PM:
>
> Do you have an example of the labels working? I guess the labels are
> changed over time through the lifecycle of the container?
>

Apologies for brutally chopping the email chain; my mail client made a
horrible hash of it.

Right now, all we are doing with Kube labels is to label each action
container with its owning invoker on startup.  This lets us delete orphaned
containers if the invoker crashes and needs to be restarted.  The labeling
happens at [1] and the removal of orphans using the labels at [2].

I think the Kube-native version of part of what you are doing with the
DistributedData for Mesos would be to add and remove additional labels to
give us the option of attaching a new invoker instance to orphaned
containers instead of just destroying them.   Interacting with the
Kubernetes API server to do a labeling operation takes around 10ms, so we
couldn't do this on a truly hot path.  But we could probably afford to
update container labels in parallel with pause/unpause operations, which
could enable re-attachment to any paused containers.

--dave

[1]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fblob%2F0b20df0f725a671f8e51c9e8793116476fd22f76%2Fcore%2Finvoker%2Fsrc%2Fmain%2Fscala%2Fwhisk%2Fcore%2Fcontainerpool%2Fkubernetes%2FKubernetesContainerFactory.scala%23L81=02%7C01%7Cddascal%40adobe.com%7Cefb609bc3ac9496ddccf08d596600540%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636580261504215781=vhkEBD%2BjkTcUDm6csPB5gBg9RZuj%2FViAwGbynBKeE2k%3D=0
[2]
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fblob%2F0b20df0f725a671f8e51c9e8793116476fd22f76%2Fcore%2Finvoker%2Fsrc%2Fmain%2Fscala%2Fwhisk%2Fcore%2Fcontainerpool%2Fkubernetes%2FKubernetesContainerFactory.scala%23L57=02%7C01%7Cddascal%40adobe.com%7Cefb609bc3ac9496ddccf08d596600540%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636580261504215781=njsaRY%2B1DoYpysh7tTMqTaL0lWyPrL8vdnuo%2Bf0sXZY%3D=0

Re: Install Api Gateway in dcos

[Dragos Dascalita Haut]

"...Is there any place I have to define the url?..."

We usually configure a DNS entry like *.gw. and have it pointing 
to the LB for the Public Mesos Agents. The Gateway then looks at the Host 
header and routes to the specific Marathon app.


"...DC/OS VERSION: 1.9.4..."

Are you planning to run Kafka inside the cluster ? We had some issues and 
upgraded to 1.9.6.



Thanks,
dragos

From: Kumar Subramanian 
Sent: Thursday, March 1, 2018 9:27:45 AM
To: dev@openwhisk.apache.org
Cc: d...@openwhisk.incubator.apache.org
Subject: Re: Install Api Gateway in dcos

Hi Dragos,
I am not sure I understood your point about “Note that by default the 
Gateway listens on this domain defined in config
at [2] ( .gw.). Once the 
controller is
installed in the cluster, it's accessible on 
controller.gw.,
assuming the Marathon app name for the OpenWhisk Controller is called
"controller" “

apigateway is the first app that I tried to install on DCOS. Is there any place 
I have to define the url? (( .gw.) and may 
be I am missing that? The only Url I mentioned is the default one in the 
Environment settings of apigateway package (during installation). The default 
value was 
"default":https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarathon.mesos%3A8080=02%7C01%7Cddascal%40adobe.com%7C13c7e7acff6d4ca2527008d57f99c65f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636555220791160814=jqCRcem3GhT%2BChCsOiUI33YxQ5euJqECQfzeSf92yCE%3D=0;
 and I kept it as is when installing.

Thanks,
Kumar.

On 3/1/18, 9:18 AM, "Kumar Subramanian"  wrote:

Hi Dragos,
Here is the DCOS Version
DC/OS VERSION: 1.9.4

Marathon Version: 1.4.7

Thanks,
Kumar.

On 2/28/18, 11:10 PM, "Dascalita Dragos"  wrote:

Hi Kumar,
Can you provide details on the DCOS version as well ?

*"...The (api gateway) service status shows waiting and there are no 
logs;
nothing getting deployed also"*
Depending on the DCOS version, you may find in the DCOS UI debug
information in regards to the resource offered by Mesos and what's 
missing.
[1] It's possible that some constraints are not matched, and Marathon is
still waiting to get offers, which would explain this effect you're 
seeing.
Note that by default the Gateway listens on this domain defined in 
config
at [2] ( .gw.). Once the 
controller is
installed in the cluster, it's accessible on 
controller.gw.,
assuming the Marathon app name for the OpenWhisk Controller is called
"controller". Same goes for the "invoker": 
invoker.gw..
Same for Exhibitor, CouchDB etc, any other app installed in the cluster 
is
exposed in this way. Of course, these domain names can be easily 
configured
differently in NGINX, I'm just explaining the default rules.

We might have to provide a DCOS universe for 1.9 or 1.10 as well. We've
tested with both versions successfully; we need to strip customizations
particular to our deployment, prior to sharing the updated universe.

HTH,
Dragos

[1] - 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__docs.mesosphere.com_1.10_monitoring_debugging_gui-2Ddebugging_%26d%3DDwIFaQ%26c%3DuilaK90D4TOVoH58JNXRgQ%26r%3DF5C8fYlpBJ270qrdwLq2iRQrPd1CLap8zItxk8laWpo%26m%3DPq6qbuCjIV0NgXrQUGgzHlpwkfQ1iR-hlycfGmbR5_4%26s%3D67DJMsbbLqbhR82ScYfqZ6g-hvjuOMnMeGOq-M6zmFQ%26e=02%7C01%7Cddascal%40adobe.com%7C13c7e7acff6d4ca2527008d57f99c65f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636555220791160814=HrhKFD6hNFtAHbQ%2BIc%2F2%2FNz4eOGlRvFblXBG5YRFtzc%3D=0=
[2] -

https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__github.com_adobe-2Dapiplatform_apigateway_blob_master_api-2Dgateway-2Dconfig_conf.d_marathon-5Fapis.conf-23L35%26d%3DDwIFaQ%26c%3DuilaK90D4TOVoH58JNXRgQ%26r%3DF5C8fYlpBJ270qrdwLq2iRQrPd1CLap8zItxk8laWpo%26m%3DPq6qbuCjIV0NgXrQUGgzHlpwkfQ1iR-hlycfGmbR5_4%26s%3D6BVUN80-tqxdqO0yYyI2IuXRoe5XKGEUUwhJhjD8J-o%26e=02%7C01%7Cddascal%40adobe.com%7C13c7e7acff6d4ca2527008d57f99c65f%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636555220791160814=7FXGWVCediKn4vRUuely%2BbquelFxHl3xvYaa97KOBJA%3D=0=



On Wed, Feb 28, 2018 at 6:08 PM Carlos Santana  
wrote:

> Hi Kumar welcome to list
>
>   Adobe team is currently deploying OpenWhisk on DCOS/Mesos I bet one 
of
> them will be able to assist you on this task and point you int he 
right
> direction
>
> The file you point is no longer on the repo, Maybe  Dragos or Tyson 
could
> also clarify that and the apigateway issues.
>
> -- Carlos
>
> On Wed, Feb 

Re: How to share an action but not its code, nor parameters

[Dragos Dascalita Haut]

Following on this thread after a quick chat with Rodric to discuss the fine 
grained approach in more details ( credits go to Rodric ).


For the fine grained permissions OpenWhisk could support unix style 
permissions: "rwx" when defining actions and packages. For example:


   wsk package create my-package --shared yes -a permissions x


This would make actions inside the shared package to be "eXecutable", and w/o 
the "r", other developers won't be able to read the code, nor the default 
parameters.  These permissions could be applied at a more granular level to the 
actions inside a package, in the same way directories and files in unix can 
have different permissions ( in this context, directory = package, and file = 
action).


I think this idea alone would solve the need to share actions with others, 
while keeping the code private. IIUC it should also fix the issue references 
bellow in the thread.


There are some aspects that result from it:

  1.  In Unix, permissions reflect 3 scopes or classes: permissions for the 
owner/user, for the group, and for others. What do these mean in the OW context 
? Do we see benefits from differentiating between these 3 ? Assuming the 
"owner" always has "rwx", there's no "group" concept at the moment in OW (we 
could assume that the "group is the namespace ?), and "others" is what we care 
about when it comes to permissions.
  2.  Implications for displaying activation results. Right now the caller of 
the shared action sees the activation results, including its logs. If the 
permission restricts to "x" only, should it have an implication on the 
activation result ?
  3.  Code vs default parameters. Should permissions differentiate the code 
from the default params or they should apply to both ?


This idea can be extended later to support groups, if OpenWhisk extends the 
namespace:user relation so that a namespace can be accessed by multiple users. 
At that point, a package, or an action, can be shared with specific developers 
belonging to a group, including specific permissions. Right now a package is 
either private, either shared with everybody.


WDYT ?


@Rodric<mailto:rod...@gmail.com>, feel free to add to what I missed.


Thanks,

dragos



From: Dragos Dascalita Haut <ddas...@adobe.com.INVALID>
Sent: Wednesday, February 14, 2018 6:58 PM
To: dev@openwhisk.apache.org
Subject: Re: How to share an action but not its code, nor parameters

"... pursuing the fine grained rights approach. ..."


I was also re-reading your email and I was thinking... it would actually be v 
nice to have this in openwhisk. In this way developers can fully control what 
they want to do with the action. Another use-case I once had was to share an 
action that exposes a key-value store; in that case the action would have the 
credentials to talk to the key-value store which I didn't want to share with 
others. Having an option to hide default params from a shared actions would 
have been handy. So a fine-grained control would help other use-cases too.


dragos

From: Rodric Rabbah <rod...@gmail.com>
Sent: Wednesday, February 14, 2018 6:39:04 PM
To: dev@openwhisk.apache.org
Subject: Re: How to share an action but not its code, nor parameters

It’s worth mentioning that going through the web action path limits the 
compositions to those that can complete is 60s. So there are other benefits to 
pursuing the fine grained rights approach.

-r

Re: How to share an action but not its code, nor parameters

[Dragos Dascalita Haut]

"... pursuing the fine grained rights approach. ..."


I was also re-reading your email and I was thinking... it would actually be v 
nice to have this in openwhisk. In this way developers can fully control what 
they want to do with the action. Another use-case I once had was to share an 
action that exposes a key-value store; in that case the action would have the 
credentials to talk to the key-value store which I didn't want to share with 
others. Having an option to hide default params from a shared actions would 
have been handy. So a fine-grained control would help other use-cases too.


dragos

From: Rodric Rabbah 
Sent: Wednesday, February 14, 2018 6:39:04 PM
To: dev@openwhisk.apache.org
Subject: Re: How to share an action but not its code, nor parameters

It’s worth mentioning that going through the web action path limits the 
compositions to those that can complete is 60s. So there are other benefits to 
pursuing the fine grained rights approach.

-r

Re: How to share an action but not its code, nor parameters

[Dragos Dascalita Haut]

"...The obvious path with composer is to create a combinator for invoking a 
URL, which could treat web actions intrinsically"

How would this work with Composer ?


I.e a 3rd party developer wants to author a composition like:

compose.sequence( "/other-namespace/pkg/action1", 
"/other-namespace/pkg/action2")


The owner of "other-namespace" doesn't want to share the src for action1 or 
action2, but wants others to be able to consume those actions.


Thanks,

dragos


From: Rodric Rabbah 
Sent: Wednesday, February 14, 2018 6:10:47 PM
To: dev@openwhisk.apache.org
Subject: Re: How to share an action but not its code, nor parameters

The obvious path with composer is to create a combinator for invoking a URL, 
which could treat web actions intrinsically.

This discussion has come up before - there’s also this related open issue on 
the topic [1]. An alternative would be fine grained privileges on assets where 
the default rights today are CRUD+Activate.

[1] 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fissues%2F2155=02%7C01%7Cddascal%40adobe.com%7C27e9ad9468524993bd2508d5741957a2%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636542574549914334=9mHPq6tunQqMsLWJlJwEgHgHX36kHCfyskfHgotGuVo%3D=0

-r

How to share an action but not its code, nor parameters

[Dragos Dascalita Haut]

I think some openwhiskers have this use-case and I'm thinking to start a thread 
to see what are the current options. and see if we can land a PR if we want to 
support this use-case.


The need is to share an action with 3rd party developers, while retaining the 
intellectual property of the code, and potentially the associated default 
parameters. 3rd party developers should be able to use the action in any 
composition (meaning to be able to invoke the action from other actions), but 
should not be able to read its code, nor its default params. The action should 
look like a blackbox from the outside. Only the namespace owners should be able 
to manage it ( create, update, delete, etc ).


It looks like this could be achieved in Openwhisk through web actions. The 
action becomes publicly accessible to any 3rd party developer that knows the 
URL, while the code and default params are not visible at all.


With Composer [1], we have a nice programming model, and actions can be easily 
referenced in compositions by their name. We can think of the name of the 
action ( //) as the "URI" in the context of 
OpenWhisk. So the question is if there's something today that allows us to 
reference an action from another namespace in a composition, w/o having to 
share the code with other developers. And if this is not possible, then WDYT if 
we make it possible ?


Thanks,

dragos


[1] - https://github.com/ibm-functions/composer

Re: Introducing myself as an incubation mentor

[Dragos Dascalita Haut]

A warm welcome Bertrand !


From: mcweeney 
Sent: Thursday, February 8, 2018 7:56:24 AM
To: dev@openwhisk.apache.org
Subject: Re: Introducing myself as an incubation mentor

+1 and thanks for agreeing to step in and help us out.


On 02/08/2018 09:38 AM, Sergio Fernández wrote:
> Welcome on board, Bertrand!
>
>
> On Wed, Feb 7, 2018 at 6:39 AM, Bertrand Delacretaz 
> wrote:
>
>> Hi OpenWhisk community,
>>
>> Felix Meschberger is busy with other things, he has asked me to
>> replace him as an incubation mentor and the OpenWhisk PPMC has
>> accepted.
>>
>> I'm happy to do so, the roster at [1] is updated and I'll do my best
>> to help OpenWhisk on its way to graduation, along with Isabel and
>> Sergio.
>>
>> As a brief introduction, I'm an ASF oldtimer, involved since the year
>> 2000, been incubation mentors for a dozen of projects. Regularly
>> active in Apache Sling currently, an OSGi-based Java Web framework
>> that I also use extensively at $work. I'm currently also on the ASF's
>> Board of Directors but that's not really relevant to my role here and
>> certainly doesn't give me more power than others here. - I'm happy to
>> be challenged when people feel like that.
>>
>> I have a blog at 
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgrep.codeconsult.ch=02%7C01%7Cmcweeney%40adobe.com%7C29325264c5b94a4e060408d56f01a536%7C71f1da39c0a84d5a8d88a67b23c30bf4%7C0%7C0%7C636536975207179427=4tsxu9OxvQKQccjdGa3qjszSis8AUm0MnOcFVnR28RE%3D=0
>>  which is mostly about
>> community topics lately.
>>
>> I do have a technical interest in OpenWhisk but haven't found time to
>> dig in much so far. Here's hoping for that!
>>
>> -Bertrand
>>
>> [1] 
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwhimsy.apache.org%2Froster%2Fppmc%2Fopenwhisk=02%7C01%7Cmcweeney%40adobe.com%7C29325264c5b94a4e060408d56f01a536%7C71f1da39c0a84d5a8d88a67b23c30bf4%7C0%7C0%7C636536975207179427=RPBgg2OeBSFkTYx8%2BwVwKTOkOJzDqiwCdmqJYlYjxCc%3D=0
>>  (restricted to ASF
>> committers)
>>

Securing Action Container communication

[Dragos Dascalita Haut]

I'm starting this thread based on our conversations in Slack [1].


This is a sensitive aspect, at least 2 folds:

  1.  Container isolation. making sure action containers can't invoke other 
containers directly, nor other system components directly (db, kafka, kube api, 
mesos api). What are the best ways to achieve this ?
  2.  Protecting restricted data on transit: securing the data plane 
communication via SSL from controller -> kafka -> invoker -> action container.  
Do we want to build this into the project, or treat it optional and only 
document it ? Either way, it would be great to brainstorm together on what are 
the best approaches. WDYT ?


Let's share our thoughts here, and then create issues for the items that we 
want to implement in OpenWhisk; if we want to treat some aspects optional, we 
can at least open issues to document possible approaches ?


Thanks,
dragos

[1] - https://openwhisk-team.slack.com/archives/C3TPCAQG1/p150670440446

Re: http enablement in nginx for local development

[Dragos Dascalita Haut]

Hi Rodric,


"...without running the full test suite against both HTTP and
HTTPS deployments, I for one can't say what will and will not work"

I assume you're referring to the test suite for the CLI, right ?


So the argument is that we can simplify the CLI tests by removing NGINX from 
the local env, assuming that w/o NGINX we can invoke the endpoints in the 
controller via HTTP b/c NGINX should only listen on port 443 ?




Thanks,
dragos

From: Rodric Rabbah 
Sent: Sunday, July 23, 2017 7:28:17 AM
To: dev@openwhisk.apache.org
Subject: http enablement in nginx for local development

If you've deployed openwhisk locally you'll have noticed that the nginx is
listening on port 443 only (https); for local development, we document a
workaround to the self-signed SSL certificate by bypassing nginx. This is
helpful for switching deployments from local to production where the latter
will have a valid certificate but the former requires some additional flags
for wsk CLI or curl to ignore the certificate.

A while ago PR 2042 was offered to open port 80 [1] and enables HTTP  as a
convenience for local dev since it avoids having to ignore the self-signed
cert. On the other hand, this path is not tested explicitly and could rot
but more so, without running the full test suite against both HTTP and
HTTPS deployments, I for one can't say what will and will not work.

At one point, we explored storing the insecure flag in the wsk CLI
properties file but ended up abandoning (see PR 1838 [2]) and instead
documenting a convenient but limited workaround instead (bypassing nginx).

[1] 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fpull%2F2042=02%7C01%7C%7C1e26f14d4bb84aee3c8108d4d1d72f8e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636364169522575347=3MZ%2FyfPbAZJ%2FqnRz8pLyVQ0MF9NahgGrZk71UNNIAMI%3D=0
[2] 
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fincubator-openwhisk%2Fpull%2F1838=02%7C01%7C%7C1e26f14d4bb84aee3c8108d4d1d72f8e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636364169522575347=VJniIpCYlvsB1743CCGMehJ6XHDZlhAmpEVDH2Ug%2F5c%3D=0

I'm inclined to continue to only support HTTPS and in fact push this
protocol even further inside the deployment.

Thoughts?  I will summarize the dev list discussion on the PR as
appropriate and then we can decide that to do with the offered patch.

-r

Openwhisk @ MesosCon North America

[Dragos Dascalita Haut]

FYI,

"OpenWhisk as a Mesos framework" is one of the highlighted sessions this year.

Tyson and I will share more on the topic, probably on separate threads, based 
on our findings.


Thanks,
dragos



From: Linux Foundation Events <no-re...@linuxfoundation.org>
Sent: Thursday, July 20, 2017 2:15 PM
To: Dragos Dascalita Haut
Subject: ANNOUNCING Program for MesosCon North America; Save $200 - Register 
Now! Expires in 5 Days.

#MESOSCON | LOS ANGELES, CA | SEPTEMBER 13-15, 2017



[The Linux 
Foundation]<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgo.linuxfoundation.org%2Fe%2F6342%2F2017-07-20%2F3w216v%2F808508153=02%7C01%7C%7C797ea5bc8776442f30f708d4cfb46e03%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636361821228963361=n8jbFU9EpyDW80p0kX8y%2FhqanXBZk1Jpu4QFK0sbOgo%3D=0>


[ANNOUNCING Program for MesosCon North America! Save $200 Through July 25 | 
Register 
Now]<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgo.linuxfoundation.org%2Fe%2F6342%2Fm-email-utm-campaign-mesosna17%2F3w2162%2F808508153=02%7C01%7C%7C797ea5bc8776442f30f708d4cfb46e03%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636361821228963361=tZ3arwryVraGKtEEuvjF6j%2Bx1hZaJCuhqGRZ66nxKHM%3D=0>

MesosCon North 
America<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgo.linuxfoundation.org%2Fe%2F6342%2Fm-email-utm-campaign-mesosna17%2F3w2164%2F808508153=02%7C01%7C%7C797ea5bc8776442f30f708d4cfb46e03%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636361821228963361=XF59B3Y5HcMAgWNuz9TD%2F41pgKBSWjXsJxoldihUHLg%3D=0>
 is an annual conference organized by the Apache Mesos community, bringing 
together users and developers to share and learn about the project and its 
growing ecosystem. The conference will feature a one-day hackathon followed by 
two days of sessions focused on the Apache Mesos Core and related technologies.
Session highlights include:

  *   How Yelp.com Runs on Mesos in AWS Spot Fleet for Fun and Profit, Kyle 
Anderson, Yelp
  *   Distributed Deep Learning on Mesos with GPUs and Gang Scheduling, Min Cai 
and Alex Sergeev, Uber
  *   DataStax Enterprise on DC/OS - Yes, it’s Possible; Customer Case Studies, 
Kathryn Erickson, DataStax and Ravi Yadav, Mesosphere
  *   Hierarchical Roles and Multi-Role Frameworks in Apache Mesos, Neil 
Conway, Mesosphere & Jay Guo, IBM
  *   Real-Time Event Processing and Handling Stateful Applications on Apache 
Mesos, Balajee Nagarajan and Venkatesh Sivasubramanian, GE Digital
  *   OpenWhisk as an Apache Mesos Framework, Tyson Norris & Dragos Dascalita 
Haut, Adobe Systems, Inc.
  *   Practical Container Scheduling: Juggling Optimizations, Guarantees, and 
Trade-Offs at Netflix, Sharma Podila, Netflix
  *   Fault Tolerant Frameworks - Making Use of CNI Without Docker, Aaron Wood, 
Verizon

View the full schedule of sessions and activities 
→<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgo.linuxfoundation.org%2Fe%2F6342%2Fm-email-utm-campaign-mesosna17%2F3w2166%2F808508153=02%7C01%7C%7C797ea5bc8776442f30f708d4cfb46e03%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636361821228963361=xI%2BF9maFYrtJAXwd%2F1C8nHCtbmOXEcSpH6LrYDVFo18%3D=0>


Save $200 when you register by July 25.

Register Now » 
<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fevents.linuxfoundation.org%2Fevents%2Fmesoscon-north-america%2Fattend%2Fregister-%3Futm_source%3Devent-promo%26utm_medium%3Demail%26utm_campaign%3Dmesosna17=02%7C01%7C%7C797ea5bc8776442f30f708d4cfb46e03%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636361821228963361=Tk6dSQ7zRwrjntx2vIaSuyJGtWRDV9X6Zh523sII75E%3D=0>



Interested in Sponsoring

Showcase your thought leadership among a vibrant open source community and 
connect with top influencers driving today's technology purchasing decisions. 
Learn how to become a sponsor 
today<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgo.linuxfoundation.org%2Fe%2F6342%2Fm-email-utm-campaign-mesosna17%2F3w2168%2F808508153=02%7C01%7C%7C797ea5bc8776442f30f708d4cfb46e03%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636361821228973365=7POuX0sBssqrU0c%2BlxwDS5VseIwM8g17rPGmECnszuo%3D=0>.




Thank You To Our Sponsors

PLATINUM
[Mesosphere]<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgo.linuxfoundation.org%2Fe%2F6342%2F2017-07-20%2F3w216b%2F808508153=02%7C01%7C%7C797ea5bc8776442f30f708d4cfb46e03%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636361821228973365=DhRf3BOONfv52H5FfXYSeZ1GgxBEMEAg%2BJNULznnZqA%3D=0>
 [Verizon Labs] 
<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgo.linuxfoundation.org%2Fe%2F6342%2FTechnologyNetwork%2F3w216d%2F808508153=02%7C01%7C%7C797ea5bc8776442f30f708d4cfb46e03%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636361821228973365=fOZq%2BsmtGo3%2FL3%2BPv1CWRZqwU46Bgzrgk59bmvBYsQM%3D=0>

COMMUNITY
 [Apache Software Foundtion] 
<https://na01.safelinks.protection.o

Re: simplifying the URLs for API calls

[Dragos Dascalita Haut]

Rodric this is a great idea to support it as part of `/v1`.


A quick comment on:


* get by id /api/v2/activations/?id=


Is this a search or a get, in which case we could maybe use:

GET /api/v2/activations/{id}


also what about providing a vanity URL to the latest activation of an 
action ?

GET /api/v2/activations/ns/pkg/action/latest

or

GET /api/v2/actions/ns/pkg/action/activations/latest



dragos

From: Rodric Rabbah 
Sent: Monday, January 16, 2017 12:15:57 PM
To: d...@openwhisk.incubator.apache.org
Subject: simplifying the URLs for API calls

With the recent changes we've made to OpenWhisk to make the mapping of keys
to namespace strictly 1-1, we can simplify the URLs for API calls (some
users have already picked up on this). I'm added a comment to an
outstanding issue about this potential improvement we can make [1] where
initially the thought was to do this as part of a v2 of the API but I think
it's entirely possible to support the cleaner URLs as part of v1 with some
nginx rewrites as well. Regardless, we should leverage the stricter mapping
and simplify the URLs because they'll be less confusing.

I'm pasting my comment below as well for your convenience.
[1] https://github.com/openwhisk/openwhisk/issues/219#issuecomment-272951007


   - *Let* ns = namespace name or “_”
   - *Let* pkg = package name or “default”
   - *Let* action = action name

basic CRUD + activate operations for actions, triggers, rules

show below for actions but intended to be the same for triggers and rules:
read get /api/v2/actions/ns/pkg/action create/update put
/api/v2/actions/ns/pkg/action delete del /api/v2/actions/ns/pkg/action activate
post /api/v2/actions/ns/pkg/action list all actions in package get
/api/v2/actions/ns/pkg list all actions in all packages in namespace get
/api/v2/actions/ns questions and considerations

   - get/put/del only the exec part of the schema e.g., "code" or "zip"?
   - want streaming for the “code” or “zip”, treat as attachments, treat as
   subresource?
   - support ?override, ?createonly, ?updateonly for put
   - list all actions in package can include bindings
   - list all actions in all packages cannot easily includes bindings
   (couchdb reasons)
   - allow for action extensions for content type and projection for result
   path as in "web" feature?

basic CRUD operations for packages get package or binding get
/api/v2/packages/ns/pkg (default not allowed) create/update package or
binding put /api/v2/actions/ns/pkg (default not allowed, cannot be
bound) delete
package or binding /api/v2/actions/ns/pkg (default not allowed) list all
packages and bindings in namespace fet /api/v2/actions/ns basic read
operations for activations get by id /api/v2/activations/?id= get by name
/api/v2/activations/?name= get all activations for action get
/api/v2/activations/ns/pkg/action list all activations for actions in
package get /api/v2/activations/ns/pkg list all activations in all packages
in namespace get /api/v2/activations/ns questions and considerations

   - get only logs, as subresource?
   - get only results, as subresource?
   - want streaming for the “logs” and the “results”, treat as attachments
   also
   - support start, end, skip, limit on all list operations, return totals
   to allow pagination
   - listing activations for packages may not include bindings

Re: Browsing the mailing lists and hostnames

[Dragos Dascalita Haut]

I've also noticed the 2 email addresses @openwhisk.apache.org and the other one 
with "incubator". Are we supposed to use both, or either one is fine ?


Thanks,
dragos

From: Carlos Santana 
Sent: Monday, January 9, 2017 1:20:01 PM
To: dev@openwhisk.apache.org
Subject: Browsing the mailing lists and hostnames

Hi I don't know if you are already aware guys but we can use pony mail to
browse and search any Apache mailing list [1]

For example to get directly to dev list use [2]

I know it's possible to do the same with the private list after login in,
but it's not enable I opened an Infra ticket [3] for someone to take look

One more thing to notice is the domain name for the lists, I did today :-)
There is @openwhisk.apache.org and @openwhisk.incubator.apache.org
when you send emails or refer in pony list.

I use google inbox to organize my mailing list and need to configure with a
To: rule for the filter to include email send to both to catch all emails.

[1]: https://lists.apache.org/
[2]: https://lists.apache.org/list.html?dev@openwhisk.apache.org
[3]:
https://issues.apache.org/jira/servicedesk/agent/INFRA/issue/INFRA-13291

Re: Passing a context object to actions

[Dragos Dascalita Haut]

Rodric, thanks for sharing more info. I've been thinking at it over the weekend.


I'm reading in the Gist comment that the idea is to do:

 1. json_data.event = data;

 2. json_data.context = ...


I'm wondering what do we do when "data" comes in a binary format. Binary 
formats could easily come into play if I think of ML, big data scenarios, and 
others. If the intent is for OpenWhisk to convert all events into a JSON format 
then it could probably mean we'd have to convert the binary to hex ending up 
with something like this:

1. json_data.event = convertToHex(data)

2. json_data.context = ...

I'm not sure this is the most efficient solution from the performance 
perspective. Users would not be able to benefit from the optimizations brought 
by various protocols (i.e. protobuf).

> The gateway doesn’t need to decode E and can treat it as an opaque value.

Let's say the Gateway can do this. There are still a few aspects to be taken 
into consideration:

- The Gateway would have to allocate memory to copy the payload into the new 
object wrapping the event + the context , provided the event (binary or not) is 
sent as a "string".

- The Gateway wouldn't be able to stream the event payload, but it would have 
to buffer it first so that it can read it and then change it. In HTTP the 
headers are sent first and could be read first; this creates an opportunity for 
performance improvements b/c not only the request could be validated without 
reading the body, it could also be stopped earlier if it's malicious. And when 
it's valid the Gateway could decorate the request with extra headers for the 
context (which would also be sent ahead of the body) then start streaming the 
body directly from the client.

> if you want to encode a function that receives three parameters g(E, C, P) 
> where P are
bound params, isn’t this equivalent to a dictionary with three properties
for each argument?

Yes from this perspective it's a fine difference. OpenWhisk internally could 
choose to use this idea too and keep the E, separated from C and P. This is 
where it gets arguable whether this is the best way to communicate the intent; 
I have to admin that probably b/c I'm used with working with a "context" object 
I'm more biased towards it; subjectively speaking it feels more natural for me 
and the performance aspects make me even more biased.

> The reason the signature of actions is
dictionary -> dictionary
is for action composition. If your signature is
(dictionary,dictionary) -> dictionary,
you cannot compose actions into a sequence for example.
I think I need some help to understand how a context object could prevent 
Openwhisk to trigger actions in a sequence.

I'm also adding an extra thought to our thread: if we want to communicate a 
"user_id" and "app_id" to the actions, but any action can edit the incoming 
event, how would we make sure that some important fields like these ones can't 
be overwritten by other actions in a sequence and they can be securely passed 
through and trusted ?

Thanks,
dragos

From: Rodric Rabbah 
Sent: Saturday, January 7, 2017 8:25:32 AM
To: dev@openwhisk.apache.org
Cc: d...@openwhisk.incubator.apache.org
Subject: Re: Passing a context object to actions

I added a comment to your gist (https://gist.github.com/ddragosd/
3afd5d8678e5dbda48b943a1125aad0c#file-zcloudfront-conf-L71-L93) that
reflects my comments about boxing the event.

Passing a context object to actions

[Dragos Dascalita Haut]

Hi,


These days I've been doing some work on creating an OpenWhisk action directly 
from a GitHub repo - issue link [1] ; initial implementation [2] . After 
successfully deploying a "hello-world" action - [3] I thought about updating 
the message with some information from an OAuth token. I've used the API 
Gateway to decorate the request with a "context" object. To achieve this I had 
to merge the context with the actual event payload. I've posted the snippet of 
the API Gateway code at [4] in a gist. Things are working fine and here is a 
sample of a response from the action; note the "context" object which was added 
by the API Gateway.


{

  "payload": "Hello, 1FF62BDC53C3662B from branch-1 !",

  "event": {

"context": {

  "identity": {

"user_id": "1FF62BDC53C3662B",

"client_id": "demo_app",

"scope": "openid,avatar"

  }

}

  }

}


As a result of this experience I've learned a few things out of which the most 
important one that I'd like to open a discussion here on the list is: the 
mechanism to pass a context to an action.


At the moment the only way is to decorate the event with extra fields and I see 
a few drawbacks to this approach:

  * The fact that we need to unpack the request body and change it has an 
impact on performance

  * If we want to send binary payloads in a special format, the API Gateway or 
some other process in the middle has to know how to decode/encode that 
payload/Content-Type. So we're limited to what these intermediary processes 
know.


The same thing happens with default action parameters; they get merged with the 
event. Basically besides the event itself there's no other way to configure the 
action with extra params or pass it a context.


I'd like to get your feedback to see if we're ok to enhance the existing 
mechanism.


As an example,  AWS Lambda allows a similar mechanism to pass a "context" to a 
function. See [5]. Clients can provide a custom "clientContext" object . 
Besides this object, the AWS API Gateway also adds its own fields into the 
"context" object; see [6].


If we want, we could take a similar approach. The way it works in AWS is 
through headers. For instance the "clientContext" is passed as a Base64 
JSON-encoded header.


In OpenWhisk actions the main method signature would allow 2 params instead of 
1 (as it is today) and it could look like:

function main(event, context)


The clientContext would be accessible as:

context.clientContext


And context.identity.* would be something reserved for the OpenWhisk API 
Gateway to populate and it could also be based on special headers like 
"X-GW-Identity".


What I like about this approach with Headers is that no intermediary process 
needs to understand the body of the request; we could also support multiple 
content types easier as well.


WDYT ?


Thanks,

dragos dascalita haut | project lead, software development | adobe cloud 
platform

[1] - https://github.com/openwhisk/openwhisk/issues/1537
[2] - https://github.com/ddragosd/openwhisk-github-deployer
[3] - 
https://github.com/ddragosd/openwhisk-hello-package/blob/branch-1/src/greeting.js
[4] - 
https://gist.github.com/ddragosd/3afd5d8678e5dbda48b943a1125aad0c#file-zcloudfront-conf-L71-L93
[5] - 
http://docs.aws.amazon.com/lambda/latest/dg/API_Invoke.html#API_Invoke_RequestSyntax
[6] - 
http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-mapping-template-reference.html#context-variable-reference