Re: A proposal for a more efficient implementation of Go actions

[Michele Sciabarra]

On Sat, Feb 10, 2018, at 4:51 PM, Carlos Santana wrote:
> I forgot to mention the use case were the user uploads source code .go
> action the /init will compiled it for them and then run the executable
> 
> Like Rodric said also the whole proxy can be all golang and no python
> involved.
> 
>
It is not too difficult to difficult to support also sources in Go. However it 
is basically the same.
Instead of running the executable you precompile it and build the "exec" before 
executing it.

 
> — Carlos
> 
> On Sat, Feb 10, 2018 at 10:47 AM Carlos Santana 
> wrote:
> 
> > Hi Michele
> >
> > I’m not super familiar with go, but I think it goes inline with what we do
> > with.
> >
> > So If I understand correctly your proposal is for the use case that user
> > provide and precompiled the executable.
> > One approach is for the web server that implements /init on port 8080 to
> > run the go executable the go executable in turn be a another web server
> > that implement the /run endpoint
> >
> > One problem I see is the /run running on port 8080 there will be a port
> > conflict.
> > There could be different options
> > To avoid having use another port like 8081 this will required changes in
> > the invoker to know about.
> > I will try to avoid changes on the invoker to start
> >
> > Another option is for the executable that implements the /run to tell the
> > server on /init to shutdown to release the port and then bind the web
> > server to port 8080 then from this point all the invokes will hit the /run
> > on the users executable action implementing the webserver.
> >
> > This can be documented as the standard api for anyone to write this web
> > server executable.
> >
> > This approach can be also be applicable to other compile languages like
> > Swift which I’m currently working which I’m currently implementing for
> > Swift 4.
> >
> > I agree providing a user land library/package like you mentioned will be
> > an improvement if user want an easy way to implement the executable
> > implementing the webserver for the /run endpoint.
> >
> > This is a good experiment and can be implemented today in user land using
> > docker action and quickly iterate and get feedback.
> >
> > This approach can be implemented with modifications to the python proxy to
> > handle the /init but not the /run
> >
> > Something that can be use for Swift to improve the mode of passing params
> > via stdin and last line of stdout and have memory with state and
> > connections and preserved improving performance.
> >
> > Similar as the go library can be offer for Swift to implement the web
> > server wrapper.
> > And other languages :-)
> >
> >
> > I will let others shine in their point of view and other alternatives
> >
> > Thank you for your interested in helping with Go support
> >
> > — Carlos
> > On Sat, Feb 10, 2018 at 9:52 AM Michele Sciabarra 
> > wrote:
> >
> >> # How Go Action are currently implemented
> >>
> >> Currently, Go Action are implemented using the generic docker support.
> >>
> >> I investigated the implementation and I found there is a python based
> >> server, listening for /init and /run requests. The /init will collect an
> >> executable and place in the current folder, while the /run will invoke the
> >> executable with popen, feeding the input and returning the output as log,
> >> and the last line as the result as a serialized json.
> >>
> >> The problem is spawning a new process for each request sound like a
> >> revival of the CGI.  I made some investigation and it is certainly not the
> >> most efficient implementation.  Basically everyone moved from CGI executing
> >> processes to servers listening  fro requests since many many years ago.
> >>
> >> Also I investigated how AWS Lambda supports go, and as I suspected it is
> >> a completely different story: they implements a server in Go listening and
> >> serving requests.
> >>
> >> The problem here is Python and Node are dynamic scripting languages,
> >> while Go is a compiled language.
> >> I checked also how Node and Python runtimes work: they are all servers,
> >> they receive the code of the function, “eval" the code and then execute it
> >> for serving requests.
> >>
> >> Go, generating an executable, cannot afford to do that. We cannot “eval”
> >> precompiled code. But it is also inefficient to spawn a new process for
> >> each function invocation.
> >>
> >> The solution here is to exec only once, when the runtime receive the
> >> executable of the function, at the /init time. Then you should replace the
> >> main executable and  serve the /run requests directly in the replaced
> >> executable. Of course this means that the replaced executable should be
> >> able to serve the /init requests too. All of this should go in a library
> >>
> >> # My proposal.
> >>
> >> Support for Go should look like to the following (very similar to Lambda
> >> support, I admit it):
> >>
> >> package main
> >>
> >> import 

Re: A proposal for a more efficient implementation of Go actions

[James Thomas]

On 12 February 2018 at 15:06, Michele Sciabarra  wrote:
> In my opinion you can have 2 ways of supporting native actions.
> One is with the "popen" that will start another process.  Another is with the 
>  replacement, the one I am trying to implement.

It'd be great to run some performance tests once you have created this
new runtime to validate the expected performance improvement.

-- 
Regards,
James Thomas

Re: A proposal for a more efficient implementation of Go actions

[James Thomas]

I've added the Apache License to the project now. Use away!

On 12 February 2018 at 14:47, Michele Sciabarra  wrote:
> Very interesting. You actually implemented the "run" handler.
>  The "init" handler is still a stub, if I understood correctly.
>
> Can I use your code? You know, you should have signed the contributor 
> agreement (as I did) and place the code under the Apache license...
>
> --
>   Michele Sciabarra
>   mich...@sciabarra.com
>
> On Mon, Feb 12, 2018, at 12:56 PM, James Thomas wrote:
>> This is a good write-up of supporting a Go-runtime. I wrote a sample
>> library to help Go a while back, it might be useful to review for a
>> starting point:
>> https://github.com/jthomas/ow
>>
>> If we have a `binary` runtime for Go, can this also be used with other
>> static binaries if they support the correct API endpoints? If so, we
>> could have a generic binary runtime to support more languages.
>>
>>
>> On 10 February 2018 at 16:37, Carlos Santana  wrote:
>> > details details ...
>> >
>> > Will help along the way
>> >
>> > Yay !! OpenWhisk Go!
>> > On Sat, Feb 10, 2018 at 10:57 AM Rodric Rabbah  wrote:
>> >
>> >>  > One problem I see is the /run running on port 8080 there will be a port
>> >> conflict.
>> >>
>> >> Michele said "exec", so the initial proxy will replace itself with a new
>> >> proxy and the compiled function.
>> >> Should avoid port conflict. A wrinkle I see is terminating the /init
>> >> connection held by the invoker (so this has to be orchestrated properly).
>> >>
>> >> There might also a need to retry the /run in case of a failed connection
>> >> (because new proxy isn't up). We had disabled this explicitly at one 
>> >> point,
>> >> but I don't foresee a show stopper yet.
>> >>
>> >> -r
>> >>
>>
>>
>>
>> --
>> Regards,
>> James Thomas



-- 
Regards,
James Thomas

Re: A proposal for a more efficient implementation of Go actions

[Michele Sciabarra]

Very interesting. You actually implemented the "run" handler.
 The "init" handler is still a stub, if I understood correctly.

Can I use your code? You know, you should have signed the contributor agreement 
(as I did) and place the code under the Apache license...

-- 
  Michele Sciabarra
  mich...@sciabarra.com

On Mon, Feb 12, 2018, at 12:56 PM, James Thomas wrote:
> This is a good write-up of supporting a Go-runtime. I wrote a sample
> library to help Go a while back, it might be useful to review for a
> starting point:
> https://github.com/jthomas/ow
> 
> If we have a `binary` runtime for Go, can this also be used with other
> static binaries if they support the correct API endpoints? If so, we
> could have a generic binary runtime to support more languages.
> 
> 
> On 10 February 2018 at 16:37, Carlos Santana  wrote:
> > details details ...
> >
> > Will help along the way
> >
> > Yay !! OpenWhisk Go!
> > On Sat, Feb 10, 2018 at 10:57 AM Rodric Rabbah  wrote:
> >
> >>  > One problem I see is the /run running on port 8080 there will be a port
> >> conflict.
> >>
> >> Michele said "exec", so the initial proxy will replace itself with a new
> >> proxy and the compiled function.
> >> Should avoid port conflict. A wrinkle I see is terminating the /init
> >> connection held by the invoker (so this has to be orchestrated properly).
> >>
> >> There might also a need to retry the /run in case of a failed connection
> >> (because new proxy isn't up). We had disabled this explicitly at one point,
> >> but I don't foresee a show stopper yet.
> >>
> >> -r
> >>
> 
> 
> 
> -- 
> Regards,
> James Thomas

Re: A proposal for a more efficient implementation of Go actions

[Michele Sciabarra]

> If we have a `binary` runtime for Go, can this also be used with other
> static binaries if they support the correct API endpoints? If so, we
> could have a generic binary runtime to support more languages.
I definitely think yes.

In my opinion you can have 2 ways of supporting native actions.
One is with the "popen" that will start another process.  Another is with the  
replacement, the one I am trying to implement.

However I think  it is possible to have something in the middle, for example 
with a named pipe.

An executable may then be started as a child process, for example, reading 
continuously the input in the file descriptor 2 line by line and writing in fd 
3. Or maybe using IPC.

After implementing the Go support we can see if this is also possible.


> 
> 
> On 10 February 2018 at 16:37, Carlos Santana  wrote:
> > details details ...
> >
> > Will help along the way
> >
> > Yay !! OpenWhisk Go!
> > On Sat, Feb 10, 2018 at 10:57 AM Rodric Rabbah  wrote:
> >
> >>  > One problem I see is the /run running on port 8080 there will be a port
> >> conflict.
> >>
> >> Michele said "exec", so the initial proxy will replace itself with a new
> >> proxy and the compiled function.
> >> Should avoid port conflict. A wrinkle I see is terminating the /init
> >> connection held by the invoker (so this has to be orchestrated properly).
> >>
> >> There might also a need to retry the /run in case of a failed connection
> >> (because new proxy isn't up). We had disabled this explicitly at one point,
> >> but I don't foresee a show stopper yet.
> >>
> >> -r
> >>
> 
> 
> 
> -- 
> Regards,
> James Thomas

Re: A proposal for a more efficient implementation of Go actions

[Michele Sciabarra]

Very interesting. You actually implemented the "run" handler.
 The "init" handler is still a stub, if I understood correctly.

Can I use your code? You know, you should have signed the contributor agreement 
(as I did) and place the code under the Apache license..

-- 
  Michele Sciabarra
  openwh...@sciabarra.com

On Mon, Feb 12, 2018, at 12:56 PM, James Thomas wrote:
> This is a good write-up of supporting a Go-runtime. I wrote a sample
> library to help Go a while back, it might be useful to review for a
> starting point:
> https://github.com/jthomas/ow
> 
> If we have a `binary` runtime for Go, can this also be used with other
> static binaries if they support the correct API endpoints? If so, we
> could have a generic binary runtime to support more languages.
> 
> 
> On 10 February 2018 at 16:37, Carlos Santana  wrote:
> > details details ...
> >
> > Will help along the way
> >
> > Yay !! OpenWhisk Go!
> > On Sat, Feb 10, 2018 at 10:57 AM Rodric Rabbah  wrote:
> >
> >>  > One problem I see is the /run running on port 8080 there will be a port
> >> conflict.
> >>
> >> Michele said "exec", so the initial proxy will replace itself with a new
> >> proxy and the compiled function.
> >> Should avoid port conflict. A wrinkle I see is terminating the /init
> >> connection held by the invoker (so this has to be orchestrated properly).
> >>
> >> There might also a need to retry the /run in case of a failed connection
> >> (because new proxy isn't up). We had disabled this explicitly at one point,
> >> but I don't foresee a show stopper yet.
> >>
> >> -r
> >>
> 
> 
> 
> -- 
> Regards,
> James Thomas

Re: A proposal for a more efficient implementation of Go actions

[James Thomas]

This is a good write-up of supporting a Go-runtime. I wrote a sample
library to help Go a while back, it might be useful to review for a
starting point:
https://github.com/jthomas/ow

If we have a `binary` runtime for Go, can this also be used with other
static binaries if they support the correct API endpoints? If so, we
could have a generic binary runtime to support more languages.


On 10 February 2018 at 16:37, Carlos Santana  wrote:
> details details ...
>
> Will help along the way
>
> Yay !! OpenWhisk Go!
> On Sat, Feb 10, 2018 at 10:57 AM Rodric Rabbah  wrote:
>
>>  > One problem I see is the /run running on port 8080 there will be a port
>> conflict.
>>
>> Michele said "exec", so the initial proxy will replace itself with a new
>> proxy and the compiled function.
>> Should avoid port conflict. A wrinkle I see is terminating the /init
>> connection held by the invoker (so this has to be orchestrated properly).
>>
>> There might also a need to retry the /run in case of a failed connection
>> (because new proxy isn't up). We had disabled this explicitly at one point,
>> but I don't foresee a show stopper yet.
>>
>> -r
>>



-- 
Regards,
James Thomas

Re: A proposal for a more efficient implementation of Go actions

[Carlos Santana]

details details ...

Will help along the way

Yay !! OpenWhisk Go!
On Sat, Feb 10, 2018 at 10:57 AM Rodric Rabbah  wrote:

>  > One problem I see is the /run running on port 8080 there will be a port
> conflict.
>
> Michele said "exec", so the initial proxy will replace itself with a new
> proxy and the compiled function.
> Should avoid port conflict. A wrinkle I see is terminating the /init
> connection held by the invoker (so this has to be orchestrated properly).
>
> There might also a need to retry the /run in case of a failed connection
> (because new proxy isn't up). We had disabled this explicitly at one point,
> but I don't foresee a show stopper yet.
>
> -r
>

Re: A proposal for a more efficient implementation of Go actions

[Rodric Rabbah]

 > One problem I see is the /run running on port 8080 there will be a port
conflict.

Michele said "exec", so the initial proxy will replace itself with a new
proxy and the compiled function.
Should avoid port conflict. A wrinkle I see is terminating the /init
connection held by the invoker (so this has to be orchestrated properly).

There might also a need to retry the /run in case of a failed connection
(because new proxy isn't up). We had disabled this explicitly at one point,
but I don't foresee a show stopper yet.

-r

Re: A proposal for a more efficient implementation of Go actions

[Carlos Santana]

I forgot to mention the use case were the user uploads source code .go
action the /init will compiled it for them and then run the executable

Like Rodric said also the whole proxy can be all golang and no python
involved.


— Carlos

On Sat, Feb 10, 2018 at 10:47 AM Carlos Santana 
wrote:

> Hi Michele
>
> I’m not super familiar with go, but I think it goes inline with what we do
> with.
>
> So If I understand correctly your proposal is for the use case that user
> provide and precompiled the executable.
> One approach is for the web server that implements /init on port 8080 to
> run the go executable the go executable in turn be a another web server
> that implement the /run endpoint
>
> One problem I see is the /run running on port 8080 there will be a port
> conflict.
> There could be different options
> To avoid having use another port like 8081 this will required changes in
> the invoker to know about.
> I will try to avoid changes on the invoker to start
>
> Another option is for the executable that implements the /run to tell the
> server on /init to shutdown to release the port and then bind the web
> server to port 8080 then from this point all the invokes will hit the /run
> on the users executable action implementing the webserver.
>
> This can be documented as the standard api for anyone to write this web
> server executable.
>
> This approach can be also be applicable to other compile languages like
> Swift which I’m currently working which I’m currently implementing for
> Swift 4.
>
> I agree providing a user land library/package like you mentioned will be
> an improvement if user want an easy way to implement the executable
> implementing the webserver for the /run endpoint.
>
> This is a good experiment and can be implemented today in user land using
> docker action and quickly iterate and get feedback.
>
> This approach can be implemented with modifications to the python proxy to
> handle the /init but not the /run
>
> Something that can be use for Swift to improve the mode of passing params
> via stdin and last line of stdout and have memory with state and
> connections and preserved improving performance.
>
> Similar as the go library can be offer for Swift to implement the web
> server wrapper.
> And other languages :-)
>
>
> I will let others shine in their point of view and other alternatives
>
> Thank you for your interested in helping with Go support
>
> — Carlos
> On Sat, Feb 10, 2018 at 9:52 AM Michele Sciabarra 
> wrote:
>
>> # How Go Action are currently implemented
>>
>> Currently, Go Action are implemented using the generic docker support.
>>
>> I investigated the implementation and I found there is a python based
>> server, listening for /init and /run requests. The /init will collect an
>> executable and place in the current folder, while the /run will invoke the
>> executable with popen, feeding the input and returning the output as log,
>> and the last line as the result as a serialized json.
>>
>> The problem is spawning a new process for each request sound like a
>> revival of the CGI.  I made some investigation and it is certainly not the
>> most efficient implementation.  Basically everyone moved from CGI executing
>> processes to servers listening  fro requests since many many years ago.
>>
>> Also I investigated how AWS Lambda supports go, and as I suspected it is
>> a completely different story: they implements a server in Go listening and
>> serving requests.
>>
>> The problem here is Python and Node are dynamic scripting languages,
>> while Go is a compiled language.
>> I checked also how Node and Python runtimes work: they are all servers,
>> they receive the code of the function, “eval" the code and then execute it
>> for serving requests.
>>
>> Go, generating an executable, cannot afford to do that. We cannot “eval”
>> precompiled code. But it is also inefficient to spawn a new process for
>> each function invocation.
>>
>> The solution here is to exec only once, when the runtime receive the
>> executable of the function, at the /init time. Then you should replace the
>> main executable and  serve the /run requests directly in the replaced
>> executable. Of course this means that the replaced executable should be
>> able to serve the /init requests too. All of this should go in a library
>>
>> # My proposal.
>>
>> Support for Go should look like to the following (very similar to Lambda
>> support, I admit it):
>>
>> package main
>>
>> import (
>> “github.com/apache/openwhisk-go-support"
>> )
>>
>> func hello() (string, error) {
>> return "Hello Whisk!", nil
>> }
>>
>> func main() {
>> openwhisk.Start(hello)
>> }
>>
>> The magic will be inside the library.
>>
>> The Start function will start a web server listening for two requests.
>>
>> Posts to /run will invoke some deserialisation code and then invoke the
>> function.
>>
>> Posts to /init will receive an executable, unzip it, look for an 

Re: A proposal for a more efficient implementation of Go actions

[Carlos Santana]

Hi Michele

I’m not super familiar with go, but I think it goes inline with what we do
with.

So If I understand correctly your proposal is for the use case that user
provide and precompiled the executable.
One approach is for the web server that implements /init on port 8080 to
run the go executable the go executable in turn be a another web server
that implement the /run endpoint

One problem I see is the /run running on port 8080 there will be a port
conflict.
There could be different options
To avoid having use another port like 8081 this will required changes in
the invoker to know about.
I will try to avoid changes on the invoker to start

Another option is for the executable that implements the /run to tell the
server on /init to shutdown to release the port and then bind the web
server to port 8080 then from this point all the invokes will hit the /run
on the users executable action implementing the webserver.

This can be documented as the standard api for anyone to write this web
server executable.

This approach can be also be applicable to other compile languages like
Swift which I’m currently working which I’m currently implementing for
Swift 4.

I agree providing a user land library/package like you mentioned will be an
improvement if user want an easy way to implement the executable
implementing the webserver for the /run endpoint.

This is a good experiment and can be implemented today in user land using
docker action and quickly iterate and get feedback.

This approach can be implemented with modifications to the python proxy to
handle the /init but not the /run

Something that can be use for Swift to improve the mode of passing params
via stdin and last line of stdout and have memory with state and
connections and preserved improving performance.

Similar as the go library can be offer for Swift to implement the web
server wrapper.
And other languages :-)


I will let others shine in their point of view and other alternatives

Thank you for your interested in helping with Go support

— Carlos
On Sat, Feb 10, 2018 at 9:52 AM Michele Sciabarra 
wrote:

> # How Go Action are currently implemented
>
> Currently, Go Action are implemented using the generic docker support.
>
> I investigated the implementation and I found there is a python based
> server, listening for /init and /run requests. The /init will collect an
> executable and place in the current folder, while the /run will invoke the
> executable with popen, feeding the input and returning the output as log,
> and the last line as the result as a serialized json.
>
> The problem is spawning a new process for each request sound like a
> revival of the CGI.  I made some investigation and it is certainly not the
> most efficient implementation.  Basically everyone moved from CGI executing
> processes to servers listening  fro requests since many many years ago.
>
> Also I investigated how AWS Lambda supports go, and as I suspected it is a
> completely different story: they implements a server in Go listening and
> serving requests.
>
> The problem here is Python and Node are dynamic scripting languages, while
> Go is a compiled language.
> I checked also how Node and Python runtimes work: they are all servers,
> they receive the code of the function, “eval" the code and then execute it
> for serving requests.
>
> Go, generating an executable, cannot afford to do that. We cannot “eval”
> precompiled code. But it is also inefficient to spawn a new process for
> each function invocation.
>
> The solution here is to exec only once, when the runtime receive the
> executable of the function, at the /init time. Then you should replace the
> main executable and  serve the /run requests directly in the replaced
> executable. Of course this means that the replaced executable should be
> able to serve the /init requests too. All of this should go in a library
>
> # My proposal.
>
> Support for Go should look like to the following (very similar to Lambda
> support, I admit it):
>
> package main
>
> import (
> “github.com/apache/openwhisk-go-support"
> )
>
> func hello() (string, error) {
> return "Hello Whisk!", nil
> }
>
> func main() {
> openwhisk.Start(hello)
> }
>
> The magic will be inside the library.
>
> The Start function will start a web server listening for two requests.
>
> Posts to /run will invoke some deserialisation code and then invoke the
> function.
>
> Posts to /init will receive an executable, unzip it, look for an "exec"
> file and then exec to it (expecting of course the server itself is
> implemented using the same library).
>
> A new Go specific runtime will built easily using the same library. Maybe
> with a default action retuning an error.
>
> If the proposal sounds acceptable I volunteer to implement it.
>
> --
>   Michele Sciabarra
>   openwh...@sciabarra.com
>

Re: A proposal for a more efficient implementation of Go actions

[Markus Thoemmes]

Reads well thought out and I don't see any immediate issues. Please go for it 
(no pun intended)!

Cheers,
Markus

Re: A proposal for a more efficient implementation of Go actions

[Rodric Rabbah]

Thanks for the excellent write up. TL;DR (if I may): implement the proxy in Go. 

Go! It of course make a lot more sense this way and the advantage of the proxy 
and function in the same language are self evident.

-t