This is a good discussion - thanks for bringing it to the dev list.
In essence, native actions push the boundary of how much of the function
abstraction we can maintain. For some of the managed runtimes which include
Node.js, Python and Java, we are able to hide the protocol you allude to in
what we loosely have called the runtime proxy. The proxy is where the
initialization and run protocols are relevant. We deliberately resisted
publishing and documenting the proxy protocol for some time.
As you observed, the initialization must handshake to the invoker - we do
this today with a generic HTTP response code of 2xx. Anything else is
treated as error. If your current proposal, this is equivalent to
{"openwhisk": 1}.
The function can only reach the run stage if initialization returns
successfully. This does not however provide any strong guarantee that the
action is well formed or valid. Only that for Node.js for example, that it
parses OK. All errors are eventually detected at the run phase.
In the work you've done, because of the nature of the native functions, one
way to look at the changes is that you're exposing the implicit proxy
protocol to the "function" itself. It must confirm that it's ready (for
whatever notion of readiness it deems valid), and only then is the run
executed.
In this way, one can say that a function execution in this model follows a
different model, where there are at least two (although can admit three)
stages: the function must implement an initializer and a run method, and if
you admit the third, a shutdown method (we don't have this today in general
but it would be nice to allow a "function" to shutdown cleanly before its
resources are killed).
These changes are a departure from the simple interface that some of the
languages have, but already for native functions, they _have_ to do
something different. Namely, they receive the input from stdin, and they
produce the result on stdout (as the last line); this is not a nice and
clean function interface compared to say Node.js, Python, or Java.
So I like to think of this and suggest it as a way of grounding the
discussion in terms of a foreign function interfaces for OpenWhisk. A
native function already breaks some of the clean abstractions. Before I
elaborate further, I will pause here for feedback.
-r
On Fri, Mar 9, 2018 at 6:13 AM, Michele Sciabarra <openwh...@sciabarra.com>
wrote:
> I just did a PR of my version of the Golang action implementation. It
> does some "breaking" changes and there is some discussion on the slack
> channel.
>
> So I report the current situation n here, looking for advices and change
> recommendations. Since I am a bit confused, if I remember well, one Apache
> rule is the mailing list is the ultimate source for the truth...
>
> It currently works this way (I call it the "pipe-loop" protocol)
>
> A golang action (or a generic binary) is expected to follow this
> "protocol":
>
> * starts with {"openwhisk": 1}
> * reads on line in stardard input, expecting a json ON A SINGLE LINE
> * process the line, emits logs in stderr (can be multiple lines)
> * outputs a line in stdout in json format ON A SINGLE LINE
> * repeat forever
>
> It is important to note this design is easy to implement and works even
> for bash scripts, but it is easy to use also perl, ruby, haskell in an
> EFFICIENT way. Indeed this bash script (with jq) is part of my tests:
>
> ---
> #!/bin/bash
> echo '{"openwhisk":1}'
> while read line
> do
> name="$(echo $line | jq -r .name)"
> logger -s "name=$name"
> hello="Hello, $name"
> logger -s "sent response"
> echo '{"hello":"'$hello'"}'
> done
> ---
>
> Things discussed:
>
> 1) remove the header {"openwhisk":1}
>
> Actually initially it was not there. But I decided to add this
> requirements because the action need to speak a protocol ANYWAY.
>
> Most important, I explain why I require it starts with "{"openwhisk: 1}".
>
> The main reason is: I start the child process at init time, and I wanted
> to detect when it does not behave properly.
>
> The simplest problem happens when the action crashes immediately. For
> example, a common reason for this problem is uploading a binary using some
> dynamic libraries not available in the runtime. For example a swift
> action. By defaults it load a lot of different libraries, it crashes
> immediately but I cannot detect it until I try to read its stdin.
>
> I can remove this requirement if someone can show me the go code to check
> that cmd.Start("true") or cmd.Start("pwd") exited 😃
>
> If it is not doable, and I skip the handshake, even if the command
> crashed, I will not detect the problem until a /run is executed and the
> action times out...
>
> Carlos say it is fine. It is ok for me but I still think an early problem
> detection would be better. Also James recommended me to provide as much as
> error detection to the user as early as possible. Kinda of conflicting
> directives here...
>
> Suggestions?
>
> 2) more checks at init time
>
> I added some sanity checks. Probably too many. I tried to detect the
> error at deployment time, not at invocation time.
>
> This is different from what currently for example dockerskeleton does.
>
> If I upload for example something wrong, like a non-zip, a non-elf
> executable, my init returns {"error": "description"}, while currently the
> dockerskeleton returns always OK.
>
> Recommendations here?
>
> 3) output to another channel the result
>
> Currently I require logs goes to stderr, and stdout is for interacting
> with the parent process.
>
> Rodric suggested to output to a separate channel (channel 3?) and use
> stdout and stderr for logs.
>
> While doable, I need to provision another pipe, and the implementation
> should probably do some syscalls to retrieve file descriptor 3. It would
> complicate implementation, while currently it is straightforward for any
> language that does not have a library available. For swift, even to flush
> stdout I needed to write "linux specific" code... I do not dare to think
> what I need to do to write in fd3...
>
> My opinion is that using stdout for I/O and stderr for logs is a better
> choice than opening another file descriptor.
>
> Thoughts here?
>
>
>
>
>
>
>
> --
> Michele Sciabarra
> openwh...@sciabarra.com
>
