Hello, Whiskers.
I already coded a first implementation of the generic binary action runner
using the pipes. And I am happy to say it works and it is efficient. I was
afraid it was not.
If there is a good reason for using Go is the awesome simplicity of the
implementation using goroutines. Also Go provides out of the box the PipeStdIn
and PipeStdout for commands so I do not have to fight with pipe creation using
unix syscalls.
I do not know how to do in python but I suspect you need to use async io and
not just Flask, and it can be a bit more complex than this...
Code is not yet really ready because I need to do a better error checks,
implementing unzip of the actions and fix an issue when I replace the action.
And I have not yet updated the documentatio. However I am eager to share the
result of the benchmark.
+----------+-------+---------+--------+----------+----------+----------+-----+------+
| Label | Hits | Average | Median | 90% Line | 95% Line | 99% Line | Min |
Max |
+----------+-------+---------+--------+----------+----------+----------+-----+------+
| Docker | 10000 | 2043 | 2038 | 2396 | 2605 | 3031 | 94 |
4016 |
| GoServer | 10000 | 4 | 4 | 7 | 8 | 16 | 1 |
61 |
| PipeExec | 10000 | 5 | 5 | 9 | 11 | 20 | 2 |
13 |
| TOTAL | 30000 | 684 | 6 | 2121 | 2258 | 2689 | 1 |
4016 |
+----------+-------+---------+--------+----------+----------+----------+-----+------+
Docker is using the current docker skeleton, GoServer is a native, all-in-one
go server for init and run, that replaces itself. PipeExec is the new code that
starts a subprocess and keep it running, feeding new requests in standard input
and getting results in standard output, line by line.
Using a native, all-in one action executor the average is 4 milliseconds, while
using pipes the average is 5 ms. I think losing one millisecond (mostly due, I
think, to the fact I decode then re-encode the json) is worth because of the
added generality. This server should work with binaries written in any
language. Minimum is 2 millisecond but strangely max is 13 ms while the max
with a native server is 61.
Below there is the code of the function, I used a slightly modified version of
the example in the IBM Cloud documentation. You do not need to use any library,
just follow the convention of reading ore line and writing one line, and log in
stderr.
Note the example it can be used both in the current dockerskeleton and in my
new implementation. If the first argument is provided, the action will run only
once. If no arguments are provided it will start a read-write loop, reading a
line, feeding to the action and outputting the result.
```
package main
import (
"bufio"
"encoding/json"
"fmt"
"os"
)
func hello(arg string) string {
var obj map[string]interface{}
json.Unmarshal([]byte(arg), &obj)
name, ok := obj["name"].(string)
if !ok {
name = "Stranger"
}
msg := map[string]string{"message": ("Hello, " + name + "!")}
res, _ := json.Marshal(msg)
return string(res)
}
func main() {
// native actions receive one argument, the JSON object as a string
if len(os.Args) > 1 {
fmt.Println(hello(os.Args[1]))
return
}
// read loop
reader := bufio.NewReader(os.Stdin)
for {
event, err := reader.ReadString('\n')
if err != nil {
break
}
fmt.Println(hello(event))
}
}
```
Ok, there is still a consistent amount of work to do to manage misbehaving
actions, I know. Working on it.
--
Michele Sciabarra
[email protected]
