Actually I want to do some benchmarks now. I want to implement a rough version of the version using pipes and compare what happens with the old docker based version, with the replacement (that is entirely native) and with the pipes.
I am very curious of the numbers that I will get. Stay tuned. -- Michele Sciabarra [email protected] On Fri, Feb 16, 2018, at 1:24 PM, Carlos Santana wrote: > This is great +1 > > But for my education and trying to understand > > What if we modify the python actionProxy that we have today to do pipes > instead? > > Then what would be the difference between using python, Go, rust scala? > > Would be interesting to do a quick comparison benchmark once a version of > the go solution is available > > — Carlos > > On Fri, Feb 16, 2018 at 4:12 AM James Thomas <[email protected]> wrote: > > > Great proposal, looks like a really good solution to the challenges > > discussed. Looking foward to seeing this in action. > > > > On 15 February 2018 at 20:20, Michele Sciabarra <[email protected]> > > wrote: > > > I spent some time thinking about my proposal to support Go Actions, and > > I tried to address some weaknesses that emerged in the discussion (again, > > thank to James Thomas for pointing them out). > > > > > > First, the most relevant, my approach requires you implement a > > supporting library able to run an http server in each language you want to > > support. So one for Swift, one for C/C++ (why not?), even one for Rust or D > > or Haskell or whatever. While it is relatively simple, it is definitely not > > a generic solution for supporting binaries. > > > > > > Second, there are some secondary problems: a “race condition”, when I > > upload an action, the current server will be closed so it won’t serve > > actions. For a very small amount of time, but it happens. Also I have some > > concerns about what happens if the user does not upload a proper > > executable. It could break the runtime, so some validation is needed. > > > > > > I hence decided to raise the bar and think to a better implementation. > > My updated proposal is now an extension of the current implementation, but > > using a pipe and keeping the "child" process running to serve actions, not > > terminate and be started again at each request. > > > > > > Instead of “exec” we should launch a process and pipe input, output and > > error. The process however should not be expected (as it is now in Docker > > actions) to read the standard input, write some logs then a serialised json > > objects and then terminate. > > > > > > Instead, the uploaded binary should be expected to: > > > > > > * read continuously the standard input line by line > > > * interpret a single line as a serialised json object > > > * write logs IN THE STANDARD ERROR > > > * write the answer in the output as a single line > > > > > > In this way we can basically use all the compiled languages using just > > the standard libraries. It will behaves like a standard command line > > function. Well, almost (it still will have to encode and decode json). > > > > > > A controlling process (of course done in Go) will receive a “/run” > > request , then it will feed to the process, read his answer and then > > return the answer. > > > > > > The implementation is basically an extension of what I already coded in > > Go (no code to throw away). It is just a bit more difficult... > > > > > > How does this updated proposal sound like? > > > > > > -- > > > Michele Sciabarra > > > [email protected] > > > > > > > > -- > > Regards, > > James Thomas > >
