Thanks Michael for raising these points. I share the same opinion and sentiment and think a branch with a clean migration story is better and makes more sense. I am not entirely convinced that the choice of language itself will make the difference vs the new architecture which is quite different and should in itself be more efficient.
-r On Tue, Aug 28, 2018 at 4:51 PM Michael Marth <mma...@adobe.com.invalid> wrote: > Hi Markus, > > IMHO what you propose below is a rather severe change in scope of this > discussion and effort. > Up until so far this was about _evolving_ the OW architecture. We have not > explicitly discussed it, but one could assume that it is at least feasible > to gradually adopt the new architecture. So there would be a smooth path > between the current state of the code base and a future one. > > Your proposal below breaks this assumption somewhat (by proposing a new > repo instead of a branch - which will inevitably make the 2 code bases > drift apart) as well as explicitly by suggesting a new implementation > language. Especially the latter would create a schism between OW-now and > OW-future. > This schism has implications like the perception of OW-now being > deprecated, the _possibility_ of no clean upgrade path, the immediate split > of the community between *-now and *-future and of course carries the risk > of the version 2 syndrome. > > I would propose to implement the future architecture in a branch and in > Scala first. If it turns out to be good, then subsequent experiments can > show or not-show if a switch of language is of additional value. That would > allow to make a decision based on data rather than anything else. > > My2c > Michael > > > On 28.08.18, 14:26, "Markus Thömmes" <markusthoem...@apache.org> wrote: > > Hi all, > > Am Mo., 27. Aug. 2018 um 20:04 Uhr schrieb David P Grove < > gro...@us.ibm.com > >: > > > > > > > > > "Markus Thömmes" <markusthoem...@apache.org> wrote on 08/23/2018 > 04:19:33 > > PM: > > > > > > > > Key point I want to make is: At some point we'll have to start to > > prototype > > > things out and see if our assumptions actually hold water. For > example, > > my > > > assumption on a work-stealing backend is pretty much in the air. > > > > > > My proposal for going forward would be: > > > 1. Create a playground for the implementation of some parts of the > system > > > (a new repository?) > > > 2. Let's build some of the things that are uncontroversial and > absolutely > > > needed in any case (ContainerRouter, ContainerManager). > > > 3. Play around with them, hook them up in different ways, see what > works > > > and what doesn't. > > > > > > Some things will need some testing out to see the scale that the > > components > > > can operate at. These things will narrow or widen the solution > space for > > > the more controversial topics around how to distribute containers > in the > > > system, how to balance between the routers, work-stealing queue: > yes/no > > etc. > > > > > > Having some simple components fleshed out could encourage > innovation and > > > creates some facts that we need to focus things into a good > direction. > > > > > > What do you think? Too early to start with this and/or the wrong > way of > > > doing it? > > > > > > > +1 for starting to prototype. It's been a good discussion and I > think > > we've identified some things that we know we don't know, so time to > > experiment and find out. > > > > > > Not sure what the best logistics are for this. Would like the work > to be > > at Apache (community visibility). I'm not sure if the best way is a > new > > repo or an experimental branch of the main repo (we could dial down > the > > level of testing on the branch to make it less cumbersome?). The > branch is > > attractive to me because it might make it easier to keep in synch > with the > > components we aren't changing. > > > > I actually think we should generate a new repository for this. Opening > PRs > etc. will then not clutter the "main" repository and we don't need to > worry > about breaking anything. > > If nobody objects I'm going to get "incubator-openwhisk-protoype" > generated > and will create a rough outline in the repository (different folders > for > different parts of the system). > > One more basic question (and this is going to be controversial): Most > of > the componentry in the execution layer will have to be build anew. I'd > like > to switch the implementation language of at least the ContainerRouter > to > Golang. Why? Because scale/performance matters a lot for them. As Dave > mentioned on multiple occasions, it will greatly matter how many > containers > such a Router can handle under load and that scale will define the > implementation alternatives we will have at hand. I therefore would > like to > optimise these Routers for minimal overhead. We could go even lower > level, > but I guess that'd be at a kinda big maintenance cost. > > I can envision the ContainerManager to be simpler to implement in > Golang as > well, at least for some of the deployment alternatives (Kubernetes > comes to > mind). The Golang based clients seemed superior to me vs. clients in > any > other language. > > As all of the communication will probably be HTTP only anyway, these > implementations should be swappable anytime. > > Comments very welcome! Let me know your opinions. > > Cheers, > Markus > > >
