Pat, would you explain more about the 'instanceId' as in `pio register --variant path/to/some-engine.json --instanceId some-REST-compatible-resource-id` ?
Currently PIO also has a concept of engineInstanceId, which is output of train. I think you are referring to different thing, right? Kenneth On Fri, Sep 16, 2016 at 12:58 PM, Pat Ferrel <p...@occamsmachete.com> wrote: > This is a great discussion topic and a great idea. > > However the cons must also be addressed, we will need to do this before > multi-tenant deploys can happen and the benefits are just as large as > removing `pio build` > > It would be great to get rid of manifest.json and put all metadata in the > store with an externally visible id so all parts of the workflow on all > machines will get the right metadata and any template specific commands > will run from anywhere on any cluster machine and in any order. All we need > is a global engine-instance id. This will make engine-instances behave more > like datasets, which are given permanent ids with `pio app new …` This > might be a new form of `pio register` and it implies a new optional param > to pio template specific commands (the instance id) but removes a lot of > misunderstandings people have and easy mistakes in workflow. > > So workflow would be: > 1) build with SBT/mvn > 2) register any time engine.json changes so make the json file an optional > param to `pio register --variant path/to/some-engine.json --instanceId > some-REST-compatible-resource-id` the instance could also be > auto-generated and output or optionally in the engine.json. `pio engine > list` lists registered instances with instanceId. The path to the binary > would be put in the instanceId and would be expected to be the same on all > cluster machines that need it. > 3) `pio train --instanceId` optional if it’s in engine.json > 4) `pio deploy --instanceId` optional if it’s in engine.json > 5) with easily recognized exceptions all the above can happen in any order > on any cluster machine and from any directory. > > This takes one big step to multi-tenancy since the instance data has an > externally visible id—call it a REST resource id… > > I bring this up not to confuse the issue but because if we change the > workflow commands we should avoid doing it often because of the disruption > it brings. > > > On Sep 16, 2016, at 10:42 AM, Donald Szeto <don...@apache.org> wrote: > > Hi all, > > I want to start the discussion of removing engine registration. How many > people actually take advantage of being able to run pio commands everywhere > outside of an engine template directory? This will be a nontrivial change > on the operational side so I want to gauge the potential impact to existing > users. > > Pros: > - Stateless build. This would work well with many PaaS. > - Eliminate the "pio build" command once and for all. > - Ability to use your own build system, i.e. Maven, Ant, Gradle, etc. > - Potentially better experience with IDE since engine templates no longer > depends on an SBT plugin. > > Cons: > - Inability to run pio engine training and deployment commands outside of > engine template directory. > - No automatic version matching of PIO binary distribution and artifacts > version used in the engine template. > - A less unified user experience: from pio-build-train-deploy to build, > then pio-train-deploy. > > Regards, > Donald > >
