I quite like Andrew's approach using an initializer, feels elegant and relatively easy to implement.
Regarding the syntax, getting a rid of the `brooklyn.initializer` and `type` is good for a discoverability point of view but that means going through a config key which feels wrong in that case. I don't have an argument for it, it's more a gut feeling. On Tue, 24 May 2016 at 09:37 John McCabe <[email protected]> wrote: > Is there any way we could get rid of the initializers/type parts, rather > than: > > - type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess > brooklyn.initializers: > - type: org.apache.brooklyn.initializer.stock.PackageInstaller > packages: > yum: curl > apt: curl-whatever=2.3.4 > > Something like: > > - type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess > packages: > yum: curl > apt: curl-whatever=2.3.4 > > > On Tue, 24 May 2016 at 09:34 Duncan Grant <[email protected]> > wrote: > > > Andrew, > > > > good point. That's much neater. Should be pretty straightforward to > > implement as well. > > > > Duncan > > > > On Mon, 23 May 2016 at 21:16 Andrew Kennedy < > > [email protected]> wrote: > > > > > Duncan, > > > > > > I like the idea here, of having some extra object that does the package > > > installation, but what about a customizer or initializer instead? You > > could > > > have something like this: > > > > > > - type: org.apache.brooklyn.entity.basic.VanillaSoftwareProcess > > > brooklyn.initializers: > > > - type: org.apache.brooklyn.initializer.stock.PackageInstaller > > > packages: > > > yum: curl > > > apt: curl-whatever=2.3.4 > > > > > > I initially thought of a JcloudsLocationCustomizer but that fails when > we > > > use BYON locations, so I am thinking of something more like a generic > > > entity customizer that runs after the entity has been created, and the > > > location is available and SSHable. There are lots of other things these > > > initializers could do, like setting up users, configuring networking, > > > mounting volumes and so on. I don't think its a big stretch to add this > > > functionality here. > > > > > > Andrew. > > > > > > On Fri, 6 May 2016 at 14:40 Duncan Grant < > [email protected] > > > > > > wrote: > > > > > > > I agree that this is a problem that needs solved but I have > misgivings > > > > about each of the proposed solutions. > > > > > > > > Firstly I think that "discovering" how to write brooklyn yaml can be > > > > difficult. It can be difficult to find things in the documentation > and > > > if > > > > you don't know to look for them in the first place then your only > hope > > is > > > > coming across an example. > > > > > > > > So if we pre-install a set of bash scripts before running the install > > > > scripts I don't see any obvious way for someone using brooklyn to > find > > > out > > > > about those scripts, and when the scripts change I imagine people > won't > > > > notice the new functionality either. > > > > Another downside to using scripts is that it will become more > difficult > > > to > > > > debug script failures. At the moment I can copy the contents of > stdin > > > and > > > > see the error associated. If I had to step into scripts in other > files > > > it > > > > could become much harder to find a failure. In my opinion this ease > of > > > > reproducibility and seeing the code I am running is one of the > > advantages > > > > of using bash over just using chef/puppet/salt/etc. > > > > If we could find a widely used, well tested bash library then some of > > > these > > > > issues might be mitigated but I've yet to find one. > > > > > > > > There is a similar issue with discovering the behaviour of the DSL > > > option. > > > > I think that the DSL only works because we keep it to a minimum and > > > really > > > > it only does one task - which is run-time configuring relationships > > > between > > > > entities. If we extend it then we have replaced java with dsl and I > > > don't > > > > think that benefits anyone. One advantage of the DSL option is that > it > > > > will generate bash so it shouldn't make debugging any harder. > > > > > > > > The advantage of adding config options to the VanillaSoftwareProcess > is > > > > that it adds to extra ways of discovering the behaviour - through the > > > > composer's autocomplete and in javadoc. I assume that this would > > > generate > > > > extra tasks so we could again debug the bash used and it might make > it > > > > easier as we might be able to see exactly which package failed to > > > install. > > > > However the big downside to this is that we are starting to make a > sort > > > of > > > > god object which contains all brooklyn functionality in one object - > > i.e. > > > > VanilllaSoftwareProcess and I'm not sure that's where we want to go. > > > > > > > > After all that I don't really have a good answer. > > > > > > > > My not too good answer would be to make more use of the brooklyn > child > > > > relationship so that a) at least the yaml could be composed from > small > > > > pieces of code b) we could write a InstallPackage entity to simplify > > the > > > > yaml. > > > > > > > > This might look something like: > > > > > > > > - type: brooklyn.entity.basic.VanillaSoftwareProcess > > > > install.command: curl somefile > somfile.txt > > > > brooklyn.config: > > > > children.startable.mode: foreground_early > > > > brooklyn.children: > > > > - type: brooklyn.entity.basic.VanillaSoftwareProcess > > > > install.command: | > > > > which curl || \ > > > > { sudo apt-get update && sudo apt-get install curl ; } > || \ > > > > { sudo yum update && sudo yum install curl ; } || \ > > > > { echo WARNING: cannot install curl && exit 1 ; } > > > > or > > > > - type: brooklyn.entity.basic.InstallPackage > > > > package: curl > > > > or > > > > - type: brooklyn.entity.basic.InstallPackage > > > > packages: > > > > yum: curl > > > > apt: someOtherCurl > > > > > > > > This way everything is discoverable through documentation, javadoc, > and > > > > autocomplete. Common requirements like curl are re-usable. Generated > > > bash > > > > is explicit and if curl fails then it will go on fire. > > > > On the other hand it requires longer yaml, isn't as readable, and due > > to > > > > the use of child parent relationships has a bit of a learning hurdle > > > > > > > > Regards > > > > > > > > Duncan > > > > > > > > > > > > On Wed, 4 May 2016 at 17:35 Thomas Bouron < > > > [email protected] > > > > > > > > > wrote: > > > > > > > > > Hi Guglielmo. > > > > > > > > > > I'm not sure to follow your idea of "stacks", could you elaborate > on > > > that > > > > > please? > > > > > > > > > > Best. > > > > > > > > > > On Tue, 3 May 2016, 13:38 Guglielmo Nigri, < > > > > > [email protected]> wrote: > > > > > > > > > > > I propose a declarative approach. For example we could add a > > > configKey > > > > to > > > > > > VanillaSoftwareProcess called requiredPackages. > > > > > > > > > > > > This way, one could just specify the prerequisite packages, sort > of > > > > > > dependencies for the software process -- this would also open up > > > > > > interesting possibilities such as “stacks” as bundled > dependencies. > > > > > > > > > > > > A declarative approach would also work for Windows (or non-bash > > > > > > environments). > > > > > > > > > > > > Cheers, > > > > > > Guglielmo > > > > > > > > > > > > > > > > > > On 3 May 2016 at 14:28, Andrea Turli < > > [email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > Great discussion guys! > > > > > > > > > > > > > > It seems that it is a shared need, and I wanted to discuss with > > you > > > > as > > > > > I > > > > > > > was not sure about my approach. > > > > > > > > > > > > > > Andrew, > > > > > > > > > > > > > > I like your proposal, thanks! > > > > > > > > > > > > > > Thanks, > > > > > > > Andrea > > > > > > > > > > > > > > > > > > > > > On 3 May 2016 at 13:05, Aled Sage <[email protected]> wrote: > > > > > > > > > > > > > > > I lean towards Andrew's approach, rather than a special > > > > > > > > $brooklyn.installPackage. Note that different distros use > > > different > > > > > > > package > > > > > > > > names sometimes, so the parameters to such a function can get > > > > > annoying. > > > > > > > > > > > > > > > > I've been hesitant about us going down the road of > > > > > > `brooklyn-commands.sh` > > > > > > > > for achieving portable blueprints. It feels like we are > > > increasing > > > > > the > > > > > > > > overlap with things like Chef, Salt, Ansible and Puppet > (which > > we > > > > can > > > > > > > build > > > > > > > > on top of with Brooklyn). > > > > > > > > > > > > > > > > However, if we keep brooklyn-commands.sh small and focused, > > then > > > > > maybe > > > > > > > > it's a good idea. > > > > > > > > > > > > > > > > Aled > > > > > > > > > > > > > > > > p.s. I want to decrease the use of > > > > > `BashCommands.installExecutable()`. > > > > > > > The > > > > > > > > bash command it generates, with all the alternatives and > > > > parentheses, > > > > > > > does > > > > > > > > not look like nice Bash. A brooklyn-commands.sh could do it > > much > > > > > > cleaner, > > > > > > > > with a properly structured multi-line if...elif... (or > > whatever). > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On 03/05/2016 11:43, Andrew Kennedy wrote: > > > > > > > > > > > > > > > >> How about an alternative approach - for all SoftwareProcess > > > > > entities, > > > > > > > >> Brooklyn will copy over a script file called > > > > `brooklyn-commands.sh` > > > > > > and > > > > > > > >> the > > > > > > > >> entity commands will soutce this script before running the > > rest > > > of > > > > > > their > > > > > > > >> configured commands. The script will contain OS agnostic > > > functions > > > > > > > written > > > > > > > >> in Bash that do things like install packages, download Curl, > > > get a > > > > > > file > > > > > > > >> from a URL etc. Then, the install config might look like > this: > > > > > > > >> > > > > > > > >> ``` > > > > > > > >> install.commands: | > > > > > > > >> brooklyn-installPackages apt:openjdk-1.8.0 > > > > > > > yum:java-1.8.0-openjdk-devel > > > > > > > >> java-1.8.0 > > > > > > > >> brooklyn-installPackages curl > > > > > > > >> brooklyn-runAsRoot cp /tmp/whatever /etc/hosts > > > > > > > >> ``` > > > > > > > >> > > > > > > > >> Andrew. > > > > > > > >> > > > > > > > >> On Tue, 3 May 2016 at 11:24 Andrea Turli < > > > > > > > [email protected]> > > > > > > > >> wrote: > > > > > > > >> > > > > > > > >> hi, > > > > > > > >>> > > > > > > > >>> I’ve been thinking about an utility to simplify the YAML > > > > blueprint > > > > > > > >>> creation: from my experience when using > > VanillaSoftwareProcess > > > is > > > > > > > >>> annoying > > > > > > > >>> to write a portable script just to install a package (say, > > > java) > > > > > > valid > > > > > > > >>> for > > > > > > > >>> apt, yum, etc so I usually write it (multiple times) just > for > > > an > > > > > OS. > > > > > > > >>> > > > > > > > >>> To increase the portability of the YAML blueprint I’d like > to > > > > > suggest > > > > > > > we > > > > > > > >>> extend the brooklyn DSL with something like: > > > > > > > >>> ``` > > > > > > > >>> $brooklyn.installPackage(“curl”) > > > > > > > >>> $brooklyn:installPackage({"apt", "openjdk-1.8.0", "yum", > > > > > > > >>> "java-1.8.0-openjdk-devel"}, "java-1.8.0") > > > > > > > >>> ``` > > > > > > > >>> instead of things like > > > > > > > >>> ``` > > > > > > > >>> which curl || \ > > > > > > > >>> { sudo apt-get update && sudo apt-get install curl > > ; } > > > > || > > > > > \ > > > > > > > >>> { sudo yum update && sudo yum install curl ; } || > \ > > > > > > > >>> { echo WARNING: cannot install curl && exit 1 ; } > > > > > > > >>> ``` > > > > > > > >>> I’m not entirely sure this feature fits well on the DSL. > > > > > > > >>> > > > > > > > >>> Alternatively, we could add a configKey to > > > VanillaSoftwareProcess > > > > > > > called > > > > > > > >>> requiredPackages for a more declarative approach > > (@googlielmo's > > > > > idea) > > > > > > > >>> > > > > > > > >>> Wdyt? > > > > > > > >>> > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > Thomas Bouron • Software Engineer @ Cloudsoft Corporation • > > > > > http://www.cloudsoftcorp.com/ > > > > > Github: https://github.com/tbouron > > > > > Twitter: https://twitter.com/eltibouron > > > > > > > > > > > > -- > > > > > > Andrew Kennedy ; Founder clocker.io project ; @grkvlt ; Cloudsoft > > > > > > -- Thomas Bouron • Software Engineer @ Cloudsoft Corporation • http://www.cloudsoftcorp.com/ Github: https://github.com/tbouron Twitter: https://twitter.com/eltibouron
