Hadrian, Let me go back for a moment to the Brooklyn location definition: IMHO it answers the question *where* to deploy an entity not *how*. If we agree with this naive definition I think I like Alex's proposal of adding 'capabilities' to a location, primarily ssh, winrm but also config mgmt tool (chef, puppet, salt, etc) and potentially package managers (apt-get, yum, npm, ...) so to enrich the metadata model attached to a location, avoiding complex locations' hierarchies, I hope.
Also I'm not totally sure Brooklyn needs to manage the elasticity of the locations themselves, main focus is to manage the application runtime by making use of whichever infra resource is already available (or provison a new one!) vs managing the underlying infra scalability per se. Wdyt? Andrea Il giorno mar 8 dic 2015 04:31 Hadrian Zbarcea <hzbar...@gmail.com> ha scritto: > Thanks Alex for the comments, pity there aren't more. I either don't fully > understand your answer or I failed to properly communicate my proposal. If > you like the direction at least there may be some merit to the idea and > it's worth exploring more. > > (1) perfectly fine > (2) I don't think the java sub/supertypes are able to properly model > brooklyn abstractions. > > Now back to my proposal ChefLocation was probably of a debatable. But let's > consider Salt or Ansible support (I put a first cut on a branch in my fork > <https://github.com/hzbarcea/incubator-brooklyn/tree/salt-location>). > SaltStack has actually two modes, the master/minion model similar to Puppet > and Chef) and a salt-ssh mode similar to ansible. It is this latter > mode/style I am referring to. I spent quite a bit of time lately with Salt > and I started to like it a lot for many reasons including the license. So, > both salt-ssh and ansible need to know the machines they need to manage > from a distance (called roster in salt and hosts in ansible). Those > machines are necessarily a subset of a location in the general case (e.g. > an EC2 based infrastructure), although in some cases, like byon one could > make the convention to use all the nodes. Therefore, every time a new node > is obtain(ed) the controller node (salt-ssh) needs to add it to its roster, > which in my mind becomes a 'specialized' location like a SaltLocation or > AnsibleLocation managed by an *Infrastructure entity. The same problem (and > model) is used by clocker. > > I believe there would be a better model to use something like a > ManagedLocation with hooks or something, but didn't get far enough to > understand what the commonalities are and for now I want to avoid shooting > in the dark. > > Another point, unrelated to the proposal, but related to the Salt thingie > and testing in general, I came to believe that we under use some cool > features in Brooklyn like the use of the SimulatedLocation. I had to > addRule for using a SimulatedDriver and there is more work to really test > propertly what's going on there, but I think it's a decent way to bring > more unit testing in and rely less on the Live or Integration tests. Btw, > this is my 3rd rewrite of the Salt support, and now I start to like the way > it looks. I have to port more code from my other local branches and then > submit a PR. > > Best, > Hadrian > > > On Thu, Dec 3, 2015 at 10:32 AM, Alex Heneveld < > alex.henev...@cloudsoftcorp.com> wrote: > > > > > Hadrian- > > > > TL;DR - I like the direction you're going in, although I'm not sure of > the > > conclustions. > > > > Firstly there is a lot of synergy with two things I've been working on > > lately: > > > > (1) typed relationships > > (2) multiple supertypes/interfaces on entities/brooklyn objects > > > > > > With (1) we're be better able model relationships between things, to say > X > > is HOSTED_ON Y, where Y is a location. And I could make queries to find > > the unique Y such that X is HOSTED_ON Y and Y is an SshMachineLocation. > Or > > a ChefLocation. Or other. We can express ON_NETWORK to model situations > > where we have multiple networks. > > > > We can also express dependencies better, so rather than say e.g. Tomcat > > has a JDBC_URL=attributeWhenReady("db") I could say that MyTomcat takes a > > required config D of type <Database>, and it has a HOSTED_ON relationship > > with D, and then JDBC_URL=attributeWhenReady(D)... . The upshot is that > a > > user won't need to edit the complex "attributeWhenReady" -- they just > have > > to provide D. You can imagine a drag-and-drop UI makes this easy! > > > > > > With (2) we can capture capabilities as interfaces -- e.g. a ChefLocation > > interface might simply define a `converge(x)` effector. Any entity which > > supports chef then advertises ChefLocation as part of his type. Same for > > Database as discussed in (1). We kinda support this currently using the > > java model, but not for yaml. The recent commits around the new Type > > Registry allows us to track a type hierarchy for arbitrary types, defined > > in yaml, in a way which neatly extends the java hierarchy. (so you can > > continue to use java types.) > > > > > > The natural conclusion of this I think is to make Locations -- and > > Policies and Enrichers and Feeds -- simple sub-types of Entities. And > > possibly nothing more than marker interfaces. Hopefully the vast > majority > > of the code which treats those as different could be removed so we treat > > them the same. > > > > > > Where I'm not sure I agree with you is when it comes to the value of > > building out ChefLocation. It is a clean underlay, but what problem does > > it solve for users? I think we want to optimize for people to write > > blueprints in yaml, like the "my-chef-mysql" example, which I think is > > really nice. > > > > Similar for package managers. I tend to the view that things like > > BashCommands.install are discouraged, except where the installation is > > relatively simple. If you want a real cross-OS blueprint use chef or > > ansible or salt. (And we make that easy.) Or you write blueprints for a > > specific OS and hard-code `apt-get`. Most clouds will give you the > choice > > of ubuntu or centos. What benefit would more PackageManagers give us in > > Brooklyn? To me it feels like we'd be trying to do what others do well. > > > > It's the modelling, grouping, interconnectedness, sensors, policies, and > > effectors that I think are our power, and systems managed at a low-level > > with chef/salt/etc become truly programmable and autonomic whan you wrap > > the ansible/puppet/cf-engine/etc with Brooklyn. (And I love the idea of > > persisting to a queryable, and visualisable, graph store BTW.) > > > > Best > > Alex > > > > > > > > On 01/12/2015 13:12, Hadrian Zbarcea wrote: > > > >> Hi, > >> > >> I was thinking about it for a few months and I still don't have a > >> complete proposal, but since discussions already go in that direction > I'll > >> put it out there and we'll see where it goes. > >> > >> There are a few motivating factors for this proposal: > >> 1. The docs say that a Location is a "A location that an entity can be > >> in". The getParent(), getChildren() methods (if one didn't know the > >> interface name) makes the abstraction more of a tree node. > >> 2. In addition to the tree like organization of Locations, there is a > >> second tree hierarchy of the kind of location (e.g. > ProvisioningLocation, > >> SimulatedLocation, AddressableLocation, etc. This piece is related to > what > >> Andrea mentioned on the other thread (i.e. we could use a > SimulatedDriver > >> to operate at a Location). > >> 3. In my mind the relation is more of a graph. > >> > >> After a few conversations with Brooklyn contributors and committers, I > >> come to believe that we should refactor a Location to mean something > that: > >> > >> - is addressable (probably in multiple ways) > >> - provides an API, actually probably more to operate at that location > >> (e.g. ssh, http, bash commands, etc). Credentials may be required to use > >> the API. > >> > >> In my model Locations are flat, i.e. no in-memory references (pointers) > >> to other locations, just URLs. A location manager (registry, need to > find a > >> reasonable name) maintains one or more data structures to manage the > >> locations. > >> > >> Example: > >> List<String> Location.getAddresses() would return something like: > >> - deployed-at:<id> (equivalent to getParent() today) > >> - geoip:(lat, long) > >> - ssh:root@localhost > >> - karaf:karaf@<ip> > >> - ssh+yum:root@localhost > >> - ssh+npm:root@localhost > >> - ssh+ansible:root@localhost > >> - winrm:<service address> > >> > >> (obviously in the above example we have a both Windows and RHEL box, > >> which is wrong, but it'll do for this proposal :) ). > >> > >> An external service would build a graph and would allow for searches on > a > >> graph given some criteria. > >> > >> Another thought I will mention is that the way our entities are > >> implemented, they give the impression that Brooklyn is more of a CM > tools > >> with the logic of deploying provided in Java. This is both not accurate > and > >> doesn't do Brooklyn justice. It is imo next to impossible to build an > >> entity that would satisfy a large majority of users. The complexity > becomes > >> prohibitive. For that reason I see Brooklyn more of an enabler for > >> implementers of entities to provide and promote their implementation. > E.g. > >> it's next to impossible to implement a best, complete entity for mysql, > and > >> it's more likely to see: hadrian-mysql (that has some characteristics, > say > >> it's super secure, deployed with bash, only works on ubuntu), and > >> andrea-mysql (which is deployed with chef and is supper fast) and > >> cipi-mysql (deployed in a cluster topology, super resilient). A user > would > >> pick the one they prefer and use it in their blueprint. This is no > >> different than the kind of ecosystem docker, chef, puppet, etc, created. > >> > >> I'll stop here hoping for some feedback. This proposal has a more > >> significant backwards compatibility impact and should be considered > >> independently of the other proposal related to locations. > >> > >> Cheers, > >> Hadrian > >> > >> > >> > >> > >> > >> > > >
