Re: Nice-to-have: Built-in Cluster Sensor List Generator

[Alex Heneveld]

Mike, All-

Rather than move the YAML into Java how about working towards being able 
to import YAML fragments?


In the simplest case this could be something like

brooklyn.enrichers:
- $brooklyn:import(cluster-list-member-addresses-enrichers.yaml)
- ... # another enricher

Where your pair of enrichers are simply saved in another file which you 
can reuse.  This practical only if we can upload a ZIP file which 
contains both our BOM and the YAML it references -- but that feels 
extremely valuable and straightforward once we have OSGi.  We could use 
it for scripts and images also.  And we could use OSGi dependencies so 
for instance "cluster-list...yaml" might be in a bundle called 
"mikes-handy-brooklyn-fragments", and then in my BOM i just declare your 
bundle as a library i need and then i can import the yaml (without 
needing my own copy).


We'd need to think through the semantics of importing but it's natural 
if importing a list into a list automatically flattens one level.  (So 
your aggregator and joiner end up as the first two entries in 
"brooklyn.enrichers",  not having the first entry in that list being 
another list which comes from the imported yaml).


We'd probably also want the ability to customize what we reference.  
Freemarker templates might be the simplest way:


  - $brooklyn:import_template:
url: cluster-list-member-addresses-enrichers.yaml
template_variables:
output_list_sensor_name:  member.host.address.list
output_string_sensor_name: 
member.host.address.list.comma_separated


An alternative to templates would be to have some way of doing deep 
merges to overwrite specific fields -- which is tempting but in my 
explorations that always gets hairy, eg when you want to remove 
something, or when lists are involved.


Another approach is to have a "brooklyn.mixins" section as part of the 
entity spec yaml, allowing you for instance to combine entity specs a la 
multiple inheritance.  So for instance I could define an entity spec 
which just gives those two enrichers.  And then when I want to use it I 
mix in that spec.  This would be also be useful for strong typing, where 
the mixin acts like an interface. So we could have an entity spec 
"Datastore" which defines a sensor "datastore.url".  Then if I write a 
"MyMysql" entity it is of supertype VanillaSoftware but it mixes this 
"Datastore" then defines the feed to populate the sensor.  Elsewhere I 
can say that MyNodeJs requires a "Datastore" type, and MyMysql would be 
eligible.  But I think this would need a lot more thought; importing is 
simpler!


Best
Alex


On 06/05/2016 09:16, Thomas Bouron wrote:

+0.5

I like the idea but I'm afraid it is too specific (i.e for DynamicClusters
only) whereas enrichers are supposed to be generic.
I would then do something like this:

- You can have this for the generic case:

```
 brooklyn.enrichers:
 - type: org.apache.brooklyn.enricher.stock.ListGenerator
   brooklyn.config:
 uniqueTag: member-address-list-generator
 enricher.sourceSensors:
 - $brooklyn:entity("entity1").sensor("member.host.address")
 - $brooklyn:entity("entity2").sensor("member.host.address")
 - $brooklyn:entity("entity3").sensor("member.host.address")
 enricher.targetSensor: $brooklyn:sensor("member.host.address.list")
```

- Or this for a cluster:

```
 brooklyn.enrichers:
 - type: org.apache.brooklyn.enricher.stock.ListGenerator
   brooklyn.config:
 uniqueTag: member-address-list-generator
 enricher.sourceCluster: $brooklyn:entity("my-cluster")
 enricher.sourceSensor: $brooklyn:sensor("member.host.address")
 enricher.targetSensor: $brooklyn:sensor("member.host.address.list")
```

WDYT?

On Fri, 6 May 2016 at 06:52 David Bush  wrote:


+1

Using this in a couple of blueprints at the moment and would be really
nice to have a concise method.

--
David Bush • Systems Integrator • Cloudsoft Corporation •
http://www.cloudsoftcorp.com 
T: 07834 127195 • SKYPE: david.c.bush


On 6 May 2016, at 01:44, Mike Zaccardo 

wrote:

Hi all,

I've encountered the following task numerous times when authoring
`DynamicCluster` blueprints: write a sensor that returns a comma

separated

list of values for each member of a cluster (e.g. the host address of

each

member).

This is currently how to achieve such a task:

  brooklyn.enrichers:
  - type: org.apache.brooklyn.enricher.stock.Aggregator
brooklyn.config:
  uniqueTag: member-address-aggregator
  enricher.aggregator.excludeBlank: true
  enricher.aggregating.fromMembers: true
  enricher.sourceSensor: $brooklyn:sensor("member.host.address")
  enricher.targetSensor:
$brooklyn:sensor("member.host.address.list")

  - type: org.apache.brooklyn.enricher.stock.Joiner
brooklyn.config:
  uniqueTag: member-address-joiner

Re: Nice-to-have: Built-in Cluster Sensor List Generator

[Thomas Bouron]

+0.5

I like the idea but I'm afraid it is too specific (i.e for DynamicClusters
only) whereas enrichers are supposed to be generic.
I would then do something like this:

   - You can have this for the generic case:

```
brooklyn.enrichers:
- type: org.apache.brooklyn.enricher.stock.ListGenerator
  brooklyn.config:
uniqueTag: member-address-list-generator
enricher.sourceSensors:
- $brooklyn:entity("entity1").sensor("member.host.address")
- $brooklyn:entity("entity2").sensor("member.host.address")
- $brooklyn:entity("entity3").sensor("member.host.address")
enricher.targetSensor: $brooklyn:sensor("member.host.address.list")
```

   - Or this for a cluster:

```
brooklyn.enrichers:
- type: org.apache.brooklyn.enricher.stock.ListGenerator
  brooklyn.config:
uniqueTag: member-address-list-generator
enricher.sourceCluster: $brooklyn:entity("my-cluster")
enricher.sourceSensor: $brooklyn:sensor("member.host.address")
enricher.targetSensor: $brooklyn:sensor("member.host.address.list")
```

WDYT?

On Fri, 6 May 2016 at 06:52 David Bush  wrote:

> +1
>
> Using this in a couple of blueprints at the moment and would be really
> nice to have a concise method.
>
> --
> David Bush • Systems Integrator • Cloudsoft Corporation •
> http://www.cloudsoftcorp.com 
> T: 07834 127195 • SKYPE: david.c.bush
>
> > On 6 May 2016, at 01:44, Mike Zaccardo 
> wrote:
> >
> > Hi all,
> >
> > I've encountered the following task numerous times when authoring
> > `DynamicCluster` blueprints: write a sensor that returns a comma
> separated
> > list of values for each member of a cluster (e.g. the host address of
> each
> > member).
> >
> > This is currently how to achieve such a task:
> >
> >  brooklyn.enrichers:
> >  - type: org.apache.brooklyn.enricher.stock.Aggregator
> >brooklyn.config:
> >  uniqueTag: member-address-aggregator
> >  enricher.aggregator.excludeBlank: true
> >  enricher.aggregating.fromMembers: true
> >  enricher.sourceSensor: $brooklyn:sensor("member.host.address")
> >  enricher.targetSensor:
> > $brooklyn:sensor("member.host.address.list")
> >
> >  - type: org.apache.brooklyn.enricher.stock.Joiner
> >brooklyn.config:
> >  uniqueTag: member-address-joiner
> >  enricher.sourceSensor:
> > $brooklyn:sensor("member.host.address.list")
> >  enricher.targetSensor:
> > $brooklyn:sensor("member.host.address.list.comma_separated")
> >  minimum: $brooklyn:config("cluster.size")
> >  separator: ","
> >  quote: false
> >
> > And here[1] is a specific example in a real blueprint.
> >
> > That's pretty verbose for a commonly required task.  Instead, it would be
> > nice to have something resembling the following:
> >
> >  brooklyn.enrichers:
> >  - type: org.apache.brooklyn.enricher.stock.ClusterListGenerator
> >brooklyn.config:
> >  uniqueTag: member-address-list-generator
> >  enricher.sourceSensor: $brooklyn:sensor("member.host.address")
> >  enricher.targetSensor:
> > $brooklyn:sensor("member.host.address.list")
> >
> > Short and sweet!  WDYT?
> >
> > -Mike
> >
> > [1]
> >
> https://github.com/cloudsoft/brooklyn-tendermint/blob/master/tendermint-mintnet.bom#L82
>
> --
Thomas Bouron • Software Engineer @ Cloudsoft Corporation •
http://www.cloudsoftcorp.com/
Github: https://github.com/tbouron
Twitter: https://twitter.com/eltibouron

Re: Nice-to-have: Built-in Cluster Sensor List Generator

[Geoff Macartney]

+1 looks like an excellent suggestion to me



Gnu PGP key - http://is.gd/uI


> On 6 May 2016, at 01:44, Mike Zaccardo  
> wrote:
> 
> Hi all,
> 
> I've encountered the following task numerous times when authoring
> `DynamicCluster` blueprints: write a sensor that returns a comma separated
> list of values for each member of a cluster (e.g. the host address of each
> member).
> 
> This is currently how to achieve such a task:
> 
>  brooklyn.enrichers:
>  - type: org.apache.brooklyn.enricher.stock.Aggregator
>brooklyn.config:
>  uniqueTag: member-address-aggregator
>  enricher.aggregator.excludeBlank: true
>  enricher.aggregating.fromMembers: true
>  enricher.sourceSensor: $brooklyn:sensor("member.host.address")
>  enricher.targetSensor:
> $brooklyn:sensor("member.host.address.list")
> 
>  - type: org.apache.brooklyn.enricher.stock.Joiner
>brooklyn.config:
>  uniqueTag: member-address-joiner
>  enricher.sourceSensor:
> $brooklyn:sensor("member.host.address.list")
>  enricher.targetSensor:
> $brooklyn:sensor("member.host.address.list.comma_separated")
>  minimum: $brooklyn:config("cluster.size")
>  separator: ","
>  quote: false
> 
> And here[1] is a specific example in a real blueprint.
> 
> That's pretty verbose for a commonly required task.  Instead, it would be
> nice to have something resembling the following:
> 
>  brooklyn.enrichers:
>  - type: org.apache.brooklyn.enricher.stock.ClusterListGenerator
>brooklyn.config:
>  uniqueTag: member-address-list-generator
>  enricher.sourceSensor: $brooklyn:sensor("member.host.address")
>  enricher.targetSensor:
> $brooklyn:sensor("member.host.address.list")
> 
> Short and sweet!  WDYT?
> 
> -Mike
> 
> [1]
> https://github.com/cloudsoft/brooklyn-tendermint/blob/master/tendermint-mintnet.bom#L82

Re: Nice-to-have: Built-in Cluster Sensor List Generator

[David Bush]

+1 

Using this in a couple of blueprints at the moment and would be really nice to 
have a concise method.

-- 
David Bush • Systems Integrator • Cloudsoft Corporation • 
http://www.cloudsoftcorp.com 
T: 07834 127195 • SKYPE: david.c.bush

> On 6 May 2016, at 01:44, Mike Zaccardo  
> wrote:
> 
> Hi all,
> 
> I've encountered the following task numerous times when authoring
> `DynamicCluster` blueprints: write a sensor that returns a comma separated
> list of values for each member of a cluster (e.g. the host address of each
> member).
> 
> This is currently how to achieve such a task:
> 
>  brooklyn.enrichers:
>  - type: org.apache.brooklyn.enricher.stock.Aggregator
>brooklyn.config:
>  uniqueTag: member-address-aggregator
>  enricher.aggregator.excludeBlank: true
>  enricher.aggregating.fromMembers: true
>  enricher.sourceSensor: $brooklyn:sensor("member.host.address")
>  enricher.targetSensor:
> $brooklyn:sensor("member.host.address.list")
> 
>  - type: org.apache.brooklyn.enricher.stock.Joiner
>brooklyn.config:
>  uniqueTag: member-address-joiner
>  enricher.sourceSensor:
> $brooklyn:sensor("member.host.address.list")
>  enricher.targetSensor:
> $brooklyn:sensor("member.host.address.list.comma_separated")
>  minimum: $brooklyn:config("cluster.size")
>  separator: ","
>  quote: false
> 
> And here[1] is a specific example in a real blueprint.
> 
> That's pretty verbose for a commonly required task.  Instead, it would be
> nice to have something resembling the following:
> 
>  brooklyn.enrichers:
>  - type: org.apache.brooklyn.enricher.stock.ClusterListGenerator
>brooklyn.config:
>  uniqueTag: member-address-list-generator
>  enricher.sourceSensor: $brooklyn:sensor("member.host.address")
>  enricher.targetSensor:
> $brooklyn:sensor("member.host.address.list")
> 
> Short and sweet!  WDYT?
> 
> -Mike
> 
> [1]
> https://github.com/cloudsoft/brooklyn-tendermint/blob/master/tendermint-mintnet.bom#L82