Re: [Puppet-dev] metaparam question

[Felix Frank]

Hi,

this is me assuming the role of the weird secluded neighbor who 
sometimes yells incoherent ramblings over the fence. Although this is 
pertinent:


For onlyif and friends, there is already 
https://tickets.puppetlabs.com/browse/PUP-1125, where John and I have 
already done some thoughtwork.


Now two years have passed, and I'm a little overwhelmed by the prospect 
of building yet another complex feature (intense fretting going on over 
here). Don't hold your breath I guess? But if anyone wants to grab it, 
I'll be glad to offer whatever limited guidance I can muster.


Cheers,
Felix

On 03/03/2016 02:24 PM, Trevor Vaughan wrote:
It really seems like we want the logical extension of the Exec checks 
on all resources with additional validation.


service { 'httpd':
  ensure => 'running',
  onlyif => '/usr/bin/probe_remote_mysql', # or 'unless'
  validate => "curl -k http://${::fqdn}/service_check.html";,
  # Ok, magical asynchronous fact update here with an ACL (ponies and 
unicorns area)

  fact_update => {
'my_unique_name' => 'w00t',
# ACLs would need to be processed server side. Let's not give the 
nodes too much power here...

# This could, perhaps, be a server setting.
'acl' => {
  'allow' => $::domain,
  'deny' => 'ALL'
}
  }


  class { 'thing_that_needs_apache':
onlyif => fact_fetch('my_unique_name', 'w00t') # 
fact_fetch(, )

  }

With the async updating of facts, and individual ACL restrictions, you 
get some really neat capabilities in terms of fine grained process 
control.


Thinking about this further, you'll probably need caching proxies for 
the fact updates though.


Maybe Puppet 8 :-D.

Thanks,

Trevor




--
You received this message because you are subscribed to the Google Groups "Puppet 
Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/5771B7F0.4050409%40Alumni.TU-Berlin.de.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[Thomas Müller]

Am Dienstag, 10. Mai 2016 10:52:55 UTC+2 schrieb Gareth Rushgrove:
>
>
> > 
> > validate or verify it works yes, what other uses did you have in mind? 
>
> I guess reporting or metrics or some sort of registration mechanism. 
> Their are other ways of doing those I realise, but as a light-weight 
> integration point this struck me as interesting. I'd probably stick 
> with validate or verify (ie. without the post_) though if it came to 
> that. 
>
> I could see modules shipping with a verify directory containing 
> scripts to run to confirm a working system too. 
>


we started deploy serverspec tests with every profile class. The tests are 
run by puppet.conf's postrun_command on dev systems - other systems run it 
daily by a cronjob. So we can get fast feedback from real sytems if 
configuration is as expected.


- Thomas

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/eab08356-db8b-4639-8747-f87e83cb6b6d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

- Original Message -
> From: "gareth rushgrove" 
> To: "puppet-dev" 
> Sent: Tuesday, 10 May, 2016 10:52:53
> Subject: Re: [Puppet-dev] metaparam question

> On 9 May 2016 at 13:58, R.I.Pienaar  wrote:
>>
>>
>> - Original Message -
>>> From: "gareth rushgrove" 
>>> To: "puppet-dev" 
>>> Sent: Monday, 9 May, 2016 14:55:12
>>> Subject: Re: [Puppet-dev] metaparam question
>>
>>> On 1 May 2016 at 19:33, R.I.Pienaar  wrote:
>>>>
>>>>
>>>> - Original Message -
>>>>> From: "R. I. Pienaar" 
>>>>> To: "puppet-dev" 
>>>>> Sent: Thursday, 3 March, 2016 09:58:42
>>>>> Subject: Re: [Puppet-dev] metaparam question
>>>>
>>>>> - Original Message -
>>>>>> From: "Erik Dalén" 
>>>>>> To: "puppet-dev" 
>>>>>> Sent: Thursday, 3 March, 2016 09:43:30
>>>>>> Subject: Re: [Puppet-dev] metaparam question
>>>>>
>>>>>> On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg  wrote:
>>>>>>
>>>>>>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>>>>>>>
>>>>>>>> hello,
>>>>>>>>
>>>>>>>> I would like to add a metaparameter - which I think is easy now via
>>>>>>>> Type.newmetaparam.
>>>>>>>>
>>>>>>>
>>>>>>> We haven't been thinking of metaparameters as a general purpose 
>>>>>>> extension
>>>>>>> point. This came up once before that I know of, about a year ago, and
>>>>>>> there's a little discussion of this in
>>>>>>> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we 
>>>>>>> reached
>>>>>>> at the time was, more or less, to explore whether the desired change 
>>>>>>> could
>>>>>>> be accomplished with a puppet function and/or a change to core puppet.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> The thing that I can't seem to find any example of though is how to
>>>>>>>> make this metaparameter do something on the nodes for all providers
>>>>>>>> or all types.
>>>>>>>
>>>>>>>
>>>>>>>> Imagine there's a metaparam that might describe how to test a resource
>>>>>>>> works, something like:
>>>>>>>>
>>>>>>>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>>>>>>>>
>>>>>>>> I'd then want to have some code that would be run on the agent nodes
>>>>>>>> for any resource that has this param set.
>>>>>>>>
>>>>>>>
>>>>>>> I don't think something like this exists per se, but 'validate' might be
>>>>>>> one such example of something worth adding to core puppet. Fwiw, one
>>>>>>> resource-specific example added not too long ago is the file type's
>>>>>>> validate_cmd:
>>>>>>>
>>>>>>>
>>>>>>> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
>>>>>>> .
>>>>>>>
>>>>>>>
>>>>>> Just to clarify, validate_cmd works differently in that it validates the
>>>>>> new contents before replacing the file.
>>>>>> I guess the generic validate metaparameter would validate the resource
>>>>>> after it has been synced.
>>>>>
>>>>> yes indeed, subtle but important.  thanks
>>>>
>>>> Late to follow up on this but I had some time to work on this again,
>>>>
>>>>   service{"http":
>>>> ensure => "running",
>>>> post_validate => "/usr/lib64/nagios/plugins/check_http -H localhost"
>>>>   }
>>>>
>>>
>>> First up, this is great. A few minor comments:
>>>
>>> * You explicitly don't trigger the script if it's a noop run or the
>>> resource failed. That obvious matches your sp

Re: [Puppet-dev] metaparam question

[Gareth Rushgrove]

On 9 May 2016 at 13:58, R.I.Pienaar  wrote:
>
>
> - Original Message -
>> From: "gareth rushgrove" 
>> To: "puppet-dev" 
>> Sent: Monday, 9 May, 2016 14:55:12
>> Subject: Re: [Puppet-dev] metaparam question
>
>> On 1 May 2016 at 19:33, R.I.Pienaar  wrote:
>>>
>>>
>>> - Original Message -
>>>> From: "R. I. Pienaar" 
>>>> To: "puppet-dev" 
>>>> Sent: Thursday, 3 March, 2016 09:58:42
>>>> Subject: Re: [Puppet-dev] metaparam question
>>>
>>>> - Original Message -
>>>>> From: "Erik Dalén" 
>>>>> To: "puppet-dev" 
>>>>> Sent: Thursday, 3 March, 2016 09:43:30
>>>>> Subject: Re: [Puppet-dev] metaparam question
>>>>
>>>>> On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg  wrote:
>>>>>
>>>>>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>>>>>>
>>>>>>> hello,
>>>>>>>
>>>>>>> I would like to add a metaparameter - which I think is easy now via
>>>>>>> Type.newmetaparam.
>>>>>>>
>>>>>>
>>>>>> We haven't been thinking of metaparameters as a general purpose extension
>>>>>> point. This came up once before that I know of, about a year ago, and
>>>>>> there's a little discussion of this in
>>>>>> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we reached
>>>>>> at the time was, more or less, to explore whether the desired change 
>>>>>> could
>>>>>> be accomplished with a puppet function and/or a change to core puppet.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> The thing that I can't seem to find any example of though is how to
>>>>>>> make this metaparameter do something on the nodes for all providers
>>>>>>> or all types.
>>>>>>
>>>>>>
>>>>>>> Imagine there's a metaparam that might describe how to test a resource
>>>>>>> works, something like:
>>>>>>>
>>>>>>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>>>>>>>
>>>>>>> I'd then want to have some code that would be run on the agent nodes
>>>>>>> for any resource that has this param set.
>>>>>>>
>>>>>>
>>>>>> I don't think something like this exists per se, but 'validate' might be
>>>>>> one such example of something worth adding to core puppet. Fwiw, one
>>>>>> resource-specific example added not too long ago is the file type's
>>>>>> validate_cmd:
>>>>>>
>>>>>>
>>>>>> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
>>>>>> .
>>>>>>
>>>>>>
>>>>> Just to clarify, validate_cmd works differently in that it validates the
>>>>> new contents before replacing the file.
>>>>> I guess the generic validate metaparameter would validate the resource
>>>>> after it has been synced.
>>>>
>>>> yes indeed, subtle but important.  thanks
>>>
>>> Late to follow up on this but I had some time to work on this again,
>>>
>>>   service{"http":
>>> ensure => "running",
>>> post_validate => "/usr/lib64/nagios/plugins/check_http -H localhost"
>>>   }
>>>
>>
>> First up, this is great. A few minor comments:
>>
>> * You explicitly don't trigger the script if it's a noop run or the
>> resource failed. That obvious matches your specific usecase but I'm
>> wondering if it matches all?
>
> yeah, as this is just playing around with the idea that choice seemed
> the right one without dwelling on it too much - I am not actually sure
> what would be right.
>
> Exec for example has all the unless/onlyif stuff and I dont think those
> are ran during a noop either - I think it assumes a value?
>
> Thinking about it, not running it during noop is the only thing thats
> safe, people might not do read only things there - though they really
> should.  And if its the first run the prereq might 

Re: [Puppet-dev] metaparam question

[Trevor Vaughan]

On a similar vein to this threadmany years ago (six?) I wanted to
introduce a semaphore-based logic construct into the language itself.

I think, with Puppet 4, it might actually be possible!

Technically, as pointed out by one of the team here, we probably want
Phasers, but I didn't know about that construct at the time :-D.

I was going to try to add support to Puppet Server itself but my Clojure-fu
is weak.

The idea was basically:

if amisafe("key_name") > 4 {
  class { 'do puppet stuff': ... }

  letsbesafe('key_name', 0)
}
else {
  class { 'do other optional puppet stuff': ... }
}

I still think it would be interesting to have and, of course, it would
require a central service to manage these things but it could theoretically
be decentralized (consul, etcd, whatever...) and I think it fits in the
general idea of this conversation.

Thanks,

Trevor

On Mon, May 9, 2016 at 2:58 PM, Trevor Vaughan 
wrote:

> Skipping actions during a noop is the only correct solution.
>
> Any time you check a cross-daemon/node state it is relatively likely that
> you're going to be changing something on the network and/or the local
> system.
>
> In many cases, this will be completely innocuous, but in others, it could
> add messages to a queue, or some other nonsense.
>
> I suppose that, if you wanted to get fancy, you could have yet another
> metaparameter which was an array that would let you specify which checks
> could be run in noop mode defaulting to an empty array.
>
> Trevor
>
> On Mon, May 9, 2016 at 8:58 AM, R.I.Pienaar  wrote:
>
>>
>>
>> - Original Message -----
>> > From: "gareth rushgrove" 
>> > To: "puppet-dev" 
>> > Sent: Monday, 9 May, 2016 14:55:12
>> > Subject: Re: [Puppet-dev] metaparam question
>>
>> > On 1 May 2016 at 19:33, R.I.Pienaar  wrote:
>> >>
>> >>
>> >> - Original Message -
>> >>> From: "R. I. Pienaar" 
>> >>> To: "puppet-dev" 
>> >>> Sent: Thursday, 3 March, 2016 09:58:42
>> >>> Subject: Re: [Puppet-dev] metaparam question
>> >>
>> >>> - Original Message -
>> >>>> From: "Erik Dalén" 
>> >>>> To: "puppet-dev" 
>> >>>> Sent: Thursday, 3 March, 2016 09:43:30
>> >>>> Subject: Re: [Puppet-dev] metaparam question
>> >>>
>> >>>> On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg 
>> wrote:
>> >>>>
>> >>>>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>> >>>>>
>> >>>>>> hello,
>> >>>>>>
>> >>>>>> I would like to add a metaparameter - which I think is easy now via
>> >>>>>> Type.newmetaparam.
>> >>>>>>
>> >>>>>
>> >>>>> We haven't been thinking of metaparameters as a general purpose
>> extension
>> >>>>> point. This came up once before that I know of, about a year ago,
>> and
>> >>>>> there's a little discussion of this in
>> >>>>> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we
>> reached
>> >>>>> at the time was, more or less, to explore whether the desired
>> change could
>> >>>>> be accomplished with a puppet function and/or a change to core
>> puppet.
>> >>>>>
>> >>>>>
>> >>>>>>
>> >>>>>> The thing that I can't seem to find any example of though is how to
>> >>>>>> make this metaparameter do something on the nodes for all providers
>> >>>>>> or all types.
>> >>>>>
>> >>>>>
>> >>>>>> Imagine there's a metaparam that might describe how to test a
>> resource
>> >>>>>> works, something like:
>> >>>>>>
>> >>>>>>service{"httpd": validate => "check_http --port 80 -H
>> localhost"}
>> >>>>>>
>> >>>>>> I'd then want to have some code that would be run on the agent
>> nodes
>> >>>>>> for any resource that has this param set.
>> >>>>>>
>> >>>>>
>> >>>>> I don't think something like this exists per se, but 'validate'
>> might be
>> >>>>> one

Re: [Puppet-dev] metaparam question

[Trevor Vaughan]

Skipping actions during a noop is the only correct solution.

Any time you check a cross-daemon/node state it is relatively likely that
you're going to be changing something on the network and/or the local
system.

In many cases, this will be completely innocuous, but in others, it could
add messages to a queue, or some other nonsense.

I suppose that, if you wanted to get fancy, you could have yet another
metaparameter which was an array that would let you specify which checks
could be run in noop mode defaulting to an empty array.

Trevor

On Mon, May 9, 2016 at 8:58 AM, R.I.Pienaar  wrote:

>
>
> - Original Message -
> > From: "gareth rushgrove" 
> > To: "puppet-dev" 
> > Sent: Monday, 9 May, 2016 14:55:12
> > Subject: Re: [Puppet-dev] metaparam question
>
> > On 1 May 2016 at 19:33, R.I.Pienaar  wrote:
> >>
> >>
> >> - Original Message -
> >>> From: "R. I. Pienaar" 
> >>> To: "puppet-dev" 
> >>> Sent: Thursday, 3 March, 2016 09:58:42
> >>> Subject: Re: [Puppet-dev] metaparam question
> >>
> >>> - Original Message -
> >>>> From: "Erik Dalén" 
> >>>> To: "puppet-dev" 
> >>>> Sent: Thursday, 3 March, 2016 09:43:30
> >>>> Subject: Re: [Puppet-dev] metaparam question
> >>>
> >>>> On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg 
> wrote:
> >>>>
> >>>>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
> >>>>>
> >>>>>> hello,
> >>>>>>
> >>>>>> I would like to add a metaparameter - which I think is easy now via
> >>>>>> Type.newmetaparam.
> >>>>>>
> >>>>>
> >>>>> We haven't been thinking of metaparameters as a general purpose
> extension
> >>>>> point. This came up once before that I know of, about a year ago, and
> >>>>> there's a little discussion of this in
> >>>>> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we
> reached
> >>>>> at the time was, more or less, to explore whether the desired change
> could
> >>>>> be accomplished with a puppet function and/or a change to core
> puppet.
> >>>>>
> >>>>>
> >>>>>>
> >>>>>> The thing that I can't seem to find any example of though is how to
> >>>>>> make this metaparameter do something on the nodes for all providers
> >>>>>> or all types.
> >>>>>
> >>>>>
> >>>>>> Imagine there's a metaparam that might describe how to test a
> resource
> >>>>>> works, something like:
> >>>>>>
> >>>>>>service{"httpd": validate => "check_http --port 80 -H localhost"}
> >>>>>>
> >>>>>> I'd then want to have some code that would be run on the agent nodes
> >>>>>> for any resource that has this param set.
> >>>>>>
> >>>>>
> >>>>> I don't think something like this exists per se, but 'validate'
> might be
> >>>>> one such example of something worth adding to core puppet. Fwiw, one
> >>>>> resource-specific example added not too long ago is the file type's
> >>>>> validate_cmd:
> >>>>>
> >>>>>
> >>>>>
> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
> >>>>> .
> >>>>>
> >>>>>
> >>>> Just to clarify, validate_cmd works differently in that it validates
> the
> >>>> new contents before replacing the file.
> >>>> I guess the generic validate metaparameter would validate the resource
> >>>> after it has been synced.
> >>>
> >>> yes indeed, subtle but important.  thanks
> >>
> >> Late to follow up on this but I had some time to work on this again,
> >>
> >>   service{"http":
> >> ensure => "running",
> >> post_validate => "/usr/lib64/nagios/plugins/check_http -H localhost"
> >>   }
> >>
> >
> > First up, this is great. A few minor comments:
> >
> > * You explicitly don't trigger the script if it's a noop run or th

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

- Original Message -
> From: "gareth rushgrove" 
> To: "puppet-dev" 
> Sent: Monday, 9 May, 2016 14:55:12
> Subject: Re: [Puppet-dev] metaparam question

> On 1 May 2016 at 19:33, R.I.Pienaar  wrote:
>>
>>
>> - Original Message -
>>> From: "R. I. Pienaar" 
>>> To: "puppet-dev" 
>>> Sent: Thursday, 3 March, 2016 09:58:42
>>> Subject: Re: [Puppet-dev] metaparam question
>>
>>> - Original Message -----
>>>> From: "Erik Dalén" 
>>>> To: "puppet-dev" 
>>>> Sent: Thursday, 3 March, 2016 09:43:30
>>>> Subject: Re: [Puppet-dev] metaparam question
>>>
>>>> On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg  wrote:
>>>>
>>>>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>>>>>
>>>>>> hello,
>>>>>>
>>>>>> I would like to add a metaparameter - which I think is easy now via
>>>>>> Type.newmetaparam.
>>>>>>
>>>>>
>>>>> We haven't been thinking of metaparameters as a general purpose extension
>>>>> point. This came up once before that I know of, about a year ago, and
>>>>> there's a little discussion of this in
>>>>> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we reached
>>>>> at the time was, more or less, to explore whether the desired change could
>>>>> be accomplished with a puppet function and/or a change to core puppet.
>>>>>
>>>>>
>>>>>>
>>>>>> The thing that I can't seem to find any example of though is how to
>>>>>> make this metaparameter do something on the nodes for all providers
>>>>>> or all types.
>>>>>
>>>>>
>>>>>> Imagine there's a metaparam that might describe how to test a resource
>>>>>> works, something like:
>>>>>>
>>>>>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>>>>>>
>>>>>> I'd then want to have some code that would be run on the agent nodes
>>>>>> for any resource that has this param set.
>>>>>>
>>>>>
>>>>> I don't think something like this exists per se, but 'validate' might be
>>>>> one such example of something worth adding to core puppet. Fwiw, one
>>>>> resource-specific example added not too long ago is the file type's
>>>>> validate_cmd:
>>>>>
>>>>>
>>>>> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
>>>>> .
>>>>>
>>>>>
>>>> Just to clarify, validate_cmd works differently in that it validates the
>>>> new contents before replacing the file.
>>>> I guess the generic validate metaparameter would validate the resource
>>>> after it has been synced.
>>>
>>> yes indeed, subtle but important.  thanks
>>
>> Late to follow up on this but I had some time to work on this again,
>>
>>   service{"http":
>> ensure => "running",
>> post_validate => "/usr/lib64/nagios/plugins/check_http -H localhost"
>>   }
>>
> 
> First up, this is great. A few minor comments:
> 
> * You explicitly don't trigger the script if it's a noop run or the
> resource failed. That obvious matches your specific usecase but I'm
> wondering if it matches all?

yeah, as this is just playing around with the idea that choice seemed
the right one without dwelling on it too much - I am not actually sure
what would be right.

Exec for example has all the unless/onlyif stuff and I dont think those
are ran during a noop either - I think it assumes a value?

Thinking about it, not running it during noop is the only thing thats
safe, people might not do read only things there - though they really
should.  And if its the first run the prereq might not be there?

> * Similar vein, the name post_validate assumes the usage is to
> 'validate' the resource. Given the power of this I can imagine other
> uses too, it might be nicer to have a more generic name?

validate or verify it works yes, what other uses did you have in mind?

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/933229095.334955.1462798703903.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[Gareth Rushgrove]

On 1 May 2016 at 19:33, R.I.Pienaar  wrote:
>
>
> - Original Message -
>> From: "R. I. Pienaar" 
>> To: "puppet-dev" 
>> Sent: Thursday, 3 March, 2016 09:58:42
>> Subject: Re: [Puppet-dev] metaparam question
>
>> - Original Message -
>>> From: "Erik Dalén" 
>>> To: "puppet-dev" 
>>> Sent: Thursday, 3 March, 2016 09:43:30
>>> Subject: Re: [Puppet-dev] metaparam question
>>
>>> On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg  wrote:
>>>
>>>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>>>>
>>>>> hello,
>>>>>
>>>>> I would like to add a metaparameter - which I think is easy now via
>>>>> Type.newmetaparam.
>>>>>
>>>>
>>>> We haven't been thinking of metaparameters as a general purpose extension
>>>> point. This came up once before that I know of, about a year ago, and
>>>> there's a little discussion of this in
>>>> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we reached
>>>> at the time was, more or less, to explore whether the desired change could
>>>> be accomplished with a puppet function and/or a change to core puppet.
>>>>
>>>>
>>>>>
>>>>> The thing that I can't seem to find any example of though is how to
>>>>> make this metaparameter do something on the nodes for all providers
>>>>> or all types.
>>>>
>>>>
>>>>> Imagine there's a metaparam that might describe how to test a resource
>>>>> works, something like:
>>>>>
>>>>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>>>>>
>>>>> I'd then want to have some code that would be run on the agent nodes
>>>>> for any resource that has this param set.
>>>>>
>>>>
>>>> I don't think something like this exists per se, but 'validate' might be
>>>> one such example of something worth adding to core puppet. Fwiw, one
>>>> resource-specific example added not too long ago is the file type's
>>>> validate_cmd:
>>>>
>>>>
>>>> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
>>>> .
>>>>
>>>>
>>> Just to clarify, validate_cmd works differently in that it validates the
>>> new contents before replacing the file.
>>> I guess the generic validate metaparameter would validate the resource
>>> after it has been synced.
>>
>> yes indeed, subtle but important.  thanks
>
> Late to follow up on this but I had some time to work on this again,
>
>   service{"http":
> ensure => "running",
> post_validate => "/usr/lib64/nagios/plugins/check_http -H localhost"
>   }
>

First up, this is great. A few minor comments:

* You explicitly don't trigger the script if it's a noop run or the
resource failed. That obvious matches your specific usecase but I'm
wondering if it matches all?
* Similar vein, the name post_validate assumes the usage is to
'validate' the resource. Given the power of this I can imagine other
uses too, it might be nicer to have a more generic name?

Gareth

> This works with the code below and I included a basic thingy that tells
> sensu about this
>
> I had to hack around a bit more than simply adding a metaparam since I had
> to wait for puppet to completely finish with the resource and then do a
> final check after it's done
>
> https://github.com/ripienaar/puppet/commit/a6c659cc3fb3555632bfe94df1411878a04fdc51
>
> This is really nice, it both makes my Puppet runs a bit better and allows
> me to get instant visibility of failures - and configurable notifications
> based on sensu.  It works with any kind of resource.
>
> Ideally the sensu notifier would be something a module can provide and you
> could register multiple event handlers but Henrik says the bindings system
> isn't initialized on the agent side at present and I didn't have time to
> hack around with that.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to puppet-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/puppet-dev/157346444.508343.1462127599948.JavaMail.zimbra%40devco.net.
> For more options, visit https://groups.google.com/d/optout.



-- 
Gareth Rushgrove
@garethr

devopsweekly.com
morethanseven.net
garethrushgrove.com

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/CAFi_6yK4ZXZqHAnm8moUuFD%2Bs%3DrXMG-kt51JN%2B%2B1EWcy_9AFzQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

- Original Message -
> From: "David Lutterkort" 
> To: "puppet-dev" 
> Sent: Saturday, 7 May, 2016 01:32:45
> Subject: Re: [Puppet-dev] metaparam question

> On Fri, May 6, 2016 at 1:02 PM, R.I.Pienaar  wrote:
> 
>> Those checks you have in AO are good but the barrier to entry is quite
>> high on
>> making those I felt
> 
> 
> Understood, and not trying to argue, but because I am curious:  what's the
> barrier to entry that keeps you from using that mechanism ? Not sure if you
> know, but you can use service resources without writing out full
> applications or an environment catalog, i.e. you can do stuff like
> 
> define db { ... }
> Db produces Sql { .. }
> 
> define web { ... }
> 
> node db_node {
>  db { one: export => Sql[one] }
> }
> 
> node web_node {
>  web { one: require => Sql[one] }
> }
> 
> which does little more than putting that sql into web_node's catalog, which
> would then trigger running the check as a gate for applying web.
> 
> Sensu/Nagios can then fish the Sql out of both db_node's and web_node's
> catalog and do their own checking.
> 
> I am mostly curious about the barrier to entry to see if there is anything
> we can do to lower it.

hmm, so that seems very verbose and how to implement the SQL check is also
quite a task.

The verbosity is because you're modelling relationships across nodes of course
I do not want to do that - these specific checks are not for cross node checks
and as in recent emails I will not use puppetserver as long as operability
features are a paid feature, so those are not what I am after.

My needs for checks are literally to augment the:

service{"x": ensure => running} this will do 'service x status' equiv, I want it
to do additional checks and I want those checks to adhere to existing 
"standards"
which might include metrics and such.

Do you have an example of using the new check framework for this purpose?

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/264011150.21817.1462693123637.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[David Lutterkort]

On Fri, May 6, 2016 at 1:02 PM, R.I.Pienaar  wrote:

> Those checks you have in AO are good but the barrier to entry is quite
> high on
> making those I felt


Understood, and not trying to argue, but because I am curious:  what's the
barrier to entry that keeps you from using that mechanism ? Not sure if you
know, but you can use service resources without writing out full
applications or an environment catalog, i.e. you can do stuff like

define db { ... }
Db produces Sql { .. }

define web { ... }

node db_node {
  db { one: export => Sql[one] }
}

node web_node {
  web { one: require => Sql[one] }
}

which does little more than putting that sql into web_node's catalog, which
would then trigger running the check as a gate for applying web.

Sensu/Nagios can then fish the Sql out of both db_node's and web_node's
catalog and do their own checking.

I am mostly curious about the barrier to entry to see if there is anything
we can do to lower it.

David

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/CAHN%2BA%2BVESgj0R0W%3DYdFyp%3DZKh43anFOR%2BaTba4rai6Lo45MsKQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

- Original Message -
> From: "David Lutterkort" 
> To: "puppet-dev" 
> Sent: Friday, 6 May, 2016 21:30:18
> Subject: Re: [Puppet-dev] metaparam question

> On Sun, May 1, 2016 at 11:33 AM, R.I.Pienaar  wrote:
> 
>> Late to follow up on this but I had some time to work on this again,
>>
> 
> No matter how late you are, there's always somebody later ;)
> 
> For the whole cross-node dependency business, we built a sqlhealthcheck
> <https://github.com/puppetlabs/puppetlabs-sqlhealthcheck> provider; it's a
> provider for a Sql service/capability type, but has the advantage that it
> is not tied to the local db service, i.e. geared towards being used by both
> on the DB node itself and nodes that need to talk to the DB.
> 
> We did discuss extending this to make it easy to incorporate nagios checks,
> but that has suffered from ENOTIME so far.

so my plan with this is 2 fold, it lets people make puppet's generate state 
check
a bit better for any resource quite cheaply

but also once this is in the catalog I can write a sensu/nagios/graphite/tool 
of 
choice thing that loads the catalog and checks everything regularly and send 
off 
results to whatever they like.

I think I'll have to figure out how to autorequire the plugin being called in
case Puppet manages it but I think this concept is pretty powerful - and indeed
should work with your cross node stuff too.

Those checks you have in AO are good but the barrier to entry is quite high on
making those I felt

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/678114459.450537.1462564928420.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[David Lutterkort]

On Sun, May 1, 2016 at 11:33 AM, R.I.Pienaar  wrote:

> Late to follow up on this but I had some time to work on this again,
>

No matter how late you are, there's always somebody later ;)

For the whole cross-node dependency business, we built a sqlhealthcheck
 provider; it's a
provider for a Sql service/capability type, but has the advantage that it
is not tied to the local db service, i.e. geared towards being used by both
on the DB node itself and nodes that need to talk to the DB.

We did discuss extending this to make it easy to incorporate nagios checks,
but that has suffered from ENOTIME so far.

David

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/CAHN%2BA%2BVFn7RCYs-b%3DWkXM8Azrq8skgguh34BB4wtkJAEyzOreA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

- Original Message -
> From: "R. I. Pienaar" 
> To: "puppet-dev" 
> Sent: Thursday, 3 March, 2016 09:58:42
> Subject: Re: [Puppet-dev] metaparam question

> - Original Message -
>> From: "Erik Dalén" 
>> To: "puppet-dev" 
>> Sent: Thursday, 3 March, 2016 09:43:30
>> Subject: Re: [Puppet-dev] metaparam question
> 
>> On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg  wrote:
>> 
>>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>>>
>>>> hello,
>>>>
>>>> I would like to add a metaparameter - which I think is easy now via
>>>> Type.newmetaparam.
>>>>
>>>
>>> We haven't been thinking of metaparameters as a general purpose extension
>>> point. This came up once before that I know of, about a year ago, and
>>> there's a little discussion of this in
>>> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we reached
>>> at the time was, more or less, to explore whether the desired change could
>>> be accomplished with a puppet function and/or a change to core puppet.
>>>
>>>
>>>>
>>>> The thing that I can't seem to find any example of though is how to
>>>> make this metaparameter do something on the nodes for all providers
>>>> or all types.
>>>
>>>
>>>> Imagine there's a metaparam that might describe how to test a resource
>>>> works, something like:
>>>>
>>>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>>>>
>>>> I'd then want to have some code that would be run on the agent nodes
>>>> for any resource that has this param set.
>>>>
>>>
>>> I don't think something like this exists per se, but 'validate' might be
>>> one such example of something worth adding to core puppet. Fwiw, one
>>> resource-specific example added not too long ago is the file type's
>>> validate_cmd:
>>>
>>>
>>> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
>>> .
>>>
>>>
>> Just to clarify, validate_cmd works differently in that it validates the
>> new contents before replacing the file.
>> I guess the generic validate metaparameter would validate the resource
>> after it has been synced.
> 
> yes indeed, subtle but important.  thanks

Late to follow up on this but I had some time to work on this again, 

  service{"http":
ensure => "running",
post_validate => "/usr/lib64/nagios/plugins/check_http -H localhost"
  }

This works with the code below and I included a basic thingy that tells
sensu about this

I had to hack around a bit more than simply adding a metaparam since I had
to wait for puppet to completely finish with the resource and then do a 
final check after it's done

https://github.com/ripienaar/puppet/commit/a6c659cc3fb3555632bfe94df1411878a04fdc51

This is really nice, it both makes my Puppet runs a bit better and allows
me to get instant visibility of failures - and configurable notifications
based on sensu.  It works with any kind of resource.

Ideally the sensu notifier would be something a module can provide and you
could register multiple event handlers but Henrik says the bindings system
isn't initialized on the agent side at present and I didn't have time to 
hack around with that.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/157346444.508343.1462127599948.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[Trevor Vaughan]

It really seems like we want the logical extension of the Exec checks on
all resources with additional validation.

service { 'httpd':
  ensure => 'running',
  onlyif => '/usr/bin/probe_remote_mysql', # or 'unless'
  validate => "curl -k http://${::fqdn}/service_check.html";,
  # Ok, magical asynchronous fact update here with an ACL (ponies and
unicorns area)
  fact_update => {
'my_unique_name' => 'w00t',
# ACLs would need to be processed server side. Let's not give the nodes
too much power here...
# This could, perhaps, be a server setting.
'acl' => {
  'allow' => $::domain,
  'deny' => 'ALL'
}
  }


  class { 'thing_that_needs_apache':
onlyif => fact_fetch('my_unique_name', 'w00t') # fact_fetch(,
)
  }

With the async updating of facts, and individual ACL restrictions, you get
some really neat capabilities in terms of fine grained process control.

Thinking about this further, you'll probably need caching proxies for the
fact updates though.

Maybe Puppet 8 :-D.

Thanks,

Trevor

On Thu, Mar 3, 2016 at 3:58 AM, R.I.Pienaar  wrote:

>
>
> - Original Message -
> > From: "Erik Dalén" 
> > To: "puppet-dev" 
> > Sent: Thursday, 3 March, 2016 09:43:30
> > Subject: Re: [Puppet-dev] metaparam question
>
> > On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg  wrote:
> >
> >> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
> >>
> >>> hello,
> >>>
> >>> I would like to add a metaparameter - which I think is easy now via
> >>> Type.newmetaparam.
> >>>
> >>
> >> We haven't been thinking of metaparameters as a general purpose
> extension
> >> point. This came up once before that I know of, about a year ago, and
> >> there's a little discussion of this in
> >> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we
> reached
> >> at the time was, more or less, to explore whether the desired change
> could
> >> be accomplished with a puppet function and/or a change to core puppet.
> >>
> >>
> >>>
> >>> The thing that I can't seem to find any example of though is how to
> >>> make this metaparameter do something on the nodes for all providers
> >>> or all types.
> >>
> >>
> >>> Imagine there's a metaparam that might describe how to test a resource
> >>> works, something like:
> >>>
> >>>service{"httpd": validate => "check_http --port 80 -H localhost"}
> >>>
> >>> I'd then want to have some code that would be run on the agent nodes
> >>> for any resource that has this param set.
> >>>
> >>
> >> I don't think something like this exists per se, but 'validate' might be
> >> one such example of something worth adding to core puppet. Fwiw, one
> >> resource-specific example added not too long ago is the file type's
> >> validate_cmd:
> >>
> >>
> >>
> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
> >> .
> >>
> >>
> > Just to clarify, validate_cmd works differently in that it validates the
> > new contents before replacing the file.
> > I guess the generic validate metaparameter would validate the resource
> > after it has been synced.
>
> yes indeed, subtle but important.  thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/1062762368.96001.1456995522124.JavaMail.zimbra%40devco.net
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Trevor Vaughan
Vice President, Onyx Point, Inc
(410) 541-6699

-- This account not approved for unencrypted proprietary information --

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoVDQ4YYkG1Gz1YxO92amk4vyN%2BWVa%3D9%2BE-psKjx9Nrykg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

- Original Message -
> From: "Erik Dalén" 
> To: "puppet-dev" 
> Sent: Thursday, 3 March, 2016 09:43:30
> Subject: Re: [Puppet-dev] metaparam question

> On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg  wrote:
> 
>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>>
>>> hello,
>>>
>>> I would like to add a metaparameter - which I think is easy now via
>>> Type.newmetaparam.
>>>
>>
>> We haven't been thinking of metaparameters as a general purpose extension
>> point. This came up once before that I know of, about a year ago, and
>> there's a little discussion of this in
>> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we reached
>> at the time was, more or less, to explore whether the desired change could
>> be accomplished with a puppet function and/or a change to core puppet.
>>
>>
>>>
>>> The thing that I can't seem to find any example of though is how to
>>> make this metaparameter do something on the nodes for all providers
>>> or all types.
>>
>>
>>> Imagine there's a metaparam that might describe how to test a resource
>>> works, something like:
>>>
>>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>>>
>>> I'd then want to have some code that would be run on the agent nodes
>>> for any resource that has this param set.
>>>
>>
>> I don't think something like this exists per se, but 'validate' might be
>> one such example of something worth adding to core puppet. Fwiw, one
>> resource-specific example added not too long ago is the file type's
>> validate_cmd:
>>
>>
>> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
>> .
>>
>>
> Just to clarify, validate_cmd works differently in that it validates the
> new contents before replacing the file.
> I guess the generic validate metaparameter would validate the resource
> after it has been synced.

yes indeed, subtle but important.  thanks

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/1062762368.96001.1456995522124.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[Erik Dalén]

On Fri, 5 Feb 2016 at 00:35 Kylo Ginsberg  wrote:

> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>
>> hello,
>>
>> I would like to add a metaparameter - which I think is easy now via
>> Type.newmetaparam.
>>
>
> We haven't been thinking of metaparameters as a general purpose extension
> point. This came up once before that I know of, about a year ago, and
> there's a little discussion of this in
> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we reached
> at the time was, more or less, to explore whether the desired change could
> be accomplished with a puppet function and/or a change to core puppet.
>
>
>>
>> The thing that I can't seem to find any example of though is how to
>> make this metaparameter do something on the nodes for all providers
>> or all types.
>
>
>> Imagine there's a metaparam that might describe how to test a resource
>> works, something like:
>>
>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>>
>> I'd then want to have some code that would be run on the agent nodes
>> for any resource that has this param set.
>>
>
> I don't think something like this exists per se, but 'validate' might be
> one such example of something worth adding to core puppet. Fwiw, one
> resource-specific example added not too long ago is the file type's
> validate_cmd:
>
>
> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
> .
>
>
Just to clarify, validate_cmd works differently in that it validates the
new contents before replacing the file.
I guess the generic validate metaparameter would validate the resource
after it has been synced.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/CAAAzDLcnH%2BGh4qqXFvpjm9kqA0uyztUv5VTWDaipZ9GQZhUAmg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[Kylo Ginsberg]

On Wed, Mar 2, 2016 at 2:50 AM, Trevor Vaughan 
wrote:

> Hi all,
>
> This is a very interesting conversation and I'm monitoring to see where it
> ends up.
>

I see what you did there ;) Thanks for the chuckle.


>
> You're going to want to have a central location for organizing this
> information between nodes since there are no guarantees that any two nodes
> can get bi-directional communications between each other.
>
> Take, for instance, a highly regulated HIPAA environment. You may be able
> to send data one way between two managed environments but bi-directional
> communication would be unable to be processed. Therefore, you would need
> some level of controlled and shared trigger between your host catalogs.
> (OK, fine, I still want language-level semaphores).
>
> Trevor
>
> On Wed, Mar 2, 2016 at 4:43 AM, R.I.Pienaar  wrote:
>
>>
>>
>> - Original Message -
>> > From: "Kylo Ginsberg" 
>> > To: "puppet-dev" 
>> > Sent: Wednesday, 2 March, 2016 06:59:08
>> > Subject: Re: [Puppet-dev] metaparam question
>>
>> > On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar  wrote:
>> >
>> >> hello,
>> >>
>> >> Thanks for the replies, think I'll reply to both in one go
>> >>
>> >> - Original Message -
>> >> > From: "Luke Kanies" 
>> >> > To: "puppet-dev" 
>> >> > Sent: Friday, February 5, 2016 6:15:28 AM
>> >> > Subject: Re: [Puppet-dev] metaparam question
>> >>
>> >> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg 
>> wrote:
>> >> >>
>> >> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>> >> >>> hello,
>> >> >>>
>> >> >>> I would like to add a metaparameter - which I think is easy now via
>> >> >>> Type.newmetaparam.
>> >> >>
>> >> >> We haven't been thinking of metaparameters as a general purpose
>> >> extension point.
>> >> >> This came up once before that I know of, about a year ago, and
>> there's
>> >> a little
>> >> >> discussion of this in
>> https://tickets.puppetlabs.com/browse/PUP-4281.
>> >> The
>> >> >> conclusion we reached at the time was, more or less, to explore
>> whether
>> >> the
>> >> >> desired change could be accomplished with a puppet function and/or a
>> >> change to
>> >> >> core puppet.
>> >> >
>> >> > I don't remember my original logic for metaparams not being an
>> extension
>> >> point.
>> >> > Given the system's support for easily loading this kind of plugin, it
>> >> should be
>> >> > easy to add.
>> >> >
>> >> > The discussion on the ticket is pretty limited. We've got multiple
>> use
>> >> cases
>> >> > here, and I have seen use cases in the past brought up, but they were
>> >> mostly on
>> >> > lists or in person, so I don't think they made it into the ticketing
>> >> system.
>> >> >
>> >> > RI - you up for a pull request to add the autoloading? Have you
>> thought
>> >> much
>> >> > about how this might go wrong?
>> >>
>> >> Unfortunately my time is pretty limited and I'll prefer to see what I
>> can
>> >> do
>> >> with this information in the catalog rather than making a merge worthy
>> >> patch
>> >> at present.
>> >>
>> >> I'm happy to just patch my actual puppet code with no modules to get to
>> >> that
>> >> point, I'll have a look at the validate_cmd in file to see if there is
>> a
>> >> way
>> >> in the provider base class to do it so all providers get it.
>> >>
>> >
>> > I kept meaning to get back to this thread and it slipped my mind. Mea
>> culpa
>> > on that.
>> >
>> > I'm curious where you got with this approach? E.g. did you end up
>> patching
>> > an additional metaparameter and were there any lessons worth sharing in
>> > that?
>>
>> Unfortunately work is a bit busy right now so I've not had time to really
>> give it
>> any screen time :(
>>
>> >> For me this being in every provider is a nice 

Re: [Puppet-dev] metaparam question

[Kylo Ginsberg]

On Wed, Mar 2, 2016 at 4:53 PM, Nan Liu  wrote:

>
>
> On Wed, Mar 2, 2016 at 1:43 AM R.I.Pienaar  wrote:
>
>>
>>
>> - Original Message -
>> > From: "Kylo Ginsberg" 
>> > To: "puppet-dev" 
>> > Sent: Wednesday, 2 March, 2016 06:59:08
>> > Subject: Re: [Puppet-dev] metaparam question
>>
>> > On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar  wrote:
>> >
>> >> hello,
>> >>
>> >> Thanks for the replies, think I'll reply to both in one go
>> >>
>> >> ----- Original Message -
>> >> > From: "Luke Kanies" 
>> >> > To: "puppet-dev" 
>> >> > Sent: Friday, February 5, 2016 6:15:28 AM
>> >> > Subject: Re: [Puppet-dev] metaparam question
>> >>
>> >> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg 
>> wrote:
>> >> >>
>> >> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>> >> >>> hello,
>> >> >>>
>> >> >>> I would like to add a metaparameter - which I think is easy now via
>> >> >>> Type.newmetaparam.
>> >> >>
>> >> >> We haven't been thinking of metaparameters as a general purpose
>> >> extension point.
>> >> >> This came up once before that I know of, about a year ago, and
>> there's
>> >> a little
>> >> >> discussion of this in
>> https://tickets.puppetlabs.com/browse/PUP-4281.
>> >> The
>> >> >> conclusion we reached at the time was, more or less, to explore
>> whether
>> >> the
>> >> >> desired change could be accomplished with a puppet function and/or a
>> >> change to
>> >> >> core puppet.
>> >> >
>> >> > I don't remember my original logic for metaparams not being an
>> extension
>> >> point.
>> >> > Given the system's support for easily loading this kind of plugin, it
>> >> should be
>> >> > easy to add.
>> >> >
>> >> > The discussion on the ticket is pretty limited. We've got multiple
>> use
>> >> cases
>> >> > here, and I have seen use cases in the past brought up, but they were
>> >> mostly on
>> >> > lists or in person, so I don't think they made it into the ticketing
>> >> system.
>> >> >
>> >> > RI - you up for a pull request to add the autoloading? Have you
>> thought
>> >> much
>> >> > about how this might go wrong?
>> >>
>> >> Unfortunately my time is pretty limited and I'll prefer to see what I
>> can
>> >> do
>> >> with this information in the catalog rather than making a merge worthy
>> >> patch
>> >> at present.
>> >>
>> >> I'm happy to just patch my actual puppet code with no modules to get to
>> >> that
>> >> point, I'll have a look at the validate_cmd in file to see if there is
>> a
>> >> way
>> >> in the provider base class to do it so all providers get it.
>> >>
>> >
>> > I kept meaning to get back to this thread and it slipped my mind. Mea
>> culpa
>> > on that.
>> >
>> > I'm curious where you got with this approach? E.g. did you end up
>> patching
>> > an additional metaparameter and were there any lessons worth sharing in
>> > that?
>>
>> Unfortunately work is a bit busy right now so I've not had time to really
>> give it
>> any screen time :(
>>
>> >> For me this being in every provider is a nice to have I guess - it
>> would be
>> >> great to be able to improve puppets ability to check if something
>> worked
>> >> in the broad sense, and it's going to apply to all resource types
>> equally.
>> >>
>> >
>> >> I could take the route that https://github.com/nanliu/puppet-transport
>> >> does
>> >> and instead of making the meta param do something use it just to store
>> this
>> >> information and then some other module do something with this
>> information
>> >> and
>> >> that would let me play with what I want, but it seems a small step to
>> >> improve
>> >> puppet in general.
>> >>
>> >

Re: [Puppet-dev] metaparam question

[Nan Liu]

On Wed, Mar 2, 2016 at 1:43 AM R.I.Pienaar  wrote:

>
>
> - Original Message -
> > From: "Kylo Ginsberg" 
> > To: "puppet-dev" 
> > Sent: Wednesday, 2 March, 2016 06:59:08
> > Subject: Re: [Puppet-dev] metaparam question
>
> > On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar  wrote:
> >
> >> hello,
> >>
> >> Thanks for the replies, think I'll reply to both in one go
> >>
> >> - Original Message -
> >> > From: "Luke Kanies" 
> >> > To: "puppet-dev" 
> >> > Sent: Friday, February 5, 2016 6:15:28 AM
> >> > Subject: Re: [Puppet-dev] metaparam question
> >>
> >> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg 
> wrote:
> >> >>
> >> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
> >> >>> hello,
> >> >>>
> >> >>> I would like to add a metaparameter - which I think is easy now via
> >> >>> Type.newmetaparam.
> >> >>
> >> >> We haven't been thinking of metaparameters as a general purpose
> >> extension point.
> >> >> This came up once before that I know of, about a year ago, and
> there's
> >> a little
> >> >> discussion of this in https://tickets.puppetlabs.com/browse/PUP-4281
> .
> >> The
> >> >> conclusion we reached at the time was, more or less, to explore
> whether
> >> the
> >> >> desired change could be accomplished with a puppet function and/or a
> >> change to
> >> >> core puppet.
> >> >
> >> > I don't remember my original logic for metaparams not being an
> extension
> >> point.
> >> > Given the system's support for easily loading this kind of plugin, it
> >> should be
> >> > easy to add.
> >> >
> >> > The discussion on the ticket is pretty limited. We've got multiple use
> >> cases
> >> > here, and I have seen use cases in the past brought up, but they were
> >> mostly on
> >> > lists or in person, so I don't think they made it into the ticketing
> >> system.
> >> >
> >> > RI - you up for a pull request to add the autoloading? Have you
> thought
> >> much
> >> > about how this might go wrong?
> >>
> >> Unfortunately my time is pretty limited and I'll prefer to see what I
> can
> >> do
> >> with this information in the catalog rather than making a merge worthy
> >> patch
> >> at present.
> >>
> >> I'm happy to just patch my actual puppet code with no modules to get to
> >> that
> >> point, I'll have a look at the validate_cmd in file to see if there is a
> >> way
> >> in the provider base class to do it so all providers get it.
> >>
> >
> > I kept meaning to get back to this thread and it slipped my mind. Mea
> culpa
> > on that.
> >
> > I'm curious where you got with this approach? E.g. did you end up
> patching
> > an additional metaparameter and were there any lessons worth sharing in
> > that?
>
> Unfortunately work is a bit busy right now so I've not had time to really
> give it
> any screen time :(
>
> >> For me this being in every provider is a nice to have I guess - it
> would be
> >> great to be able to improve puppets ability to check if something worked
> >> in the broad sense, and it's going to apply to all resource types
> equally.
> >>
> >
> >> I could take the route that https://github.com/nanliu/puppet-transport
> >> does
> >> and instead of making the meta param do something use it just to store
> this
> >> information and then some other module do something with this
> information
> >> and
> >> that would let me play with what I want, but it seems a small step to
> >> improve
> >> puppet in general.
> >>
> >
> > I'm intrigued by the idea of a generic 'validate' or 'health check'. It
> > sounds like you're thinking of it as separate from the rest of the
> > type/provider, i.e. an external command that could be run, but I also
> > wonder (this is half-baked) if, rather than a separate metaparameter, it
> > could use a standard provider method to do the validation. I.e. when
> puppet
> > is applying the catalog it has to essentially ask the same question that
> a
>

Re: [Puppet-dev] metaparam question

[Trevor Vaughan]

Now that I'm thinking more about it, I 100% want this.

There have been quite a few occasions where I didn't want to pass back an
epic ton of fact data just to change a small amount of logic.

Being able to bookend my resources without a hacky exec chain would be
great.

Propagating that check to other resources would be kind of neat too but
that's probably easier done with class chaining anyway.

Trevor

On Wed, Mar 2, 2016 at 6:00 AM, R.I.Pienaar  wrote:

>
>
> - Original Message -
> > From: "Trevor Vaughan" 
> > To: "puppet-dev" 
> > Sent: Wednesday, 2 March, 2016 11:50:27
> > Subject: Re: [Puppet-dev] metaparam question
>
> > Hi all,
> >
> > This is a very interesting conversation and I'm monitoring to see where
> it
> > ends up.
> >
> > You're going to want to have a central location for organizing this
> > information between nodes since there are no guarantees that any two
> nodes
> > can get bi-directional communications between each other.
> >
> > Take, for instance, a highly regulated HIPAA environment. You may be able
> > to send data one way between two managed environments but bi-directional
> > communication would be unable to be processed. Therefore, you would need
> > some level of controlled and shared trigger between your host catalogs.
> > (OK, fine, I still want language-level semaphores).
>
> quite, that's what the puppet orchastrator and node catalogs will do
> eventually.
>
> In the mean time as a building block what I am proposing:
>
>   service{"your_app":
> ensure => "running",
> validate => "check_my_app -H localhost -p 80"
>   }
>
> Might connect to a local web app that was configured with its DB details
> and it
> might hit something like /status which might return:
>
>{"database": "ok", "health": "ok", "metrics": {"requests": 1234,
> "avg_response_time": 0.2}}
>
> and so you know the service up and happy and working much more than just a:
>
>service you_app start
>
> would know.  At a deeper level additional per resource reporting could
> include
> things like this additional status and using the nagios metric protocol
> you could
> communicate the metrics requests and avg_response_time back to Puppet for
> surfacing
> in the various consoles.
>
> I'd as a side effect feed that into let say sensu as well so your other
> monitoring
> will automagically become aware of this resoure and its status.
>
> >
> > Trevor
> >
> > On Wed, Mar 2, 2016 at 4:43 AM, R.I.Pienaar  wrote:
> >
> >>
> >>
> >> - Original Message -
> >> > From: "Kylo Ginsberg" 
> >> > To: "puppet-dev" 
> >> > Sent: Wednesday, 2 March, 2016 06:59:08
> >> > Subject: Re: [Puppet-dev] metaparam question
> >>
> >> > On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar  wrote:
> >> >
> >> >> hello,
> >> >>
> >> >> Thanks for the replies, think I'll reply to both in one go
> >> >>
> >> >> - Original Message -
> >> >> > From: "Luke Kanies" 
> >> >> > To: "puppet-dev" 
> >> >> > Sent: Friday, February 5, 2016 6:15:28 AM
> >> >> > Subject: Re: [Puppet-dev] metaparam question
> >> >>
> >> >> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg 
> >> wrote:
> >> >> >>
> >> >> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar 
> wrote:
> >> >> >>> hello,
> >> >> >>>
> >> >> >>> I would like to add a metaparameter - which I think is easy now
> via
> >> >> >>> Type.newmetaparam.
> >> >> >>
> >> >> >> We haven't been thinking of metaparameters as a general purpose
> >> >> extension point.
> >> >> >> This came up once before that I know of, about a year ago, and
> >> there's
> >> >> a little
> >> >> >> discussion of this in
> https://tickets.puppetlabs.com/browse/PUP-4281
> >> .
> >> >> The
> >> >> >> conclusion we reached at the time was, more or less, to explore
> >> whether
> >> >> the
> >> >> >> desired change could be accomplished with a puppet function
> and/or a
> >> >>

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

- Original Message -
> From: "Trevor Vaughan" 
> To: "puppet-dev" 
> Sent: Wednesday, 2 March, 2016 11:50:27
> Subject: Re: [Puppet-dev] metaparam question

> Hi all,
> 
> This is a very interesting conversation and I'm monitoring to see where it
> ends up.
> 
> You're going to want to have a central location for organizing this
> information between nodes since there are no guarantees that any two nodes
> can get bi-directional communications between each other.
> 
> Take, for instance, a highly regulated HIPAA environment. You may be able
> to send data one way between two managed environments but bi-directional
> communication would be unable to be processed. Therefore, you would need
> some level of controlled and shared trigger between your host catalogs.
> (OK, fine, I still want language-level semaphores).

quite, that's what the puppet orchastrator and node catalogs will do eventually.

In the mean time as a building block what I am proposing:

  service{"your_app":
ensure => "running",
validate => "check_my_app -H localhost -p 80"
  }

Might connect to a local web app that was configured with its DB details and it
might hit something like /status which might return:

   {"database": "ok", "health": "ok", "metrics": {"requests": 1234, 
"avg_response_time": 0.2}}

and so you know the service up and happy and working much more than just a:

   service you_app start

would know.  At a deeper level additional per resource reporting could include
things like this additional status and using the nagios metric protocol you 
could
communicate the metrics requests and avg_response_time back to Puppet for 
surfacing
in the various consoles.

I'd as a side effect feed that into let say sensu as well so your other 
monitoring
will automagically become aware of this resoure and its status.

> 
> Trevor
> 
> On Wed, Mar 2, 2016 at 4:43 AM, R.I.Pienaar  wrote:
> 
>>
>>
>> - Original Message -
>> > From: "Kylo Ginsberg" 
>> > To: "puppet-dev" 
>> > Sent: Wednesday, 2 March, 2016 06:59:08
>> > Subject: Re: [Puppet-dev] metaparam question
>>
>> > On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar  wrote:
>> >
>> >> hello,
>> >>
>> >> Thanks for the replies, think I'll reply to both in one go
>> >>
>> >> - Original Message -
>> >> > From: "Luke Kanies" 
>> >> > To: "puppet-dev" 
>> >> > Sent: Friday, February 5, 2016 6:15:28 AM
>> >> > Subject: Re: [Puppet-dev] metaparam question
>> >>
>> >> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg 
>> wrote:
>> >> >>
>> >> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>> >> >>> hello,
>> >> >>>
>> >> >>> I would like to add a metaparameter - which I think is easy now via
>> >> >>> Type.newmetaparam.
>> >> >>
>> >> >> We haven't been thinking of metaparameters as a general purpose
>> >> extension point.
>> >> >> This came up once before that I know of, about a year ago, and
>> there's
>> >> a little
>> >> >> discussion of this in https://tickets.puppetlabs.com/browse/PUP-4281
>> .
>> >> The
>> >> >> conclusion we reached at the time was, more or less, to explore
>> whether
>> >> the
>> >> >> desired change could be accomplished with a puppet function and/or a
>> >> change to
>> >> >> core puppet.
>> >> >
>> >> > I don't remember my original logic for metaparams not being an
>> extension
>> >> point.
>> >> > Given the system's support for easily loading this kind of plugin, it
>> >> should be
>> >> > easy to add.
>> >> >
>> >> > The discussion on the ticket is pretty limited. We've got multiple use
>> >> cases
>> >> > here, and I have seen use cases in the past brought up, but they were
>> >> mostly on
>> >> > lists or in person, so I don't think they made it into the ticketing
>> >> system.
>> >> >
>> >> > RI - you up for a pull request to add the autoloading? Have you
>> thought
>> >> much
>> >> > about how this might go wrong?
>> >>
>> >>

Re: [Puppet-dev] metaparam question

[Trevor Vaughan]

Hi all,

This is a very interesting conversation and I'm monitoring to see where it
ends up.

You're going to want to have a central location for organizing this
information between nodes since there are no guarantees that any two nodes
can get bi-directional communications between each other.

Take, for instance, a highly regulated HIPAA environment. You may be able
to send data one way between two managed environments but bi-directional
communication would be unable to be processed. Therefore, you would need
some level of controlled and shared trigger between your host catalogs.
(OK, fine, I still want language-level semaphores).

Trevor

On Wed, Mar 2, 2016 at 4:43 AM, R.I.Pienaar  wrote:

>
>
> - Original Message -
> > From: "Kylo Ginsberg" 
> > To: "puppet-dev" 
> > Sent: Wednesday, 2 March, 2016 06:59:08
> > Subject: Re: [Puppet-dev] metaparam question
>
> > On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar  wrote:
> >
> >> hello,
> >>
> >> Thanks for the replies, think I'll reply to both in one go
> >>
> >> - Original Message -
> >> > From: "Luke Kanies" 
> >> > To: "puppet-dev" 
> >> > Sent: Friday, February 5, 2016 6:15:28 AM
> >> > Subject: Re: [Puppet-dev] metaparam question
> >>
> >> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg 
> wrote:
> >> >>
> >> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
> >> >>> hello,
> >> >>>
> >> >>> I would like to add a metaparameter - which I think is easy now via
> >> >>> Type.newmetaparam.
> >> >>
> >> >> We haven't been thinking of metaparameters as a general purpose
> >> extension point.
> >> >> This came up once before that I know of, about a year ago, and
> there's
> >> a little
> >> >> discussion of this in https://tickets.puppetlabs.com/browse/PUP-4281
> .
> >> The
> >> >> conclusion we reached at the time was, more or less, to explore
> whether
> >> the
> >> >> desired change could be accomplished with a puppet function and/or a
> >> change to
> >> >> core puppet.
> >> >
> >> > I don't remember my original logic for metaparams not being an
> extension
> >> point.
> >> > Given the system's support for easily loading this kind of plugin, it
> >> should be
> >> > easy to add.
> >> >
> >> > The discussion on the ticket is pretty limited. We've got multiple use
> >> cases
> >> > here, and I have seen use cases in the past brought up, but they were
> >> mostly on
> >> > lists or in person, so I don't think they made it into the ticketing
> >> system.
> >> >
> >> > RI - you up for a pull request to add the autoloading? Have you
> thought
> >> much
> >> > about how this might go wrong?
> >>
> >> Unfortunately my time is pretty limited and I'll prefer to see what I
> can
> >> do
> >> with this information in the catalog rather than making a merge worthy
> >> patch
> >> at present.
> >>
> >> I'm happy to just patch my actual puppet code with no modules to get to
> >> that
> >> point, I'll have a look at the validate_cmd in file to see if there is a
> >> way
> >> in the provider base class to do it so all providers get it.
> >>
> >
> > I kept meaning to get back to this thread and it slipped my mind. Mea
> culpa
> > on that.
> >
> > I'm curious where you got with this approach? E.g. did you end up
> patching
> > an additional metaparameter and were there any lessons worth sharing in
> > that?
>
> Unfortunately work is a bit busy right now so I've not had time to really
> give it
> any screen time :(
>
> >> For me this being in every provider is a nice to have I guess - it
> would be
> >> great to be able to improve puppets ability to check if something worked
> >> in the broad sense, and it's going to apply to all resource types
> equally.
> >>
> >
> >> I could take the route that https://github.com/nanliu/puppet-transport
> >> does
> >> and instead of making the meta param do something use it just to store
> this
> >> information and then some other module do something with this
> information
> >> and
> >> that would let me play 

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

- Original Message -
> From: "Kylo Ginsberg" 
> To: "puppet-dev" 
> Sent: Wednesday, 2 March, 2016 06:59:08
> Subject: Re: [Puppet-dev] metaparam question

> On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar  wrote:
> 
>> hello,
>>
>> Thanks for the replies, think I'll reply to both in one go
>>
>> - Original Message -
>> > From: "Luke Kanies" 
>> > To: "puppet-dev" 
>> > Sent: Friday, February 5, 2016 6:15:28 AM
>> > Subject: Re: [Puppet-dev] metaparam question
>>
>> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg  wrote:
>> >>
>> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>> >>> hello,
>> >>>
>> >>> I would like to add a metaparameter - which I think is easy now via
>> >>> Type.newmetaparam.
>> >>
>> >> We haven't been thinking of metaparameters as a general purpose
>> extension point.
>> >> This came up once before that I know of, about a year ago, and there's
>> a little
>> >> discussion of this in https://tickets.puppetlabs.com/browse/PUP-4281.
>> The
>> >> conclusion we reached at the time was, more or less, to explore whether
>> the
>> >> desired change could be accomplished with a puppet function and/or a
>> change to
>> >> core puppet.
>> >
>> > I don't remember my original logic for metaparams not being an extension
>> point.
>> > Given the system's support for easily loading this kind of plugin, it
>> should be
>> > easy to add.
>> >
>> > The discussion on the ticket is pretty limited. We've got multiple use
>> cases
>> > here, and I have seen use cases in the past brought up, but they were
>> mostly on
>> > lists or in person, so I don't think they made it into the ticketing
>> system.
>> >
>> > RI - you up for a pull request to add the autoloading? Have you thought
>> much
>> > about how this might go wrong?
>>
>> Unfortunately my time is pretty limited and I'll prefer to see what I can
>> do
>> with this information in the catalog rather than making a merge worthy
>> patch
>> at present.
>>
>> I'm happy to just patch my actual puppet code with no modules to get to
>> that
>> point, I'll have a look at the validate_cmd in file to see if there is a
>> way
>> in the provider base class to do it so all providers get it.
>>
> 
> I kept meaning to get back to this thread and it slipped my mind. Mea culpa
> on that.
> 
> I'm curious where you got with this approach? E.g. did you end up patching
> an additional metaparameter and were there any lessons worth sharing in
> that?

Unfortunately work is a bit busy right now so I've not had time to really give 
it
any screen time :(

>> For me this being in every provider is a nice to have I guess - it would be
>> great to be able to improve puppets ability to check if something worked
>> in the broad sense, and it's going to apply to all resource types equally.
>>
> 
>> I could take the route that https://github.com/nanliu/puppet-transport
>> does
>> and instead of making the meta param do something use it just to store this
>> information and then some other module do something with this information
>> and
>> that would let me play with what I want, but it seems a small step to
>> improve
>> puppet in general.
>>
> 
> I'm intrigued by the idea of a generic 'validate' or 'health check'. It
> sounds like you're thinking of it as separate from the rest of the
> type/provider, i.e. an external command that could be run, but I also
> wonder (this is half-baked) if, rather than a separate metaparameter, it
> could use a standard provider method to do the validation. I.e. when puppet
> is applying the catalog it has to essentially ask the same question that a
> 'validate' method would to figure out if it needs to remediate, so can the
> same need be met with some refinement of the generic type/provider api?

As a POC yes, I'd like it external - there's a ton of Nagios plugins out there
and they would magically all be useful without any extra work, with the time
I have minimising the yaks have value :)

But there are common patterns to these things, so for sure you could mark
up a module/resource with some additional properties or allow provider authors
to do better than we do today - but then they more or less can already code 
their
modules to do in d

Re: [Puppet-dev] metaparam question

[Kylo Ginsberg]

On Fri, Feb 5, 2016 at 12:02 AM, R.I.Pienaar  wrote:

> hello,
>
> Thanks for the replies, think I'll reply to both in one go
>
> - Original Message -
> > From: "Luke Kanies" 
> > To: "puppet-dev" 
> > Sent: Friday, February 5, 2016 6:15:28 AM
> > Subject: Re: [Puppet-dev] metaparam question
>
> >> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg  wrote:
> >>
> >>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
> >>> hello,
> >>>
> >>> I would like to add a metaparameter - which I think is easy now via
> >>> Type.newmetaparam.
> >>
> >> We haven't been thinking of metaparameters as a general purpose
> extension point.
> >> This came up once before that I know of, about a year ago, and there's
> a little
> >> discussion of this in https://tickets.puppetlabs.com/browse/PUP-4281.
> The
> >> conclusion we reached at the time was, more or less, to explore whether
> the
> >> desired change could be accomplished with a puppet function and/or a
> change to
> >> core puppet.
> >
> > I don't remember my original logic for metaparams not being an extension
> point.
> > Given the system's support for easily loading this kind of plugin, it
> should be
> > easy to add.
> >
> > The discussion on the ticket is pretty limited. We've got multiple use
> cases
> > here, and I have seen use cases in the past brought up, but they were
> mostly on
> > lists or in person, so I don't think they made it into the ticketing
> system.
> >
> > RI - you up for a pull request to add the autoloading? Have you thought
> much
> > about how this might go wrong?
>
> Unfortunately my time is pretty limited and I'll prefer to see what I can
> do
> with this information in the catalog rather than making a merge worthy
> patch
> at present.
>
> I'm happy to just patch my actual puppet code with no modules to get to
> that
> point, I'll have a look at the validate_cmd in file to see if there is a
> way
> in the provider base class to do it so all providers get it.
>

I kept meaning to get back to this thread and it slipped my mind. Mea culpa
on that.

I'm curious where you got with this approach? E.g. did you end up patching
an additional metaparameter and were there any lessons worth sharing in
that?


>
> For me this being in every provider is a nice to have I guess - it would be
> great to be able to improve puppets ability to check if something worked
> in the broad sense, and it's going to apply to all resource types equally.
>

> I could take the route that https://github.com/nanliu/puppet-transport
> does
> and instead of making the meta param do something use it just to store this
> information and then some other module do something with this information
> and
> that would let me play with what I want, but it seems a small step to
> improve
> puppet in general.
>

I'm intrigued by the idea of a generic 'validate' or 'health check'. It
sounds like you're thinking of it as separate from the rest of the
type/provider, i.e. an external command that could be run, but I also
wonder (this is half-baked) if, rather than a separate metaparameter, it
could use a standard provider method to do the validation. I.e. when puppet
is applying the catalog it has to essentially ask the same question that a
'validate' method would to figure out if it needs to remediate, so can the
same need be met with some refinement of the generic type/provider api?

(And btw, thanks for the pointer to puppet-transport. I looked at that long
ago and had forgotten (or more likely didn't internalize) the use there of
a do-nothing metaparameter to store connection information.)


> I've told many people this story so no doubt you remember the context Luke
> but what I'd like to do is allow people to attach these checks to resources
> and of course use them in catalog apply stage to get a deeper sense of
> 'done'
> in the same way that the File validate_cmd does but later I want to consume
> the catalog in other tools and extract the same - or rather hopefully speak
> to some API and get this.
>
> Imagine I had a bridge to Sensu where it loads the catalog and just checks
> everything that has a validate attached for free with no configuration
> required,
> this is actually really easy with sensu as it doesnt require you to declare
> the checks upfront on the server and all you have to do is write JSON to a
> socket for every check, it'll promiscuously take them and show them and do
> alerts etc.
>
> Once you have 

Re: [Puppet-dev] metaparam question

[R.I.Pienaar]

hello,

Thanks for the replies, think I'll reply to both in one go

- Original Message -
> From: "Luke Kanies" 
> To: "puppet-dev" 
> Sent: Friday, February 5, 2016 6:15:28 AM
> Subject: Re: [Puppet-dev] metaparam question

>> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg  wrote:
>> 
>>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>>> hello,
>>> 
>>> I would like to add a metaparameter - which I think is easy now via
>>> Type.newmetaparam.
>> 
>> We haven't been thinking of metaparameters as a general purpose extension 
>> point.
>> This came up once before that I know of, about a year ago, and there's a 
>> little
>> discussion of this in https://tickets.puppetlabs.com/browse/PUP-4281. The
>> conclusion we reached at the time was, more or less, to explore whether the
>> desired change could be accomplished with a puppet function and/or a change 
>> to
>> core puppet.
> 
> I don't remember my original logic for metaparams not being an extension 
> point.
> Given the system's support for easily loading this kind of plugin, it should 
> be
> easy to add.
> 
> The discussion on the ticket is pretty limited. We've got multiple use cases
> here, and I have seen use cases in the past brought up, but they were mostly 
> on
> lists or in person, so I don't think they made it into the ticketing system.
> 
> RI - you up for a pull request to add the autoloading? Have you thought much
> about how this might go wrong?

Unfortunately my time is pretty limited and I'll prefer to see what I can do
with this information in the catalog rather than making a merge worthy patch
at present.

I'm happy to just patch my actual puppet code with no modules to get to that
point, I'll have a look at the validate_cmd in file to see if there is a way
in the provider base class to do it so all providers get it.

For me this being in every provider is a nice to have I guess - it would be
great to be able to improve puppets ability to check if something worked 
in the broad sense, and it's going to apply to all resource types equally.

I could take the route that https://github.com/nanliu/puppet-transport does
and instead of making the meta param do something use it just to store this
information and then some other module do something with this information and
that would let me play with what I want, but it seems a small step to improve
puppet in general.

I've told many people this story so no doubt you remember the context Luke
but what I'd like to do is allow people to attach these checks to resources
and of course use them in catalog apply stage to get a deeper sense of 'done'
in the same way that the File validate_cmd does but later I want to consume 
the catalog in other tools and extract the same - or rather hopefully speak
to some API and get this.

Imagine I had a bridge to Sensu where it loads the catalog and just checks
everything that has a validate attached for free with no configuration required,
this is actually really easy with sensu as it doesnt require you to declare
the checks upfront on the server and all you have to do is write JSON to a
socket for every check, it'll promiscuously take them and show them and do
alerts etc. 

Once you have this information and can surface it additionally into the Puppet
eco system you can gather the state regularly like a monitoring system does
and it becomes quite interesting later down the line when perhaps you can 
use this to influence the node assignment logic in the orchastrator so that
a Web app will be configured to talk to a known good DB server - or that
you can detect that the web app should be reconfigured once it's chosen
DB isn't good anymore.  That's quite far down the line though, I think just
being able to do the monitoring part into sensu would already be a huge win
for many.



> 
>>> The thing that I can't seem to find any example of though is how to
>>> make this metaparameter do something on the nodes for all providers
>>> or all types.
>>> 
>>> Imagine there's a metaparam that might describe how to test a resource
>>> works, something like:
>>> 
>>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>>> 
>>> I'd then want to have some code that would be run on the agent nodes
>>> for any resource that has this param set.
>> 
>> I don't think something like this exists per se, but 'validate' might be one
>> such example of something worth adding to core puppet. Fwiw, one
>> resource-specific example added not too long ago is the file type's
>> validate_cmd:

Re: [Puppet-dev] metaparam question

[Luke Kanies]

> On Feb 4, 2016, at 3:35 PM, Kylo Ginsberg  wrote:
> 
>> On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:
>> hello,
>> 
>> I would like to add a metaparameter - which I think is easy now via
>> Type.newmetaparam.
> 
> We haven't been thinking of metaparameters as a general purpose extension 
> point. This came up once before that I know of, about a year ago, and there's 
> a little discussion of this in 
> https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we reached at 
> the time was, more or less, to explore whether the desired change could be 
> accomplished with a puppet function and/or a change to core puppet.

I don't remember my original logic for metaparams not being an extension point. 
Given the system's support for easily loading this kind of plugin, it should be 
easy to add.

The discussion on the ticket is pretty limited. We've got multiple use cases 
here, and I have seen use cases in the past brought up, but they were mostly on 
lists or in person, so I don't think they made it into the ticketing system.

RI - you up for a pull request to add the autoloading? Have you thought much 
about how this might go wrong?

>> The thing that I can't seem to find any example of though is how to
>> make this metaparameter do something on the nodes for all providers
>> or all types.
>> 
>> Imagine there's a metaparam that might describe how to test a resource
>> works, something like:
>> 
>>service{"httpd": validate => "check_http --port 80 -H localhost"}
>> 
>> I'd then want to have some code that would be run on the agent nodes
>> for any resource that has this param set.
> 
> I don't think something like this exists per se, but 'validate' might be one 
> such example of something worth adding to core puppet. Fwiw, one 
> resource-specific example added not too long ago is the file type's 
> validate_cmd: 
> 
> https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd.
> 
> Also, I'm guessing you're aware but another tool in the toolbox *might* 
> include the postrun_command setting (which is *not* per-provider as you were 
> thinking):
> 
> https://docs.puppetlabs.com/puppet/latest/reference/configuration.html#postruncommand

Ryan and others on his team have discussed ways of using the new 
consume/produce system for doing service validation. I don't think the whole 
system is quite in place, but there are some good ideas getting close.

That doesn't settle the generalized plugin capability issue, though.

> More broadly, there has been some discussion on a few threads and tickets 
> about making the agent lifecycle provide some more useful hooks, so I'd love 
> to hear more ideas about how to improve this.

> 
> Thanks!
> Kylo
> 
>> 
>> Does anyone have any hints or know of an example of this?
>> 
>> thanks
>> 
>> ---
>> R.I.Pienaar
>> 
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Puppet Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to puppet-dev+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/puppet-dev/70254077.345165.1454514435921.JavaMail.zimbra%40devco.net.
>> For more options, visit https://groups.google.com/d/optout.
> 
> 
> 
> -- 
> Kylo Ginsberg | k...@puppetlabs.com | irc: kylo | twitter: @kylog
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to puppet-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/puppet-dev/CALsUZFHniTYyYU0ThP-Jk37f6QXroJ0P0-K0PkbRPXMsdB6v0Q%40mail.gmail.com.
> For more options, visit https://groups.google.com/d/optout.

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/FB2D2883-CE00-4D61-ACC2-0B59F807A4D5%40puppetlabs.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Puppet-dev] metaparam question

[Kylo Ginsberg]

On Wed, Feb 3, 2016 at 7:47 AM, R.I.Pienaar  wrote:

> hello,
>
> I would like to add a metaparameter - which I think is easy now via
> Type.newmetaparam.
>

We haven't been thinking of metaparameters as a general purpose extension
point. This came up once before that I know of, about a year ago, and
there's a little discussion of this in
https://tickets.puppetlabs.com/browse/PUP-4281. The conclusion we reached
at the time was, more or less, to explore whether the desired change could
be accomplished with a puppet function and/or a change to core puppet.


>
> The thing that I can't seem to find any example of though is how to
> make this metaparameter do something on the nodes for all providers
> or all types.


> Imagine there's a metaparam that might describe how to test a resource
> works, something like:
>
>service{"httpd": validate => "check_http --port 80 -H localhost"}
>
> I'd then want to have some code that would be run on the agent nodes
> for any resource that has this param set.
>

I don't think something like this exists per se, but 'validate' might be
one such example of something worth adding to core puppet. Fwiw, one
resource-specific example added not too long ago is the file type's
validate_cmd:

https://docs.puppetlabs.com/puppet/latest/reference/type.html#file-attribute-validate_cmd
.

Also, I'm guessing you're aware but another tool in the toolbox *might*
include the postrun_command setting (which is *not* per-provider as you
were thinking):

https://docs.puppetlabs.com/puppet/latest/reference/configuration.html#postruncommand

More broadly, there has been some discussion on a few threads and tickets
about making the agent lifecycle provide some more useful hooks, so I'd
love to hear more ideas about how to improve this.

Thanks!
Kylo


> Does anyone have any hints or know of an example of this?
>
> thanks
>
> ---
> R.I.Pienaar
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-dev+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/puppet-dev/70254077.345165.1454514435921.JavaMail.zimbra%40devco.net
> .
> For more options, visit https://groups.google.com/d/optout.
>



-- 
Kylo Ginsberg | k...@puppetlabs.com | irc: kylo | twitter: @kylog

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/CALsUZFHniTYyYU0ThP-Jk37f6QXroJ0P0-K0PkbRPXMsdB6v0Q%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[Puppet-dev] metaparam question

[R.I.Pienaar]

hello,

I would like to add a metaparameter - which I think is easy now via 
Type.newmetaparam.

The thing that I can't seem to find any example of though is how to
make this metaparameter do something on the nodes for all providers
or all types.

Imagine there's a metaparam that might describe how to test a resource
works, something like:

   service{"httpd": validate => "check_http --port 80 -H localhost"}

I'd then want to have some code that would be run on the agent nodes
for any resource that has this param set.

Does anyone have any hints or know of an example of this?

thanks

---
R.I.Pienaar

-- 
You received this message because you are subscribed to the Google Groups 
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to puppet-dev+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/puppet-dev/70254077.345165.1454514435921.JavaMail.zimbra%40devco.net.
For more options, visit https://groups.google.com/d/optout.