Ok, I've started trying to work up some code to outline the change below...
However I'm hitting an issue whereby the catalogue is failing due to:
*"Error: Failed to apply catalog: Parameter ensure failed on
Netapp_snapmirror[actint-star-nactl02:/vol/v_puppet_db01_redo/q_puppet_db01_redo]:
Invalid value "initializing". Valid values are present, absent."*
I've updated my *ensurable *block to be:
ensurable do
desc "Netapp Snapmirror resource state. Valid values are: present,
absent, initializing."
defaultto(:present)
newvalue(:present) do
if provider.initialized
#provider.create
provider.progress
else
# provider.create
provider.progress
end
end
newvalue(:absent) do
provider.destroy
end
newvalue(:initializing) do
provider.progress
end
end
I've tried restarting the puppetmaster service several times, however keep
coming back with that error :(
Any ideas??
Cheers
Gavin
On Tuesday, 19 February 2013 08:40:42 UTC, Gavin Williams wrote:
>
> Nan
>
> That looks like a great way of working around this requirement...
>
> One quick q - how does '*provider.initialized?*' get set?
>
> Other than that, looks great... Will start working up some code now :)
>
> Cheers
> Gav
>
>
> On 19 February 2013 01:47, Nan Liu <[email protected]> wrote:
>
>> On Mon, Feb 18, 2013 at 3:21 AM, fatmcgav <[email protected]> wrote:
>>
>>>
>>> On 18 February 2013 10:50, Andy Parker <[email protected]> wrote:
>>>
>>>> I just took a look and see that you got no responses on puppet-users.
>>>> That is unfortunate :(
>>>>
>>>> On Mon, Feb 18, 2013 at 12:13 AM, Gavin Williams <[email protected]>wrote:
>>>>
>>>>> Morning All
>>>>>
>>>>> I posted this on Puppet-users a few days ago, but I thought i'd post
>>>>> it on here aswell to get a Dev's view-point...
>>>>>
>>>>> Firstly, apologies for the length of this post, however I thought it
>>>>> probably most useful to fully outline the challenge and the desired
>>>>> result...
>>>>>
>>>>> Ok, so we're in the process of Puppetizing our Oracle/NetApp platform
>>>>> for Live/DR running.
>>>>>
>>>>> The current manual process, upon setting up a new database, a set of
>>>>> volumes are created to contain the various Oracle DB elements, and these
>>>>> are then SnapMirror'd to the DR site.
>>>>> This SnapMirror process requires a period of time to copy the base
>>>>> data over... This time period is directly relational to the amount of
>>>>> data
>>>>> required... I.e. a copy of 20Gb may take an hour, 200Gb may take 10
>>>>> hours...
>>>>> During this period, the SnapMirror resource is in an 'initializing'
>>>>> state. Once the data copy is complete, then the resource will change to
>>>>> an
>>>>> 'initialized' state.
>>>>> The next step in the process is then to break the relationship so that
>>>>> the DR end can be used in a R/W mode...
>>>>>
>>>>> Now, in order to Puppetize this, I need to be able to replicate the
>>>>> above behaviour...
>>>>> I've got Puppet to create and initialize the relationship, and that
>>>>> works as expected. However Puppet doesn't currently care about the
>>>>> relationship state. Now that's easy enough to add in as a new property
>>>>> against the type/provider.
>>>>>
>>>>
>>>> Based on how you are describing this, I'm not sure that expressing it
>>>> as a parameter is best. It sounds like you are describing a situation
>>>> where
>>>> there are a few states that you care about, but transitioning between
>>>> those
>>>> states requires sitting in other "non-interesting" states for a while.
>>>> Describing the "non-interesting" states pushes the management of those
>>>> state transitions outside of puppet and possibly makes them harder to work
>>>> with.
>>>>
>>>
>>> Ok, that makes sense... Unless I do lots of masking and mapping of the
>>> intermediate status' into something that Puppet knows, but again, that adds
>>> complication etc...
>>>
>>>
>>>>
>>>>
>>>>> However what I'm struggling to understand is how, or if it's even
>>>>> possible, to automate the switch from 'Initialized' state to a 'Broken'
>>>>> state upon completion of the initialization stage???
>>>>>
>>>>>
>>>> Yeah. Normally puppet deals with achieving the desired state in a
>>>> single run of puppet. So one possible solution is to have puppet block! I
>>>> really don't think that in this situation that would be a good idea, since
>>>> it would leave everything else on the machine unmanaged for an unknown
>>>> length of time.
>>>>
>>>
>>> Yeh, we could be looking at transfer times of 24-48 hours on some of our
>>> larger datasets, so wouldn't want Puppet blocking for that long a period...
>>>
>>
>> So just to explore this a bit. An ensurable resource by default have a
>> present absent state, and the transition between them is pretty
>> straightforward.
>>
>> present -> absent (def destroy)
>> absent -> present (def create)
>>
>> I'm assuming present -> absent is short enough you can wait for the
>> process to complete, so create is the only problematic state.
>>
>> For now the closest thing appears to be a transition state that fails
>> (intending to block dependent resource):
>>
>> absent -> initializing -> present
>>
>> The custom ensurable block:
>> ensurable do
>> newvalue(:present) do
>> unless provider.initialized?
>> provider.create
>> else
>> provider.progress
>> end
>> end
>>
>> newvalue(:absent) do
>> provider.destroy
>> end
>>
>> newvalue(:initializing) do
>> provider.progress
>> end
>> end
>>
>> So when a resource is in an initializing state just report back the
>> progress status and fail:
>> def progress
>> percent = File.stat(resource[:name]).size / 100.0
>> fail("Creation in progress #{percent}% complete.")
>> end
>>
>> Here's the output example:
>>
>> # initial create
>> $ puppet apply tests/transition.pp
>> err: /Stage[main]//Transition[/tmp/demo_a]/ensure: change from absent to
>> present failed: Creation in progress 0.0% complete.
>> notice: /Stage[main]//Notify[complete]: Dependency
>> Transition[/tmp/demo_a] has failures: true
>> warning: /Stage[main]//Notify[complete]: Skipping because of failed
>> dependencies
>> notice: Finished catalog run in 0.08 seconds
>>
>> # in progress
>> $ puppet apply tests/transition.pp
>> err: /Stage[main]//Transition[/tmp/demo_a]/ensure: change from absent to
>> present failed: Creation in progress 12% complete.
>> notice: /Stage[main]//Notify[complete]: Dependency
>> Transition[/tmp/demo_a] has failures: true
>> warning: /Stage[main]//Notify[complete]: Skipping because of failed
>> dependencies
>> notice: Finished catalog run in 0.08 seconds
>>
>> # finished:
>> $ puppet apply tests/transition.pp
>> notice: complete
>> notice: /Stage[main]//Notify[complete]/message: defined 'message' as
>> 'complete'
>> notice: Finished catalog run in 0.08 seconds
>>
>> Now these databases definitions are currently driven from a YAML
>>>>> backend which maintains information such as database name, volume
>>>>> information, primary netapp controller, replication netapp controller,
>>>>> etc... Currently, this YAML file is just a file on the puppet master...
>>>>> However there are ambitions to move this into a more dynamic backend,
>>>>> such
>>>>> as CouchDB or similar... So that opens the possibility to automatically
>>>>> update the YAML resource state.. However Puppet still needs to be able to
>>>>> support updating that backend based on the information it gets from the
>>>>> actual resource...
>>>>>
>>>>> So to flow it out:
>>>>>
>>>>> 1. Create a new database in backend ->
>>>>> 2. Puppet creates volumes on primary ->
>>>>> 3. Data is added to volumes ->
>>>>> 4. Backend updated to indicate replication is required ->
>>>>> 5. Puppet creates volumes on Secondary and adds Snapmirror
>>>>> relationship ->
>>>>> 6. Snapmirror initializes in background ->
>>>>> 7. Puppet periodically runs against network device and checks
>>>>> resource state ->
>>>>> 8. Backend resource state is updated following each run? ->
>>>>> 9. Snapmirror initialization completes ->
>>>>> 10. Puppet runs, detects new resource state and then triggers
>>>>> break?
>>>>> 11. Backend resource state updated to 'broken'?
>>>>>
>>>>> Now 1 to 7 above are fine, but 8 to 11 are where I get a bit unsure...
>>>>>
>>>> I think you have most of the picture here. Puppet manages some of the
>>>> transitions between states in order to get to that final "broken" state.
>>>> Using defined resource types or parameterized classes won't get you there
>>>> since the information about whether the next step of the management of the
>>>> resource can be taken is on the node. As you said earlier, it is once the
>>>> snapmirror process reaches the "initialized" state that puppet should
>>>> finish its job.
>>>>
>>>> Since the data needs to come from the node, then there are a couple of
>>>> choices:
>>>> * a custom fact: doesn't seem good since you would be encoding in
>>>> facter the presence of particular resources
>>>> * an ENC the probes the Snapmirror system: seems doable, but once
>>>> again encodes the presence of particular resources outside the manifests
>>>> * a custom type: probably the best solution, the replication itself
>>>> is a kind of resource that you want to manage, and what needs to be done
>>>> is
>>>> heavily dependent on the current state and desired state of the resource
>>>>
>>>> So I would suggest creating a custom type and provider for a
>>>> "replicated data" resource, or even try splitting it up into several
>>>> different resources. Doing this will let you make the final transition
>>>> without having to change the catalog.
>>>>
>>>> I'll admit, though, that puppet doesn't really have a concept of an "in
>>>> progress" convergence of a resource, so I'm not sure how the report will
>>>> work out for these kinds of resources. I suspect that it would show a
>>>> change every time that puppet runs and the replication is still in
>>>> progress.
>>>>
>>>
>> The problem is failing is a bit misleading. Certainly an interesting use
>> case if we can mark the resource as pending and subsequent resources simply
>> noop, but as it stands we can't do anything like this:
>>
>> $ puppet apply tests/transition.pp
>> warning: Could not retrieve fact fqdn
>> notice: /Stage[main]//Transition[/tmp/demo_a]/ensure: current_value
>> absent, should be present Progress: 0.0 % (pending)
>> notice: /Stage[main]//Notify[complete]/message: current_value absent,
>> should be complete (noop)
>> notice: Finished catalog run in 0.08 seconds
>>
>> Andy, is this worth filing a feature request?
>>
>> Thanks,
>>
>> Nan
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Puppet Developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to [email protected].
>> To post to this group, send email to [email protected].
>> Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/puppet-dev?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
