Hey Trevor, I've been able to reproduce your problem and I see what's going on. Since this sort of post hoc catalog serialization isn't something that's done a lot (or at all) there are some sharp edges; I'm going to head into this code base with a big stick and I'm going to hit things really hard until they work. It might take me a few days but when I've got things a bit more sorted out I'll let you know what I've figured out.
Adrien On Thu, Mar 24, 2016 at 6:48 AM Trevor Vaughan <[email protected]> wrote: > Hey Adrian, > > The code is pretty simple right now and is as follows: > > ### Type > > module Puppet > newtype(:catalog_test) do > @doc = "Save a micro-catalog" > > newparam(:name) do > isnamevar > desc "Resource name to trigger off of" > end > > newproperty(:resource_type) do > desc "The resource type" > end > end > end > > ### Provider > > Puppet::Type.type(:catalog_test).provide(:micro_catalog) do > desc 'Save a micro-catalog' > > def resource_type > my_catalog = Puppet::Resource::Catalog.new > > target_resource = @resource.catalog.resources.find{|x| > (x.type == @resource[:resource_type].downcase.to_sym) && > (x.name == @resource[:name]) > } > > if target_resource > > @resource.catalog.relationship_graph.dependencies(target_resource).each do > |res| > my_catalog.add_resource(res) > end > > my_catalog.add_resource(@resource) > > > @resource.catalog.relationship_graph.dependents(target_resource).each do > |res| > my_catalog.add_resource(res) > end > end > > # Tinker Point > require 'pry' > binding.pry > return @resource[:resource_type] > end > end > > ### Test File > > file { '/tmp' : ensure => 'directory' } > > file { '/tmp/foo1': content => 'blah' } > > file { '/tmp/stack': ensure => 'directory' } > > file { '/tmp/stack/test1': content => 'test' } > > file { '/usr/test': > content => 'test', > require => File['/tmp/stack'] > } > > file { '/var/trigger': > content => 'test', > notify => File['/tmp/stack/test1'] > } > > catalog_test { '/var/trigger': > resource_type => 'file' > } > > Thanks for the help, > > Trevor > > On Wed, Mar 23, 2016 at 2:43 PM, Adrien Thebo <[email protected]> > wrote: > >> The exact issues that you may run into depend on how the catalog was >> generated and what it's storing; in my scenario I'm using the RAL to >> collect Puppet::Type instances, I'm converting them to Puppet::Resource >> instances, and then inserting those into the catalog. Trying to >> individually convert resources in the catalog might be messy because you'll >> need to delete the type you're converting and then add the Resource >> instance; it's doable but there are some logistics around that. >> >> Do you have code that you can provide to demonstrate what you're doing? >> Given a RAL catalog you should be able to get a resource catalog in some >> form, as for reactivating a catalog I'm not sure what you mean by >> "reactivate". And lastly, the "resource catalog" is effectively the only >> means of serializing a catalog right now; if you want to store a catalog >> you'll almost certainly have to convert your data to a resource catalog, or >> figure out how you can make Puppet::Type instances serializable. >> >> On Tue, Mar 22, 2016 at 8:43 AM Trevor Vaughan <[email protected]> >> wrote: >> >>> Hey Adrian, >>> >>> Thanks for the response. I tried the >>> Puppet::Resource::Catalog#to_resource method but ended up with an error on >>> 'copy_as_resource' not found. >>> >>> I also tried the 'to_resource' method on all of the resource in my >>> catalog individually but that didn't seem to get me anywhere either. >>> >>> Looking at the links that you posted, I suppose what I'm trying to do is >>> actually save a RAL catalog and then reactivate it via the command line at >>> a later point. Do you know if this is possible? I don't really need a >>> compiler-level resource catalog since this won't be shipped off of the >>> local system. >>> >>> Thanks, >>> >>> Trevor >>> >>> On Mon, Mar 21, 2016 at 11:16 AM, Adrien Thebo <[email protected]> >>> wrote: >>> >>>> I've been working on a project with similar goals and ran into the same >>>> thing. The core of the issue is that there are two catalogs, the resource >>>> catalog and the RAL catalog. The resource catalog is what the Puppet >>>> compiler emits and contains Puppet::Resource instances; this is what's >>>> actually sent to the agent and what you'll see if you run `puppet master >>>> --compile`. The RAL catalog is generated when the catalog is actually being >>>> applied ( >>>> https://github.com/puppetlabs/puppet/blob/master/lib/puppet/configurer.rb#L106), >>>> and converts Puppet::Resource instances to Puppet::Type instances. >>>> Puppet::Resource instances are meant to be serialized, but Puppet::Type >>>> instances aren't - I'm pretty sure that the error you're seeing is because >>>> Puppet is trying to serialize Puppet::Type instances that don't have the >>>> correct method defined. >>>> >>>> There is a document in the Puppet source ( >>>> https://github.com/puppetlabs/puppet/blob/master/docs/catalogs.md) >>>> that contains a bit more information about the different types of catalogs. >>>> >>>> That being said there's a couple of ways of converting things around to >>>> a more easily serializable format. >>>> >>>> One option is to individually convert Puppet::Type instances to >>>> Puppet::Resource instances as you find them; I've been doing them with >>>> something like this: >>>> >>>> Puppet::Type.type(:service).instances.map { |res| res.to_resource } >>>> >>>> Alternately, if you have an existing catalog and you want to convert >>>> and serialize that wholesale, you should be able to use >>>> `Puppet::Resource::Catalog#to_resource` ( >>>> https://github.com/puppetlabs/puppet/blob/master/lib/puppet/resource/catalog.rb#L490) >>>> to do this. I haven't tested this out but it seems like it should work. >>>> >>>> There are a couple of caveats in this - for instance things get a >>>> little bit screwy with some resource types such as files, but if you run >>>> into issues with too many parameters being generated/not enough parameters >>>> being generated, let me know. >>>> >>>> If you make progress in this area let me know; since I'm doing some >>>> poking around in this area I would be interested in seeing what you find >>>> out as well. >>>> >>>> >>>> >>>> On Sat, Mar 19, 2016 at 12:04 PM Trevor Vaughan <[email protected]> >>>> wrote: >>>> >>>>> Hi All, >>>>> >>>>> I'm doing some experiments with extracted catalog snippets but seem to >>>>> be having issues on saving the new catalog after the fact. >>>>> >>>>> I have a Puppet::Resource::Catalog object that has content and >>>>> relationships but when I attempt to save it using 'to_pson' I get an >>>>> "undefined method `to_pson_data_hash`" for any given resource in the >>>>> catalog. >>>>> >>>>> As an aside, if I just run @resource.catalog.to_pson from within a >>>>> provider, I end up with the same error. >>>>> >>>>> Any help would be appreciated. >>>>> >>>>> Thanks, >>>>> >>>>> Trevor >>>>> >>>>> -- >>>>> Trevor Vaughan >>>>> Vice President, Onyx Point, Inc >>>>> (410) 541-6699 >>>>> >>>>> -- This account not approved for unencrypted proprietary information -- >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Puppet Developers" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUnc589eJ3nujpF0G38%3D6hAHOMHVShU7xRU9KfzS%2BTjQg%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoUnc589eJ3nujpF0G38%3D6hAHOMHVShU7xRU9KfzS%2BTjQg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> -- >>>> Adrien Thebo | Puppet Labs >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Puppet Developers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> >>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/puppet-dev/CALVJ9SKOK4HyZyOMYT%3DF31m7GxdL0jVvmYvKhzUAPBb5qhn0wg%40mail.gmail.com >>>> <https://groups.google.com/d/msgid/puppet-dev/CALVJ9SKOK4HyZyOMYT%3DF31m7GxdL0jVvmYvKhzUAPBb5qhn0wg%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>> . >>> >>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Trevor Vaughan >>> Vice President, Onyx Point, Inc >>> (410) 541-6699 >>> >>> -- This account not approved for unencrypted proprietary information -- >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Puppet Developers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU7jp3SBnL2%3DOpa3abixQwPuUvWDgh6VEGRNntGjO8zsw%40mail.gmail.com >>> <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoU7jp3SBnL2%3DOpa3abixQwPuUvWDgh6VEGRNntGjO8zsw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> -- >> Adrien Thebo | Puppet Labs >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Puppet Developers" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJ6OjCnWG5TVqOMNFJF8Jc3kXvhGP9wgubg2CwET1Fy%3DA%40mail.gmail.com >> <https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJ6OjCnWG5TVqOMNFJF8Jc3kXvhGP9wgubg2CwET1Fy%3DA%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . > > >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Trevor Vaughan > Vice President, Onyx Point, Inc > (410) 541-6699 > > -- This account not approved for unencrypted proprietary information -- > > -- > You received this message because you are subscribed to the Google Groups > "Puppet Developers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWBPxK5fSfB5CGFZWyWc_PN7htatdChp4S%3DbZ9PayEiSg%40mail.gmail.com > <https://groups.google.com/d/msgid/puppet-dev/CANs%2BFoWBPxK5fSfB5CGFZWyWc_PN7htatdChp4S%3DbZ9PayEiSg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Adrien Thebo | Puppet Labs -- You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/puppet-dev/CALVJ9SJe2bkJT9W07kRO4YcOtPAm35BqMYFJVvWKxuVgQoq8xQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
