On 17/10/08 18:18, Luke Kanies wrote:
> On Oct 17, 2008, at 9:36 AM, Brice Figureau wrote:
>>> [...]
>>> The first query will make the resource non-virtual, and I think that
>>> means the second query will null-op, which means the override
>>> wouldn't
>>> happen.
>>>
>>> Right?
>>>
>>> As expected, I just ran this code in your 1088 branch, and just the
>>> first override took affect.
>> Yes, you're right.
>> Is it something I should change?
>> But then, when to mark the resource non virtual anymore?
>
> I, um, really don't know, so let's talk a bit about what the querying
> really does.
>
> Initially, the querying syntax was envisioned as a way to find
> resources in the database -- exported resources existed long before
> virtual resources. When a customer came up with the need for virtual
> resources (which, in retrospect, based on tools I'd written before
> Puppet, I should have created as part of Puppet's initial design), I
> modified the support for exported resources to support virtual
> resources as a subset.
Oh, I thought virtual resources were invented before exported resource.
I've always seen exported resources as a kind of virtual resources...
> The querying for exported resources was viewed as a simplified way to
> query a database. When the same querying was reused for virtual
> resources, it became a means of interacting with a collection of
> resources in memory, restricted to those that were marked virtual.
>
> The question becomes, should this querying be viewed as a means of
> interacting with all resources or memory, or should it retains its
> current focus on virtual and exported resources?
If the syntax changes to something different, then the meaning should
change. If we keep the current syntax, then the meaning should be the same.
> This is obviously a language design question, and it kind of brings up
> the question of whether we should step back a bit and view this whole
> querying/overriding a bit more holistically. It might be that we can
> simplify the available syntaxes into something that can coherently
> cover normal resources, virtual resources, and exported resources.
>
> What we need is:
> * Something to find exported resources in an external resource
> repository and add them to the current catalog
> * Something to find virtual resources in the current catalog and mark
> them non-virtual
> * Something to find resources in the current catalog and apply overrides
>
> At this point, I'm open to completely new syntaxes to cover all three
> of these needs, but I don't have any sudden inspiration myself.
> My original method for supporting two query types -- add an additional
> set of brackets -- clearly won't work in this case, because it would
> just be stupid.
>
> The only real I can think of right now (after, admittedly, not much
> thinking about it) is:
>
> * Keep the two similar query syntaxes right now, with this behaviour:
> * Resource <| |> operates on resources in the catalog. Any found
> resources that happen to be marked virtual will be marked non-virtual,
> and all overrides will always be applied.
> * Resource <<| |>> operates in resources in external resource
> repositories (currently only StoreConfigs). Any found resources will
> be added to the local catalog, and all overrides will be applied.
>
> I'd kind of like to find a way to reduce to a single query syntax
> (rather than the two forms with different numbers of brackets), but
> that would require having the ability to indicate a given query should
> operate against a resource repository vs. the local catalog. One of
> the reasons I'd like this is that I think we're going to need the
> ability to specify which repository to operate against. I can see a
> day where you'd have resources scattered around different databases
> and applications, and you'd want to integrate Puppet with all of
> them. It's reasonable to start with the restriction that Puppet can
> only be configured to talk to one of those repositories at once, but
> it'd be nice to eventually support connecting to multiple repositories.
>
> All I can think of for this is something like:
>
> Resource(source = active_record) <| foo == bar |>
>
> And that's just horrible. Horrible.
I didn't think too much about it yet.
I'm throwing this out of my head without thinking about it, so please be
indulgeant.
Basically right now:
* Resource[title] is representing a completely determined collection
of 1 resource from the catalog.
* Resource<| query |> is representing a collection of virtual resources
* Resource<<| query |>> is a collection of imported resources
All these syntax can be overriden by adding an "override block".
It should be possible to "merge" all this in the same unified syntax.
We could have a syntax that means "collection" as a new puppet type.
We could even store a collection in a puppet variable.
If the result of the collection is "not used" (ie a statement instead of
a rvalue), then it means "realize/import/override"
Let's say we have a language structure that "builds" a collection. We
have to say
* where this collection comes (catalog, external source of any kind,
virtual)
* selecting/filtering query
* any override
Maybe we can introduce a new keyword?
collect<Resource>(source)[query] { override }
Or maybe write it simply like this:
Resource(source)[query] { override }
I always thought the <| |> and <<| |>> syntax was not fitting perfectly
in the puppet language, hence my proposal to have the query in [] to
match the Resource[title] syntax.
When the query is a simple string, then it would be expanded as "title
== string" which maps back to the current "resource selection".
The source can be either:
* empty -> means catalog, and we're back to the simpler syntax
* catalog, same as empty
* virtual
* any defined external source coming from puppet.conf, with the only
known type at this stage is active_record or storeconfigs. The
puppet.conf syntax should be specified too.
Since it might be cumbersome to add the source everywhere, we could have
shorter syntax (see below).
There's no need to change the override syntax which is clear and
understandable and works fine.
So to summarize, here's how I see it:
* Catalog resources:
Resource[query] { override }
Ex:
File["/tmp/test"] { mode => 0666 }
or
User[ uid == 1000 ] { groups => staff }
or
User(catalog)[ uid == 1000 ] { groups => staff }
* Virtual resources
Resource(virtual)[query] { override }
Ex:
File(virtual)["/tmp/test"] { override }
or
File(virtual)[ tag == test ] { override }
We can have a short syntax for virtual:
@File[ query ] { override }
or in its simplest form:
@File
Used alone, it means "realize"
It should also be possible to write:
$collection = @File
realize( $collection )
to achieve the same thing
* Exported resources
File(storedconfigs)[ tag == test ] { override }
if we one day have more than one external source.
Let's say, I have an ldap source of resource defined in my puppet.conf, ala:
[mysource]
sourcetype = ldap
ldaphost= localhost
Then I could use the following in my manifests:
File(mysource)[ tag == test ] { override }
And for storeconfigs, we can have a shorter syntax:
@@File[ query ] {override}
or, in its simplest form:
@@File
which means all resources of type File that are exported in storeconfigs.
Comments, and flames accepted :-)
> Barring any brilliant ideas from others, I'd say just provide the
> behaviour change described above (the local resource query operates
> against all resources, and marks virtual resources as non-virtual).
OK, so instead of acting only on virtual resources, we scan both virtual
resources and catalog resources, select them, override them accordingly
and if we ever encounter one that is virtual, we realize it.
Is it right?
> Note that the evaluate_generators method in Parser::Compiler should be
> evaluated to make sure its behaviour still makes sense with this
> change, along with the 'fail_on_unevaluated' method in the same class.
I'll try to assess this later this week. But I think evaluate_generators
will still be used by storeconfigs resources, right?
Thanks,
--
Brice Figureau
Days of Wonder
http://www.daysofwonder.com
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
