Hi,
I'm doing another pass on the subject (I like to fight for my
ideas ;-)).
You can dump it if it doesn't make any sense as I'm clearly lacking some
deserved sleep :-).
On Mon, 2008-10-20 at 19:09 -0500, Luke Kanies wrote:
> On Oct 18, 2008, at 7:40 AM, Brice Figureau wrote:
> > [snipped]
> > We could have a syntax that means "collection" as a new puppet type.
> > We could even store a collection in a puppet variable.
> > If the result of the collection is "not used" (ie a statement
> > instead of
> > a rvalue), then it means "realize/import/override"
>
> If we store a collection in a variable, does that mean it doesn't get
> evaluated? If we do so, does that mean we can do things like:
>
> $variable { foo => bar }
I guess so. Note that I wrote this without really thinking about the
whole consequences on puppet :-)
> Is there a difference in what the collection does (other than,
> obviously, the override) between a collection with and without an
> override? I.e., if we wanted to collect virtual resources, realize
> them, and override them, would it take two collections (one to
> realize, one to override)? I think I know the answer, based on the
> text below, but figured I'd ask.
I don't think so. You can override and realize (or collect) at the same
time in the same syntax, isn't it?
> If we have a query with no source (or @ symbols) and no override, what
> does it mean? Is it a language warning?
It is a language error if used as a plain statement or left value.
In the current puppet, you can't write an isolated:
File["/tmp/toto"]
It's the exact same thing, it has no meaning.
It _could_ be used as a reference to a collection in what I originally
had in mind, if used as an rvalue.
OK, maybe this has no use or no meaning, but the "collection" syntax
would allow to express:
file {
...
require => @File[ tag == "ldap" ]
}
> >
> > Let's say we have a language structure that "builds" a collection. We
> > have to say
> > * where this collection comes (catalog, external source of any kind,
> > virtual)
> > * selecting/filtering query
> > * any override
> >
> > Maybe we can introduce a new keyword?
> > collect<Resource>(source)[query] { override }
> >
> > Or maybe write it simply like this:
> > Resource(source)[query] { override }
> >
> > I always thought the <| |> and <<| |>> syntax was not fitting
> > perfectly
> > in the puppet language, hence my proposal to have the query in [] to
> > match the Resource[title] syntax.
> > When the query is a simple string, then it would be expanded as "title
> > == string" which maps back to the current "resource selection".
>
> I disagree with the <||> syntax not being awesome, but I'm apparently
> the only person who thinks that.
lol :-)
That's not the syntax that is bizarre, it just doesn't look "puppet" to
me (ok I'm used to it now, so it looks "puppet" now).
I don't mind if you want to keep this syntax for queries.
But in my mind writing:
File["/tmp/toto"]
is a generalization of writing
File <| title == "/tmp/toto" |>
even if in the fact it is totally different (I mean in puppet as of
now).
My vision of the things is much more:
* there are no more "collect" statement (ie Resource<||> or
Resource<<||>>)
* there is a syntax expressing: reference to a collection (ie
Resource(source)[query])
* collection references can be realized or collected, which in turns
means realizing the collection or collect it.
* collection references can be used where a resource reference is used.
Because a resource reference is not more than a reference to a
collection of only 1 non virtual resource.
* you can apply overrides to such reference, which means, once it is
collected/realized (or used) it is overriden. This would also trigger
collection/realization. This can only be used as a whole statement of
course (no rvalue).
> But yes, I've also thought that the Resource["title"] syntax could be
> equivalent to Resource[title == "title"].
>
> That would get a bit hard to parse, I think, since I'm pretty sure the
> title can be any rvalue, and supporting either an rvalue by itself or
> a full query syntax might be a touch hard.
Yes, I understand where the problem is. If you want to go this way you
can build up on the expression stuff I contributed. It deals with
rvalues being part of boolean expressions.
Then the grammar could be written as:
collection RBRACK expression LBRACK
The difficulty is then to tell to the expression code to produce a query
code matching the original query that would be executed in
Collector.match?, which the expression system doesn't yet support (but I
don't see why it wouldn't be possible).
> >
> > The source can be either:
> > * empty -> means catalog, and we're back to the simpler syntax
> > * catalog, same as empty
> > * virtual
> > * any defined external source coming from puppet.conf, with the only
> > known type at this stage is active_record or storeconfigs. The
> > puppet.conf syntax should be specified too.
> >
> > Since it might be cumbersome to add the source everywhere, we could
> > have
> > shorter syntax (see below).
> >
> > There's no need to change the override syntax which is clear and
> > understandable and works fine.
> >
> > So to summarize, here's how I see it:
> >
> > * Catalog resources:
> >
> > Resource[query] { override }
> >
> > Ex:
> > File["/tmp/test"] { mode => 0666 }
> > or
> > User[ uid == 1000 ] { groups => staff }
> > or
> > User(catalog)[ uid == 1000 ] { groups => staff }
>
> This seems fine, except maybe that last bit. Puppet's language
> doesn't have any concept of keywords like 'catalog', and I'm a bit
> uncomfortable with the function-like syntax. I don't know what else
> to suggest for it, but those two things make me hesitate.
OK, why not using a syntax ala C++ templates:
User<catalog>[ uid == 1000 ] { groups => staff }
catalog would just be a reserved word for a specific source. If you
don't want to call it catalog, you can call it node (it is the resources
defined in the current node after all).
> >
> > * Virtual resources
> >
> > Resource(virtual)[query] { override }
> >
> > Ex:
> > File(virtual)["/tmp/test"] { override }
> > or
> > File(virtual)[ tag == test ] { override }
> >
> > We can have a short syntax for virtual:
> > @File[ query ] { override }
> >
> > or in its simplest form:
> > @File
>
> Interesting idea -- reusing the @ signs for virtual/exported for the
> collections too -- I like that. I'm not totally against the (virtual)
> syntax, but...
The (virtual) or (catalog) syntax is just to unify the whole syntax into
something that is general:
Resource(source)[query] { override }
With the (source) being optional which means catalog when missing.
Or with short syntax, prefixed by @ or @@.
The { override } block would also be optional. If used, it adds an
override pass on the realized, collected or referenced resource.
> >
> > Used alone, it means "realize"
> > It should also be possible to write:
> >
> > $collection = @File
> > realize( $collection )
> > to achieve the same thing
>
> Interesting; ok.
>
> >
> > * Exported resources
> >
> > File(storedconfigs)[ tag == test ] { override }
> >
> > if we one day have more than one external source.
> > Let's say, I have an ldap source of resource defined in my
> > puppet.conf, ala:
> >
> > [mysource]
> > sourcetype = ldap
> > ldaphost= localhost
> >
> > Then I could use the following in my manifests:
> >
> > File(mysource)[ tag == test ] { override }
>
> puppet.conf doesn't currently supporting having this kind of named
> collection of parameters -- the sections are for environments and
> nothing else, at this point. I'm not sure how easily I could add one,
> either.
Maybe there could be a sources.conf ala fileserver.conf which would
contain the defined sources (minus the mandatory ones, like catalog and
storedconfig or active_record). If you generalize the storeconfigs
system to allow resource storage in other scheme, you'll need to have a
way to configure it one way or another.
> >
> > And for storeconfigs, we can have a shorter syntax:
> > @@File[ query ] {override}
> >
> > or, in its simplest form:
> > @@File
> >
> > which means all resources of type File that are exported in
> > storeconfigs.
>
> Wouldn't this find all exported resources, and, um, do nothing with
> them?
Yes, as I presented the whole thing, I think you're right.
> Or are you suggesting that the query syntaxes could be used to export/
> virtualize resources?
That would break my aforementionned unifying rule saying that all
collections are in fact references to collection :-(
I guess you'd have to:
collect( @@File )
which is not as good as the current syntax.
Or allow a straight isolated "collection reference" (ie exported or
virtual, but not catalog) to realize/collect itself?
> >
> > Comments, and flames accepted :-)
> >
> >> Barring any brilliant ideas from others, I'd say just provide the
> >> behaviour change described above (the local resource query operates
> >> against all resources, and marks virtual resources as non-virtual).
> >
> > OK, so instead of acting only on virtual resources, we scan both
> > virtual
> > resources and catalog resources, select them, override them
> > accordingly
> > and if we ever encounter one that is virtual, we realize it.
> > Is it right?
>
> Yeah, that's what I was recommending.
That's something that should be doable without touching too much code in
collector.rb.
What would happen if I change collector.rb to _always_ return all
resources matching the query beit virtual or real (but not exported)?
Wouldn't that produce twice the same resources?
I think I have to study the whole process a little bit more.
> >
> >> Note that the evaluate_generators method in Parser::Compiler should
> >> be
> >> evaluated to make sure its behaviour still makes sense with this
> >> change, along with the 'fail_on_unevaluated' method in the same
> >> class.
> >
> > I'll try to assess this later this week. But I think
> > evaluate_generators
> > will still be used by storeconfigs resources, right?
>
> Every collection would be evaluated in the evaluate_generators method,
> and any overrides would need to end up in the fail_on_unevaluated
> method (although I'm not exactly sure if it's even necessary for
> overrides related to collections -- I suppose the overrides would
> happen when the collections are evaluated).
I don't think it matters, in the end the resources get overriden in
Compiler.finish(). I'm not sure that moving the overriding directly in
evaluate_collections (or evaluate_generators) brings us any advantage.
Or maybe I'm wrong and I don't see your point?
--
Brice Figureau <[EMAIL PROTECTED]>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Puppet Developers" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/puppet-dev?hl=en
-~----------~----~----~----~------~----~------~--~---
