I think keeping the ability to accept ad-hoc inputs (at least for now) is a good idea =). This will (among other thing) make the job of writing custom service template less cumbersome. Just mentioning again that this is the place I see a possible justification for ad-hoc 'additions', as we don't have an operation type, in contrast to other entities.
the suggestion of 'cfg' is only valid if we enable ad-hoc inputs, so if at the end we will drop this support, I don't think 'cfg' is a good idea. By the way, do you see a way of not being able to derive the value of an intrinsic function's 'result'? (I know it is not currently supported) On Thu, Nov 9, 2017 at 9:02 PM, Tal Liron <[email protected]> wrote: > Thanks for keeping the discussion going, Avia. > > Yeah, I do not think that my interpretation is rock solid at all and it's > definitely forced. It's not hard to find examples in the spec that > contradict my interpretation, but also there are others that contradict the > opposite. I think we can all agree at least that the TOSCA spec writers did > not pin down this topic well enough so it's necessary for us to fill in the > gaps. I'm still sure that my interpretation is the most inherently correct > because it preserves the OOP base contract. If we allow ad-hoc inputs we > are making it that much harder to inherit existing node types while > expecting their existing implementations to continue working. If we allow > ah-hoc inputs, why not go ahead and allow a complete free-for-all here and > allow dynamic monkey patching of derived types? (This is actually what the > NFV csd04 profile is trying to do: "deprecate" parts of > tosca.nodes.Compute, which of course will break tooling that relies on that > contract.) The whole point of having type derivation in TOSCA seems to be > to allow reuse of tools, and I insist that is especially true for operation > inputs. > > Your idea of supporting a "cfg" (or similar) input in ARIA is something we > considered. (I don't think it's something we will have in the ARIA profile, > because we need it supported by *all* operations, including the normative > ones, and including custom ones created by template writers: so it would be > have to be hardcoded support in the ARIA codebase.) However, the problem > with this idea is that non-ARIA TOSCA parsers will fail. The whole idea of > using the "dependencies" hack was to allow such templates to at least parse > successfully with other parsers. Their are pros and cons to each approach, > though, and I'm open to reopening this. > > By the way, the next commit to ARIA adds a configuration option for the > parser: whether to support "ad-hoc inputs" or not. The default is to allow > them: something that as you know I am very reluctant to do, but feel we > have no choice at present. But, to do this properly, I think we need to > also find a way to preserve the type in intrinsic functions, as mentioned > in this discussion. It would still leave cases where it's possible to have > type-less inputs, but I guess that's just how it is right now. > > [as for the side note: there are *lots* of bugs in the included TOSCA > examples, including outright typos, so it's hard to consider them as > canonical...] > > > On Sat, Nov 4, 2017 at 9:26 AM, Avia Efrat <[email protected]> wrote: > > > I know this is an old thread, but since this is an important issue > (and > > I'm doing a review of old and interesting mails), I thought I'll take a > > shot at a reply. > > > > I'm not relying on one example in the spec. actually, we can see that > > inputs are defined under the normative "Standard" and "Configure" > > interfaces and their operations many time throughout the spec. Isn't the > > fact that inputs are assigned to interfaces and operations that do not > have > > the corresponding definition, in addition to the sentence from > [3.5.13.3], > > in addition to the interfaces being are the only entities in TOSCA that > do > > not have explicit distinction between definition and assignment enough to > > suggest that this may have been intentional? > > > > These usages of untyped inputs are still there in v1.2, and > unfortunately, > > artifact definitions still do not have a properties-like field. > > > > I liked your idea suggesting that "untyped" inputs get their type > > definition from the intrinsic functions, that seems to settle between the > > unusualness of interface definitions, and the fact that TOSCA gives us > the > > notion that assignment should be derived from a definition. This could > also > > be a (temporary?) solution to the lack of "operation type" concept, since > > it does not seem to solved in v1.2. > > > > In addition, I agree that this does not solve the problem of your > proposed > > inputs/meta inputs (inputs/configurations) distinction. Just a thought, > we > > could decide that an input with a special name such as "cfg" could be a > map > > containing "configurations", while other inputs are treated as regular > > inputs. This could part of the ARIA profile. > > > > [side note: > > You said: > > And indeed this is exactly what we see example 2.14.2, too, where we see > > property assignments. But this example snippet doesn't prove much of > > anything, because we don't see the node type defined, just used here. > > The "wp_db_port" input could/should be declared there. > > But in fact this type is defined in [8.3 > > <http://docs.oasis-open.org/tosca/TOSCA-Simple-Profile- > > YAML/v1.0/os/TOSCA-Simple-Profile-YAML-v1.0-os.html#_Toc471725259>], > > The section for non-normative types used for examples throughout the > spec. > > This section also exist in v1.2, with no changes to the aforementioned > node > > type] > > > > On Mon, Sep 11, 2017 at 8:01 PM, Tal Liron <[email protected]> wrote: > > > > > The single sentence you mention in the 3.5.13.3 is the only place that > > > *might* be implying that ad hoc, type-less input assignments are > allowed, > > > but I actually think it could have a different reading. What i meant > is: > > > "...that do not necessarily have a property definition defined in its > > > corresponding *interface* type." I think what is implied is what I > > mention > > > in level #2: a node template could have assignments that are not in the > > > interface type, *but obviously they have to be defined in the node > type*. > > > Obvious, I say, because they have to be defined somewhere: an undefined > > > property assignment makes little sense in TOSCA. So let me rewrite that > > > sentence with more clarity, without (I think) changing its original > > > meaning: > > > > > > "Template authors MAY provide property assignments on operation inputs > on > > > templates that do not necessarily have a property definition defined in > > its > > > corresponding interface type by adding an input definition to the node > > > type." > > > > > > To be more honest here: when I say "I think what is implied" I have to > > > wonder if the author was completely clear on what was implied. Like > many > > > parts of spec, there's a lot of theory that clashes in practice. We are > > > confused, but it could be that the author was confused as well. By > > > implementing the spec we have a responsibility to pin down what's > fuzzy. > > > > > > The "partial" differentiation as you call it in 3.5.13.1 is what I > meant: > > > in some places you use property definitions, in other assignments. The > > > language is very clear that you use definitions with types and > > assignments > > > with templates. They didn't explicitly have an "operation assignment" > > > grammar, but it's clear that there are two different grammars. When I > say > > > "operation assignment", it's a shorthand for the kind of operation > > > definition that you find in templates which uses property assignments. > > This > > > is consistent with everything else in TOSCA. > > > > > > And indeed this is exactly what we see example 2.14.2, too, where we > see > > > property assignments. But this example snippet doesn't prove much of > > > anything, because we don't see the node type defined, just used here. > > > The "wp_db_port" > > > input could/should be declared there. The fact that ARIA accepts this > has > > > always been very problematic to me: we here have an input that is > > untyped. > > > I consider this a bug. As we discussed, this particular example *might* > > > make a bit of sense because we are using an intrinsic function, which > > could > > > carry the type with it... but nowhere in the spec is any mechanism like > > > that explained. Many of the examples in the TOSCA spec are wrong or > > > incomplete, too, so I wouldn't be shocked if this one is wrong as well. > > > > > > Anyway, none of this addresses the core issue in my opinion, which I > keep > > > returning to: these kinds of parameters we add here -- SSH user, > > password, > > > etc. -- are meant as *directives to our execution plugin*, conceptually > > > very different from operation inputs. Mixing them together is confusing > > > (what do you do with name overlaps?) as well as a major security > concern. > > > These two kinds of values simply should not be mixed together. Indeed, > in > > > TOSCA 1.2 they might end up being artifact properties. > > > > > > > > > On Mon, Sep 11, 2017 at 11:25 AM, Avia Efrat <[email protected]> wrote: > > > > > > > In contrary to most of the TOSCA entities, TOSCA does not > differentiate > > > > between 'definition' and 'assignment' in the context of operations. > > There > > > > are only "operation definitions" [3.5.13]. Logically, there is a > > partial > > > > differentiation [3.5.13.1], where inputs in node type operations are > > > > expected to be property definitions, and inputs in node template > > > operation > > > > are expected to be property assignments (which are actual value > > > > assignments). Both of these options are listed under "operation > > > > definition". > > > > > > > > Under [3.5.13.3] it is explicitly stated that > > > > "Template authors MAY provide property *assignments* on operation > > inputs > > > on > > > > *templates* that do not necessarily have a property definition > defined > > in > > > > its corresponding type." (my emphasis) > > > > > > > > That is, from this paragraph, I think that it is clear that you can > > > define > > > > operation inputs in node templates (level #3) without them being > > defined > > > in > > > > the node template's type. In fact, [2.14.2] is an example of doing > just > > > > that. And out parser treats such a syntax as valid, see: > > > > https://github.com/apache/incubator-ariatosca/blob/ > > > > master/tests/parser/test_tosca_simple_v1_0/test_end2end.py#L73 > > > > > > > > Me thinking it was a good idea to construct TOSCA that way is another > > > thing > > > > =) > > > > > > > > On Mon, Sep 11, 2017 at 6:35 PM, Tal Liron <[email protected]> wrote: > > > > > > > > > Feel free to change the wiki, Ran, to whatever name you find > > > appropriate. > > > > > > > > > > I think what Avia discovered is not new to us and actually doesn't > > > solve > > > > > the problem, unfortunately. Let me go over what is clearly allowed > > and > > > > not > > > > > allowed in TOSCA, confusing because there are a few levels of > > > inheritance > > > > > here. > > > > > > > > > > 1. Interface types. Obviously, you are allowed to inherit an > > interface > > > > type > > > > > and add or override inputs. (ARIA insists that overridden input > types > > > be > > > > > derived from what it is they are overriding, too, to keep the OO > > > contract > > > > > intact.) > > > > > > > > > > 2. Node types. In the "interfaces" section you can define several > > > > > interfaces of various types. Here, again, TOSCA lets you > add/override > > > > > inputs. Though note here that the line of inheritance is quite > > complex: > > > > you > > > > > can override inputs from the interface type, but *also* from the > > parent > > > > > node type. (ARIA here has to do some work to make sure that you are > > > doing > > > > > it all OK and not breaking the OO contract, it's a rather complex > > part > > > of > > > > > the parser code.) > > > > > > > > > > I think the above is what Avia discovered. However, the third level > > is > > > > > locked to us: > > > > > > > > > > 3. Node templates. Unfortunately, here we can not add inputs ad > hoc. > > > The > > > > > "interfaces" section here is not the same DSL format as those > above: > > it > > > > is > > > > > operation *assignments* rather than operation *definitions*. When > you > > > > > assign input values here, they are validated against the defined > > types. > > > > It > > > > > would make no sense in TOSCA to just assign values without a type. > > > > > > > > > > So, because we can't add inputs at level #3, we still have a > problem: > > > we > > > > > would have to derive new node types for every type of execution. > SSH > > > > would > > > > > require its own node types, Juju would require its own node types, > > > Puppet > > > > > would require its own node types, etc. And that's for *all* your > node > > > > > types. This seems extremely un-scalable. > > > > > > > > > > But also, as I try to explain in the wiki page, I insist that these > > > kinds > > > > > of configuration parameters are essentially not the same as > operation > > > > > inputs. They are not meant to be used by the operation itself > > (script, > > > > > charm, recipe, etc.), rather by the mechanism that executes the > > > operation > > > > > (SSH, Juju, Puppet, etc.). Especially I point out the security > hole: > > > you > > > > > don't want an SSH password exposed and sent over the wire to the > > script > > > > > itself. It is simply not an input. > > > > > > > > > > By the way, it seems that there's some acknowledgement by other > folk > > in > > > > > OASIS about this gap in TOSCA, and there's an interest to use > > artifact > > > > > types in TOSCA 1.2 as a way to solve this problem. I don't think > > it's a > > > > bit > > > > > awkward, but at least there will be a standard solution. > > > > > > > > > > > > > > > On Sun, Sep 10, 2017 at 2:13 AM, Ran Ziv <[email protected]> wrote: > > > > > > > > > > > Avia's mentioned at one point that we might have misunderstood > the > > > spec > > > > > at > > > > > > this section, and that in fact it can be possible to pass > arbitrary > > > > > inputs > > > > > > into operations regardless of the interface definition - which > > would > > > > mean > > > > > > this notion of "configuration" might be unnecessary. > > > > > > > > > > > > Also, note that the doc page is talking about "executors", which > is > > > > > > confusing as that's a different concept in ARIA (see the base > > > executor > > > > > > class); Supposedly up until now we've simply called these > > "operation > > > > > > plugins". > > > > > > > > > > > > > > > > > > On Sat, Sep 9, 2017 at 1:43 AM, Tal Liron <[email protected]> > wrote: > > > > > > > > > > > > > Yes: > > > > > > > https://cwiki.apache.org/confluence/display/ARIATOSCA/ > > > > > > > Execution+Configuration > > > > > > > > > > > > > > On Fri, Sep 8, 2017 at 4:45 PM, DeWayne Filppi < > > > [email protected]> > > > > > > > wrote: > > > > > > > > > > > > > > > I see in the examples a list of dependencies for scripts in > > > > > operations, > > > > > > > for > > > > > > > > example: > > > > > > > > > > > > > > > > implementation: > > > > > > > > primary: scripts/configure.sh > > > > > > > > dependencies: > > > > > > > > - "ssh.user > { get_input: ssh_username }" > > > > > > > > - "ssh.key_filename > { get_input: > > > private_key_path > > > > > }" > > > > > > > > - "ssh.address > { get_attribute: [ > virtual_ip, > > > > > > > > floating_ip_address ] }" > > > > > > > > > > > > > > > > The spec seems to indicate that the dependency list is for > > > > resources > > > > > > that > > > > > > > > need to be made available so the main script can be run. > > Clearly > > > > > that > > > > > > > > isn't the case in the example. Is this '>' syntax documented > > > > > anywhere? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
