Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
On Thu, Feb 19, 2015 at 12:24:14PM +0600, Renat Akhmerov wrote: Guys, I really appreciate the input of you all. We decided that ideally we need to agree on that syntax within days, not weeks or months. But anyway, since we started this discussion yesterday I just want to give us extra 1-2 days to play with all these thoughts in our heads. Just one additional maybe a little bit crazy idea to the pile wea**ve already made: What if we officially allow more than one delimiter syntax? Why stick to just one? Because consistency is a good thing, it aids readability, reduces developer confusion, encourages consistent style, etc, etc. If you've ever worked on a perl codebase of any size, I'm sure you'll appreciate the zen of python: There should be one, and preferably only one obvious way to do it My 0.02 :) Steve __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Hi !
From those three I'd choose only { ... }, it looks better for YAML while
'%' sign looks foreign for YAML. Moreover, it needs extra spaces for
writing expressions:
Compare:
1. %$.var + 1%
2. % $.var + 1 %
3. {$.var + 1}
One more point from me: We can't do things just beacuse it is familiar with
N things and it should be so.
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Folks, one more thing to consider: the next big release of yaql (1.0,
coming really soon) will get support of curly braces (by default - to
initialise dictionaries in the following way:
v0.2/v0.3 syntax: dict(key1=value1, key2=value2)
v1 syntax: {key1=value1, key2=value2} (the old syntax works in v1 as well)
So, {} will become a valid yaql expression (empty dict). I believe it is
not a big deal to parse that correctly and differentiate between outer
markup and an expression contained within, however statements like {{}}
may be a little ugly.
--
Regards,
Alexander Tivelkov
On Thu, Feb 19, 2015 at 1:13 PM, Nikolay Makhotkin nmakhot...@mirantis.com
wrote:
Hi !
From those three I'd choose only { ... }, it looks better for YAML while
'%' sign looks foreign for YAML. Moreover, it needs extra spaces for
writing expressions:
Compare:
1. %$.var + 1%
2. % $.var + 1 %
3. {$.var + 1}
One more point from me: We can't do things just beacuse it is familiar
with N things and it should be so.
Best Regards,
Nikolay
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Hi again,
Sorry, I started writing this email before Angus replied so I will shoot it as
is and then we can continue…
So after discussing all the options again with a small group of team members we
came to the following things:
Syntax options that we’d like to discuss further
% 1 + 1 % # pro- ruby/js/puppet/chef familiarity con - spaces, and % is too
large symbol
{1 + 1} # pro - less spaces, con - no familiarity
? 1 + 1 ? # php familiarity, need spaces
The primary criteria to select these 3 options is that they are YAML
compatible. Technically they all would solve our problems (primarily no
embracing quotes needed like in Ansible so no ambiguity on data types).
The secondary criteria is syntax symmetry. After all I agree with Patrick's
point about better readability when we have opening and closing sequences alike.
Some additional details can be found in [0]
[0] https://etherpad.openstack.org/p/mistral-YAQL-delimiters
https://etherpad.openstack.org/p/mistral-YAQL-delimiters
Renat Akhmerov
@ Mirantis Inc.
On 18 Feb 2015, at 07:37, Patrick Hoolboom patr...@stackstorm.com wrote:
My main concern with the {} delimiters in YAQL is that the curly brace
already has a defined use within YAML. We most definitely will eventually
run in to parsing errors with whatever delimiter we choose but I don't feel
that it should conflict with the markup language it is directly embedded in.
It gets quite difficult to, at a glance, identify YAQL expressions. % %
may appear ugly to some but I feel that it works as a clear delimiter of both
the beginning AND the end of the YAQL query. The options that only escape the
beginning look fine in small examples like this but the workflows that we
have written or seen in the wild tend to have some fairly large expressions.
If the opening and closing delimiters don't match, it gets quite difficult to
read.
From: Anastasia Kuznetsova akuznets...@mirantis.com
mailto:akuznets...@mirantis.com
Subject: Re: [openstack-dev] [Mistral] Changing expression delimiters in
Mistral DSL
Date: February 17, 2015 at 8:28:27 AM PST
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
mailto:openstack-dev@lists.openstack.org
Reply-To: OpenStack Development Mailing List \(not for usage questions\)
openstack-dev@lists.openstack.org
mailto:openstack-dev@lists.openstack.org
As for me, I think that % ... % is not an elegant solution and looks
massive because of '%' sign. Also I agree with Renat, that % ... % reminds
HTML/Jinja2 syntax.
I am not sure that similarity with something should be one of the main
criteria, because we don't know who will use Mistral.
I like:
- {1 + $.var} Renat's example
- variant with using some functions (item 2 in Dmitry's list): { yaql:
“1+1+$.my.var 100” } or yaql: 'Hello' + $.name
- my two cents, maybe we can use something like: result: - Hello + $.name
-
Regards,
Anastasia Kuznetsova
On Tue, Feb 17, 2015 at 1:17 PM, Nikolay Makhotkin nmakhot...@mirantis.com
mailto:nmakhot...@mirantis.com wrote:
Some suggestions from me:
1. y 1 + $.var # (short from yaql).
2. { 1 + $.var } # as for me, looks more elegant than % %. And visually
it is more strong
A also like p7 and p8 suggested by Renat.
On Tue, Feb 17, 2015 at 11:43 AM, Renat Akhmerov rakhme...@mirantis.com
mailto:rakhme...@mirantis.com wrote:
One more:
p9: \{1 + $.var} # That’s pretty much what
https://review.openstack.org/#/c/155348/
https://review.openstack.org/#/c/155348/ addresses but it’s not exactly
that. Note that we don’t have to put it in quotes in this case to deal with
YAML {} semantics, it’s just a string
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 13:37, Renat Akhmerov rakhme...@mirantis.com
mailto:rakhme...@mirantis.com wrote:
Along with % % syntax here are some other alternatives that I checked for
YAML friendliness with my short comments:
p1: ${1 + $.var}# Here it’s bad that $ sign is used for two
different things
p2: ~{1 + $.var}# ~ is easy to miss in a text
p3: ^{1 + $.var}# For someone may be associated with regular
expressions
p4: ?{1 + $.var}
p5: {1 + $.var} # This is kinda crazy
p6: e{1 + $.var}# That looks a pretty interesting option to me, “e”
could mean “expression” here.
p7: yaql{1 + $.var} # This is interesting because it would give a clear and
easy mechanism to plug in other expression languages, “yaql” here is a used
dialect for the following expression
p8: y{1 + $.var}# “y” here is just shortened “yaql
Any ideas and thoughts would be really appreciated!
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 12:53, Renat Akhmerov rakhme...@mirantis.com
mailto:rakhme...@mirantis.com wrote:
Dmitri,
I agree with all your reasonings and fully support the idea of changing
the syntax now as well as changing
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
On Tue, Feb 17, 2015 at 7:06 AM, Dmitri Zimine dzim...@stackstorm.com
wrote:
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML
from {1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments
and suggestions welcome.
DETAILS:
-
We faced a number of problems with using YAQL expressions in Mistral DSL:
[1] must handle any YAQL, not only the ones started with $; [2] must
preserve types and [3] must comply with YAML. We fixed these problems by
applying Ansible style syntax, requiring quotes around delimiters (e.g.
“{1+$.my.yaql.var}”). However, it lead to unbearable confusion in DSL
readability, in regards to types:
publish:
intvalue1: {1+1}” # Confusing: you expect quotes to be string.
intvalue2: {int(1+1)}” # Even this doestn’ clean the confusion
whatisthis:{$.x + $.y}” # What type would this return?
We got a very strong push back from users in the filed on this syntax.
The crux of the problem is using { } as delimiters YAML. It is plain wrong
to use the reserved character. The clean solution is to find a delimiter
that won’t conflict with YAML.
Criteria for selecting best alternative are:
1) Consistently applies to to all cases of using YAML in DSL
2) Complies with YAML
3) Familiar to target user audience - openstack and devops
We prefer using two-char delimiters to avoid requiring extra escaping
within the expressions.
The current winner is % %. It fits YAML well. It is familiar to
openstack/devops as this is used for embedding Ruby expressions in Puppet
and Chef (for instance, [4]). It plays relatively well across all cases of
using expressions in Mistral (see examples in [5]):
A really long time ago I posted this patch for Heat:
https://review.openstack.org/#/c/41858/2/doc/source/template_guide/functions.rst
(adds a jinja2 function to Heat http://jinja.pocoo.org/docs/dev/)
I also used % %, it seems to be what people use when using jinja2 on yaml.
This was rejected because of security concerns of Jinja2.
ALTERNATIVES considered:
--
1) Use Ansible-like syntax:
http://docs.ansible.com/YAMLSyntax.html#gotchas
Rejected for confusion around types. See above.
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a concept of typed variables
to borrow from (please correct me if I missed it). But they have functions:
function: { function_name: {foo: [parameter1, parameter 2], bar:xxx”}}.
Applied to Mistral, it would look like:
publish:
- bool_var: { yaql: “1+1+$.my.var 100” }
You *could* have the expression as a list, like this (but might not work in
all cases):
{ yaql: [1, +, 1, $.my.var, , 100] }
Generally in Heat we make the functions and args a natural part of the yaml
so it's not one big string that gets parsed separately.
Tho' it would be nice to have a common approach to this, so I am partial to
the one you have here.
-Angus
Not bad, but currently rejected as it reads worse than delimiter-based
syntax, especially in simplified one-line action invocation.
3) paired with other symbols: php-styoe ? ..?
*REFERENCES: *
--
[1] Allow arbitrary YAQL expressions, not just ones started with $ :
https://github.com/stackforge/mistral/commit/5c10fb4b773cd60d81ed93aec33345c0bf8f58fd
[2] Use Ansible-like syntax to make YAQL expressions YAML complient
https://github.com/stackforge/mistral/commit/d9517333b1fc9697d4847df33d3b774f881a111b
[3] Preserving types in YAQL
https://github.com/stackforge/mistral/blob/d9517333b1fc9697d4847df33d3b774f881a111b/mistral/tests/unit/test_expressions.py#L152-L184
[4]Using % % in Puppet
https://docs.puppetlabs.com/guides/templating.html#erb-is-plain-text-with-embedded-ruby
[5] Etherpad with discussion
https://etherpad.openstack.org/p/mistral-YAQL-delimiters
[6] Blueprint
https://blueprints.launchpad.net/mistral/+spec/yaql-delimiters
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Zane, Angus, thanks for your input!
This functional based syntax is consisted for dev use. The trouble is it ITSELF
needs to be delimited. Without pissing off the users :) Consider two key usage:
input parameters and shorthand syntax for action input. That’s why we are
looking for two-char symmetric (opening + closing) delimiters.
task_with_full_syntax_input:
action: std.ssh
input:
cmd = awk '{ print % $.my_var % }' /etc/passwd
task_with_shorthand_action_input:
action: std.ssh cmd=awk '{ print \”% $.my_var %\ }' /etc/passwd”
using function-like we still will have to do
action: std.ssh cmd=“awk '{ print \”% yaql {$.my_var} %\ }' /etc/passwd”
that only adds confusion IMO.
The full set of usages for YAQL in DSL is here:
https://etherpad.openstack.org/p/mistral-YAQL-delimiters
DZ.
On Feb 18, 2015, at 7:22 AM, Zane Bitter zbit...@redhat.com wrote:
On 16/02/15 16:06, Dmitri Zimine wrote:
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a concept of typed
variables to borrow from (please correct me if I missed it). But they
have functions: function: { function_name: {foo: [parameter1, parameter
2], bar:xxx”}}. Applied to Mistral, it would look like:
publish:
- bool_var: { yaql: “1+1+$.my.var 100” }
Not bad, but currently rejected as it reads worse than delimiter-based
syntax, especially in simplified one-line action invocation.
Note that you don't actually need the quotes there, so this would be
equivalent:
publish:
- bool_var: {yaql: 1+1+$.my.var 100}
FWIW I am partial to this or to Renat's p7 suggestion:
publish:
- bool_var: yaql{1+1+$.my.var 100}
Both offer the flexibility to introduce new syntax in the future without
breaking backwards compatibility.
cheers,
Zane.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
point about better readability when we have opening and closing sequences
alike.
Some additional details can be found in [0]
[0] https://etherpad.openstack.org/p/mistral-YAQL-delimiters
https://etherpad.openstack.org/p/mistral-YAQL-delimiters
Renat Akhmerov
@ Mirantis Inc.
On 18 Feb 2015, at 07:37, Patrick Hoolboom patr...@stackstorm.com
mailto:patr...@stackstorm.com wrote:
My main concern with the {} delimiters in YAQL is that the curly brace
already has a defined use within YAML. We most definitely will eventually
run in to parsing errors with whatever delimiter we choose but I don't
feel that it should conflict with the markup language it is directly
embedded in. It gets quite difficult to, at a glance, identify YAQL
expressions. % % may appear ugly to some but I feel that it works as a
clear delimiter of both the beginning AND the end of the YAQL query. The
options that only escape the beginning look fine in small examples like
this but the workflows that we have written or seen in the wild tend to
have some fairly large expressions. If the opening and closing delimiters
don't match, it gets quite difficult to read.
From: Anastasia Kuznetsova akuznets...@mirantis.com
mailto:akuznets...@mirantis.com
Subject: Re: [openstack-dev] [Mistral] Changing expression delimiters
in Mistral DSL
Date: February 17, 2015 at 8:28:27 AM PST
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
mailto:openstack-dev@lists.openstack.org
Reply-To: OpenStack Development Mailing List \(not for usage
questions\) openstack-dev@lists.openstack.org
mailto:openstack-dev@lists.openstack.org
As for me, I think that % ... % is not an elegant solution and looks
massive because of '%' sign. Also I agree with Renat, that % ... %
reminds HTML/Jinja2 syntax.
I am not sure that similarity with something should be one of the main
criteria, because we don't know who will use Mistral.
I like:
- {1 + $.var} Renat's example
- variant with using some functions (item 2 in Dmitry's list): { yaql:
“1+1+$.my.var 100” } or yaql: 'Hello' + $.name
- my two cents, maybe we can use something like: result: - Hello +
$.name -
Regards,
Anastasia Kuznetsova
On Tue, Feb 17, 2015 at 1:17 PM, Nikolay Makhotkin
nmakhot...@mirantis.com mailto:nmakhot...@mirantis.com wrote:
Some suggestions from me:
1. y 1 + $.var # (short from yaql).
2. { 1 + $.var } # as for me, looks more elegant than % %. And
visually it is more strong
A also like p7 and p8 suggested by Renat.
On Tue, Feb 17, 2015 at 11:43 AM, Renat Akhmerov rakhme...@mirantis.com
mailto:rakhme...@mirantis.com wrote:
One more:
p9: \{1 + $.var} # That’s pretty much what
https://review.openstack.org/#/c/155348/
https://review.openstack.org/#/c/155348/ addresses but it’s not exactly
that. Note that we don’t have to put it in quotes in this case to deal
with YAML {} semantics, it’s just a string
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 13:37, Renat Akhmerov rakhme...@mirantis.com
mailto:rakhme...@mirantis.com wrote:
Along with % % syntax here are some other alternatives that I checked
for YAML friendliness with my short comments:
p1: ${1 + $.var} # Here it’s bad that $ sign is used for two
different things
p2: ~{1 + $.var} # ~ is easy to miss in a text
p3: ^{1 + $.var} # For someone may be associated with regular
expressions
p4: ?{1 + $.var}
p5: {1 + $.var}# This is kinda crazy
p6: e{1 + $.var} # That looks a pretty interesting option to me, “e”
could mean “expression” here.
p7: yaql{1 + $.var} # This is interesting because it would give a
clear and easy mechanism to plug in other expression languages, “yaql”
here is a used dialect for the following expression
p8: y{1 + $.var} # “y” here is just shortened “yaql
Any ideas and thoughts would be really appreciated!
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 12:53, Renat Akhmerov rakhme...@mirantis.com
mailto:rakhme...@mirantis.com wrote:
Dmitri,
I agree with all your reasonings and fully support the idea of changing
the syntax now as well as changing system’s API a little bit due to
recently found issues in the current engine design that don’t allow us,
for example, to fully implement ‘with-items’ (although that’s a little
bit different story).
Just a general note about all changes happening now: Once we release
kilo stable release our API, DSL of version 2 must be 100% stable. I
was hoping to stabilize it much earlier but the start of production use
revealed a number of things (I think this is normal) which we need to
address, but not later than the end of Kilo.
As far as % % syntax. I see that it would solve a number of problems
(YAML friendliness, type ambiguity) but my only not strong argument is
that it doesn’t look that elegant in YAML
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
On Feb 18, 2015, at 3:07 PM, Dmitri Zimine dzim...@stackstorm.com wrote:
Syntax options that we’d like to discuss further
% 1 + 1 % # pro- ruby/js/puppet/chef familiarity con - spaces, and % is
too large symbol
{1 + 1} # pro - less spaces, con - no familiarity
? 1 + 1 ? # php familiarity, need spaces
The primary criteria to select these 3 options is that they are YAML
compatible. Technically they all would solve our problems (primarily no
embracing quotes needed like in Ansible so no ambiguity on data types).
The secondary criteria is syntax symmetry. After all I agree with Patrick's
point about better readability when we have opening and closing sequences
alike.
To me, another critical criteria is familiarity: target users - openstack
developers and devops, familiar with the delimiters.
That is why of the three above I prefer % % .
It is commonly used in Puppet/Chef [1], Ruby, Javascript. One won’t be
surprised to see it and won’t need to change the muscle memory to type
open/closed characters especially when working on say Puppet and Mistral at
the same time (not unlikely).
[1]
https://docs.puppetlabs.com/guides/templating.html#erb-is-plain-text-with-embedded-ruby
I have been lurking on this thread, and just wanted to toss in $0.02 as you all
deliberate. In truth, any of the options Renat highlights would be fine, and
the points made to arrive at the final choices are sound. The end result will
types will be explicit, and that is great. In light of this though, using the
% % syntax is still ideal if only for one reason: friction.
In a recent discussion with a colleague of mine, he told me that in his daily
job, he is so busy and slammed with operations tasks, his measure of a tool he
will use is whether it provides value within 30-60 minutes. Otherwise, there is
a fire somewhere else that needs to be put out and cannot be bothered.
To be frank, there is no way that this proposed syntax change and how it is
ultimately decorated is going to be a game changer to how future users will
evaluate Mistral. But, at 3am in the morning during a production outage where
an Ops admin is hotpatching a workflow to get things moving again... That
disparity does in fact matter. Eyes are already trained to look for % %, the
large amount of spacing draws the eyes, and it's a known function in other
existing ops toolkits. Less friction to adoption/less friction to
troubleshoot.I think these are all pluses. I want to be drawn to where things
are changing or dynamic in my templates to aid in troubleshooting.
Again, this is just another data point to throw out. While users are busy
trying to absorb all that is workflow creation/design... Having as many
likenesses and anchors to existing tools can certainly not hurt adoption.
Thank you all for your efforts!
On Feb 18, 2015, at 3:20 AM, Renat Akhmerov rakhme...@mirantis.com wrote:
Hi again,
Sorry, I started writing this email before Angus replied so I will shoot it
as is and then we can continue…
So after discussing all the options again with a small group of team members
we came to the following things:
Syntax options that we’d like to discuss further
% 1 + 1 % # pro- ruby/js/puppet/chef familiarity con - spaces, and % is
too large symbol
{1 + 1} # pro - less spaces, con - no familiarity
? 1 + 1 ? # php familiarity, need spaces
The primary criteria to select these 3 options is that they are YAML
compatible. Technically they all would solve our problems (primarily no
embracing quotes needed like in Ansible so no ambiguity on data types).
The secondary criteria is syntax symmetry. After all I agree with Patrick's
point about better readability when we have opening and closing sequences
alike.
Some additional details can be found in [0]
[0] https://etherpad.openstack.org/p/mistral-YAQL-delimiters
Renat Akhmerov
@ Mirantis Inc.
On 18 Feb 2015, at 07:37, Patrick Hoolboom patr...@stackstorm.com wrote:
My main concern with the {} delimiters in YAQL is that the curly brace
already has a defined use within YAML. We most definitely will eventually
run in to parsing errors with whatever delimiter we choose but I don't
feel that it should conflict with the markup language it is directly
embedded in. It gets quite difficult to, at a glance, identify YAQL
expressions. % % may appear ugly to some but I feel that it works as a
clear delimiter of both the beginning AND the end of the YAQL query. The
options that only escape the beginning look fine in small examples like
this but the workflows that we have written or seen in the wild tend to
have some fairly large expressions. If the opening and closing delimiters
don't match, it gets quite difficult to read.
From: Anastasia Kuznetsova akuznets...@mirantis.com
Subject: Re: [openstack-dev] [Mistral] Changing expression delimiters
in Mistral DSL
Date: February
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
On 16/02/15 16:06, Dmitri Zimine wrote:
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a concept of typed
variables to borrow from (please correct me if I missed it). But they
have functions: function: { function_name: {foo: [parameter1, parameter
2], bar:xxx”}}. Applied to Mistral, it would look like:
publish:
- bool_var: { yaql: “1+1+$.my.var 100” }
Not bad, but currently rejected as it reads worse than delimiter-based
syntax, especially in simplified one-line action invocation.
Note that you don't actually need the quotes there, so this would be
equivalent:
publish:
- bool_var: {yaql: 1+1+$.my.var 100}
FWIW I am partial to this or to Renat's p7 suggestion:
publish:
- bool_var: yaql{1+1+$.my.var 100}
Both offer the flexibility to introduce new syntax in the future without
breaking backwards compatibility.
cheers,
Zane.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Syntax options that we’d like to discuss further
% 1 + 1 % # pro- ruby/js/puppet/chef familiarity con - spaces, and % is too
large symbol
{1 + 1} # pro - less spaces, con - no familiarity
? 1 + 1 ? # php familiarity, need spaces
The primary criteria to select these 3 options is that they are YAML
compatible. Technically they all would solve our problems (primarily no
embracing quotes needed like in Ansible so no ambiguity on data types).
The secondary criteria is syntax symmetry. After all I agree with Patrick's
point about better readability when we have opening and closing sequences
alike.
To me, another critical criteria is familiarity: target users - openstack
developers and devops, familiar with the delimiters.
That is why of the three above I prefer % % .
It is commonly used in Puppet/Chef [1], Ruby, Javascript. One won’t be
surprised to see it and won’t need to change the muscle memory to type
open/closed characters especially when working on say Puppet and Mistral at the
same time (not unlikely).
[1]
https://docs.puppetlabs.com/guides/templating.html#erb-is-plain-text-with-embedded-ruby
On Feb 18, 2015, at 3:20 AM, Renat Akhmerov rakhme...@mirantis.com wrote:
Hi again,
Sorry, I started writing this email before Angus replied so I will shoot it
as is and then we can continue…
So after discussing all the options again with a small group of team members
we came to the following things:
Syntax options that we’d like to discuss further
% 1 + 1 % # pro- ruby/js/puppet/chef familiarity con - spaces, and % is too
large symbol
{1 + 1} # pro - less spaces, con - no familiarity
? 1 + 1 ? # php familiarity, need spaces
The primary criteria to select these 3 options is that they are YAML
compatible. Technically they all would solve our problems (primarily no
embracing quotes needed like in Ansible so no ambiguity on data types).
The secondary criteria is syntax symmetry. After all I agree with Patrick's
point about better readability when we have opening and closing sequences
alike.
Some additional details can be found in [0]
[0] https://etherpad.openstack.org/p/mistral-YAQL-delimiters
Renat Akhmerov
@ Mirantis Inc.
On 18 Feb 2015, at 07:37, Patrick Hoolboom patr...@stackstorm.com wrote:
My main concern with the {} delimiters in YAQL is that the curly brace
already has a defined use within YAML. We most definitely will eventually
run in to parsing errors with whatever delimiter we choose but I don't feel
that it should conflict with the markup language it is directly embedded in.
It gets quite difficult to, at a glance, identify YAQL expressions. % %
may appear ugly to some but I feel that it works as a clear delimiter of
both the beginning AND the end of the YAQL query. The options that only
escape the beginning look fine in small examples like this but the workflows
that we have written or seen in the wild tend to have some fairly large
expressions. If the opening and closing delimiters don't match, it gets
quite difficult to read.
From: Anastasia Kuznetsova akuznets...@mirantis.com
Subject: Re: [openstack-dev] [Mistral] Changing expression delimiters in
Mistral DSL
Date: February 17, 2015 at 8:28:27 AM PST
To: OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
Reply-To: OpenStack Development Mailing List \(not for usage questions\)
openstack-dev@lists.openstack.org
As for me, I think that % ... % is not an elegant solution and looks
massive because of '%' sign. Also I agree with Renat, that % ... %
reminds HTML/Jinja2 syntax.
I am not sure that similarity with something should be one of the main
criteria, because we don't know who will use Mistral.
I like:
- {1 + $.var} Renat's example
- variant with using some functions (item 2 in Dmitry's list): { yaql:
“1+1+$.my.var 100” } or yaql: 'Hello' + $.name
- my two cents, maybe we can use something like: result: - Hello +
$.name -
Regards,
Anastasia Kuznetsova
On Tue, Feb 17, 2015 at 1:17 PM, Nikolay Makhotkin
nmakhot...@mirantis.com wrote:
Some suggestions from me:
1. y 1 + $.var # (short from yaql).
2. { 1 + $.var } # as for me, looks more elegant than % %. And
visually it is more strong
A also like p7 and p8 suggested by Renat.
On Tue, Feb 17, 2015 at 11:43 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:
One more:
p9: \{1 + $.var}# That’s pretty much what
https://review.openstack.org/#/c/155348/ addresses but it’s not exactly
that. Note that we don’t have to put it in quotes in this case to deal with
YAML {} semantics, it’s just a string
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 13:37, Renat Akhmerov rakhme...@mirantis.com wrote:
Along with % % syntax here are some other alternatives that I checked
for YAML friendliness with my short comments:
p1: ${1 + $.var
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
As a user of Mistral pretty regularly these days, I certainly prefers % %. I agree with the other comments on devops familiarity. And looking this from another angle, it's certainly easier to type % % then the other options, especially if you have to do this over and over again. LOL Although, I am interested in the security concerns of this use in Jinja2. __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Out of those three I still prefer % %. The main reason I like it is
familiarity. Also the question mark makes me think of a wildcard, and I
don't want to use curly braces because of all the aforementioned reasons
(already has a meaning in YAML).
On Wed, Feb 18, 2015 at 3:07 PM, Dmitri Zimine dzim...@stackstorm.com
wrote:
*Syntax options that we’d like to discuss **further *
% 1 + 1 % # pro- ruby/js/puppet/chef familiarity con - spaces, and % is
too large symbol
{1 + 1} # pro - less spaces, con - no familiarity
? 1 + 1 ? # php familiarity, need spaces
The primary criteria to select these 3 options is that they are YAML
compatible. Technically they all would solve our problems (primarily no
embracing quotes needed like in Ansible so no ambiguity on data types).
The secondary criteria is syntax symmetry. After all I agree with
Patrick's point about better readability when we have opening and closing
sequences alike.
To me, another critical criteria is familiarity: target users - openstack
developers and devops, familiar with the delimiters.
That is why of the three above I prefer % % .
It is commonly used in Puppet/Chef [1], Ruby, Javascript. One won’t be
surprised to see it and won’t need to change the muscle memory to type
open/closed characters especially when working on say Puppet and Mistral at
the same time (not unlikely).
[1]
https://docs.puppetlabs.com/guides/templating.html#erb-is-plain-text-with-embedded-ruby
On Feb 18, 2015, at 3:20 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:
Hi again,
Sorry, I started writing this email before Angus replied so I will shoot
it as is and then we can continue…
So after discussing all the options again with a small group of team
members we came to the following things:
*Syntax options that we’d like to discuss **further *
% 1 + 1 % # pro- ruby/js/puppet/chef familiarity con - spaces, and % is
too large symbol
{1 + 1} # pro - less spaces, con - no familiarity
? 1 + 1 ? # php familiarity, need spaces
The primary criteria to select these 3 options is that they are YAML
compatible. Technically they all would solve our problems (primarily no
embracing quotes needed like in Ansible so no ambiguity on data types).
The secondary criteria is syntax symmetry. After all I agree with
Patrick's point about better readability when we have opening and closing
sequences alike.
Some additional details can be found in [0]
[0] https://etherpad.openstack.org/p/mistral-YAQL-delimiters
Renat Akhmerov
@ Mirantis Inc.
On 18 Feb 2015, at 07:37, Patrick Hoolboom patr...@stackstorm.com wrote:
My main concern with the {} delimiters in YAQL is that the curly brace
already has a defined use within YAML. We most definitely will eventually
run in to parsing errors with whatever delimiter we choose but I don't feel
that it should conflict with the markup language it is directly embedded
in. It gets quite difficult to, at a glance, identify YAQL expressions.
% % may appear ugly to some but I feel that it works as a clear
delimiter of both the beginning AND the end of the YAQL query. The options
that only escape the beginning look fine in small examples like this but
the workflows that we have written or seen in the wild tend to have some
fairly large expressions. If the opening and closing delimiters don't
match, it gets quite difficult to read.
*From: *Anastasia Kuznetsova akuznets...@mirantis.com
*Subject: **Re: [openstack-dev] [Mistral] Changing expression
delimiters in Mistral DSL*
*Date: *February 17, 2015 at 8:28:27 AM PST
*To: *OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
*Reply-To: *OpenStack Development Mailing List \(not for usage
questions\) openstack-dev@lists.openstack.org
As for me, I think that % ... % is not an elegant solution and looks
massive because of '%' sign. Also I agree with Renat, that % ... %
reminds HTML/Jinja2 syntax.
I am not sure that similarity with something should be one of the main
criteria, because we don't know who will use Mistral.
I like:
- {1 + $.var} Renat's example
- variant with using some functions (item 2 in Dmitry's list): { yaql:
“1+1+$.my.var 100” } or yaql: 'Hello' + $.name
- my two cents, maybe we can use something like: result: - Hello +
$.name -
Regards,
Anastasia Kuznetsova
On Tue, Feb 17, 2015 at 1:17 PM, Nikolay Makhotkin
nmakhot...@mirantis.com wrote:
Some suggestions from me:
1. y 1 + $.var # (short from yaql).
2. { 1 + $.var } # as for me, looks more elegant than % %. And
visually it is more strong
A also like p7 and p8 suggested by Renat.
On Tue, Feb 17, 2015 at 11:43 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:
One more:
p9: \{1 + $.var} # That’s pretty much what
https://review.openstack.org/#/c/155348/ addresses but it’s not
exactly that. Note that we don’t have to put it in quotes in this case to
deal with YAML {} semantics, it’s
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
My main concern with the {} delimiters in YAQL is that the curly brace
already has a defined use within YAML. We most definitely will eventually
run in to parsing errors with whatever delimiter we choose but I don't feel
that it should conflict with the markup language it is directly embedded
in. It gets quite difficult to, at a glance, identify YAQL expressions.
% % may appear ugly to some but I feel that it works as a clear
delimiter of both the beginning AND the end of the YAQL query. The options
that only escape the beginning look fine in small examples like this but
the workflows that we have written or seen in the wild tend to have some
fairly large expressions. If the opening and closing delimiters don't
match, it gets quite difficult to read.
*From: *Anastasia Kuznetsova akuznets...@mirantis.com
*Subject: **Re: [openstack-dev] [Mistral] Changing expression
delimiters in Mistral DSL*
*Date: *February 17, 2015 at 8:28:27 AM PST
*To: *OpenStack Development Mailing List (not for usage questions)
openstack-dev@lists.openstack.org
*Reply-To: *OpenStack Development Mailing List \(not for usage
questions\) openstack-dev@lists.openstack.org
As for me, I think that % ... % is not an elegant solution and looks
massive because of '%' sign. Also I agree with Renat, that % ... %
reminds HTML/Jinja2 syntax.
I am not sure that similarity with something should be one of the main
criteria, because we don't know who will use Mistral.
I like:
- {1 + $.var} Renat's example
- variant with using some functions (item 2 in Dmitry's list): { yaql:
“1+1+$.my.var 100” } or yaql: 'Hello' + $.name
- my two cents, maybe we can use something like: result: - Hello +
$.name -
Regards,
Anastasia Kuznetsova
On Tue, Feb 17, 2015 at 1:17 PM, Nikolay Makhotkin
nmakhot...@mirantis.com wrote:
Some suggestions from me:
1. y 1 + $.var # (short from yaql).
2. { 1 + $.var } # as for me, looks more elegant than % %. And
visually it is more strong
A also like p7 and p8 suggested by Renat.
On Tue, Feb 17, 2015 at 11:43 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:
One more:
p9: \{1 + $.var} # That’s pretty much what
https://review.openstack.org/#/c/155348/ addresses but it’s not exactly
that. Note that we don’t have to put it in quotes in this case to deal with
YAML {} semantics, it’s just a string
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 13:37, Renat Akhmerov rakhme...@mirantis.com wrote:
Along with % % syntax here are some other alternatives that I checked
for YAML friendliness with my short comments:
p1: ${1 + $.var} # Here it’s bad that $ sign is used for two
different things
p2: ~{1 + $.var} # ~ is easy to miss in a text
p3: ^{1 + $.var} # For someone may be associated with regular
expressions
p4: ?{1 + $.var}
p5: {1 + $.var} # This is kinda crazy
p6: e{1 + $.var} # That looks a pretty interesting option to me, “e”
could mean “expression” here.
p7: yaql{1 + $.var} # This is interesting because it would give a clear
and easy mechanism to plug in other expression languages, “yaql” here is a
used dialect for the following expression
p8: y{1 + $.var} # “y” here is just shortened “yaql
Any ideas and thoughts would be really appreciated!
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 12:53, Renat Akhmerov rakhme...@mirantis.com wrote:
Dmitri,
I agree with all your reasonings and fully support the idea of changing
the syntax now as well as changing system’s API a little bit due to
recently found issues in the current engine design that don’t allow us, for
example, to fully implement ‘with-items’ (although that’s a little bit
different story).
Just a general note about all changes happening now: *Once we release
kilo stable release our API, DSL of version 2 must be 100% stable*. I
was hoping to stabilize it much earlier but the start of production use
revealed a number of things (I think this is normal) which we need to
address, but not later than the end of Kilo.
As far as % % syntax. I see that it would solve a number of problems
(YAML friendliness, type ambiguity) but my only not strong argument is that
it doesn’t look that elegant in YAML as it looks, for example, in ERB
templates. It really reminds me XML/HTML and looks like a bear in a grocery
store (tried to make it close to one old russian saying :) ). So just for
this only reason I’d suggest we think about other alternatives, maybe not
so familiar to Ruby/Chef/Puppet users but looking better with YAML and at
the same time being YAML friendly.
I would be good if we could here more feedback on this, especially from
people who started using Mistral.
Thanks
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 03:06, Dmitri Zimine dzim...@stackstorm.com wrote:
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML
from {1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
As for me, I think that % ... % is not an elegant solution and looks
massive because of '%' sign. Also I agree with Renat, that % ... %
reminds HTML/Jinja2 syntax.
I am not sure that similarity with something should be one of the main
criteria, because we don't know who will use Mistral.
I like:
- {1 + $.var} Renat's example
- variant with using some functions (item 2 in Dmitry's list): { yaql:
“1+1+$.my.var 100” } or yaql: 'Hello' + $.name
- my two cents, maybe we can use something like: result: - Hello +
$.name -
Regards,
Anastasia Kuznetsova
On Tue, Feb 17, 2015 at 1:17 PM, Nikolay Makhotkin nmakhot...@mirantis.com
wrote:
Some suggestions from me:
1. y 1 + $.var # (short from yaql).
2. { 1 + $.var } # as for me, looks more elegant than % %. And
visually it is more strong
A also like p7 and p8 suggested by Renat.
On Tue, Feb 17, 2015 at 11:43 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:
One more:
p9: \{1 + $.var} # That’s pretty much what
https://review.openstack.org/#/c/155348/ addresses but it’s not exactly
that. Note that we don’t have to put it in quotes in this case to deal with
YAML {} semantics, it’s just a string
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 13:37, Renat Akhmerov rakhme...@mirantis.com wrote:
Along with % % syntax here are some other alternatives that I checked
for YAML friendliness with my short comments:
p1: ${1 + $.var} # Here it’s bad that $ sign is used for two
different things
p2: ~{1 + $.var} # ~ is easy to miss in a text
p3: ^{1 + $.var} # For someone may be associated with regular
expressions
p4: ?{1 + $.var}
p5: {1 + $.var} # This is kinda crazy
p6: e{1 + $.var} # That looks a pretty interesting option to me, “e”
could mean “expression” here.
p7: yaql{1 + $.var} # This is interesting because it would give a clear
and easy mechanism to plug in other expression languages, “yaql” here is a
used dialect for the following expression
p8: y{1 + $.var} # “y” here is just shortened “yaql
Any ideas and thoughts would be really appreciated!
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 12:53, Renat Akhmerov rakhme...@mirantis.com wrote:
Dmitri,
I agree with all your reasonings and fully support the idea of changing
the syntax now as well as changing system’s API a little bit due to
recently found issues in the current engine design that don’t allow us, for
example, to fully implement ‘with-items’ (although that’s a little bit
different story).
Just a general note about all changes happening now: *Once we release
kilo stable release our API, DSL of version 2 must be 100% stable*. I
was hoping to stabilize it much earlier but the start of production use
revealed a number of things (I think this is normal) which we need to
address, but not later than the end of Kilo.
As far as % % syntax. I see that it would solve a number of problems
(YAML friendliness, type ambiguity) but my only not strong argument is that
it doesn’t look that elegant in YAML as it looks, for example, in ERB
templates. It really reminds me XML/HTML and looks like a bear in a grocery
store (tried to make it close to one old russian saying :) ). So just for
this only reason I’d suggest we think about other alternatives, maybe not
so familiar to Ruby/Chef/Puppet users but looking better with YAML and at
the same time being YAML friendly.
I would be good if we could here more feedback on this, especially from
people who started using Mistral.
Thanks
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 03:06, Dmitri Zimine dzim...@stackstorm.com wrote:
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML
from {1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments
and suggestions welcome.
DETAILS:
-
We faced a number of problems with using YAQL expressions in Mistral DSL:
[1] must handle any YAQL, not only the ones started with $; [2] must
preserve types and [3] must comply with YAML. We fixed these problems by
applying Ansible style syntax, requiring quotes around delimiters (e.g.
“{1+$.my.yaql.var}”). However, it lead to unbearable confusion in DSL
readability, in regards to types:
publish:
intvalue1: {1+1}” # Confusing: you expect quotes to be string.
intvalue2: {int(1+1)}” # Even this doestn’ clean the confusion
whatisthis:{$.x + $.y}” # What type would this return?
We got a very strong push back from users in the filed on this syntax.
The crux of the problem is using { } as delimiters YAML. It is plain
wrong to use the reserved character. The clean solution is to find a
delimiter that won’t conflict with YAML.
Criteria for selecting best alternative are:
1) Consistently applies to to all cases of using YAML in DSL
2) Complies with YAML
3) Familiar to target user audience - openstack and devops
We prefer using
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
One more:
p9: \{1 + $.var}# That’s pretty much what
https://review.openstack.org/#/c/155348/
https://review.openstack.org/#/c/155348/ addresses but it’s not exactly that.
Note that we don’t have to put it in quotes in this case to deal with YAML {}
semantics, it’s just a string
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 13:37, Renat Akhmerov rakhme...@mirantis.com wrote:
Along with % % syntax here are some other alternatives that I checked for
YAML friendliness with my short comments:
p1: ${1 + $.var} # Here it’s bad that $ sign is used for two different
things
p2: ~{1 + $.var} # ~ is easy to miss in a text
p3: ^{1 + $.var} # For someone may be associated with regular expressions
p4: ?{1 + $.var}
p5: {1 + $.var} # This is kinda crazy
p6: e{1 + $.var} # That looks a pretty interesting option to me, “e”
could mean “expression” here.
p7: yaql{1 + $.var} # This is interesting because it would give a clear and
easy mechanism to plug in other expression languages, “yaql” here is a used
dialect for the following expression
p8: y{1 + $.var} # “y” here is just shortened “yaql
Any ideas and thoughts would be really appreciated!
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 12:53, Renat Akhmerov rakhme...@mirantis.com
mailto:rakhme...@mirantis.com wrote:
Dmitri,
I agree with all your reasonings and fully support the idea of changing the
syntax now as well as changing system’s API a little bit due to recently
found issues in the current engine design that don’t allow us, for example,
to fully implement ‘with-items’ (although that’s a little bit different
story).
Just a general note about all changes happening now: Once we release kilo
stable release our API, DSL of version 2 must be 100% stable. I was hoping
to stabilize it much earlier but the start of production use revealed a
number of things (I think this is normal) which we need to address, but not
later than the end of Kilo.
As far as % % syntax. I see that it would solve a number of problems (YAML
friendliness, type ambiguity) but my only not strong argument is that it
doesn’t look that elegant in YAML as it looks, for example, in ERB
templates. It really reminds me XML/HTML and looks like a bear in a grocery
store (tried to make it close to one old russian saying :) ). So just for
this only reason I’d suggest we think about other alternatives, maybe not so
familiar to Ruby/Chef/Puppet users but looking better with YAML and at the
same time being YAML friendly.
I would be good if we could here more feedback on this, especially from
people who started using Mistral.
Thanks
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 03:06, Dmitri Zimine dzim...@stackstorm.com
mailto:dzim...@stackstorm.com wrote:
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML
from {1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments and
suggestions welcome.
DETAILS:
-
We faced a number of problems with using YAQL expressions in Mistral DSL:
[1] must handle any YAQL, not only the ones started with $; [2] must
preserve types and [3] must comply with YAML. We fixed these problems by
applying Ansible style syntax, requiring quotes around delimiters (e.g.
“{1+$.my.yaql.var}”). However, it lead to unbearable confusion in DSL
readability, in regards to types:
publish:
intvalue1: {1+1}” # Confusing: you expect quotes to be string.
intvalue2: {int(1+1)}” # Even this doestn’ clean the confusion
whatisthis:{$.x + $.y}” # What type would this return?
We got a very strong push back from users in the filed on this syntax.
The crux of the problem is using { } as delimiters YAML. It is plain wrong
to use the reserved character. The clean solution is to find a delimiter
that won’t conflict with YAML.
Criteria for selecting best alternative are:
1) Consistently applies to to all cases of using YAML in DSL
2) Complies with YAML
3) Familiar to target user audience - openstack and devops
We prefer using two-char delimiters to avoid requiring extra escaping
within the expressions.
The current winner is % %. It fits YAML well. It is familiar to
openstack/devops as this is used for embedding Ruby expressions in Puppet
and Chef (for instance, [4]). It plays relatively well across all cases of
using expressions in Mistral (see examples in [5]):
ALTERNATIVES considered:
--
1) Use Ansible-like syntax: http://docs.ansible.com/YAMLSyntax.html#gotchas
http://docs.ansible.com/YAMLSyntax.html#gotchas
Rejected for confusion around types. See above.
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a concept of typed variables
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Some suggestions from me:
1. y 1 + $.var # (short from yaql).
2. { 1 + $.var } # as for me, looks more elegant than % %. And
visually it is more strong
A also like p7 and p8 suggested by Renat.
On Tue, Feb 17, 2015 at 11:43 AM, Renat Akhmerov rakhme...@mirantis.com
wrote:
One more:
p9: \{1 + $.var} # That’s pretty much what
https://review.openstack.org/#/c/155348/ addresses but it’s not exactly
that. Note that we don’t have to put it in quotes in this case to deal with
YAML {} semantics, it’s just a string
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 13:37, Renat Akhmerov rakhme...@mirantis.com wrote:
Along with % % syntax here are some other alternatives that I checked
for YAML friendliness with my short comments:
p1: ${1 + $.var} # Here it’s bad that $ sign is used for two
different things
p2: ~{1 + $.var} # ~ is easy to miss in a text
p3: ^{1 + $.var} # For someone may be associated with regular
expressions
p4: ?{1 + $.var}
p5: {1 + $.var} # This is kinda crazy
p6: e{1 + $.var} # That looks a pretty interesting option to me, “e”
could mean “expression” here.
p7: yaql{1 + $.var} # This is interesting because it would give a clear
and easy mechanism to plug in other expression languages, “yaql” here is a
used dialect for the following expression
p8: y{1 + $.var} # “y” here is just shortened “yaql
Any ideas and thoughts would be really appreciated!
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 12:53, Renat Akhmerov rakhme...@mirantis.com wrote:
Dmitri,
I agree with all your reasonings and fully support the idea of changing
the syntax now as well as changing system’s API a little bit due to
recently found issues in the current engine design that don’t allow us, for
example, to fully implement ‘with-items’ (although that’s a little bit
different story).
Just a general note about all changes happening now: *Once we release
kilo stable release our API, DSL of version 2 must be 100% stable*. I was
hoping to stabilize it much earlier but the start of production use
revealed a number of things (I think this is normal) which we need to
address, but not later than the end of Kilo.
As far as % % syntax. I see that it would solve a number of problems
(YAML friendliness, type ambiguity) but my only not strong argument is that
it doesn’t look that elegant in YAML as it looks, for example, in ERB
templates. It really reminds me XML/HTML and looks like a bear in a grocery
store (tried to make it close to one old russian saying :) ). So just for
this only reason I’d suggest we think about other alternatives, maybe not
so familiar to Ruby/Chef/Puppet users but looking better with YAML and at
the same time being YAML friendly.
I would be good if we could here more feedback on this, especially from
people who started using Mistral.
Thanks
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 03:06, Dmitri Zimine dzim...@stackstorm.com wrote:
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML
from {1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments
and suggestions welcome.
DETAILS:
-
We faced a number of problems with using YAQL expressions in Mistral DSL:
[1] must handle any YAQL, not only the ones started with $; [2] must
preserve types and [3] must comply with YAML. We fixed these problems by
applying Ansible style syntax, requiring quotes around delimiters (e.g.
“{1+$.my.yaql.var}”). However, it lead to unbearable confusion in DSL
readability, in regards to types:
publish:
intvalue1: {1+1}” # Confusing: you expect quotes to be string.
intvalue2: {int(1+1)}” # Even this doestn’ clean the confusion
whatisthis:{$.x + $.y}” # What type would this return?
We got a very strong push back from users in the filed on this syntax.
The crux of the problem is using { } as delimiters YAML. It is plain wrong
to use the reserved character. The clean solution is to find a delimiter
that won’t conflict with YAML.
Criteria for selecting best alternative are:
1) Consistently applies to to all cases of using YAML in DSL
2) Complies with YAML
3) Familiar to target user audience - openstack and devops
We prefer using two-char delimiters to avoid requiring extra escaping
within the expressions.
The current winner is % %. It fits YAML well. It is familiar to
openstack/devops as this is used for embedding Ruby expressions in Puppet
and Chef (for instance, [4]). It plays relatively well across all cases of
using expressions in Mistral (see examples in [5]):
ALTERNATIVES considered:
--
1) Use Ansible-like syntax:
http://docs.ansible.com/YAMLSyntax.html#gotchas
Rejected for confusion around types. See above.
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Along with % % syntax here are some other alternatives that I checked for
YAML friendliness with my short comments:
p1: ${1 + $.var}# Here it’s bad that $ sign is used for two different
things
p2: ~{1 + $.var}# ~ is easy to miss in a text
p3: ^{1 + $.var}# For someone may be associated with regular expressions
p4: ?{1 + $.var}
p5: {1 + $.var} # This is kinda crazy
p6: e{1 + $.var}# That looks a pretty interesting option to me, “e”
could mean “expression” here.
p7: yaql{1 + $.var} # This is interesting because it would give a clear and
easy mechanism to plug in other expression languages, “yaql” here is a used
dialect for the following expression
p8: y{1 + $.var}# “y” here is just shortened “yaql
Any ideas and thoughts would be really appreciated!
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 12:53, Renat Akhmerov rakhme...@mirantis.com wrote:
Dmitri,
I agree with all your reasonings and fully support the idea of changing the
syntax now as well as changing system’s API a little bit due to recently
found issues in the current engine design that don’t allow us, for example,
to fully implement ‘with-items’ (although that’s a little bit different
story).
Just a general note about all changes happening now: Once we release kilo
stable release our API, DSL of version 2 must be 100% stable. I was hoping to
stabilize it much earlier but the start of production use revealed a number
of things (I think this is normal) which we need to address, but not later
than the end of Kilo.
As far as % % syntax. I see that it would solve a number of problems (YAML
friendliness, type ambiguity) but my only not strong argument is that it
doesn’t look that elegant in YAML as it looks, for example, in ERB templates.
It really reminds me XML/HTML and looks like a bear in a grocery store (tried
to make it close to one old russian saying :) ). So just for this only reason
I’d suggest we think about other alternatives, maybe not so familiar to
Ruby/Chef/Puppet users but looking better with YAML and at the same time
being YAML friendly.
I would be good if we could here more feedback on this, especially from
people who started using Mistral.
Thanks
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 03:06, Dmitri Zimine dzim...@stackstorm.com
mailto:dzim...@stackstorm.com wrote:
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML
from {1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments and
suggestions welcome.
DETAILS:
-
We faced a number of problems with using YAQL expressions in Mistral DSL:
[1] must handle any YAQL, not only the ones started with $; [2] must
preserve types and [3] must comply with YAML. We fixed these problems by
applying Ansible style syntax, requiring quotes around delimiters (e.g.
“{1+$.my.yaql.var}”). However, it lead to unbearable confusion in DSL
readability, in regards to types:
publish:
intvalue1: {1+1}” # Confusing: you expect quotes to be string.
intvalue2: {int(1+1)}” # Even this doestn’ clean the confusion
whatisthis:{$.x + $.y}” # What type would this return?
We got a very strong push back from users in the filed on this syntax.
The crux of the problem is using { } as delimiters YAML. It is plain wrong
to use the reserved character. The clean solution is to find a delimiter
that won’t conflict with YAML.
Criteria for selecting best alternative are:
1) Consistently applies to to all cases of using YAML in DSL
2) Complies with YAML
3) Familiar to target user audience - openstack and devops
We prefer using two-char delimiters to avoid requiring extra escaping within
the expressions.
The current winner is % %. It fits YAML well. It is familiar to
openstack/devops as this is used for embedding Ruby expressions in Puppet
and Chef (for instance, [4]). It plays relatively well across all cases of
using expressions in Mistral (see examples in [5]):
ALTERNATIVES considered:
--
1) Use Ansible-like syntax: http://docs.ansible.com/YAMLSyntax.html#gotchas
http://docs.ansible.com/YAMLSyntax.html#gotchas
Rejected for confusion around types. See above.
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a concept of typed variables to
borrow from (please correct me if I missed it). But they have functions:
function: { function_name: {foo: [parameter1, parameter 2], bar:xxx”}}.
Applied to Mistral, it would look like:
publish:
- bool_var: { yaql: “1+1+$.my.var 100” }
Not bad, but currently rejected as it reads worse than delimiter-based
syntax, especially in simplified one-line action invocation.
3) paired with other symbols: php-styoe ?
[openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML from
{1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments and
suggestions welcome.
DETAILS:
-
We faced a number of problems with using YAQL expressions in Mistral DSL: [1]
must handle any YAQL, not only the ones started with $; [2] must preserve types
and [3] must comply with YAML. We fixed these problems by applying Ansible
style syntax, requiring quotes around delimiters (e.g. “{1+$.my.yaql.var}”).
However, it lead to unbearable confusion in DSL readability, in regards to
types:
publish:
intvalue1: {1+1}” # Confusing: you expect quotes to be string.
intvalue2: {int(1+1)}” # Even this doestn’ clean the confusion
whatisthis:{$.x + $.y}” # What type would this return?
We got a very strong push back from users in the filed on this syntax.
The crux of the problem is using { } as delimiters YAML. It is plain wrong to
use the reserved character. The clean solution is to find a delimiter that
won’t conflict with YAML.
Criteria for selecting best alternative are:
1) Consistently applies to to all cases of using YAML in DSL
2) Complies with YAML
3) Familiar to target user audience - openstack and devops
We prefer using two-char delimiters to avoid requiring extra escaping within
the expressions.
The current winner is % %. It fits YAML well. It is familiar to
openstack/devops as this is used for embedding Ruby expressions in Puppet and
Chef (for instance, [4]). It plays relatively well across all cases of using
expressions in Mistral (see examples in [5]):
ALTERNATIVES considered:
--
1) Use Ansible-like syntax: http://docs.ansible.com/YAMLSyntax.html#gotchas
Rejected for confusion around types. See above.
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a concept of typed variables to
borrow from (please correct me if I missed it). But they have functions:
function: { function_name: {foo: [parameter1, parameter 2], bar:xxx”}}.
Applied to Mistral, it would look like:
publish:
- bool_var: { yaql: “1+1+$.my.var 100” }
Not bad, but currently rejected as it reads worse than delimiter-based syntax,
especially in simplified one-line action invocation.
3) paired with other symbols: php-styoe ? ..?
REFERENCES:
--
[1] Allow arbitrary YAQL expressions, not just ones started with $ :
https://github.com/stackforge/mistral/commit/5c10fb4b773cd60d81ed93aec33345c0bf8f58fd
[2] Use Ansible-like syntax to make YAQL expressions YAML complient
https://github.com/stackforge/mistral/commit/d9517333b1fc9697d4847df33d3b774f881a111b
[3] Preserving types in YAQL
https://github.com/stackforge/mistral/blob/d9517333b1fc9697d4847df33d3b774f881a111b/mistral/tests/unit/test_expressions.py#L152-L184
[4]Using % % in Puppet
https://docs.puppetlabs.com/guides/templating.html#erb-is-plain-text-with-embedded-ruby
[5] Etherpad with discussion
https://etherpad.openstack.org/p/mistral-YAQL-delimiters
[6] Blueprint https://blueprints.launchpad.net/mistral/+spec/yaql-delimiters
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Mistral] Changing expression delimiters in Mistral DSL
Dmitri,
I agree with all your reasonings and fully support the idea of changing the
syntax now as well as changing system’s API a little bit due to recently found
issues in the current engine design that don’t allow us, for example, to fully
implement ‘with-items’ (although that’s a little bit different story).
Just a general note about all changes happening now: Once we release kilo
stable release our API, DSL of version 2 must be 100% stable. I was hoping to
stabilize it much earlier but the start of production use revealed a number of
things (I think this is normal) which we need to address, but not later than
the end of Kilo.
As far as % % syntax. I see that it would solve a number of problems (YAML
friendliness, type ambiguity) but my only not strong argument is that it
doesn’t look that elegant in YAML as it looks, for example, in ERB templates.
It really reminds me XML/HTML and looks like a bear in a grocery store (tried
to make it close to one old russian saying :) ). So just for this only reason
I’d suggest we think about other alternatives, maybe not so familiar to
Ruby/Chef/Puppet users but looking better with YAML and at the same time being
YAML friendly.
I would be good if we could here more feedback on this, especially from people
who started using Mistral.
Thanks
Renat Akhmerov
@ Mirantis Inc.
On 17 Feb 2015, at 03:06, Dmitri Zimine dzim...@stackstorm.com wrote:
SUMMARY:
We are changing the syntax for inlining YAQL expressions in Mistral YAML from
{1+$.my.var} (or “{1+$.my.var}”) to % 1+$.my.var %
Below I explain the rationale and the criteria for the choice. Comments and
suggestions welcome.
DETAILS:
-
We faced a number of problems with using YAQL expressions in Mistral DSL: [1]
must handle any YAQL, not only the ones started with $; [2] must preserve
types and [3] must comply with YAML. We fixed these problems by applying
Ansible style syntax, requiring quotes around delimiters (e.g.
“{1+$.my.yaql.var}”). However, it lead to unbearable confusion in DSL
readability, in regards to types:
publish:
intvalue1: {1+1}” # Confusing: you expect quotes to be string.
intvalue2: {int(1+1)}” # Even this doestn’ clean the confusion
whatisthis:{$.x + $.y}” # What type would this return?
We got a very strong push back from users in the filed on this syntax.
The crux of the problem is using { } as delimiters YAML. It is plain wrong to
use the reserved character. The clean solution is to find a delimiter that
won’t conflict with YAML.
Criteria for selecting best alternative are:
1) Consistently applies to to all cases of using YAML in DSL
2) Complies with YAML
3) Familiar to target user audience - openstack and devops
We prefer using two-char delimiters to avoid requiring extra escaping within
the expressions.
The current winner is % %. It fits YAML well. It is familiar to
openstack/devops as this is used for embedding Ruby expressions in Puppet and
Chef (for instance, [4]). It plays relatively well across all cases of using
expressions in Mistral (see examples in [5]):
ALTERNATIVES considered:
--
1) Use Ansible-like syntax: http://docs.ansible.com/YAMLSyntax.html#gotchas
http://docs.ansible.com/YAMLSyntax.html#gotchas
Rejected for confusion around types. See above.
2) Use functions, like Heat HOT or TOSCA:
HOT templates and TOSCA doesn’t seem to have a concept of typed variables to
borrow from (please correct me if I missed it). But they have functions:
function: { function_name: {foo: [parameter1, parameter 2], bar:xxx”}}.
Applied to Mistral, it would look like:
publish:
- bool_var: { yaql: “1+1+$.my.var 100” }
Not bad, but currently rejected as it reads worse than delimiter-based
syntax, especially in simplified one-line action invocation.
3) paired with other symbols: php-styoe ? ..?
REFERENCES:
--
[1] Allow arbitrary YAQL expressions, not just ones started with $ :
https://github.com/stackforge/mistral/commit/5c10fb4b773cd60d81ed93aec33345c0bf8f58fd
https://github.com/stackforge/mistral/commit/5c10fb4b773cd60d81ed93aec33345c0bf8f58fd
[2] Use Ansible-like syntax to make YAQL expressions YAML complient
https://github.com/stackforge/mistral/commit/d9517333b1fc9697d4847df33d3b774f881a111b
https://github.com/stackforge/mistral/commit/d9517333b1fc9697d4847df33d3b774f881a111b
[3] Preserving types in YAQL
https://github.com/stackforge/mistral/blob/d9517333b1fc9697d4847df33d3b774f881a111b/mistral/tests/unit/test_expressions.py#L152-L184
https://github.com/stackforge/mistral/blob/d9517333b1fc9697d4847df33d3b774f881a111b/mistral/tests/unit/test_expressions.py#L152-L184
[4]Using % % in Puppet
https://docs.puppetlabs.com/guides/templating.html#erb-is-plain-text-with-embedded-ruby
