Jira (HI-482) String literals should literally be literal
Title: Message Title
Julian Meier commented on HI-482
Re: String literals should literally be literal
Just a note: I did just run into the same issue. I try to use following string in hieradata:
"secure/services/%{literal('%')}{::service}/%{literal('%')}{::stack}/%{literal('%')}{::role}"
The issue for this is HI-469.
Add Comment
This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)
--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title Lee Lowder commented on HI-482 Re: String literals should literally be literal I really feel that closing this is the wrong thing to do, especially since the reason seems to be "this is hard, so no." Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title Kyle Harper commented on HI-482 Re: String literals should literally be literal I understand that YAML and JSON don't treat double or single quotes differently; it's all up to hiera as to how the backend data is being interpreted. It's hiera that chose to look at single-quoted strings as interpretative. A common practice with Puppet features that break current conventions is: create a config option, leave the option default set to the current behavior, and monitor feedback and adoption rates. If the argument was that it would be very time consuming and you don't have resources for it, that's one thing. Otherwise, with all due respect, I feel this should get consideration from more than a single person before being shut down because of potential "incompatibilities" that can be completely avoided with a new option in hiera.yaml. A string-literal form is a very basic tenant of interpretive systems, including Puppet DSL. Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title
R.I.Pienaar commented on HI-482
Re: String literals should literally be literal
"it's all up to hiera as to how the backend data is being interpreted"
It's not really, hiera does YAML.parse("foo.yaml") and gets already parsed data back. It has literally no idea and no possibility of an idea without writing its own YAML parser if the string was single/double quoted. And that yaml parser would then not be portable meaning your data files become hiera data files and not yaml.
Add Comment
This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)
--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title
Ben Ford commented on HI-482
Re: String literals should literally be literal
Would double quoting be an acceptable compromise?
---
message: "'this string gets returned exactly, including the single quotes around it.'"
irb(main):001:0> require 'yaml'
=> true
irb(main):002:0> p YAML.load(File.read('foo.yaml'))
{"message"=>"'this string gets returned exactly, including the single quotes around it.'"}
I'm not sure that's a reasonable solution either, because the ' and the " blend into one another so easily. But it could work.
Add Comment
Jira (HI-482) String literals should literally be literal
Title: Message Title
Kyle Harper commented on HI-482
Re: String literals should literally be literal
R.I.Pienaar Thanks for the demonstration.
Henrik Lindberg I see what you're saying now, and why hiera can't do what I was originally thinking.
It might be worth linking this issue on certain documentation pages which discuss Hiera. There's already a note that values must be quoted, and I know some recent updates helped cover the RewriteRule issue by using % {literal('%')}
, but it would help edify others to note:
1. Hiera doesn't parse YAML or JSON, ruby does. Therefore Hiera is subject to whatever ruby returns. (Show the example above from R.I.Pienaar). 2. Hiera's interpretations purposely remain simple, only interpreting %{} blocks.
Anyway, thanks for all the feedback.
Add Comment
This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)
--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title Henrik Lindberg commented on HI-482 Re: String literals should literally be literal I should have given this some more thought before closing. I was so focused on the notion of double/single quoted string in the data format that I did not think about how to actually solve the problem. In fact, since we just introduced lookup_options in puppet's lookup function, and for automatic data binding. The lookup_options could be expanded to allow mapping if a string should be verbatim or not. That would also work with existing providers. See ticket(s) for lookup_options. If you like that idea, we can change this ticket into providing support for a "verbatim" option. (Ping R.I.Pienaar Kyle Harper). Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title Henrik Lindberg commented on HI-482 Re: String literals should literally be literal sure - it should be able to have the entire string as an argument to `literal`. I know there are other problems in hiera with rules that have problems seeing multiple '%'. The new implementation in "data in modules" does not have that problem (afaik). That way you can have both verbatim/literal and interpolation in the same string. Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title
R.I.Pienaar commented on HI-482
Re: String literals should literally be literal
Henrik Lindberg that's an interesting option but I do not know if it's really something you want to set on the key?
One common use case for the literal() function I see from people is that they want to for example set some piece of data that usually is fine but in one case there's a %{} in the particular piece of data they want to avoid parsing - sometimes even just on one specific piece of data rather than all data for a key.
Not saying it wouldnt be a good option - just not sure if it will address the common problem I've seen, perhaps others can throw in some more use cases they'd see for this ability?
Add Comment
This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)
--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at http://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title
R.I.Pienaar commented on HI-482
Re: String literals should literally be literal
A small example:
>> p YAML.load("foo: 'bar'");p YAML.load('foo: "bar"')
{"foo"=>"bar"}
{"foo"=>"bar"}
In both cases - single or double quotes - hiera just get a ruby data structure back. It has no insight into where that data come from or how the data file looked. Data files are YAML standard compliant, this is just how it is.
Add Comment
This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc)
--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com.
To post to this group, send email to puppet-bugs@googlegroups.com.
Visit this group at
Jira (HI-482) String literals should literally be literal
Title: Message Title Henrik Lindberg commented on HI-482 Re: String literals should literally be literal I know for a fact that it is impossible to differentiate one kind of string from another. So it is much harder than just "hard" since it is neither hiera nor puppet that is transforming the YAML or JSON into objects. When hiera gets hold of them they are just strings. What is left is only syntax in the actual strings. Hiera currently squats on everything in terms of syntax and data types - so yeah, this is impossible without causing incompatible breakage. We would need to create a custom YAML or JSON parser in order to treat single or double quoted string differently. Alternatively encode the data differently by adding a top data file scope that is for instructions instead of keys (to allow some keys to be uninterpreted). The later can be achieved by implementing a custom backend - say by having data under they keys "verbatim:" and "processed" (or somesuch). Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title Henrik Lindberg commented on HI-482 Re: String literals should literally be literal There are no difference between single quoted and double quoted strings in YAML and JSON. There is no simple solution to this problem since the "programming language" here is a data format and any changes to hiera interpretation of data is sensitive and will have incompatibilities that would silently just deliver different data to users. I am therefore closing this as a "won't fix". Add Comment This message was sent by Atlassian JIRA (v6.4.12#64027-sha1:e3691cc) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.
Jira (HI-482) String literals should literally be literal
Title: Message Title
Kyle Harper created an issue
Hiera / HI-482
String literals should literally be literal
Issue Type:
Improvement
Assignee:
Unassigned
Created:
2015/11/23 7:21 PM
Priority:
Normal
Reporter:
Kyle Harper
Hiera treats any quoted string as a candidate for interpolation. This is in contrast to most languages (including Puppet DSL) wherein single-quotes mean a string should be taken literally.
With $::environment == production my_key: "My environment is %{::environment}" ==> My environment is production my_key: 'My environment is %{::environment}' ==> My environment is production
The latter should evaluate to: ==> My environment is %{::environment}
The limited interference of Hiera on the YAML back-end data has prevented this lack of behavior from being a bigger issue; but as Hiera grows over time it's likely this will help users avoid other problems with unintended expressions in data.
A current example problem that has been discussed in other tickets (
HI-127
) was the Apache rewrite rule/cond string. A few unsupported solutions were presented, but the final official response was to use a function: my_key: "RewriteCond % {literal('%')} {REMOTE_HOST} =host1" ==> RewriteCond %{REMOTE_HOST}
=host1
While it works, it seems significantly more complex than is necessary; it also has the expense of an entire function call just to place a percent sign. (Not to mentioned we're further mixing code inside data, but that's another issue entirely...).
Thanks for your consideration.
