Issue #11246 has been reported by Jeff McCune.
----------------------------------------
Bug #11246: Puppet should work with UTF-8 encoded ERB templates
https://projects.puppetlabs.com/issues/11246
Author: Jeff McCune
Status: Accepted
Priority: High
Assignee: Jeff McCune
Category: ruby19
Target version: 2.7.x
Affected Puppet version: 2.7.6
Keywords: support535 support utf unicode encoding internationalization template
zaml yaml report
Branch:
# Overview #
A support customer reported this issue with Puppet 2.7.6 and Ruby 1.9.2-p290.
When using an ERB template with the template() function Puppet throws an
exception. Reproducing this error I've determined the issue comes from writing
out the YAML report.
# Steps to reproduce #
Using this simple class in a module the error is triggered when producing the
serialized report:
support535/manifests/utf8template.pp
<pre>
# Class: support535::utf8template
#
# This class uses the template() function with a UTF-8 encoded ERB file. The
# manifest itself does not actually contain UTF8 encoded strings.
#
# Parameters:
#
# Actions:
#
# Requires:
#
# Sample Usage:
#
class support535::utf8template {
notify { "utf8template":
message => template("support535/snowman.erb"),
}
}
# EOF
</pre>
support535/templates/snowman.erb
<pre>
This is a simple text file with a snowman.
The snowman is here: (☃) EOL
EOF
</pre>
# Actual Behavior #
With reporting turned on:
<pre>
% puppet apply -v --debug --trace -e 'include support535::utf8template'
info: Loading facts in facter_dot_d
info: Loading facts in root_home
debug: importing '/vagrant/modules/support535/manifests/init.pp' in environment
production
debug: importing '/vagrant/modules/support535/manifests/utf8template.pp' in
environment production
debug: Automatically imported support535::utf8template from
support535/utf8template into production
debug: Scope(Class[Support535::Utf8template]): Retrieving template
support535/snowman.erb
debug: template[/vagrant/modules/support535/templates/snowman.erb]: Bound
template variables for /vagrant/modules/support535/templates/snowman.erb in
0.00 seconds
debug: template[/vagrant/modules/support535/templates/snowman.erb]:
Interpolated template /vagrant/modules/support535/templates/snowman.erb in 0.00
seconds
debug: Creating default schedules
debug: Failed to load library 'selinux' for feature 'selinux'
debug: Failed to load library 'shadow' for feature 'libshadow'
debug: Failed to load library 'ldap' for feature 'ldap'
debug: /File[/Users/jeff/.puppet/var]: Autorequiring File[/Users/jeff/.puppet]
debug: /File[/Users/jeff/.puppet/var/log]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/state]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/run]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/lib]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/ssl/certs]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/ssl/public_keys]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl/certificate_requests]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl/private_keys]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl/private]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl/certs/ca.pem]: Autorequiring
File[/Users/jeff/.puppet/var/ssl/certs]
debug: /File[/Users/jeff/.puppet/var/ssl/crl.pem]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/state/state.yaml]: Autorequiring
File[/Users/jeff/.puppet/var/state]
debug: /File[/Users/jeff/.puppet/var/client_yaml]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/client_data]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/clientbucket]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/state/last_run_summary.yaml]:
Autorequiring File[/Users/jeff/.puppet/var/state]
debug: /File[/Users/jeff/.puppet/var/state/last_run_report.yaml]: Autorequiring
File[/Users/jeff/.puppet/var/state]
debug: /File[/Users/jeff/.puppet/var/state/graphs]: Autorequiring
File[/Users/jeff/.puppet/var/state]
debug: /File[/Users/jeff/.puppet/var/facts]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: Finishing transaction 70320110466340
debug: Loaded state in 0.00 seconds
debug: Loaded state in 0.00 seconds
info: Applying configuration version '1323223267'
debug: /Schedule[daily]: Skipping device resources because running on a host
debug: /Schedule[monthly]: Skipping device resources because running on a host
debug: /Schedule[hourly]: Skipping device resources because running on a host
notice: This is a simple text file with a snowman.
The snowman is here: (☃) EOL
EOF
notice: /Stage[main]/Support535::Utf8template/Notify[utf8template]/message:
defined 'message' as 'This is a simple text file with a snowman.
The snowman is here: (☃) EOL
EOF
'
debug: /Stage[main]/Support535::Utf8template/Notify[utf8template]: The
container Class[Support535::Utf8template] will propagate my refresh event
debug: Class[Support535::Utf8template]: The container Stage[main] will
propagate my refresh event
debug: /Schedule[never]: Skipping device resources because running on a host
debug: /Schedule[weekly]: Skipping device resources because running on a host
debug: /Schedule[puppet]: Skipping device resources because running on a host
debug: Finishing transaction 70320141681140
debug: Storing state
debug: Stored state in 0.04 seconds
notice: Finished catalog run in 0.06 seconds
debug: Finishing transaction 70320143015880
debug: Received report to process from maynard.puppetlabs.lan
debug: Processing report from maynard.puppetlabs.lan with processor
Puppet::Reports::Store
/vagrant/src/puppet/lib/puppet/util/zaml.rb:221:in `gsub'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:221:in `escaped_for_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:249:in `block in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:229:in `to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:142:in `block (3 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `each'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `block (2 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:133:in `block in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:131:in `to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `block (3 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `each'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `block (2 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:282:in `block in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:281:in `to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:142:in `block (3 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `each'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `block (2 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:133:in `block in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:131:in `to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:22:in `dump'
/vagrant/src/puppet/lib/puppet/util/monkey_patches.rb:32:in `to_yaml'
/vagrant/src/puppet/lib/puppet/reports/store.rb:34:in `block in process'
/vagrant/src/puppet/lib/puppet/reports/store.rb:33:in `open'
/vagrant/src/puppet/lib/puppet/reports/store.rb:33:in `process'
/vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:37:in `block in
process'
/vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:55:in `block in
processors'
/vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:53:in `each'
/vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:53:in `processors'
/vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:30:in `process'
/vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:14:in `save'
/vagrant/src/puppet/lib/puppet/indirector/indirection.rb:265:in `save'
/vagrant/src/puppet/lib/puppet/configurer.rb:174:in `send_report'
/vagrant/src/puppet/lib/puppet/configurer.rb:168:in `run'
/vagrant/src/puppet/lib/puppet/application/apply.rb:215:in `main'
/vagrant/src/puppet/lib/puppet/application/apply.rb:135:in `run_command'
/vagrant/src/puppet/lib/puppet/application.rb:306:in `block (2 levels) in run'
/vagrant/src/puppet/lib/puppet/application.rb:410:in `hook'
/vagrant/src/puppet/lib/puppet/application.rb:306:in `block in run'
/vagrant/src/puppet/lib/puppet/application.rb:401:in `exit_on_fail'
/vagrant/src/puppet/lib/puppet/application.rb:306:in `run'
/vagrant/src/puppet/lib/puppet/util/command_line.rb:69:in `execute'
/vagrant/src/puppet/bin/puppet:4:in `<main>'
warning: Could not write report for maynard.puppetlabs.lan at
/Users/jeff/.puppet/var/reports/maynard.puppetlabs.lan/201112070201.yaml:
incompatible encoding regexp match (ASCII-8BIT regexp with UTF-8 string)
/vagrant/src/puppet/lib/puppet/util/zaml.rb:221:in `gsub'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:221:in `escaped_for_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:249:in `block in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:229:in `to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:142:in `block (3 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `each'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `block (2 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:133:in `block in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:131:in `to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `block (3 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `each'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `block (2 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:282:in `block in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:281:in `to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:142:in `block (3 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `each'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `block (2 levels) in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:133:in `block in to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:131:in `to_zaml'
/vagrant/src/puppet/lib/puppet/util/zaml.rb:22:in `dump'
/vagrant/src/puppet/lib/puppet/util/monkey_patches.rb:38:in `dump'
/vagrant/src/puppet/lib/puppet/indirector/yaml.rb:73:in `to_yaml'
/vagrant/src/puppet/lib/puppet/indirector/yaml.rb:38:in `block in save'
/vagrant/src/puppet/lib/puppet/util/file_locking.rb:42:in `block (3 levels) in
writelock'
/vagrant/src/puppet/lib/puppet/external/lock.rb:34:in `lock_exclusive'
/vagrant/src/puppet/lib/puppet/util/file_locking.rb:38:in `block (2 levels) in
writelock'
/vagrant/src/puppet/lib/puppet/util/file_locking.rb:37:in `open'
/vagrant/src/puppet/lib/puppet/util/file_locking.rb:37:in `block in writelock'
/vagrant/src/puppet/lib/puppet/util.rb:41:in `block in synchronize_on'
/Users/jeff/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/sync.rb:225:in
`sync_synchronize'
/vagrant/src/puppet/lib/puppet/util.rb:41:in `synchronize_on'
/vagrant/src/puppet/lib/puppet/util/file_locking.rb:36:in `writelock'
/vagrant/src/puppet/lib/puppet/indirector/yaml.rb:38:in `save'
/vagrant/src/puppet/lib/puppet/indirector/indirection.rb:268:in `save'
/vagrant/src/puppet/lib/puppet/configurer.rb:174:in `send_report'
/vagrant/src/puppet/lib/puppet/configurer.rb:168:in `run'
/vagrant/src/puppet/lib/puppet/application/apply.rb:215:in `main'
/vagrant/src/puppet/lib/puppet/application/apply.rb:135:in `run_command'
/vagrant/src/puppet/lib/puppet/application.rb:306:in `block (2 levels) in run'
/vagrant/src/puppet/lib/puppet/application.rb:410:in `hook'
/vagrant/src/puppet/lib/puppet/application.rb:306:in `block in run'
/vagrant/src/puppet/lib/puppet/application.rb:401:in `exit_on_fail'
/vagrant/src/puppet/lib/puppet/application.rb:306:in `run'
/vagrant/src/puppet/lib/puppet/util/command_line.rb:69:in `execute'
/vagrant/src/puppet/bin/puppet:4:in `<main>'
err: Could not send report: incompatible encoding regexp match (ASCII-8BIT
regexp with UTF-8 string)
</pre>
# Work Around #
Turning reporting off prevents the error since the serialization to YAML
doesn't occur.
<pre>
% puppet apply -v --debug --trace --no-report -e 'include
support535::utf8template'
info: Loading facts in facter_dot_d
info: Loading facts in root_home
debug: importing '/vagrant/modules/support535/manifests/init.pp' in environment
production
debug: importing '/vagrant/modules/support535/manifests/utf8template.pp' in
environment production
debug: Automatically imported support535::utf8template from
support535/utf8template into production
debug: Scope(Class[Support535::Utf8template]): Retrieving template
support535/snowman.erb
debug: template[/vagrant/modules/support535/templates/snowman.erb]: Bound
template variables for /vagrant/modules/support535/templates/snowman.erb in
0.00 seconds
debug: template[/vagrant/modules/support535/templates/snowman.erb]:
Interpolated template /vagrant/modules/support535/templates/snowman.erb in 0.00
seconds
debug: Creating default schedules
debug: Failed to load library 'selinux' for feature 'selinux'
debug: Failed to load library 'shadow' for feature 'libshadow'
debug: Failed to load library 'ldap' for feature 'ldap'
debug: /File[/Users/jeff/.puppet/var]: Autorequiring File[/Users/jeff/.puppet]
debug: /File[/Users/jeff/.puppet/var/log]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/state]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/run]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/lib]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/ssl/certs]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/ssl/public_keys]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl/certificate_requests]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl/private_keys]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl/private]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/ssl/certs/ca.pem]: Autorequiring
File[/Users/jeff/.puppet/var/ssl/certs]
debug: /File[/Users/jeff/.puppet/var/ssl/crl.pem]: Autorequiring
File[/Users/jeff/.puppet/var/ssl]
debug: /File[/Users/jeff/.puppet/var/state/state.yaml]: Autorequiring
File[/Users/jeff/.puppet/var/state]
debug: /File[/Users/jeff/.puppet/var/client_yaml]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/client_data]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/clientbucket]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: /File[/Users/jeff/.puppet/var/state/last_run_summary.yaml]:
Autorequiring File[/Users/jeff/.puppet/var/state]
debug: /File[/Users/jeff/.puppet/var/state/last_run_report.yaml]: Autorequiring
File[/Users/jeff/.puppet/var/state]
debug: /File[/Users/jeff/.puppet/var/state/graphs]: Autorequiring
File[/Users/jeff/.puppet/var/state]
debug: /File[/Users/jeff/.puppet/var/facts]: Autorequiring
File[/Users/jeff/.puppet/var]
debug: Finishing transaction 70250873848000
debug: Loaded state in 0.00 seconds
debug: Loaded state in 0.00 seconds
info: Applying configuration version '1323223324'
debug: /Schedule[daily]: Skipping device resources because running on a host
debug: /Schedule[monthly]: Skipping device resources because running on a host
debug: /Schedule[hourly]: Skipping device resources because running on a host
notice: This is a simple text file with a snowman.
The snowman is here: (☃) EOL
EOF
notice: /Stage[main]/Support535::Utf8template/Notify[utf8template]/message:
defined 'message' as 'This is a simple text file with a snowman.
The snowman is here: (☃) EOL
EOF
'
debug: /Stage[main]/Support535::Utf8template/Notify[utf8template]: The
container Class[Support535::Utf8template] will propagate my refresh event
debug: Class[Support535::Utf8template]: The container Stage[main] will
propagate my refresh event
debug: /Schedule[never]: Skipping device resources because running on a host
debug: /Schedule[weekly]: Skipping device resources because running on a host
debug: /Schedule[puppet]: Skipping device resources because running on a host
debug: Finishing transaction 70250870270220
debug: Storing state
debug: Stored state in 0.03 seconds
notice: Finished catalog run in 0.05 seconds
</pre>
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://projects.puppetlabs.com/my/account
--
You received this message because you are subscribed to the Google Groups
"Puppet Bugs" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/puppet-bugs?hl=en.
