Issue #11246 has been updated by Jeff McCune. Branch set to https://github.com/jeffmccune/puppet/tree/ticket/2.7.x/535_utf8_templates_fail
# Topic Branch # I have a fix that appears to solve this issue in the topic branch `ticket/2.7.x/535_utf8_templates_fail` in my fork of Puppet. I need to write some tests and it should be ready for a pull request. Branch: <https://github.com/jeffmccune/puppet/tree/ticket/2.7.x/535_utf8_templates_fail> -Jeff ---------------------------------------- Bug #11246: Puppet should work with UTF-8 encoded ERB templates https://projects.puppetlabs.com/issues/11246 Author: Jeff McCune Status: Accepted Priority: High Assignee: Jeff McCune Category: ruby19 Target version: 2.7.x Affected Puppet version: 2.7.6 Keywords: support535 support utf unicode encoding internationalization template zaml yaml report Branch: https://github.com/jeffmccune/puppet/tree/ticket/2.7.x/535_utf8_templates_fail # Overview # A support customer reported this issue with Puppet 2.7.6 and Ruby 1.9.2-p290. When using an ERB template with the template() function Puppet throws an exception. Reproducing this error I've determined the issue comes from writing out the YAML report. # Steps to reproduce # Using this simple class in a module the error is triggered when producing the serialized report: support535/manifests/utf8template.pp <pre> # Class: support535::utf8template # # This class uses the template() function with a UTF-8 encoded ERB file. The # manifest itself does not actually contain UTF8 encoded strings. # # Parameters: # # Actions: # # Requires: # # Sample Usage: # class support535::utf8template { notify { "utf8template": message => template("support535/snowman.erb"), } } # EOF </pre> support535/templates/snowman.erb <pre> This is a simple text file with a snowman. The snowman is here: (☃) EOL EOF </pre> # Actual Behavior # With reporting turned on: <pre> % puppet apply -v --debug --trace -e 'include support535::utf8template' info: Loading facts in facter_dot_d info: Loading facts in root_home debug: importing '/vagrant/modules/support535/manifests/init.pp' in environment production debug: importing '/vagrant/modules/support535/manifests/utf8template.pp' in environment production debug: Automatically imported support535::utf8template from support535/utf8template into production debug: Scope(Class[Support535::Utf8template]): Retrieving template support535/snowman.erb debug: template[/vagrant/modules/support535/templates/snowman.erb]: Bound template variables for /vagrant/modules/support535/templates/snowman.erb in 0.00 seconds debug: template[/vagrant/modules/support535/templates/snowman.erb]: Interpolated template /vagrant/modules/support535/templates/snowman.erb in 0.00 seconds debug: Creating default schedules debug: Failed to load library 'selinux' for feature 'selinux' debug: Failed to load library 'shadow' for feature 'libshadow' debug: Failed to load library 'ldap' for feature 'ldap' debug: /File[/Users/jeff/.puppet/var]: Autorequiring File[/Users/jeff/.puppet] debug: /File[/Users/jeff/.puppet/var/log]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/state]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/run]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/lib]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/ssl/certs]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/ssl/public_keys]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl/certificate_requests]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl/private_keys]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl/private]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl/certs/ca.pem]: Autorequiring File[/Users/jeff/.puppet/var/ssl/certs] debug: /File[/Users/jeff/.puppet/var/ssl/crl.pem]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/state/state.yaml]: Autorequiring File[/Users/jeff/.puppet/var/state] debug: /File[/Users/jeff/.puppet/var/client_yaml]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/client_data]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/clientbucket]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/state/last_run_summary.yaml]: Autorequiring File[/Users/jeff/.puppet/var/state] debug: /File[/Users/jeff/.puppet/var/state/last_run_report.yaml]: Autorequiring File[/Users/jeff/.puppet/var/state] debug: /File[/Users/jeff/.puppet/var/state/graphs]: Autorequiring File[/Users/jeff/.puppet/var/state] debug: /File[/Users/jeff/.puppet/var/facts]: Autorequiring File[/Users/jeff/.puppet/var] debug: Finishing transaction 70320110466340 debug: Loaded state in 0.00 seconds debug: Loaded state in 0.00 seconds info: Applying configuration version '1323223267' debug: /Schedule[daily]: Skipping device resources because running on a host debug: /Schedule[monthly]: Skipping device resources because running on a host debug: /Schedule[hourly]: Skipping device resources because running on a host notice: This is a simple text file with a snowman. The snowman is here: (☃) EOL EOF notice: /Stage[main]/Support535::Utf8template/Notify[utf8template]/message: defined 'message' as 'This is a simple text file with a snowman. The snowman is here: (☃) EOL EOF ' debug: /Stage[main]/Support535::Utf8template/Notify[utf8template]: The container Class[Support535::Utf8template] will propagate my refresh event debug: Class[Support535::Utf8template]: The container Stage[main] will propagate my refresh event debug: /Schedule[never]: Skipping device resources because running on a host debug: /Schedule[weekly]: Skipping device resources because running on a host debug: /Schedule[puppet]: Skipping device resources because running on a host debug: Finishing transaction 70320141681140 debug: Storing state debug: Stored state in 0.04 seconds notice: Finished catalog run in 0.06 seconds debug: Finishing transaction 70320143015880 debug: Received report to process from maynard.puppetlabs.lan debug: Processing report from maynard.puppetlabs.lan with processor Puppet::Reports::Store /vagrant/src/puppet/lib/puppet/util/zaml.rb:221:in `gsub' /vagrant/src/puppet/lib/puppet/util/zaml.rb:221:in `escaped_for_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:249:in `block in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only' /vagrant/src/puppet/lib/puppet/util/zaml.rb:229:in `to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:142:in `block (3 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `each' /vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `block (2 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested' /vagrant/src/puppet/lib/puppet/util/zaml.rb:133:in `block in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only' /vagrant/src/puppet/lib/puppet/util/zaml.rb:131:in `to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `block (3 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `each' /vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `block (2 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested' /vagrant/src/puppet/lib/puppet/util/zaml.rb:282:in `block in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only' /vagrant/src/puppet/lib/puppet/util/zaml.rb:281:in `to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:142:in `block (3 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `each' /vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `block (2 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested' /vagrant/src/puppet/lib/puppet/util/zaml.rb:133:in `block in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only' /vagrant/src/puppet/lib/puppet/util/zaml.rb:131:in `to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:22:in `dump' /vagrant/src/puppet/lib/puppet/util/monkey_patches.rb:32:in `to_yaml' /vagrant/src/puppet/lib/puppet/reports/store.rb:34:in `block in process' /vagrant/src/puppet/lib/puppet/reports/store.rb:33:in `open' /vagrant/src/puppet/lib/puppet/reports/store.rb:33:in `process' /vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:37:in `block in process' /vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:55:in `block in processors' /vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:53:in `each' /vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:53:in `processors' /vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:30:in `process' /vagrant/src/puppet/lib/puppet/indirector/report/processor.rb:14:in `save' /vagrant/src/puppet/lib/puppet/indirector/indirection.rb:265:in `save' /vagrant/src/puppet/lib/puppet/configurer.rb:174:in `send_report' /vagrant/src/puppet/lib/puppet/configurer.rb:168:in `run' /vagrant/src/puppet/lib/puppet/application/apply.rb:215:in `main' /vagrant/src/puppet/lib/puppet/application/apply.rb:135:in `run_command' /vagrant/src/puppet/lib/puppet/application.rb:306:in `block (2 levels) in run' /vagrant/src/puppet/lib/puppet/application.rb:410:in `hook' /vagrant/src/puppet/lib/puppet/application.rb:306:in `block in run' /vagrant/src/puppet/lib/puppet/application.rb:401:in `exit_on_fail' /vagrant/src/puppet/lib/puppet/application.rb:306:in `run' /vagrant/src/puppet/lib/puppet/util/command_line.rb:69:in `execute' /vagrant/src/puppet/bin/puppet:4:in `<main>' warning: Could not write report for maynard.puppetlabs.lan at /Users/jeff/.puppet/var/reports/maynard.puppetlabs.lan/201112070201.yaml: incompatible encoding regexp match (ASCII-8BIT regexp with UTF-8 string) /vagrant/src/puppet/lib/puppet/util/zaml.rb:221:in `gsub' /vagrant/src/puppet/lib/puppet/util/zaml.rb:221:in `escaped_for_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:249:in `block in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only' /vagrant/src/puppet/lib/puppet/util/zaml.rb:229:in `to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:142:in `block (3 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `each' /vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `block (2 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested' /vagrant/src/puppet/lib/puppet/util/zaml.rb:133:in `block in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only' /vagrant/src/puppet/lib/puppet/util/zaml.rb:131:in `to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `block (3 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `each' /vagrant/src/puppet/lib/puppet/util/zaml.rb:286:in `block (2 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested' /vagrant/src/puppet/lib/puppet/util/zaml.rb:282:in `block in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only' /vagrant/src/puppet/lib/puppet/util/zaml.rb:281:in `to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:142:in `block (3 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `each' /vagrant/src/puppet/lib/puppet/util/zaml.rb:138:in `block (2 levels) in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:39:in `nested' /vagrant/src/puppet/lib/puppet/util/zaml.rb:133:in `block in to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:90:in `first_time_only' /vagrant/src/puppet/lib/puppet/util/zaml.rb:131:in `to_zaml' /vagrant/src/puppet/lib/puppet/util/zaml.rb:22:in `dump' /vagrant/src/puppet/lib/puppet/util/monkey_patches.rb:38:in `dump' /vagrant/src/puppet/lib/puppet/indirector/yaml.rb:73:in `to_yaml' /vagrant/src/puppet/lib/puppet/indirector/yaml.rb:38:in `block in save' /vagrant/src/puppet/lib/puppet/util/file_locking.rb:42:in `block (3 levels) in writelock' /vagrant/src/puppet/lib/puppet/external/lock.rb:34:in `lock_exclusive' /vagrant/src/puppet/lib/puppet/util/file_locking.rb:38:in `block (2 levels) in writelock' /vagrant/src/puppet/lib/puppet/util/file_locking.rb:37:in `open' /vagrant/src/puppet/lib/puppet/util/file_locking.rb:37:in `block in writelock' /vagrant/src/puppet/lib/puppet/util.rb:41:in `block in synchronize_on' /Users/jeff/.rvm/rubies/ruby-1.9.2-p290/lib/ruby/1.9.1/sync.rb:225:in `sync_synchronize' /vagrant/src/puppet/lib/puppet/util.rb:41:in `synchronize_on' /vagrant/src/puppet/lib/puppet/util/file_locking.rb:36:in `writelock' /vagrant/src/puppet/lib/puppet/indirector/yaml.rb:38:in `save' /vagrant/src/puppet/lib/puppet/indirector/indirection.rb:268:in `save' /vagrant/src/puppet/lib/puppet/configurer.rb:174:in `send_report' /vagrant/src/puppet/lib/puppet/configurer.rb:168:in `run' /vagrant/src/puppet/lib/puppet/application/apply.rb:215:in `main' /vagrant/src/puppet/lib/puppet/application/apply.rb:135:in `run_command' /vagrant/src/puppet/lib/puppet/application.rb:306:in `block (2 levels) in run' /vagrant/src/puppet/lib/puppet/application.rb:410:in `hook' /vagrant/src/puppet/lib/puppet/application.rb:306:in `block in run' /vagrant/src/puppet/lib/puppet/application.rb:401:in `exit_on_fail' /vagrant/src/puppet/lib/puppet/application.rb:306:in `run' /vagrant/src/puppet/lib/puppet/util/command_line.rb:69:in `execute' /vagrant/src/puppet/bin/puppet:4:in `<main>' err: Could not send report: incompatible encoding regexp match (ASCII-8BIT regexp with UTF-8 string) </pre> # Work Around # Turning reporting off prevents the error since the serialization to YAML doesn't occur. <pre> % puppet apply -v --debug --trace --no-report -e 'include support535::utf8template' info: Loading facts in facter_dot_d info: Loading facts in root_home debug: importing '/vagrant/modules/support535/manifests/init.pp' in environment production debug: importing '/vagrant/modules/support535/manifests/utf8template.pp' in environment production debug: Automatically imported support535::utf8template from support535/utf8template into production debug: Scope(Class[Support535::Utf8template]): Retrieving template support535/snowman.erb debug: template[/vagrant/modules/support535/templates/snowman.erb]: Bound template variables for /vagrant/modules/support535/templates/snowman.erb in 0.00 seconds debug: template[/vagrant/modules/support535/templates/snowman.erb]: Interpolated template /vagrant/modules/support535/templates/snowman.erb in 0.00 seconds debug: Creating default schedules debug: Failed to load library 'selinux' for feature 'selinux' debug: Failed to load library 'shadow' for feature 'libshadow' debug: Failed to load library 'ldap' for feature 'ldap' debug: /File[/Users/jeff/.puppet/var]: Autorequiring File[/Users/jeff/.puppet] debug: /File[/Users/jeff/.puppet/var/log]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/state]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/run]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/lib]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/ssl/certs]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/ssl/public_keys]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl/certificate_requests]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl/private_keys]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl/private]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/ssl/certs/ca.pem]: Autorequiring File[/Users/jeff/.puppet/var/ssl/certs] debug: /File[/Users/jeff/.puppet/var/ssl/crl.pem]: Autorequiring File[/Users/jeff/.puppet/var/ssl] debug: /File[/Users/jeff/.puppet/var/state/state.yaml]: Autorequiring File[/Users/jeff/.puppet/var/state] debug: /File[/Users/jeff/.puppet/var/client_yaml]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/client_data]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/clientbucket]: Autorequiring File[/Users/jeff/.puppet/var] debug: /File[/Users/jeff/.puppet/var/state/last_run_summary.yaml]: Autorequiring File[/Users/jeff/.puppet/var/state] debug: /File[/Users/jeff/.puppet/var/state/last_run_report.yaml]: Autorequiring File[/Users/jeff/.puppet/var/state] debug: /File[/Users/jeff/.puppet/var/state/graphs]: Autorequiring File[/Users/jeff/.puppet/var/state] debug: /File[/Users/jeff/.puppet/var/facts]: Autorequiring File[/Users/jeff/.puppet/var] debug: Finishing transaction 70250873848000 debug: Loaded state in 0.00 seconds debug: Loaded state in 0.00 seconds info: Applying configuration version '1323223324' debug: /Schedule[daily]: Skipping device resources because running on a host debug: /Schedule[monthly]: Skipping device resources because running on a host debug: /Schedule[hourly]: Skipping device resources because running on a host notice: This is a simple text file with a snowman. The snowman is here: (☃) EOL EOF notice: /Stage[main]/Support535::Utf8template/Notify[utf8template]/message: defined 'message' as 'This is a simple text file with a snowman. The snowman is here: (☃) EOL EOF ' debug: /Stage[main]/Support535::Utf8template/Notify[utf8template]: The container Class[Support535::Utf8template] will propagate my refresh event debug: Class[Support535::Utf8template]: The container Stage[main] will propagate my refresh event debug: /Schedule[never]: Skipping device resources because running on a host debug: /Schedule[weekly]: Skipping device resources because running on a host debug: /Schedule[puppet]: Skipping device resources because running on a host debug: Finishing transaction 70250870270220 debug: Storing state debug: Stored state in 0.03 seconds notice: Finished catalog run in 0.05 seconds </pre> -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://projects.puppetlabs.com/my/account -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/puppet-bugs?hl=en.
