Jira (PUP-6217) site.pp with file resource containing UTF8 failing on certain platforms

Ethan Brown (JIRA) Sat, 10 Jun 2017 19:43:15 -0700

Title: Message Title

Ethan Brown commented on

PUP-6217

I was able to finally track down what's going on here (which spawned an effort to get ci:test:git working on OSX / tickets PUP-7648 and BKR-1139).

Based on the efforts in

~~PUP-5756~~ the logging is now more resilient than it was previously per https://github.com/puppetlabs/puppet/commit/c47be2d3b3d002f02cbed6edee8dfee119152b46. So on current master branch, the failure no longer occurs because Puppet handles it better, like:

Info: Using configured environment 'supports_utf8_vth5kn2g'

      Info: Retrieving pluginfacts

      Info: Retrieving plugin

      Info: Caching catalog for xdf0d3anl5c5vv3.delivery.puppetlabs.net

      Info: Applying configuration version '1497147050'

      Notice: /Stage[main]/Main/File[/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_vth5kn2g.XXXXXX.WrOekzFs]/content: Received a log message with invalid encoding:"\n--- /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_vth5kn2g.XXXXXX.WrOekzFs\t2017-06-10 19:08:40.000000000 -0700\n+++ /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/puppet-file20170610-662-1dmapk8\t2017-06-10 19:10:51.000000000 -0700\n@@ -0,0 +1 @@\n+M\xC3\xB8nti Pyth\xC3\xB8n ik den H\xC3\xB8lie Gr\xC3\xA4ilen, y\xC3\xABr? \xE2\x82\xAC \xE2\x80\xB0 \xE3\x84\x98 \xE4\xB8\x87 \xE7\xAB\xB9 \xC3\x9C \xC3\x96\n"

      Backtrace:

      /Library/Ruby/Site/2.0.0/puppet/util/log.rb:304:in `initialize'

      /Library/Ruby/Site/2.0.0/puppet/util/log.rb:87:in `new'

      /Library/Ruby/Site/2.0.0/puppet/util/log.rb:87:in `create'

      /Library/Ruby/Site/2.0.0/puppet/util/logging.rb:12:in `send_log'

      /Library/Ruby/Site/2.0.0/puppet/util/logging.rb:21:in `block (2 levels) in <module:Logging>'

      /Library/Ruby/Site/2.0.0/puppet/type/file/data_sync.rb:44:in `block in checksum_insync?'

      /Library/Ruby/Site/2.0.0/puppet/type/file/data_sync.rb:19:in `write_temporarily'

      /Library/Ruby/Site/2.0.0/puppet/type/file/data_sync.rb:43:in `checksum_insync?'

      /Library/Ruby/Site/2.0.0/puppet/type/file/content.rb:91:in `insync?'

      /Library/Ruby/Site/2.0.0/puppet/property.rb:277:in `safe_insync?'

      /Library/Ruby/Site/2.0.0/puppet/transaction/resource_harness.rb:115:in `sync_if_needed'

      Info: Computing checksum on file /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_vth5kn2g.XXXXXX.WrOekzFs

      Info: /Stage[main]/Main/File[/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_vth5kn2g.XXXXXX.WrOekzFs]: Filebucketed /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_vth5kn2g.XXXXXX.WrOekzFs to puppet with sum d41d8cd98f00b204e9800998ecf8427e

      Notice: /Stage[main]/Main/File[/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_vth5kn2g.XXXXXX.WrOekzFs]/content: content changed '{md5}d41d8cd98f00b204e9800998ecf8427e' to '{md5}f1c1271f5e21eba48374b96e43caddc1'

      Notice: Applied catalog in 0.08 seconds

However, this still points to an internal issue. With a little debugging, it's easy to see what's going on here:

xqkeu5wzs8j44o8.delivery.puppetlabs.net (osx1011-64-1) 19:35:36$ ruby -e "puts Encoding.default_external"

      US-ASCII

    xqkeu5wzs8j44o8.delivery.puppetlabs.net (osx1011-64-1) executed in 0.19 seconds

=> #<Beaker::Result:0x007fa4766db5d8

 @cmd="ruby -e \"puts Encoding.default_external\"",

 @exit_code=0,

 @host="osx1011-64-1",

 @output="US-ASCII\n",

 @raw_output="US-ASCII\n",

 @raw_stderr="",

 @raw_stdout="US-ASCII\n",

 @stderr="",

 @stdout="US-ASCII\n">

Looking at the host environment variables shows missing LANG or LC variables:

[2] pry(#<Beaker::TestCase>)> on(agent, 'env').stdout.chomp.split("\n")

    xqkeu5wzs8j44o8.delivery.puppetlabs.net (osx1011-64-1) 19:37:39$ env

      SHELL=/bin/sh

      TMPDIR=/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/

      SSH_CLIENT=10.32.130.9 56634 22

      USER=root

      SSH_AUTH_SOCK=/tmp/ssh-MhSIVDz3jP/agent.379

      MAIL=/var/mail/root

      PATH=PATH:/usr/local/bin:/usr/bin:/opt/puppet-git-repos/hiera/bin:/usr/bin:/bin:/usr/sbin:/sbin

      PWD=/private/var/root

      SHLVL=1

      HOME=/var/root

      LOGNAME=root

      SSH_CONNECTION=10.32.130.9 56634 10.32.112.46 22

      _=/usr/bin/env

Therefore, its evident that Ruby believes it is in US-ASCII encoding because the LANG variable is not. If the host has ~/.ssh/environment modified like ssh -i ~/.ssh/id_rsa-jenkins [email protected] 'echo "LANG=en_US.UTF-8" >> ~/.ssh/environment' to add LANG=en_US.UTF-8, then behavior changes:

[4] pry(#<Beaker::TestCase>)> on agent, 'ruby -e "puts Encoding.default_external"'

    xqkeu5wzs8j44o8.delivery.puppetlabs.net (osx1011-64-1) 19:40:40$ ruby -e "puts Encoding.default_external"

      UTF-8

    xqkeu5wzs8j44o8.delivery.puppetlabs.net (osx1011-64-1) executed in 0.19 seconds

=> #<Beaker::Result:0x007fa47b498988

 @cmd="ruby -e \"puts Encoding.default_external\"",

 @exit_code=0,

 @host="osx1011-64-1",

 @output="UTF-8\n",

 @raw_output="UTF-8\n",

 @raw_stderr="",

 @raw_stdout="UTF-8\n",

 @stderr="",

 @stdout="UTF-8\n">

And then the test passes as expected:

Info: Using configured environment 'supports_utf8_s3df0w7h'

      Info: Retrieving pluginfacts

      Info: Retrieving plugin

      Info: Caching catalog for xqkeu5wzs8j44o8.delivery.puppetlabs.net

      Info: Applying configuration version '1497148873'

      Notice: /Stage[main]/Main/File[/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_s3df0w7h.XXXXXX.riqWxy3Z]/content:

      --- /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_s3df0w7h.XXXXXX.riqWxy3Z	2017-06-10 19:35:08.000000000 -0700

      +++ /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/puppet-file20170610-683-1w1xyo7	2017-06-10 19:41:13.000000000 -0700

      @@ -0,0 +1 @@

      +Mønti Pythøn ik den Hølie Gräilen, yër? € ‰ ㄘ 万 竹 Ü Ö

      Info: Computing checksum on file /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_s3df0w7h.XXXXXX.riqWxy3Z

      Info: /Stage[main]/Main/File[/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_s3df0w7h.XXXXXX.riqWxy3Z]: Filebucketed /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_s3df0w7h.XXXXXX.riqWxy3Z to puppet with sum d41d8cd98f00b204e9800998ecf8427e

      Notice: /Stage[main]/Main/File[/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/supports_utf8_s3df0w7h.XXXXXX.riqWxy3Z]/content: content changed '{md5}d41d8cd98f00b204e9800998ecf8427e' to '{md5}f1c1271f5e21eba48374b96e43caddc1'

      Notice: Applied catalog in 0.07 seconds

So the solution is to land another change in Beaker that ensures LANG is set as nodes are provisioned - there is no bug in Puppet here.

Add Comment

This message was sent by Atlassian JIRA

--
You received this message because you are subscribed to the Google Groups "Puppet Bugs" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
To post to this group, send email to [email protected].
Visit this group at https://groups.google.com/group/puppet-bugs.
For more options, visit https://groups.google.com/d/optout.

Jira (PUP-6217) site.pp with file resource containing UTF8 failing on certain platforms

Reply via email to