Bug#894854: ansible: Ansible 2.5.0 fails to gather facts on hosts which don't have dmidecode

2018-04-05 Thread Jeremy Sowden
On 2018-04-04, at 21:41:52 +0100, Jeremy Sowden wrote:
> Since upgrading Ansible to 2.5.0, it has been failing to gather facts
> on my servers which do not have dmidecode installed.  I diffed the
> 2.4.3 and 2.5.0 versions of
> ansible/module_utils/facts/virtual/linux.py and found the following
> change:
>
> @@ -220,6 +233,17 @@
>  virtual_facts['virtualization_role'] = 'guest'
>  return virtual_facts
>
> +# In older Linux Kernel versions, /sys filesystem is not
> available
> +# dmidecode is the safest option to parse virtualization
> related values

Obviously this bug doesn't affect Ansible when trying to gather facts
from newer hosts, since it does not fall back to trying to run
dmidecode.

I forgot to provide a test-case.  I've attached logs of failed and
successful attempts to gather facts from one of my amd64 servers.  The
only difference between the two attempts is that I installed dmidecode
before the successful one.

I've snipped the actual facts returned by the successful attempt.
[azazel@celephais:~/work/ansible] $ ssh kadath apt-cache policy dmidecode
dmidecode:
  Installed: (none)
  Candidate: 3.0-4
  Version table:
 3.0-4 500
500 http://mirror.ox.ac.uk/debian stable/main amd64 Packages
500 http://ftp.uk.debian.org/debian stable/main amd64 Packages
500 http://ftp.debian.org/debian stable/main amd64 Packages
 2.12-3 500
500 http://mirror.ox.ac.uk/debian oldstable/main amd64 Packages
500 http://ftp.uk.debian.org/debian oldstable/main amd64 Packages
500 http://ftp.debian.org/debian oldstable/main amd64 Packages
[azazel@celephais:~/work/ansible] $ cat inventories/dmidecode_test
[all]
  192.168.96.1
[azazel@celephais:~/work/ansible] $ cat dmidecode_test_playbook.yml
---
- hosts : all
  gather_facts : True

  vars :
  roles :
[azazel@celephais:~/work/ansible] $ ansible-playbook -vvv -i 
inventories/dmidecode_test dmidecode_test_playbook.yml
ansible-playbook 2.5.0
  config file = /etc/ansible/ansible.cfg
  configured module search path = [u'/home/azazel/.ansible/plugins/modules', 
u'/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python2.7/dist-packages/ansible
  executable location = /usr/bin/ansible-playbook
  python version = 2.7.14+ (default, Mar 13 2018, 15:23:44) [GCC 7.3.0]
Using /etc/ansible/ansible.cfg as config file
Parsed /home/azazel/work/ansible/inventories/dmidecode_test inventory source 
with ini plugin
 ___
< PLAYBOOK: dmidecode_test_playbook.yml >
 ---
\   ^__^
 \  (oo)\___
(__)\   )\/\
||w |
|| ||

1 plays in dmidecode_test_playbook.yml
 
< PLAY [all] >
 
\   ^__^
 \  (oo)\___
(__)\   )\/\
||w |
|| ||

 
< TASK [Gathering Facts] >
 
\   ^__^
 \  (oo)\___
(__)\   )\/\
||w |
|| ||

task path: /home/azazel/work/ansible/dmidecode_test_playbook.yml:2
Using module file 
/usr/lib/python2.7/dist-packages/ansible/modules/system/setup.py
<192.168.96.1> ESTABLISH SSH CONNECTION FOR USER: None
<192.168.96.1> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o 
KbdInteractiveAuthentication=no -o 
PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o 
PasswordAuthentication=no -o ConnectTimeout=10 -o 
ControlPath=/home/azazel/.ansible/cp/f62fa23a25 192.168.96.1 '/bin/sh -c 
'"'"'echo ~ && sleep 0'"'"''
<192.168.96.1> (0, '/home/azazel\n', '')
<192.168.96.1> ESTABLISH SSH CONNECTION FOR USER: None
<192.168.96.1> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o 
KbdInteractiveAuthentication=no -o 
PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o 
PasswordAuthentication=no -o ConnectTimeout=10 -o 
ControlPath=/home/azazel/.ansible/cp/f62fa23a25 192.168.96.1 '/bin/sh -c '"'"'( 
umask 77 && mkdir -p "` echo 
/home/azazel/.ansible/tmp/ansible-tmp-1522913112.18-247490260293391 `" && echo 
ansible-tmp-1522913112.18-247490260293391="` echo 
/home/azazel/.ansible/tmp/ansible-tmp-1522913112.18-247490260293391 `" ) && 
sleep 0'"'"''
<192.168.96.1> (0, 
'ansible-tmp-1522913112.18-247490260293391=/home/azazel/.ansible/tmp/ansible-tmp-1522913112.18-247490260293391\n',
 '')
<192.168.96.1> PUT 
/home/azazel/.ansible/tmp/ansible-local-29652szfdDu/tmpePQ_Xy TO 
/home/azazel/.ansible/tmp/ansible-tmp-1522913112.18-247490260293391/setup.py
<192.168.96.1> SSH: EXEC sftp -b - -C -o ControlMaster=auto -o 
ControlPersist=60s -o KbdInteractiveAuthentication=no -o 
PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o 
PasswordAuthentication=no -o ConnectTimeout=10 -o 
ControlPath=/home/azazel/.ansible/cp/f62fa23a25 

Bug#894854: ansible: Ansible 2.5.0 fails to gather facts on hosts which don't have dmidecode

2018-04-04 Thread Jeremy Sowden
Package: ansible
Version: 2.5.0+dfsg-1
Severity: important
Tags: patch upstream

Dear Maintainer,

*** Reporter, please consider answering these questions, where appropriate ***

   * What led up to the situation?
   * What exactly did you do (or not do) that was effective (or
 ineffective)?
   * What was the outcome of this action?
   * What outcome did you expect instead?

*** End of the template - remove these template lines ***

Since upgrading Ansible to 2.5.0, it has been failing to gather facts on my
servers which do not have dmidecode installed.  I diffed the 2.4.3 and 2.5.0
versions of ansible/module_utils/facts/virtual/linux.py and found the following
change:

@@ -220,6 +233,17 @@
 virtual_facts['virtualization_role'] = 'guest'
 return virtual_facts

+# In older Linux Kernel versions, /sys filesystem is not available
+# dmidecode is the safest option to parse virtualization related values
+dmi_bin = self.module.get_bin_path('dmidecode')
+(rc, out, err) = self.module.run_command('%s -s system-product-name' % 
dmi_bin)
+if rc == 0:
+# Strip out commented lines (specific dmidecode output)
+vendor_name = ''.join([line.strip() for line in out.splitlines() 
if not line.startswith('#')])
+if vendor_name in ['VMware Virtual Platform', 'VMware7,1']:
+virtual_facts['virtualization_type'] = 'VMware'
+virtual_facts['virtualization_role'] = 'guest'
+

If dmidecode is not installed, dmi_bin is None and the call to
self.module.run_command fails with a fatal error.  Compare this to the running
of dmidecode in ansible/module_utils/facts/hardware/linux.py (ll. 304ff.):

# Fall back to using dmidecode, if available
dmi_bin = self.module.get_bin_path('dmidecode')
DMI_DICT = {
'bios_date': 'bios-release-date',
'bios_version': 'bios-version',
'form_factor': 'chassis-type',
'product_name': 'system-product-name',
'product_serial': 'system-serial-number',
'product_uuid': 'system-uuid',
'product_version': 'system-version',
'system_vendor': 'system-manufacturer'
}
for (k, v) in DMI_DICT.items():
if dmi_bin is not None:
(rc, out, err) = self.module.run_command('%s -s %s' % 
(dmi_bin, v))
if rc == 0:
# Strip out commented lines (specific dmidecode output)
thisvalue = ''.join([line for line in out.splitlines() 
if not line.startswith('#')])
try:
json.dumps(thisvalue)
except UnicodeDecodeError:
thisvalue = "NA"

dmi_facts[k] = thisvalue
else:
dmi_facts[k] = 'NA'
else:
dmi_facts[k] = 'NA'

In this case, the code checks whether dmi_bin is None before attempting to run
it.

I think the new code is also missing a return statement.

I have installed dmidecode on my amd64 hosts, but I have an armel on which
dmidecode is not available.

-- System Information:
Debian Release: buster/sid
  APT prefers testing
  APT policy: (990, 'testing'), (900, 'stable'), (500, 'oldoldstable-updates'), 
(99, 'unstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 4.14.0-2-amd64 (SMP w/4 CPU cores)
Locale: LANG=en_GB.UTF-8, LC_CTYPE=en_GB.UTF-8 (charmap=UTF-8) (ignored: LC_ALL 
set to en_GB.UTF-8), LANGUAGE=en_GB:en (charmap=UTF-8) (ignored: LC_ALL set to 
en_GB.UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages ansible depends on:
ii  python2.7.14-4
ii  python-crypto 2.6.1-8
ii  python-cryptography   2.1.4-1
ii  python-httplib2   0.9.2+dfsg-1
ii  python-jinja2 2.10-1
ii  python-netaddr0.7.19-1
ii  python-paramiko   2.4.0-1
ii  python-pkg-resources  39.0.1-1
ii  python-yaml   3.12-1+b1

Versions of packages ansible recommends:
pn  python-jmespath   
pn  python-kerberos   
pn  python-libcloud   
pn  python-selinux
pn  python-winrm  
pn  python-xmltodict  

Versions of packages ansible suggests:
ii  cowsay   3.03+dfsg2-4
pn  sshpass  

-- no debconf information
diff --git a/lib/ansible/module_utils/facts/virtual/linux.py 
b/lib/ansible/module_utils/facts/virtual/linux.py
index a0ad545..7b0b233 100644
--- a/lib/ansible/module_utils/facts/virtual/linux.py
+++ b/lib/ansible/module_utils/facts/virtual/linux.py
@@ -236,13 +236,15 @@ class LinuxVirtual(Virtual):
 # In older Linux Kernel versions, /sys filesystem is not available
 # dmidecode is the safest option to parse virtualization related values
 dmi_bin = self.module.get_bin_path('dmidecode')
-