Public bug reported:
Using:
- SLE12SP1
- xen 4.7
- nova 13.1.2.dev68 (stable-mitaka tarball)
When configuring nova-compute to use the rootwrap daemon and using Xen with
libvirt as hypervisor I get the following error when booting a VM:
2016-10-12 15:54:34.216 17936 INFO nova.compute.claims
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Claim successful
2016-10-12 15:54:34.458 17936 INFO nova.virt.osinfo
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Cannot load Libosinfo: (No module named
Libosinfo)
2016-10-12 15:54:34.479 17936 WARNING oslo_config.cfg
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Option "username" from group "neutron"
is deprecated. Use option "user-name" from group "neutron".
2016-10-12 15:54:34.751 17936 INFO nova.virt.libvirt.driver
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Creating image
2016-10-12 15:54:34.758 17936 INFO nova.utils
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Executing RootwrapDaemonHelper.execute
cmd=[u'touch -c
/var/lib/nova/instances/_base/309100c6d00d13edba007a0dde00e9889ce0410a']
kwargs=[{'run_as_root': True}]
2016-10-12 15:54:34.795 17936 INFO oslo_rootwrap.client
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Spawned new rootwrap daemon process
with pid=17984
2016-10-12 15:54:36.060 17936 INFO nova.utils
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Executing RootwrapDaemonHelper.execute
cmd=[u'xend status'] kwargs=[{'run_as_root': True}]
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Instance failed to spawn
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Traceback (most recent call last):
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in
_build_resources
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] yield resources
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2064, in
_build_and_run_instance
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] block_device_info=block_device_info)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2790, in
spawn
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] write_to_disk=True)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4746, in
_get_guest_xml
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] context)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4605, in
_get_guest_config
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] flavor, guest.os_type)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3640, in
_get_guest_storage_config
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] inst_type)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3593, in
_get_guest_disk_config
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] self._host.get_version())
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 176,
in libvirt_info
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] self.is_block_dev)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 133, in
pick_disk_driver_name
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] run_as_root=True,
check_exit_code=True)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 56, in
execute
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] return utils.execute(*args, **kwargs)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/utils.py", line 386, in execute
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] *cmd, **kwargs)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/utils.py", line 336, in execute
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] cmd, process_input)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/oslo_rootwrap/client.py", line 135, in execute
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] res = proxy.run_one_command(cmd,
stdin)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File "<string>", line 2, in
run_one_command
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 774, in _callmethod
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] raise convert_to_error(kind, result)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] RemoteError:
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5]
---------------------------------------------------------------------------
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Unserializable message: ('#ERROR',
FilterMatchNotExecutable())
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5]
---------------------------------------------------------------------------
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5]
2016-10-12 15:54:36.067 17936 INFO nova.compute.manager
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Terminating instance
2016-10-12 15:54:36.074 17936 INFO nova.virt.libvirt.driver [-] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] During wait destroy, instance disappeared.
2016-10-12 15:54:36.126 17936 INFO nova.virt.libvirt.driver
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Deleting instance files
/var/lib/nova/instances/98dbdd8b-ae17-4861-bffe-cc48042f93e5_del
2016-10-12 15:54:36.189 17936 INFO nova.virt.libvirt.driver
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Deletion of
/var/lib/nova/instances/98dbdd8b-ae17-4861-bffe-cc48042f93e5_del complete
** Affects: nova
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1632768
Title:
rootwrap daemon with libvirt/xen not working
Status in OpenStack Compute (nova):
New
Bug description:
Using:
- SLE12SP1
- xen 4.7
- nova 13.1.2.dev68 (stable-mitaka tarball)
When configuring nova-compute to use the rootwrap daemon and using Xen with
libvirt as hypervisor I get the following error when booting a VM:
2016-10-12 15:54:34.216 17936 INFO nova.compute.claims
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Claim successful
2016-10-12 15:54:34.458 17936 INFO nova.virt.osinfo
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Cannot load Libosinfo: (No module named
Libosinfo)
2016-10-12 15:54:34.479 17936 WARNING oslo_config.cfg
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Option "username" from group "neutron"
is deprecated. Use option "user-name" from group "neutron".
2016-10-12 15:54:34.751 17936 INFO nova.virt.libvirt.driver
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Creating image
2016-10-12 15:54:34.758 17936 INFO nova.utils
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Executing RootwrapDaemonHelper.execute
cmd=[u'touch -c
/var/lib/nova/instances/_base/309100c6d00d13edba007a0dde00e9889ce0410a']
kwargs=[{'run_as_root': True}]
2016-10-12 15:54:34.795 17936 INFO oslo_rootwrap.client
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Spawned new rootwrap daemon process
with pid=17984
2016-10-12 15:54:36.060 17936 INFO nova.utils
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] Executing RootwrapDaemonHelper.execute
cmd=[u'xend status'] kwargs=[{'run_as_root': True}]
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Instance failed to spawn
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Traceback (most recent call last):
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2218, in
_build_resources
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] yield resources
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2064, in
_build_and_run_instance
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] block_device_info=block_device_info)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2790, in
spawn
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] write_to_disk=True)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4746, in
_get_guest_xml
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] context)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4605, in
_get_guest_config
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] flavor, guest.os_type)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3640, in
_get_guest_storage_config
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] inst_type)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3593, in
_get_guest_disk_config
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] self._host.get_version())
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/imagebackend.py", line 176,
in libvirt_info
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] self.is_block_dev)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 133, in
pick_disk_driver_name
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] run_as_root=True,
check_exit_code=True)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/virt/libvirt/utils.py", line 56, in
execute
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] return utils.execute(*args, **kwargs)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/utils.py", line 386, in execute
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] *cmd, **kwargs)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/nova/utils.py", line 336, in execute
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] cmd, process_input)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib/python2.7/site-packages/oslo_rootwrap/client.py", line 135, in execute
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] res = proxy.run_one_command(cmd,
stdin)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File "<string>", line 2, in
run_one_command
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] File
"/usr/lib64/python2.7/multiprocessing/managers.py", line 774, in _callmethod
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] raise convert_to_error(kind, result)
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] RemoteError:
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5]
---------------------------------------------------------------------------
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Unserializable message: ('#ERROR',
FilterMatchNotExecutable())
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5]
---------------------------------------------------------------------------
2016-10-12 15:54:36.062 17936 ERROR nova.compute.manager [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5]
2016-10-12 15:54:36.067 17936 INFO nova.compute.manager
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Terminating instance
2016-10-12 15:54:36.074 17936 INFO nova.virt.libvirt.driver [-] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] During wait destroy, instance disappeared.
2016-10-12 15:54:36.126 17936 INFO nova.virt.libvirt.driver
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Deleting instance files
/var/lib/nova/instances/98dbdd8b-ae17-4861-bffe-cc48042f93e5_del
2016-10-12 15:54:36.189 17936 INFO nova.virt.libvirt.driver
[req-5f1c8974-b449-41fa-806b-73f705ed1634 47640c082746419f87ae498f7bdab44e
08f3d1224d1845cda767a2193594c3d7 - - -] [instance:
98dbdd8b-ae17-4861-bffe-cc48042f93e5] Deletion of
/var/lib/nova/instances/98dbdd8b-ae17-4861-bffe-cc48042f93e5_del complete
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1632768/+subscriptions
--
Mailing list: https://launchpad.net/~yahoo-eng-team
Post to : [email protected]
Unsubscribe : https://launchpad.net/~yahoo-eng-team
More help : https://help.launchpad.net/ListHelp
