Francesco Romani has uploaded a new change for review. Change subject: profiling: Add support for memory profiling ......................................................................
profiling: Add support for memory profiling There are many python memory profilers, but most of them are tailored for interactive usage, so they are poorly suited for a daemon, like VDSM. This patch adds support for dowser: <http://www.aminus.net/wiki/Dowser> <https://pypi.python.org/pypi/dowser/0.2> which has the following benefits which makes it ina better position to be used with VDSM: + self contained, includes WEB UI + allows live monitoring + tailored for cherrypy usage, so friendly towards server applications. + easily portable (pure package) It must be noted that dowser has some drawbacks as well - significat dependencies, both in number and in size (with respect to VDSM standards): cherrypy, PIL - based on gc module As per the cpu profile already added, this code is meant to be used only as debug aid or in development environments. The feature is controlled by a config tunable and disabled by default; the feature disable itself if any dependency is missed. Change-Id: Ib56b65513e0118b68cd43791bf655c928d6a26e2 Signed-off-by: Francesco Romani <[email protected]> --- M debian/vdsm-python.install M lib/vdsm/Makefile.am A lib/vdsm/profile_mem.py M vdsm.spec.in M vdsm/vdsm 5 files changed, 89 insertions(+), 0 deletions(-) git pull ssh://gerrit.ovirt.org:29418/vdsm refs/changes/19/32019/1 diff --git a/debian/vdsm-python.install b/debian/vdsm-python.install index 8d11bfc..f429a91 100644 --- a/debian/vdsm-python.install +++ b/debian/vdsm-python.install @@ -15,6 +15,7 @@ ./usr/lib/python2.7/dist-packages/vdsm/netlink/link.py ./usr/lib/python2.7/dist-packages/vdsm/netlink/route.py ./usr/lib/python2.7/dist-packages/vdsm/profile.py +./usr/lib/python2.7/dist-packages/vdsm/profile_mem.py ./usr/lib/python2.7/dist-packages/vdsm/qemuimg.py ./usr/lib/python2.7/dist-packages/vdsm/sslutils.py ./usr/lib/python2.7/dist-packages/vdsm/tool/__init__.py diff --git a/lib/vdsm/Makefile.am b/lib/vdsm/Makefile.am index 4bebf28..04c8018 100644 --- a/lib/vdsm/Makefile.am +++ b/lib/vdsm/Makefile.am @@ -31,6 +31,7 @@ netconfpersistence.py \ netinfo.py \ profile.py \ + profile_mem.py \ qemuimg.py \ sslutils.py \ utils.py \ diff --git a/lib/vdsm/profile_mem.py b/lib/vdsm/profile_mem.py new file mode 100755 index 0000000..a60a833 --- /dev/null +++ b/lib/vdsm/profile_mem.py @@ -0,0 +1,83 @@ +# +# Copyright 2014 Red Hat, Inc. +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program; if not, write to the Free Software +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA +# +# Refer to the README and COPYING files for full details of the license +# + +""" +This module provides memory profiling. +""" + +import threading + +from vdsm.config import config + +try: + # dowser dependencies + import cherrypy + from PIL import Image, ImageDraw + # dowser profiler itself + import dowser +except IOError: + _SUPPORTED = False +else: + _SUPPORTED = True + +Image # makes pyflakes happy +ImageDraw # same + + +_thread = None + + +def memory_usage_server(port=None): + cherrypy.tree.mount(dowser.Root()) + + port = (config.getint('vars', 'profile_memory_port') + if port is None else port) + + cherrypy.config.update({ + 'environment': 'embedded', + 'server.socket_port': port}) + + cherrypy.engine.start() + + +def start(): + """ Starts application memory profiling """ + if _SUPPORTED and is_enabled(): + _start_profiling() + + +def stop(): + """ Stops application memory profiling """ + pass # not yet needed + + +def is_enabled(): + return config.getboolean('vars', 'profile_memory_enable') + + +def is_running(): + return _thread is not None and _thread.is_alive() + + +def _start_profiling(): + global _thread + _thread = threading.Thread(name='memprofile', target=memory_usage_server) + _thread.daemon = True + _thread.start() diff --git a/vdsm.spec.in b/vdsm.spec.in index 5f9488c..233eee6 100644 --- a/vdsm.spec.in +++ b/vdsm.spec.in @@ -1184,6 +1184,7 @@ %{python_sitearch}/%{vdsm_name}/netlink/link.py* %{python_sitearch}/%{vdsm_name}/netlink/route.py* %{python_sitearch}/%{vdsm_name}/profile.py* +%{python_sitearch}/%{vdsm_name}/profile_mem.py* %{python_sitearch}/%{vdsm_name}/qemuimg.py* %{python_sitearch}/%{vdsm_name}/netconfpersistence.py* %{python_sitearch}/%{vdsm_name}/sslutils.py* diff --git a/vdsm/vdsm b/vdsm/vdsm index e575295..619e57e 100755 --- a/vdsm/vdsm +++ b/vdsm/vdsm @@ -34,6 +34,7 @@ from vdsm.config import config from vdsm import libvirtconnection from vdsm import profile +from vdsm import profile_mem from storage.dispatcher import Dispatcher from storage.hsm import HSM @@ -69,6 +70,7 @@ zombiereaper.registerSignalHandler() profile.start() + profile_mem.start() libvirtconnection.start_event_loop() @@ -85,6 +87,7 @@ while running[0]: signal.pause() + profile_mem.stop() profile.stop() finally: cif.prepareForShutdown() -- To view, visit http://gerrit.ovirt.org/32019 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: newchange Gerrit-Change-Id: Ib56b65513e0118b68cd43791bf655c928d6a26e2 Gerrit-PatchSet: 1 Gerrit-Project: vdsm Gerrit-Branch: master Gerrit-Owner: Francesco Romani <[email protected]> _______________________________________________ vdsm-patches mailing list [email protected] https://lists.fedorahosted.org/mailman/listinfo/vdsm-patches
