wido commented on code in PR #14320: URL: https://github.com/apache/cloudstack/pull/14320#discussion_r4257181289
########## scripts/vm/hypervisor/kvm/kvmvmactivity_rbd.py: ########## @@ -0,0 +1,282 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Ceph RBD flavor of kvmvmactivity.sh. +# +# On NFS/SharedMountPoint storage, VM disk activity is detected via the mtime +# of the volume files on the shared mount point. RBD volumes aren't files on +# a mount point, so instead activity is detected via RBD watchers: as long as +# qemu has an RBD image open (i.e. a VM using that volume is running +# somewhere), the image will have a live watcher. The most recent +# suspect-time/watcher-state is persisted as a RADOS object (per host) +# in place of the "ac-<host>" file used by the NFS/SMP scripts. +# +# The volumes are given in -u, or in a file with -U, which has no limit on the number +# of volumes: one argument of a command is limited to 128 KiB, which is about 3500 +# volumes. The file has the uuids separated by commas or by lines. +# +# The cephx key is not an argument, as it would be visible in the process list: +# it is read from the environment variable CEPH_KEY, and is given to rados and +# rbd in the environment variable CEPH_ARGS. +# +# The watchers of the volumes are listed over one connection to the Ceph cluster +# with the Python bindings of librados and librbd (python3-rados and python3-rbd), +# which is much faster than "rbd status" for each volume when a host has many +# volumes. If the bindings are not installed or can not be used, "rbd status" is +# run for each volume instead. +# +# Exit status: 0 with ALIVE or DEAD in the output, 2 if the arguments are not +# valid or if the state of the host could not be found out, which is not a DEAD host. + +import argparse +import os +import re +import subprocess +import sys +import time + +try: + import rados + import rbd +except ImportError: + rados = None + rbd = None + +KEY_ENV = "CEPH_KEY" +CONNECT_TIMEOUT = 60 + + +class Parser(argparse.ArgumentParser): + def error(self, message): + self.print_usage(sys.stderr) + sys.exit(1) + + +def ceph(tool, args, env, data=None): + """Runs rados or rbd and returns its exit status and its output.""" + try: + result = subprocess.run([tool] + args, input=data, stdout=subprocess.PIPE, + stderr=subprocess.DEVNULL, env=env, universal_newlines=True) + except OSError: + return 127, "" + return result.returncode, result.stdout + + +class WatchersError(Exception): + """The watchers of a volume could not be listed: it is not known whether the host is alive.""" + + +class BindingsWatchers: + """Lists the watchers of volumes over one connection, with the Python bindings of librados and librbd.""" + + def __init__(self, mon_hosts, pool, user, key): + if rados is None or rbd is None or not hasattr(rbd.Image, "watchers_list"): + raise RuntimeError("the Python bindings of librados and librbd can not be used") + conf = {"mon_host": mon_hosts} + if key: + conf["key"] = key + self.cluster = rados.Rados(rados_id=user if user else None, conf=conf) + self.ioctx = None + try: + self.cluster.connect(timeout=CONNECT_TIMEOUT) Review Comment: You should set the options 'rados_osd_op_timeout' and 'rados_mon_op_timeout' to something like 30 seconds. Otherwise RADOS operations can block for ever if the client can't reach the Ceph cluster for what ever reason. ########## scripts/vm/hypervisor/kvm/kvmheartbeat_rbd.py: ########## @@ -0,0 +1,133 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# Ceph RBD flavor of kvmheartbeat.sh/kvmsmpheartbeat.sh. +# +# There is no shared POSIX mount point to write a heartbeat file to when the +# primary storage pool is Ceph RBD, so the heartbeat timestamp is instead +# stored as a small RADOS object (one object per host) in the same RBD pool. +# Any host with a working path to the Ceph cluster can write/read this object, +# which gives the same semantics as the NFS/SharedMountPoint heartbeat file. +# +# The cephx key is not an argument, as it would be visible in the process list: +# it is read from the environment variable CEPH_KEY, and is given to rados in +# the environment variable CEPH_ARGS. + +import argparse +import os +import re +import subprocess +import sys +import time + +KEY_ENV = "CEPH_KEY" + + +class Parser(argparse.ArgumentParser): + def error(self, message): + self.print_usage(sys.stderr) + sys.exit(1) + + +def rados(args, env, data=None): + """Runs rados and returns its exit status and its output.""" + try: + result = subprocess.run(["rados"] + args, input=data, stdout=subprocess.PIPE, Review Comment: Why would we run the rados command here and not use the native librados in Python to write to the RADOS cluster? And don't forget the 'rados_osd_op_timeout' and 'rados_mon_op_timeout' options here as well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
