Hello community, here is the log from the commit of package crmsh for openSUSE:Factory checked in at 2015-10-19 22:51:28 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/crmsh (Old) and /work/SRC/openSUSE:Factory/.crmsh.new (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "crmsh" Changes: -------- --- /work/SRC/openSUSE:Factory/crmsh/crmsh.changes 2015-10-12 10:02:44.000000000 +0200 +++ /work/SRC/openSUSE:Factory/.crmsh.new/crmsh.changes 2015-10-20 00:05:26.000000000 +0200 @@ -1,0 +2,22 @@ +Thu Oct 15 05:18:06 UTC 2015 - [email protected] + +- Update to version 2.2.0~rc3+git.1444854254.fc37f7f: + + high: utils: Handle time zones in parse_time (bsc#949511) + + medium: cibconfig: Fix sanity check for attribute-based fencing topology (#110) + + medium: ui_script: Optionally print common params + + medium: hb_report: Remove reference to function name in event patterns (bsc#942906) + + medium: report: Make transitions without end stretch to 2525 + + doc: add missing <> to fencing_topology syntax + + doc: add missing backslash in fencing_topology example + + doc: add explanatory comments to fencing_topology + + doc: Update the scripts documentation + + doc: Fix unclosed block in scripts documentation + +------------------------------------------------------------------- +Mon Oct 12 14:51:37 UTC 2015 - [email protected] + +- Update to version 2.2.0~rc3+git.1444661352.14fa72b: + + high: scripts: Determine output format of script correctly (bsc#949980) + + high: cibconfig: Fix bug with node/resource collision + +------------------------------------------------------------------- Old: ---- crmsh-2.2.0~rc3+git.1444340345.59850ca.tar.bz2 New: ---- crmsh-2.2.0~rc3+git.1444854254.fc37f7f.tar.bz2 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ crmsh.spec ++++++ --- /var/tmp/diff_new_pack.mvUPdr/_old 2015-10-20 00:05:27.000000000 +0200 +++ /var/tmp/diff_new_pack.mvUPdr/_new 2015-10-20 00:05:27.000000000 +0200 @@ -36,7 +36,7 @@ Summary: High Availability cluster command-line interface License: GPL-2.0+ Group: %{pkg_group} -Version: 2.2.0~rc3+git.1444340345.59850ca +Version: 2.2.0~rc3+git.1444854254.fc37f7f Release: 0 Url: http://crmsh.github.io Source0: %{name}-%{version}.tar.bz2 ++++++ _servicedata ++++++ --- /var/tmp/diff_new_pack.mvUPdr/_old 2015-10-20 00:05:27.000000000 +0200 +++ /var/tmp/diff_new_pack.mvUPdr/_new 2015-10-20 00:05:27.000000000 +0200 @@ -1,4 +1,4 @@ <servicedata> <service name="tar_scm"> <param name="url">git://github.com/ClusterLabs/crmsh.git</param> - <param name="changesrevision">59850ca9ed07b3e965170b1fb50712ea1bfa502f</param></service></servicedata> \ No newline at end of file + <param name="changesrevision">fc37f7f872a065147e3c0c32e962a924cad109b3</param></service></servicedata> \ No newline at end of file ++++++ crmsh-2.2.0~rc3+git.1444340345.59850ca.tar.bz2 -> crmsh-2.2.0~rc3+git.1444854254.fc37f7f.tar.bz2 ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/doc/crm.8.adoc new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/doc/crm.8.adoc --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/doc/crm.8.adoc 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/doc/crm.8.adoc 2015-10-15 07:18:06.000000000 +0200 @@ -2761,20 +2761,26 @@ Usage: ............... -fencing_topology stonith_resources [stonith_resources ...] -fencing_topology fencing_order [fencing_order ...] +fencing_topology <stonith_resources> [<stonith_resources> ...] +fencing_topology <fencing_order> [<fencing_order> ...] -fencing_order :: target stonith_resources [stonith_resources ...] +fencing_order :: <target> <stonith_resources> [<stonith_resources> ...] stonith_resources :: <rsc>[,<rsc>...] target :: <node>: | attr:<node-attribute>=<value> ............... Example: ............... +# Only kill the power if poison-pill fails fencing_topology poison-pill power + +# As above for node-a, but a different strategy for node-b fencing_topology \ - node-a: poison-pill power + node-a: poison-pill power \ node-b: ipmi serial + +# Fencing anything on rack 1 requires fencing via both APC 1 and 2, +# to defeat the redundancy provided by two separate UPS units. fencing_topology attr:rack=1 apc01,apc02 ............... diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/doc/website-v1/scripts.adoc new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/doc/website-v1/scripts.adoc --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/doc/website-v1/scripts.adoc 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/doc/website-v1/scripts.adoc 2015-10-15 07:18:06.000000000 +0200 @@ -2,7 +2,7 @@ :source-highlighter: pygments .Version information -NOTE: This section applies to `crmsh 2.0+` only. +NOTE: This section applies to `crmsh 2.2+` only. == Introduction == @@ -36,16 +36,52 @@ function through the usual SSH channels used for system maintenance, requiring no additional software to be installed or maintained. +For many scripts that only configure cluster resources or only perform +changes on the local machine, the use of SSH is not necessary. These +scripts can be used even if there is no way for `crmsh` to reach the +other nodes other than through the cluster configuration. + +NOTE: The scripts functionality in `crmsh` has been greatly expanded +and improved in `crmsh` 2.2. Many new scripts have been added, and in +addition the scripts are now used as the backend for the wizards +functionality in HAWK, the HA web interface. For more information, see +https://github.com/ClusterLabs/hawk. + == Usage == Scripts are available through the `cluster` sub-level in the crm shell. Some scripts have custom commands linked to them for -convenience, such as the `init`, `join` and `remove` commands for -creating new clusters, introducing new nodes into the cluster and for -removing nodes from a running cluster. - -Other scripts can be accessed through the `script` sub-level inside -`cluster`. +convenience, such as the `init`, `add` and `remove` commands available +in the `cluster` sublevel, for creating new clusters, introducing new +nodes into the cluster and for removing nodes from a running cluster. + +Other scripts can be accessed through the `script` sub-level. + +=== Common Parameters === + +Which parameters a script accepts varies from script to +script. However, there is a set of parameters that are common to all +scripts. These parameters can be passed to any script. + +`nodes`:: + List of nodes to execute the script for +`dry_run`:: + If set, simulate execution only + (default: no) +`action`:: + If set, only execute a single action (index, as returned by verify) +`statefile`:: + When single-stepping, the state is saved in the given file +`user`:: + Run script as the given user +`sudo`:: + If set, crm will prompt for a sudo password and use sudo when appropriate + (default: no) +`port`:: + Port to connect on +`timeout`:: + Execution timeout in seconds + (default: 600) === List available scripts === @@ -56,37 +92,88 @@ list ......... -The available scripts are listed along with a short description. +The available scripts are listed along with a short +description. Optionally, the arguments +all+ or +names+ can be +used. Without the +all+ flag, some scripts that are used by `crmsh` to +implement certain commands are hidden from view. With the +names+ +flag, only a plain list of script names is printed. === Script description === -To get more details about a script, run the `describe` command. For -example, to get more information about what the `health` script does +To get more details about a script, run the `show` command. For +example, to get more information about what the `virtual-ip` script does and what parameters it accepts, use the following command: ......... # crm script -describe health +show virtual-ip ......... -`describe` will print a longer explanation for the script, along with +`show` will print a longer description of the script, along with a +list of parameters divided into _steps_. Each script is divided into a +series of steps which are performed in order. Some steps may not +accept any parameters, but for those that do, the available parameters +are listed here. + +By default, only a basic subset of the available parameters is printed +in order to make the scripts easier to use. By passing `all` to the +`show` command, the advanced parameters are also shown. In addition, +there is a list of common parameters + +`show` will print a longer explanation for the script, along with a list of parameters, each parameter having a description, a note saying if it is an optional or required parameter, and if optional, what the default value is. +=== Verifying parameters === + +Since a script potentially performs a series of actions and may fail +for various reasons at any point, it is advisable to review the +actions that a script will perform before actually running it. To do +this, the `verify` command can be used. + +Pass the parameters that you would pass to `run`, and `verify` will +check that the parameter values are OK, as well as print the sequence +of steps that will be performed given the particular parameter values +given. + +The following is an example showing how to verify the creation of a +Virtual IP resource, using the `virtual-ip` script: + +.......... +# crm script +verify virtual-ip id=my-virtual-ip ip=192.168.0.10 +.......... + +`crmsh` will print something similar to the following output: + +........... +1. Configure cluster resources + + primitive my-virtual-ip ocf:heartbeat:IPaddr2 + ip="192.168.0.10" + op start timeout="20" op stop timeout="20" + op monitor interval="10" timeout="20" +........... + +In this particular case, there is only a single step, and that step +configures a primitive resource. Other scripts may configure multiple +resources and constraints, or may perform multiple steps in sequence. + === Running a script === To run a script, all required parameters and any optional parameters that should have values other than the default should be provided as -`key=value` pairs on the command line. The following example shows how -to call the `health` script with verbose output enabled: +`key=value` pairs on the command line. + +The following example shows how to create a Virtual IP resource using +the `virtual-ip` script: ........ # crm script -run health verbose=true +run virtual-ip id=my-virtual-ip ip=192.168.0.10 ........ - ==== Single-stepping a script ==== It is possible to run a script action-by-action, with manual intervention @@ -135,6 +222,17 @@ === How scripts work, in detail === +NOTE: The implementation of cluster scripts was revised between +`crmsh` 2.0 and `crmsh` 2.2. This section describes the revised +cluster script format. The old format is still accepted by `crmsh`. + +A cluster script consists of four main sections: + +. The name and description of the script. +. Any other scripts or agents included by this script, and any parameter value overrides to those provided by the included script. +. A set of parameters accepted by the script itself, in addition to those accepted by any scripts or agents included in the script. +. A sequence of actions which the script will perform. + When the script runs, the actions defined in `main.yml` as described below are executed one at a time. Each action prescribes a modification that is applied to the cluster. Some actions work by @@ -230,8 +328,8 @@ to read and modify, while at the same time be compatible with JSON. To learn more, see http:://yaml.org/[yaml.org]. -Here is an example `main.yml` file, heavily commented to explain what -each section means. +Here is an example `main.yml` file which wraps the resource agent +`ocf:heartbeat:IPaddr2`. [source,yaml] ---- @@ -243,83 +341,112 @@ # is less than 2.2, the script is assumed to be a legacy # script (specified in the format used before crmsh 2.2). - version: 2.2 - shortdesc: Check uptime of nodes - longdesc: > - This script will fetch the uptime of - all nodes and report which node has been - up the longest. + shortdesc: Virtual IP + category: Basic + include: + - agent: ocf:heartbeat:IPaddr2 + name: virtual-ip + parameters: + - name: id + type: resource + required: true + - name: ip + type: ip_address + required: true + - name: cidr_netmask + type: integer + required: false + - name: broadcast + type: ip_address + required: false + ops: | + op start timeout="20" op stop timeout="20" + op monitor interval="10" timeout="20" + actions: + - include: virtual-ip +---- + +For a bigger example, here is the `apache` agent which includes +multiple optional steps, the optional installation of packages, +defines multiple cluster resources and potentially calls bash commands +on each of the cluster nodes. + +[source,yaml] +---- +# Copyright (C) 2009 Dejan Muhamedagic +# Copyright (C) 2015 Kristoffer Gronlund +# +# License: GNU General Public License (GPL) +--- +- version: 2.2 + category: Server + shortdesc: Apache Webserver + longdesc: | + Configure a resource group containing a virtual IP address and + an instance of the Apache web server. + + You can optionally configure a Filesystem resource which will be + mounted before the web server is started. + + You can also optionally configure a database resource which will + be started before the web server but after mounting the optional + filesystem. + include: + - agent: ocf:heartbeat:apache + name: apache + longdesc: | + The Apache configuration file specified here must be available via the + same path on all cluster nodes, and Apache must be configured with + mod_status enabled. If in doubt, try running Apache manually via + its init script first, and ensure http://localhost:80/server-status is + accessible. + ops: | + op start timeout="40" + op stop timeout="60" + op monitor interval="10" timeout="20" + - script: virtual-ip + shortdesc: The IP address configured here will start before the Apache instance. + parameters: + - name: id + value: "{{id}}-vip" + - script: filesystem + shortdesc: Optional filesystem mounted before the web server is started. + required: false + - script: database + shortdesc: Optional database started before the web server is started. + required: false parameters: - # Parameters must have a name. - # If a default value is provided, the parameter - # is considered optional. Parameters without a - # default value must be provided when running the - # script. - # To require a parameter to be explicitly provided - # by the user, set required to true. - # To require the value of the parameter to be unique - # across the cluster, set unique to true. This setting - # is not enforced by crmsh, but can be useful as - # documentation. - - name: show_all - shortdesc: Show all uptimes - longdesc: Enable to print all uptimes, not only a summary. + - name: install + type: boolean + shortdesc: Install and configure apache value: false - required: true - unique: false - steps: - # Steps consist of a descriptive name and an action which - # calls a script to do its work. The script should be an - # executable file located in the same folder as main.yml. - # - # Script files can be written in any language, as long as - # the cluster nodes know how to execute them. - # - # These are the valid actions: - # cib: - # Apply the given CIB configuration. The configuration - # can refer to script variables using a mustaschioed - # syntax described in the documentation. - # install: - # Install the given space-separated list of packages - # using the system package manager. - # service: - # Manages system services using the system init tools. - # The argument should be a space-separated list of - # <service>:<state> pairs. - # call: - # Runs a shell command either on the current node or - # on all nodes in the cluster. If the shell command - # fails, the action fails as well. - # crm: - # Runs the given crm command line. - # copy: - # Copy a file to all of the cluster nodes. - # collect: - # Runs on all nodes. Should not perform changes, only - # gather and return information. - # validate: - # Runs on the local node only. Should report problems - # that would prevent further progress. If validate returns - # a map of values, matching script parameters are updated - # to reflect those values. - # apply: - # Runs on all nodes. Applies changes. - # If the dry_run flag is set, script execution stops - # before the first apply action. - # - # apply_local: - # Runs on the local node only. Otherwise same as apply. - # - # report: - # Runs on the local node only. Output from this step is - # printed, not saved as input to the following steps. - # This output does not have to be in JSON format. - - name: Fetch uptime - collect: fetch.py - - name: Report uptime - report: report.py + actions: + - install: + - apache2 + shortdesc: Install the apache package + when: install + - service: + - apache: disable + shortdesc: Let cluster manage apache + when: install + - call: a2enmod status; true + shortdesc: Enable status module + when: install + - include: filesystem + - include: database + - include: virtual-ip + - include: apache + - cib: | + group g-{{id}} + {{filesystem:id}} + {{database:id}} + {{virtual-ip:id}} + {{id}} ---- +The language for referring to parameter values in `cib` actions is +described below. + === Command arguments === The actions that accept a command as argument must not refer to diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/hb_report/hb_report.in new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/hb_report/hb_report.in --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/hb_report/hb_report.in 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/hb_report/hb_report.in 2015-10-15 07:18:06.000000000 +0200 @@ -32,15 +32,15 @@ # Important events # # Patterns format: -# title extended_regexp +# title extended_regexp # NB: don't use spaces in titles or regular expressions! EVENT_PATTERNS=" -membership crmd.*ccm_event.*(NEW|LOST)|pcmk_peer_update.*(lost|memb): -quorum crmd.*crm_update_quorum:.Updating.quorum.status|crmd.*ais.disp.*quorum.(lost|ac?quir) -pause Process.pause.detected -resources lrmd.*rsc:(start|stop) -stonith crmd.*te_fence_node.*Exec|stonith-ng.*log_oper.*reboot|stonithd.*(requests|(Succeeded|Failed).to.STONITH|result=) -start_stop Configuration.validated..Starting.heartbeat|Corosync.Cluster.Engine|Executive.Service.RELEASE|crm_shutdown:.Requesting.shutdown|pcmk_shutdown:.Shutdown.complete +membership crmd.*(NEW|LOST)|pcmk_peer_update.*(lost|memb): +quorum crmd.*Updating.quorum.status|crmd.*ais.disp.*quorum.(lost|ac?quir) +pause Process.pause.detected +resources lrmd.*(start|stop) +stonith crmd.*Exec|stonith-ng.*log_oper.*reboot|stonithd.*(requests|(Succeeded|Failed).to.STONITH|result=) +start_stop Configuration.validated..Starting.heartbeat|Corosync.Cluster.Engine|Executive.Service.RELEASE|Requesting.shutdown|Shutdown.complete " init_tmpfiles diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/cibconfig.py new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/cibconfig.py --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/cibconfig.py 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/cibconfig.py 2015-10-15 07:18:06.000000000 +0200 @@ -1957,7 +1957,7 @@ return utils.get_check_rc() rc = 0 nl = self.node.findall("fencing-level") - for target in [x.get("target") for x in nl]: + for target in [x.get("target") for x in nl if x.get("target") is not None]: if target.lower() not in [id.lower() for id in cib_factory.node_id_list()]: common_warn("%s: target %s not a node" % (self.obj_id, target)) rc = 1 @@ -2127,13 +2127,25 @@ return obj.obj_type return None + def _is_node(self, nid): + for obj in self.objset.all_set: + if obj.obj_id == nid and obj.obj_type == 'node': + return True + return False + + def _is_resource(self, nid): + for obj in self.objset.all_set: + if obj.obj_id == nid and obj.obj_type != 'node': + return True + return False + def _obj_nodes(self): return oset([n for n in self.objset.obj_ids - if self._obj_type(n) == 'node']) + if self._is_node(n)]) def _obj_resources(self): return oset([n for n in self.objset.obj_ids - if self._obj_type(n) != 'node']) + if self._is_resource(n)]) def _is_edit_valid(self, id_set, existing): ''' diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/report.py new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/report.py --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/report.py 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/report.py 2015-10-15 07:18:06.000000000 +0200 @@ -16,6 +16,7 @@ from .utils import file2str, shortdate, acquire_lock, append_file, ext_cmd, shorttime from .utils import page_string, release_lock, rmdir_r, parse_time, get_cib_attributes from .utils import is_pcmk_118, pipe_cmd_nosudo, file_find_by_name, get_stdout, quote +from .utils import make_datetime_naive, datetime_to_timestamp _HAS_PARALLAX = False try: @@ -67,7 +68,7 @@ if t is None: return None elif isinstance(t, datetime.datetime): - return convert_dt(t) + return datetime_to_timestamp(t) return t @@ -81,13 +82,14 @@ # strptime returns a time_struct tm = time.strptime(' '.join([YEAR] + s.split()[0:3]), "%Y %b %d %H:%M:%S") - return time.mktime(tm) + ts = time.mktime(tm) except: # try the rfc5424 try: - return convert_dt(parse_time(s.split()[0])) + ts = datetime_to_timestamp(parse_time(s.split()[0])) except Exception: common_debug("malformed line: %s" % s) return None + return ts _syslog2node_formats = (re.compile(r'\w+ \d+ \d+:\d+:\d+ (?:\[\d+\])? (\w+)'), @@ -274,18 +276,6 @@ return l -def convert_dt(dt): - """ - Convert a datetime object into a floating-point second value - """ - try: - ts = time.mktime(dt.timetuple()) - ts += dt.microsecond / 1000000.0 - return ts - except: - return None - - class LogSyslog(object): ''' Slice log, search log. @@ -337,8 +327,10 @@ start = log_seek(f, self.from_ts) end = log_seek(f, self.to_ts, to_end=True) if start == -1 or end == -1: + common_debug("%s is a bad log" % (log)) bad_logs.append(log) else: + common_debug("%s start=%s, end=%s" % (log, start, end)) self.startpos[f] = start self.endpos[f] = end for log in bad_logs: @@ -435,7 +427,7 @@ def human_date(dt): 'Some human date representation. Date defaults to now.' if not dt: - dt = datetime.datetime.now() + dt = make_datetime_naive(datetime.datetime.now()) # drop microseconds return re.sub("[.].*", "", "%s %s" % (dt.date(), dt.time())) @@ -595,7 +587,7 @@ self.end_ts = syslog_ts(end_msg) else: common_warn("end of transition %s not found in logs (transition not complete yet?)" % self) - self.end_ts = self.start_ts + self.end_ts = (datetime.datetime(2525, 1, 1) - datetime.datetime(1970, 1, 1)).total_seconds() def __str__(self): return self.get_node_file() diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/scripts.py new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/scripts.py --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/scripts.py 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/scripts.py 2015-10-15 07:18:06.000000000 +0200 @@ -262,27 +262,27 @@ def collect(self): "input: shell command" - self._run.run_command(self._nodes or 'all', self._value) + self._run.run_command(self._nodes or 'all', self._value, True) self._run.record_json() def validate(self): "input: shell command" - self._run.run_command(self._nodes, self._value) + self._run.run_command(None, self._value, True) self._run.validate_json() def apply(self): "input: shell command" - self._run.run_command(self._nodes or 'all', self._value) + self._run.run_command(self._nodes or 'all', self._value, True) self._run.record_json() def apply_local(self): "input: shell command" - self._run.run_command(self._nodes, self._value) + self._run.run_command(None, self._value, True) self._run.record_json() def report(self): "input: shell command" - self._run.run_command(self._nodes, self._value) + self._run.run_command(None, self._value, False) self._run.report_result() def call(self): @@ -1160,7 +1160,7 @@ # TODO: remove common params? # Pass them in a separate list of options? # Right now these names are basically reserved.. -def _common_params(): +def common_params(): "Parameters common to all cluster scripts" return [('nodes', None, 'List of nodes to execute the script for'), ('dry_run', 'no', 'If set, simulate execution only'), @@ -1174,7 +1174,7 @@ def _common_param_default(name): - for param, default, _ in _common_params(): + for param, default, _ in common_params(): if param == name: return default return None @@ -1403,7 +1403,7 @@ # pass as flags to command line def _split_commons(params): - ret, cdict = {}, dict([(c, d) for c, d, _ in _common_params()]) + ret, cdict = {}, dict([(c, d) for c, d, _ in common_params()]) for key, value in params.iteritems(): if key in cdict: cdict[key] = value @@ -1769,15 +1769,15 @@ self.dstfile, self.opts) - def run_command(self, nodes, command): + def run_command(self, nodes, command, is_json_output): "called by Actions" cmdline = 'cd "%s"; ./%s' % (self.workdir, command) if not self._update_state(): raise ValueError("Failed when updating input, aborting.") - self.call(nodes, cmdline) + self.call(nodes, cmdline, is_json_output) def copy_file(self, nodes, src, dst): - if nodes == 'all': + if not self._is_local(nodes): ok = _copy_to_all(self.printer, self.workdir, self.hosts, @@ -1798,7 +1798,7 @@ "called by Actions" if self.result is not None: if not self.result: - self.result = '' + self.result = {} self.data.append(self.result) self.rc = True else: @@ -1851,11 +1851,22 @@ prompt = "sudo password: " self.sudo_pass = getpass.getpass(prompt=prompt) - def call(self, nodes, cmdline): + def _is_local(self, nodes): + islocal = False if nodes == 'all': - self.result = self._process_remote(cmdline) + pass + elif nodes is not None and nodes != []: + islocal = nodes == [self.local_node_name()] else: - self.result = self._process_local(cmdline) + islocal = True + self.printer.debug("is_local (%s): %s" % (nodes, islocal)) + return islocal + + def call(self, nodes, cmdline, is_json_output=False): + if not self._is_local(nodes): + self.result = self._process_remote(cmdline, is_json_output) + else: + self.result = self._process_local(cmdline, is_json_output) self.rc = self.result not in (False, None) def execute_shell(self, nodes, cmdscript): @@ -1872,7 +1883,7 @@ tmpf = self.str2tmp(cmdscript) _chmodx(tmpf) - if nodes == 'all': + if not self._is_local(nodes): ok = _copy_to_remote_dirs(self.printer, self.hosts, tmpf, @@ -1881,10 +1892,10 @@ self.result = False else: cmdline = 'cd "%s"; %s' % (self.workdir, tmpf) - self.result = self._process_remote(cmdline) + self.result = self._process_remote(cmdline, False) else: cmdline = 'cd "%s"; %s' % (self.workdir, tmpf) - self.result = self._process_local(cmdline) + self.result = self._process_local(cmdline, False) self.rc = self.result not in (None, False) def str2tmp(self, s): @@ -1908,7 +1919,7 @@ return return fn - def _process_remote(self, cmdline): + def _process_remote(self, cmdline, is_json_output): """ Handle an action that executes on all nodes """ @@ -1938,20 +1949,24 @@ if rc != 0: self.printer.error(host, "Remote error (rc=%s) %s%s" % (rc, out, err)) ok = False - else: + elif is_json_output: action_result[host] = json.loads(out) + else: + action_result[host] = out if self.local_node: - ret = self._process_local(cmdline) + ret = self._process_local(cmdline, False) if ret is None: ok = False - else: + elif is_json_output: action_result[self.local_node_name()] = json.loads(ret) + else: + action_result[self.local_node_name()] = ret if ok: self.printer.debug("Result: %s" % repr(action_result)) return action_result return None - def _process_local(self, cmdline): + def _process_local(self, cmdline, is_json_output): """ Handle an action that executes locally """ @@ -1968,7 +1983,9 @@ if rc != 0: self.printer.error(self.local_node_name(), "Error (%d): %s" % (rc, err)) return None - self.printer.debug("%s" % repr(out)) + self.printer.debug("Result(local): %s" % repr(out)) + if is_json_output: + out = json.loads(out) return out def local_node_name(self): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/ui_history.py new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/ui_history.py --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/ui_history.py 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/ui_history.py 2015-10-15 07:18:06.000000000 +0200 @@ -18,7 +18,7 @@ from . import options from .cibconfig import mkset_obj, cib_factory from .msg import common_err, common_debug, common_info -from .msg import syntax_err, bad_usage +from .msg import syntax_err from . import report from . import cmd_status diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/ui_script.py new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/ui_script.py --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/ui_script.py 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/ui_script.py 2015-10-15 07:18:06.000000000 +0200 @@ -138,7 +138,6 @@ ret += describe_param(p, _scoped_name(context, p['name']), all) for i, step in enumerate(s.get('steps', [])): ret += describe_step(icontext + [i], context, step, all) - ret += '\n' return ret @@ -238,13 +237,21 @@ 'shortdesc': str(script['shortdesc']), 'longdesc': scripts.format_desc(script['longdesc']), 'steps': "\n".join((describe_step([i], [], s, all) for i, s in enumerate(script['steps'])))} - print("""%(name)s (%(category)s) + output = """%(name)s (%(category)s) %(shortdesc)s %(longdesc)s %(steps)s -""" % vals) +""" % vals + if all: + output += "Common Parameters\n\n" + for name, defval, desc in scripts.common_params(): + output += " %s\n" % (name) + output += " %s\n" % (desc) + if defval is not None: + output += " (default: %s)\n" % (defval) + utils.page_string(output) @command.completers(compl.call(scripts.list_scripts)) def do_verify(self, context, name, *args): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/utils.py new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/utils.py --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/modules/utils.py 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/modules/utils.py 2015-10-15 07:18:06.000000000 +0200 @@ -1013,11 +1013,45 @@ return [x for x in cl if x] +def datetime_is_aware(dt): + """ + Determines if a given datetime.datetime is aware. + + The logic is described in Python's docs: + http://docs.python.org/library/datetime.html#datetime.tzinfo + """ + return dt and dt.tzinfo is not None and dt.tzinfo.utcoffset(dt) is not None + + +def make_datetime_naive(dt): + """ + Ensures that the datetime is + not time zone-aware + """ + if dt and datetime_is_aware(dt): + return dt.replace(tzinfo=None) - dt.utcoffset() + return dt + + +def datetime_to_timestamp(dt): + """ + Convert a datetime object into a floating-point second value + """ + try: + return (make_datetime_naive(dt) - datetime.datetime(1970, 1, 1)).total_seconds() + except Exception as e: + common_err("datetime_to_timestamp error: %s" % (e)) + return None + + def parse_time(t): ''' Try to make sense of the user provided time spec. Use dateutil if available, otherwise strptime. Return the datetime value. + + Also does time zone elimination by passing the datetime + through a timestamp conversion if necessary ''' try: import dateutil.parser @@ -1032,6 +1066,11 @@ except ValueError, msg: common_err("no dateutil, please provide times as printed by date(1)") return None + if datetime_is_aware(dt): + ts = datetime_to_timestamp(dt) + if ts is None: + return None + dt = datetime.datetime.fromtimestamp(ts) return dt diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/testcases/commit.exp new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/testcases/commit.exp --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/testcases/commit.exp 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/testcases/commit.exp 2015-10-15 07:18:06.000000000 +0200 @@ -27,17 +27,17 @@ .INP: commit .EXT crm_resource --show-metadata ocf:heartbeat:Dummy .INP: rename p3 pp3 -INFO: 21: resource references in colocation:cl1 updated -INFO: 21: resource references in location:l1 updated -INFO: 21: resource references in order:o1 updated +INFO: 21: modified colocation:cl1 from p3 to pp3 +INFO: 21: modified location:l1 from p3 to pp3 +INFO: 21: modified order:o1 from p3 to pp3 .INP: commit .INP: rename pp3 p3 -INFO: 23: resource references in colocation:cl1 updated -INFO: 23: resource references in location:l1 updated -INFO: 23: resource references in order:o1 updated +INFO: 23: modified colocation:cl1 from pp3 to p3 +INFO: 23: modified location:l1 from pp3 to p3 +INFO: 23: modified order:o1 from pp3 to p3 .INP: delete c1 -INFO: 24: resource references in colocation:cl1 updated -INFO: 24: resource references in order:o1 updated +INFO: 24: modified colocation:cl1 from c1 to g1 +INFO: 24: modified order:o1 from c1 to g1 .INP: commit .INP: group g2 d1 d2 .INP: commit diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/testcases/delete.exp new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/testcases/delete.exp --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/testcases/delete.exp 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/testcases/delete.exp 2015-10-15 07:18:06.000000000 +0200 @@ -21,7 +21,7 @@ location d1-pref d1 100: node1 .INP: _test .INP: rename d1 p1 -INFO: 13: resource references in location:d1-pref updated +INFO: 13: modified location:d1-pref from d1 to p1 .INP: show node node1 primitive d2 ocf:pacemaker:Dummy @@ -62,7 +62,7 @@ .INP: primitive d2 ocf:pacemaker:Dummy .INP: _test .INP: group g1 d2 d1 -INFO: 29: resource references in location:d1-pref updated +INFO: 29: modified location:d1-pref from d1 to g1 .INP: delete d2 .INP: show node node1 @@ -76,7 +76,7 @@ location d1-pref g1 100: node1 .INP: _test .INP: delete g1 -INFO: 33: resource references in location:d1-pref updated +INFO: 33: modified location:d1-pref from g1 to d1 .INP: show node node1 primitive d1 ocf:pacemaker:Dummy @@ -94,12 +94,12 @@ .INP: # delete a group which is in a clone .INP: primitive d2 ocf:pacemaker:Dummy .INP: group g1 d2 d1 -INFO: 38: resource references in location:d1-pref updated +INFO: 38: modified location:d1-pref from d1 to g1 .INP: clone c1 g1 -INFO: 39: resource references in location:d1-pref updated +INFO: 39: modified location:d1-pref from g1 to c1 .INP: delete g1 -INFO: 40: resource references in location:d1-pref updated -INFO: 40: resource references in location:d1-pref updated +INFO: 40: modified location:d1-pref from c1 to g1 +INFO: 40: modified location:d1-pref from g1 to d2 .INP: show node node1 primitive d1 ocf:pacemaker:Dummy @@ -112,14 +112,14 @@ location d1-pref d2 100: node1 .INP: _test .INP: group g1 d2 d1 -INFO: 43: resource references in location:d1-pref updated +INFO: 43: modified location:d1-pref from d2 to g1 .INP: clone c1 g1 -INFO: 44: resource references in location:d1-pref updated +INFO: 44: modified location:d1-pref from g1 to c1 .INP: _test .INP: # delete group from a clone (again) .INP: delete g1 -INFO: 47: resource references in location:d1-pref updated -INFO: 47: resource references in location:d1-pref updated +INFO: 47: modified location:d1-pref from c1 to g1 +INFO: 47: modified location:d1-pref from g1 to d2 .INP: show node node1 primitive d1 ocf:pacemaker:Dummy @@ -132,13 +132,13 @@ location d1-pref d2 100: node1 .INP: _test .INP: group g1 d2 d1 -INFO: 50: resource references in location:d1-pref updated +INFO: 50: modified location:d1-pref from d2 to g1 .INP: clone c1 g1 -INFO: 51: resource references in location:d1-pref updated +INFO: 51: modified location:d1-pref from g1 to c1 .INP: # delete primitive and its group and their clone .INP: delete d2 d1 c1 g1 -INFO: 53: resource references in location:d1-pref updated -INFO: 53: resource references in location:d1-pref updated +INFO: 53: modified location:d1-pref from c1 to g1 +INFO: 53: modified location:d1-pref from g1 to d2 INFO: 53: hanging location:d1-pref deleted .INP: show node node1 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/testcases/resource.exp new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/testcases/resource.exp --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/testcases/resource.exp 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/testcases/resource.exp 2015-10-15 07:18:06.000000000 +0200 @@ -130,6 +130,7 @@ .SETENV showobj=p0 .TRY resource param p0 set a0 "1 2 3" .EXT crm_resource -r 'p0' -p 'a0' -v '1 2 3' + Set 'p0' option: id=p0-instance_attributes-a0 set=p0-instance_attributes name=a0=1 2 3 .INP: configure .INP: _regtest on @@ -194,6 +195,7 @@ .TRY resource meta p0 set m0 123 .EXT crm_resource --meta -r 'p0' -p 'm0' -v '123' + Set 'p0' option: id=p0-meta_attributes-m0 set=p0-meta_attributes name=m0=123 .INP: configure .INP: _regtest on diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/testcases/scripts.exp new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/testcases/scripts.exp --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/testcases/scripts.exp 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/testcases/scripts.exp 2015-10-15 07:18:06.000000000 +0200 @@ -245,7 +245,6 @@ Broadcast address - .INP: verify virtual-ip id=foo ip=10.0.0.5 1. Configure cluster resources diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/unittests/test_bugs.py new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/unittests/test_bugs.py --- old/crmsh-2.2.0~rc3+git.1444340345.59850ca/test/unittests/test_bugs.py 2015-10-08 23:44:05.000000000 +0200 +++ new/crmsh-2.2.0~rc3+git.1444854254.fc37f7f/test/unittests/test_bugs.py 2015-10-15 07:18:06.000000000 +0200 @@ -655,6 +655,50 @@ @with_setup(setup_func, teardown_func) +def test_id_collision_breakage_3(): + from crmsh import clidisplay + + obj = cibconfig.mkset_obj() + assert obj is not None + with clidisplay.nopretty(): + original_cib = obj.repr() + print original_cib + + obj = cibconfig.mkset_obj() + assert obj is not None + ok = obj.save("""node node1 +primitive node1 Dummy params fake=something + """) + assert ok + + print "** baseline" + obj = cibconfig.mkset_obj() + assert obj is not None + with clidisplay.nopretty(): + print obj.repr() + + obj = cibconfig.mkset_obj() + assert obj is not None + ok = obj.save("""primitive node1 Dummy params fake=something-else + """, no_remove=True, method='update') + assert ok + + print "** end" + + obj = cibconfig.mkset_obj() + assert obj is not None + ok = obj.save(original_cib, no_remove=False, method='replace') + assert ok + obj = cibconfig.mkset_obj() + with clidisplay.nopretty(): + print "*** ORIGINAL" + print original_cib + print "*** NOW" + print obj.repr() + assert original_cib == obj.repr() + + +@with_setup(setup_func, teardown_func) def test_id_collision_breakage_2(): from crmsh import clidisplay @@ -747,3 +791,22 @@ print "*** NOW" print obj.repr() assert original_cib == obj.repr() + + +@with_setup(setup_func, teardown_func) +def test_bug_110(): + """ + configuring attribute-based fencing-topology + """ + factory.create_object(*"primitive stonith-libvirt stonith:null".split()) + factory.create_object(*"primitive fence-nova stonith:null".split()) + cmd = "fencing_topology attr:OpenStack-role=compute stonith-libvirt,fence-nova".split() + ok = factory.create_object(*cmd) + assert ok + obj = cibconfig.mkset_obj() + assert obj is not None + + for o in obj.obj_set: + if o.node.tag == 'fencing-topology': + assert o.check_sanity() == 0 +
