Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package python-subprocrunner for
openSUSE:Factory checked in at 2021-06-05 23:31:47
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/python-subprocrunner (Old)
and /work/SRC/openSUSE:Factory/.python-subprocrunner.new.1898 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-subprocrunner"
Sat Jun 5 23:31:47 2021 rev:7 rq:897730 version:1.6.0
Changes:
--------
---
/work/SRC/openSUSE:Factory/python-subprocrunner/python-subprocrunner.changes
2021-05-17 18:45:15.356638565 +0200
+++
/work/SRC/openSUSE:Factory/.python-subprocrunner.new.1898/python-subprocrunner.changes
2021-06-05 23:32:17.088516498 +0200
@@ -1,0 +2,27 @@
+Sat Jun 5 13:14:13 UTC 2021 - Martin Hauke <[email protected]>
+
+- Update to version 1.6.0
+ * Add __repr__ method to Retry/SubprocessRunner classes
+ * Fix to properly suppress debug logs when quiet=True
+ * Fixed to save command history at runtime
+
+-------------------------------------------------------------------
+Fri Jun 4 20:34:17 UTC 2021 - Martin Hauke <[email protected]>
+
+- Update to version 1.5.0
+ * Add no_retry_returncodes parameter to Retry class
+- Update to version 1.4.2
+ * Fix retry processing
+ * Modify log messages
+- Update to version 1.4.1
+ * Fix retry processing
+- Update to version 1.4.0
+ * Add quiet mode support for SubprocessRunner/Retry
+ * Modify a retry log message
+- Update to version 1.3.0
+ * Add timeout keyword argument to SubprocessRunner.run:
+ * Add support for retry functionality to SubprocessRunner.run
+ * Add SubprocessRunner.Retry class
+ * Modify type annotation of SubprocessRunner.run return value
+
+-------------------------------------------------------------------
Old:
----
subprocrunner-1.2.2.tar.gz
New:
----
subprocrunner-1.6.0.tar.gz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ python-subprocrunner.spec ++++++
--- /var/tmp/diff_new_pack.ipK9HF/_old 2021-06-05 23:32:17.572517340 +0200
+++ /var/tmp/diff_new_pack.ipK9HF/_new 2021-06-05 23:32:17.576517347 +0200
@@ -19,7 +19,7 @@
%{?!python_module:%define python_module() python-%{**} python3-%{**}}
%define skip_python2 1
Name: python-subprocrunner
-Version: 1.2.2
+Version: 1.6.0
Release: 0
Summary: A Python wrapper library for subprocess module
License: MIT
@@ -37,6 +37,7 @@
# SECTION test requirements
BuildRequires: %{python_module loguru >= 0.4.1}
BuildRequires: %{python_module mbstrdecoder >= 1.0.0}
+BuildRequires: %{python_module pytest-mock}
BuildRequires: %{python_module pytest}
BuildRequires: %{python_module six}
BuildRequires: %{python_module typepy}
++++++ subprocrunner-1.2.2.tar.gz -> subprocrunner-1.6.0.tar.gz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/PKG-INFO
new/subprocrunner-1.6.0/PKG-INFO
--- old/subprocrunner-1.2.2/PKG-INFO 2021-05-05 09:10:08.561095700 +0200
+++ new/subprocrunner-1.6.0/PKG-INFO 2021-06-05 13:14:36.910000000 +0200
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: subprocrunner
-Version: 1.2.2
+Version: 1.6.0
Summary: A Python wrapper library for subprocess module.
Home-page: https://github.com/thombashi/subprocrunner
Author: Tsuyoshi Hombashi
@@ -30,13 +30,13 @@
:target: https://pypi.org/project/subprocrunner
:alt: Supported Python implementations
- .. image::
https://img.shields.io/travis/thombashi/subprocrunner/master.svg?label=Linux/macOS%20CI
- :target: https://travis-ci.org/thombashi/subprocrunner
- :alt: Linux/macOS CI status
-
- .. image::
https://img.shields.io/appveyor/ci/thombashi/subprocrunner/master.svg?label=Windows%20CI
- :target:
https://ci.appveyor.com/project/thombashi/subprocrunner/branch/master
- :alt: Windows CI status
+ .. image::
https://github.com/thombashi/subprocrunner/workflows/Tests/badge.svg
+ :target:
https://github.com/thombashi/subprocrunner/actions/workflows/tests.yml
+ :alt: Test result of Linux/macOS/Windows
+
+ .. image::
https://github.com/thombashi/subprocrunner/actions/workflows/lint.yml/badge.svg
+ :target:
https://github.com/thombashi/subprocrunner/actions/workflows/lint.yml
+ :alt: Lint result
.. image::
https://coveralls.io/repos/github/thombashi/subprocrunner/badge.svg?branch=master
:target:
https://coveralls.io/github/thombashi/subprocrunner?branch=master
@@ -52,45 +52,57 @@
from subprocrunner import SubprocessRunner
- runner = SubprocessRunner("echo test")
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stdout: {:s}".format(runner.stdout))
-
- runner = SubprocessRunner("ls __not_exist_dir__")
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stderr: {:s}".format(runner.stderr))
-
+ runner = SubprocessRunner(["echo", "test"])
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stdout: {runner.stdout}")
+
+ runner = SubprocessRunner(["ls", "__not_exist_dir__"])
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stderr: {runner.stderr}")
+
:Output:
.. code::
- command: echo test
+ SubprocessRunner(command='echo test', returncode='not yet
executed')
return code: 0
stdout: test
-
- command: ls __not_exist_dir__
+
+ SubprocessRunner(command='ls __not_exist_dir__',
returncode='not yet executed')
return code: 2
stderr: ls: cannot access '__not_exist_dir__': No such file or
directory
+ Execute a command with retry
+ --------------------------------------------------------
+
+ :Sample Code:
+ .. code:: python
+
+ from subprocrunner import Retry, SubprocessRunner
+
+ SubprocessRunner(command).run(retry=Retry(total=3,
backoff_factor=0.2, jitter=0.2))
+
dry run
----------------------------
+ Commands are not actually run when passing ``dry_run=True`` to
``SubprocessRunner`` class constructor.
+
:Sample Code:
.. code:: python
from subprocrunner import SubprocessRunner
runner = SubprocessRunner("echo test", dry_run=True)
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stdout: {:s}".format(runner.stdout))
-
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stdout: {runner.stdout}")
+
:Output:
.. code::
- command: echo test
+ SubprocessRunner(command='echo test', returncode='not yet
executed', dryrun=True)
return code: 0
- stdout:
+ stdout:
Get execution command history
--------------------------------------------------------
@@ -101,10 +113,10 @@
SubprocessRunner.clear_history()
SubprocessRunner.is_save_history = True
-
- SubprocessRunner("echo hoge").run()
- SubprocessRunner("echo foo").run()
-
+
+ SubprocessRunner(["echo", "hoge"]).run()
+ SubprocessRunner(["echo", "foo"]).run()
+
print("\n".join(SubprocessRunner.get_history()))
:Output:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/README.rst
new/subprocrunner-1.6.0/README.rst
--- old/subprocrunner-1.2.2/README.rst 2021-03-20 05:14:16.000000000 +0100
+++ new/subprocrunner-1.6.0/README.rst 2021-06-05 13:14:22.000000000 +0200
@@ -20,13 +20,13 @@
:target: https://pypi.org/project/subprocrunner
:alt: Supported Python implementations
-.. image::
https://img.shields.io/travis/thombashi/subprocrunner/master.svg?label=Linux/macOS%20CI
- :target: https://travis-ci.org/thombashi/subprocrunner
- :alt: Linux/macOS CI status
-
-.. image::
https://img.shields.io/appveyor/ci/thombashi/subprocrunner/master.svg?label=Windows%20CI
- :target:
https://ci.appveyor.com/project/thombashi/subprocrunner/branch/master
- :alt: Windows CI status
+.. image:: https://github.com/thombashi/subprocrunner/workflows/Tests/badge.svg
+ :target:
https://github.com/thombashi/subprocrunner/actions/workflows/tests.yml
+ :alt: Test result of Linux/macOS/Windows
+
+.. image::
https://github.com/thombashi/subprocrunner/actions/workflows/lint.yml/badge.svg
+ :target:
https://github.com/thombashi/subprocrunner/actions/workflows/lint.yml
+ :alt: Lint result
.. image::
https://coveralls.io/repos/github/thombashi/subprocrunner/badge.svg?branch=master
:target: https://coveralls.io/github/thombashi/subprocrunner?branch=master
@@ -42,45 +42,57 @@
from subprocrunner import SubprocessRunner
- runner = SubprocessRunner("echo test")
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stdout: {:s}".format(runner.stdout))
-
- runner = SubprocessRunner("ls __not_exist_dir__")
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stderr: {:s}".format(runner.stderr))
-
+ runner = SubprocessRunner(["echo", "test"])
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stdout: {runner.stdout}")
+
+ runner = SubprocessRunner(["ls", "__not_exist_dir__"])
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stderr: {runner.stderr}")
+
:Output:
.. code::
- command: echo test
+ SubprocessRunner(command='echo test', returncode='not yet executed')
return code: 0
stdout: test
-
- command: ls __not_exist_dir__
+
+ SubprocessRunner(command='ls __not_exist_dir__', returncode='not yet
executed')
return code: 2
stderr: ls: cannot access '__not_exist_dir__': No such file or
directory
+Execute a command with retry
+--------------------------------------------------------
+
+:Sample Code:
+ .. code:: python
+
+ from subprocrunner import Retry, SubprocessRunner
+
+ SubprocessRunner(command).run(retry=Retry(total=3, backoff_factor=0.2,
jitter=0.2))
+
dry run
----------------------------
+Commands are not actually run when passing ``dry_run=True`` to
``SubprocessRunner`` class constructor.
+
:Sample Code:
.. code:: python
from subprocrunner import SubprocessRunner
runner = SubprocessRunner("echo test", dry_run=True)
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stdout: {:s}".format(runner.stdout))
-
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stdout: {runner.stdout}")
+
:Output:
.. code::
- command: echo test
+ SubprocessRunner(command='echo test', returncode='not yet executed',
dryrun=True)
return code: 0
- stdout:
+ stdout:
Get execution command history
--------------------------------------------------------
@@ -91,10 +103,10 @@
SubprocessRunner.clear_history()
SubprocessRunner.is_save_history = True
-
- SubprocessRunner("echo hoge").run()
- SubprocessRunner("echo foo").run()
-
+
+ SubprocessRunner(["echo", "hoge"]).run()
+ SubprocessRunner(["echo", "foo"]).run()
+
print("\n".join(SubprocessRunner.get_history()))
:Output:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/subprocrunner-1.2.2/requirements/test_requirements.txt
new/subprocrunner-1.6.0/requirements/test_requirements.txt
--- old/subprocrunner-1.2.2/requirements/test_requirements.txt 2021-03-20
05:14:16.000000000 +0100
+++ new/subprocrunner-1.6.0/requirements/test_requirements.txt 2021-06-05
13:14:22.000000000 +0200
@@ -1,2 +1,3 @@
pytest
+pytest-mock
typepy
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/subprocrunner/__init__.py
new/subprocrunner-1.6.0/subprocrunner/__init__.py
--- old/subprocrunner-1.2.2/subprocrunner/__init__.py 2021-03-20
05:14:16.000000000 +0100
+++ new/subprocrunner-1.6.0/subprocrunner/__init__.py 2021-06-05
13:14:22.000000000 +0200
@@ -8,3 +8,4 @@
from ._subprocess_runner import SubprocessRunner
from ._which import Which
from .error import CalledProcessError, CommandError
+from .retry import Retry
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/subprocrunner/__version__.py
new/subprocrunner-1.6.0/subprocrunner/__version__.py
--- old/subprocrunner-1.2.2/subprocrunner/__version__.py 2021-05-05
08:33:26.000000000 +0200
+++ new/subprocrunner-1.6.0/subprocrunner/__version__.py 2021-06-05
13:14:22.000000000 +0200
@@ -1,6 +1,6 @@
__author__ = "Tsuyoshi Hombashi"
__copyright__ = "Copyright 2016, {}".format(__author__)
__license__ = "MIT License"
-__version__ = "1.2.2"
+__version__ = "1.6.0"
__maintainer__ = __author__
__email__ = "[email protected]"
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/subprocrunner/_logger/_logger.py
new/subprocrunner-1.6.0/subprocrunner/_logger/_logger.py
--- old/subprocrunner-1.2.2/subprocrunner/_logger/_logger.py 2021-05-05
07:06:17.000000000 +0200
+++ new/subprocrunner-1.6.0/subprocrunner/_logger/_logger.py 2021-06-05
13:14:22.000000000 +0200
@@ -40,6 +40,7 @@
"CRITICAL": logger.critical,
}
+ log_level = log_level.strip().upper()
method = method_table.get(log_level)
if method is None:
raise ValueError("unknown log level: {}".format(log_level))
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/subprocrunner-1.2.2/subprocrunner/_subprocess_runner.py
new/subprocrunner-1.6.0/subprocrunner/_subprocess_runner.py
--- old/subprocrunner-1.2.2/subprocrunner/_subprocess_runner.py 2021-05-05
07:59:09.000000000 +0200
+++ new/subprocrunner-1.6.0/subprocrunner/_subprocess_runner.py 2021-06-05
13:14:22.000000000 +0200
@@ -9,13 +9,14 @@
import subprocess
import traceback
from subprocess import PIPE
-from typing import Dict, List, Optional, Pattern, Sequence, Union, cast
+from typing import Dict, List, Optional, Pattern, Sequence, Union, cast # noqa
from mbstrdecoder import MultiByteStrDecoder
from ._logger import DEFAULT_ERROR_LOG_LEVEL, get_logging_method
from ._which import Which
from .error import CalledProcessError, CommandError
+from .retry import Retry
from .typing import Command
@@ -32,6 +33,7 @@
"""
_DRY_RUN_OUTPUT = ""
+ _RETRY_ATTEMPT_KEY = "__retry_attempt__"
default_error_log_level = DEFAULT_ERROR_LOG_LEVEL
default_is_dry_run = False
@@ -57,6 +59,7 @@
error_log_level: Optional[str] = None,
ignore_stderr_regexp: Optional[Pattern] = None,
dry_run: Optional[bool] = None,
+ quiet: bool = False,
) -> None:
self.__command = [] # type: Union[str, Sequence[str]]
@@ -74,23 +77,33 @@
self.__dry_run = dry_run
else:
self.__dry_run = self.default_is_dry_run
- self.__stdout = None # type: Union[str, bytes, None]
- self.__stderr = None # type: Union[str, bytes, None]
- self.__returncode = None
+ self.__stdout = None # type: Optional[str]
+ self.__stderr = None # type: Optional[str]
+ self.__returncode = None # type: Optional[int]
self.__ignore_stderr_regexp = ignore_stderr_regexp
- self.__debug_logging_method = get_logging_method()
+ self.__debug_logging_method = get_logging_method("QUIET" if quiet else
"DEBUG")
- if error_log_level is not None:
+ if quiet:
+ self.error_log_level = "QUIET"
+ elif error_log_level is not None:
self.error_log_level = error_log_level
else:
self.error_log_level = self.default_error_log_level
- if self.is_save_history:
- if len(self.__command_history) >= self.history_size:
- self.__command_history.pop(0)
+ self.__quiet = quiet
- self.__command_history.append(command)
+ def __repr__(self) -> str:
+ params = [
+ "command='{}'".format(self.command_str),
+ "returncode={}".format(
+ self.returncode if self.returncode is not None else "'not yet
executed'"
+ ),
+ ]
+ if self.dry_run:
+ params.append("dry_run={}".format(self.dry_run))
+
+ return "SubprocessRunner({})".format(", ".join(params))
@property
def dry_run(self) -> bool:
@@ -108,11 +121,11 @@
return " ".join(self.__command)
@property
- def stdout(self) -> Union[str, bytes, None]:
+ def stdout(self) -> Optional[str]:
return self.__stdout
@property
- def stderr(self) -> Union[str, bytes, None]:
+ def stderr(self) -> Optional[str]:
return self.__stderr
@property
@@ -127,28 +140,18 @@
def error_log_level(self, log_level: Optional[str]):
self.__error_logging_method = get_logging_method(log_level)
- def run(self, **kwargs) -> Optional[int]:
- self.__verify_command()
-
- check = kwargs.pop("check", None)
- env = kwargs.pop("env", None)
+ def _run(self, env, check: bool, timeout: Optional[float] = None,
**kwargs) -> int:
+ self.__save_command()
+
self.__debug_print_command(retry_attept=kwargs.get(self._RETRY_ATTEMPT_KEY))
- if self.dry_run:
- self.__stdout = self._DRY_RUN_OUTPUT
- self.__stderr = self._DRY_RUN_OUTPUT
- self.__returncode = 0
-
- self.__debug_logging_method("dry-run: {}".format(self.command))
-
- return self.__returncode
-
- self.__debug_print_command()
+ if self._RETRY_ATTEMPT_KEY in kwargs:
+ kwargs.pop(self._RETRY_ATTEMPT_KEY)
try:
proc = subprocess.Popen(
self.command,
shell=self.__is_shell,
- env=self.__get_env(env),
+ env=env,
stdin=PIPE,
stdout=PIPE,
stderr=PIPE,
@@ -158,11 +161,11 @@
self.command, shell=self.__is_shell, stdin=PIPE, stdout=PIPE,
stderr=PIPE
)
- self.__stdout, self.__stderr = proc.communicate(**kwargs)
+ stdout, stderr = proc.communicate(timeout=timeout, **kwargs)
self.__returncode = proc.returncode
- self.__stdout = MultiByteStrDecoder(self.__stdout).unicode_str
- self.__stderr = MultiByteStrDecoder(self.__stderr).unicode_str
+ self.__stdout = MultiByteStrDecoder(stdout).unicode_str
+ self.__stderr = MultiByteStrDecoder(stderr).unicode_str
if self.returncode == 0:
return 0
@@ -172,45 +175,83 @@
self.__ignore_stderr_regexp
and self.__ignore_stderr_regexp.search(self.stderr) is not None
):
- return self.returncode
+ return self.__returncode
except AttributeError:
pass
+ self.__error_logging_method(
+ "command='{}', returncode={}, stderr={!r}".format(
+ self.command_str, self.returncode, self.stderr
+ )
+ )
+
if check is True:
- # stdout and stderr attributes added since Python 3.5
raise CalledProcessError(
- returncode=self.returncode,
+ returncode=self.__returncode,
cmd=self.command_str,
output=self.stdout,
stderr=self.stderr,
)
- # pytype: disable=attribute-error
- self.__error_logging_method(
- "command='{}', returncode={}, stderr={!r}".format(
- self.command, self.returncode, self.stderr
- )
+ return self.__returncode
+
+ def run(self, timeout: Optional[float] = None, retry: Retry = None,
**kwargs) -> int:
+ self.__verify_command()
+
+ check = kwargs.pop("check", False)
+ env = self.__get_env(kwargs.pop("env", None))
+
+ if self.dry_run:
+ self.__stdout = self._DRY_RUN_OUTPUT
+ self.__stderr = self._DRY_RUN_OUTPUT
+ self.__returncode = 0
+
+ self.__save_command()
+ self.__debug_print_command()
+
+ return self.__returncode
+
+ returncode = self._run(
+ env=env, check=check if retry is None else False, timeout=timeout,
**kwargs
)
- # pytype: enable=attribute-error
+ if retry is None or returncode in [0] + retry.no_retry_returncodes:
+ return returncode
+
+ for i in range(retry.total):
+ retry.sleep_before_retry(
+ attempt=i + 1,
+ logging_method=self.__debug_logging_method,
+ retry_target=self.command_str,
+ )
+ kwargs[self._RETRY_ATTEMPT_KEY] = i + 1
+
+ returncode = self._run(env=env, check=False, timeout=timeout,
**kwargs)
+ if returncode in [0] + retry.no_retry_returncodes:
+ return returncode
+
+ if check is True:
+ raise CalledProcessError(
+ returncode=self.__returncode, # type: ignore
+ cmd=self.command_str,
+ output=self.stdout,
+ stderr=self.stderr,
+ )
- return self.returncode
+ return self.__returncode # type: ignore
def popen(self, std_in: Optional[int] = None, env: Optional[Dict[str,
str]] = None):
self.__verify_command()
+ self.__debug_print_command()
if self.dry_run:
self.__stdout = self._DRY_RUN_OUTPUT
self.__stderr = self._DRY_RUN_OUTPUT
self.__returncode = 0
- self.__debug_logging_method("dry-run: {}".format(self.command))
-
return subprocess.CompletedProcess(
args=[], returncode=self.__returncode, stdout=self.__stdout,
stderr=self.__stderr
)
- self.__debug_print_command()
-
try:
process = subprocess.Popen(
self.command,
@@ -243,6 +284,15 @@
Which(base_command).verify()
+ def __save_command(self):
+ if not self.is_save_history:
+ return
+
+ if len(self.__command_history) >= self.history_size:
+ self.__command_history.pop(0)
+
+ self.__command_history.append(self.command_str)
+
@staticmethod
def __get_env(env=None):
if env is not None:
@@ -253,8 +303,19 @@
return os.environ
- def __debug_print_command(self) -> None:
- message_list = [self.command_str]
+ def __debug_print_command(self, retry_attept: Optional[int] = None) ->
None:
+ if self.__quiet:
+ return
+
+ message_list = []
+
+ if self.dry_run:
+ message_list.append("dryrun: ")
+
+ if retry_attept is not None:
+ message_list.append("retry-attempt={}: {}".format(retry_attept,
self.command_str))
+ else:
+ message_list.append(self.command_str)
if self.is_output_stacktrace:
message_list.append("".join(traceback.format_stack()[:-2]))
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/subprocrunner/error.py
new/subprocrunner-1.6.0/subprocrunner/error.py
--- old/subprocrunner-1.2.2/subprocrunner/error.py 2021-05-05
07:59:36.000000000 +0200
+++ new/subprocrunner-1.6.0/subprocrunner/error.py 2021-06-05
13:14:22.000000000 +0200
@@ -3,8 +3,8 @@
"""
-import subprocess
-import sys
+# keep the following line for backward compatibility
+from subprocess import CalledProcessError # noqa
from typing import Optional
from .typing import Command
@@ -24,13 +24,3 @@
self.__errno = kwargs.pop("errno", None)
super().__init__(*args)
-
-
-class CalledProcessError(subprocess.CalledProcessError):
- def __init__(self, *args, **kwargs) -> None:
- if sys.version_info[0:2] <= (3, 4):
- # stdout and stderr attribute added to
subprocess.CalledProcessError since Python 3.5
- self.stdout = kwargs.pop("stdout", None)
- self.stderr = kwargs.pop("stderr", None)
-
- super().__init__(*args, **kwargs)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/subprocrunner/retry.py
new/subprocrunner-1.6.0/subprocrunner/retry.py
--- old/subprocrunner-1.2.2/subprocrunner/retry.py 1970-01-01
01:00:00.000000000 +0100
+++ new/subprocrunner-1.6.0/subprocrunner/retry.py 2021-06-05
13:14:22.000000000 +0200
@@ -0,0 +1,72 @@
+import time
+from random import uniform
+from typing import Callable, List, Optional
+
+
+class Retry:
+ def __init__(
+ self,
+ total: int = 3,
+ backoff_factor: float = 0.2,
+ jitter: float = 0.2,
+ no_retry_returncodes: Optional[List[int]] = None,
+ quiet: bool = False,
+ ) -> None:
+ self.total = total
+ self.__backoff_factor = backoff_factor
+ self.__jitter = jitter
+ self.__quiet = quiet
+
+ if self.total <= 0:
+ raise ValueError("total must be greater than zero")
+
+ if self.__backoff_factor <= 0:
+ raise ValueError("backoff_factor must be greater than zero")
+
+ if self.__jitter <= 0:
+ raise ValueError("jitter must be greater than zero")
+
+ if no_retry_returncodes:
+ self.no_retry_returncodes = no_retry_returncodes
+ else:
+ self.no_retry_returncodes = []
+
+ def __repr__(self) -> str:
+ msgs = [
+ "total={}".format(self.total),
+ "backoff-factor={}".format(self.__backoff_factor),
+ "jitter={}".format(self.__jitter),
+ ]
+
+ if self.no_retry_returncodes:
+
msgs.append("no-retry-returncodes={}".format(self.no_retry_returncodes))
+
+ return "Retry({})".format(", ".join(msgs))
+
+ def calc_backoff_time(self, attempt: int) -> float:
+ sleep_duration = self.__backoff_factor * (2 ** max(0, attempt - 1))
+ sleep_duration += uniform(0.5 * self.__jitter, 1.5 * self.__jitter)
+
+ return sleep_duration
+
+ def sleep_before_retry(
+ self,
+ attempt: int,
+ logging_method: Optional[Callable] = None,
+ retry_target: Optional[str] = None,
+ ) -> float:
+ sleep_duration = self.calc_backoff_time(attempt)
+
+ if logging_method and not self.__quiet:
+ if retry_target:
+ msg = "Retrying '{}' in ".format(retry_target)
+ else:
+ msg = "Retrying in "
+
+ msg += "{:.2f} seconds ... (attempt={}/{})".format(sleep_duration,
attempt, self.total)
+
+ logging_method(msg)
+
+ time.sleep(sleep_duration)
+
+ return sleep_duration
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/subprocrunner.egg-info/PKG-INFO
new/subprocrunner-1.6.0/subprocrunner.egg-info/PKG-INFO
--- old/subprocrunner-1.2.2/subprocrunner.egg-info/PKG-INFO 2021-05-05
09:10:08.000000000 +0200
+++ new/subprocrunner-1.6.0/subprocrunner.egg-info/PKG-INFO 2021-06-05
13:14:36.000000000 +0200
@@ -1,6 +1,6 @@
Metadata-Version: 2.1
Name: subprocrunner
-Version: 1.2.2
+Version: 1.6.0
Summary: A Python wrapper library for subprocess module.
Home-page: https://github.com/thombashi/subprocrunner
Author: Tsuyoshi Hombashi
@@ -30,13 +30,13 @@
:target: https://pypi.org/project/subprocrunner
:alt: Supported Python implementations
- .. image::
https://img.shields.io/travis/thombashi/subprocrunner/master.svg?label=Linux/macOS%20CI
- :target: https://travis-ci.org/thombashi/subprocrunner
- :alt: Linux/macOS CI status
-
- .. image::
https://img.shields.io/appveyor/ci/thombashi/subprocrunner/master.svg?label=Windows%20CI
- :target:
https://ci.appveyor.com/project/thombashi/subprocrunner/branch/master
- :alt: Windows CI status
+ .. image::
https://github.com/thombashi/subprocrunner/workflows/Tests/badge.svg
+ :target:
https://github.com/thombashi/subprocrunner/actions/workflows/tests.yml
+ :alt: Test result of Linux/macOS/Windows
+
+ .. image::
https://github.com/thombashi/subprocrunner/actions/workflows/lint.yml/badge.svg
+ :target:
https://github.com/thombashi/subprocrunner/actions/workflows/lint.yml
+ :alt: Lint result
.. image::
https://coveralls.io/repos/github/thombashi/subprocrunner/badge.svg?branch=master
:target:
https://coveralls.io/github/thombashi/subprocrunner?branch=master
@@ -52,45 +52,57 @@
from subprocrunner import SubprocessRunner
- runner = SubprocessRunner("echo test")
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stdout: {:s}".format(runner.stdout))
-
- runner = SubprocessRunner("ls __not_exist_dir__")
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stderr: {:s}".format(runner.stderr))
-
+ runner = SubprocessRunner(["echo", "test"])
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stdout: {runner.stdout}")
+
+ runner = SubprocessRunner(["ls", "__not_exist_dir__"])
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stderr: {runner.stderr}")
+
:Output:
.. code::
- command: echo test
+ SubprocessRunner(command='echo test', returncode='not yet
executed')
return code: 0
stdout: test
-
- command: ls __not_exist_dir__
+
+ SubprocessRunner(command='ls __not_exist_dir__',
returncode='not yet executed')
return code: 2
stderr: ls: cannot access '__not_exist_dir__': No such file or
directory
+ Execute a command with retry
+ --------------------------------------------------------
+
+ :Sample Code:
+ .. code:: python
+
+ from subprocrunner import Retry, SubprocessRunner
+
+ SubprocessRunner(command).run(retry=Retry(total=3,
backoff_factor=0.2, jitter=0.2))
+
dry run
----------------------------
+ Commands are not actually run when passing ``dry_run=True`` to
``SubprocessRunner`` class constructor.
+
:Sample Code:
.. code:: python
from subprocrunner import SubprocessRunner
runner = SubprocessRunner("echo test", dry_run=True)
- print("command: {:s}".format(runner.command))
- print("return code: {:d}".format(runner.run()))
- print("stdout: {:s}".format(runner.stdout))
-
+ print(runner)
+ print(f"return code: {runner.run()}")
+ print(f"stdout: {runner.stdout}")
+
:Output:
.. code::
- command: echo test
+ SubprocessRunner(command='echo test', returncode='not yet
executed', dryrun=True)
return code: 0
- stdout:
+ stdout:
Get execution command history
--------------------------------------------------------
@@ -101,10 +113,10 @@
SubprocessRunner.clear_history()
SubprocessRunner.is_save_history = True
-
- SubprocessRunner("echo hoge").run()
- SubprocessRunner("echo foo").run()
-
+
+ SubprocessRunner(["echo", "hoge"]).run()
+ SubprocessRunner(["echo", "foo"]).run()
+
print("\n".join(SubprocessRunner.get_history()))
:Output:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/subprocrunner-1.2.2/subprocrunner.egg-info/SOURCES.txt
new/subprocrunner-1.6.0/subprocrunner.egg-info/SOURCES.txt
--- old/subprocrunner-1.2.2/subprocrunner.egg-info/SOURCES.txt 2021-05-05
09:10:08.000000000 +0200
+++ new/subprocrunner-1.6.0/subprocrunner.egg-info/SOURCES.txt 2021-06-05
13:14:36.000000000 +0200
@@ -12,6 +12,7 @@
subprocrunner/_which.py
subprocrunner/error.py
subprocrunner/py.typed
+subprocrunner/retry.py
subprocrunner/typing.py
subprocrunner.egg-info/PKG-INFO
subprocrunner.egg-info/SOURCES.txt
@@ -22,5 +23,6 @@
subprocrunner/_logger/_logger.py
subprocrunner/_logger/_null_logger.py
test/test_logger.py
+test/test_retry.py
test/test_subproc_runner.py
test/test_which.py
\ No newline at end of file
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore'
old/subprocrunner-1.2.2/subprocrunner.egg-info/requires.txt
new/subprocrunner-1.6.0/subprocrunner.egg-info/requires.txt
--- old/subprocrunner-1.2.2/subprocrunner.egg-info/requires.txt 2021-05-05
09:10:08.000000000 +0200
+++ new/subprocrunner-1.6.0/subprocrunner.egg-info/requires.txt 2021-06-05
13:14:36.000000000 +0200
@@ -5,5 +5,6 @@
[test]
pytest
+pytest-mock
typepy
loguru<1,>=0.4.1
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/test/test_retry.py
new/subprocrunner-1.6.0/test/test_retry.py
--- old/subprocrunner-1.2.2/test/test_retry.py 1970-01-01 01:00:00.000000000
+0100
+++ new/subprocrunner-1.6.0/test/test_retry.py 2021-06-05 13:14:22.000000000
+0200
@@ -0,0 +1,54 @@
+import pytest
+
+from subprocrunner.retry import Retry
+
+
+class Test_Retry_repr:
+ def test_normal(self):
+ assert (
+ str(Retry(backoff_factor=0.5, jitter=0.5))
+ == "Retry(total=3, backoff-factor=0.5, jitter=0.5)"
+ )
+
+
+class Test_Retry_calc_backoff_time:
+ @pytest.mark.parametrize(
+ ["attempt"],
+ [
+ [0],
+ [1],
+ [2],
+ [3],
+ [4],
+ ],
+ )
+ def test_normal(self, attempt):
+ LOOP = 100
+ coef = 2 ** max(0, attempt - 1)
+ backoff_factor = 1
+ jitter = 0.5
+ base_time = backoff_factor * coef
+ retry = Retry(backoff_factor=backoff_factor, jitter=jitter)
+
+ for _i in range(LOOP):
+ assert (
+ (base_time - jitter * 0.5)
+ <= retry.calc_backoff_time(attempt=attempt)
+ <= (base_time + jitter * 1.5)
+ )
+
+
+class Test_Retry_sleep_before_retry:
+ def test_normal(self):
+ attempt = 1
+ coef = 2 ** max(0, attempt - 1)
+ backoff_factor = 0.1
+ jitter = 0.1
+ base_time = backoff_factor * coef
+ retry = Retry(backoff_factor=backoff_factor, jitter=jitter)
+
+ assert (
+ (base_time - jitter * 0.5)
+ <= retry.sleep_before_retry(attempt=attempt)
+ <= (base_time + jitter * 1.5)
+ )
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/test/test_subproc_runner.py
new/subprocrunner-1.6.0/test/test_subproc_runner.py
--- old/subprocrunner-1.2.2/test/test_subproc_runner.py 2021-05-05
07:06:24.000000000 +0200
+++ new/subprocrunner-1.6.0/test/test_subproc_runner.py 2021-06-05
13:14:22.000000000 +0200
@@ -9,13 +9,15 @@
import re
import subprocess
import sys
-from subprocess import PIPE, CalledProcessError
+from subprocess import PIPE
import pytest
from typepy import is_not_null_string, is_null_string
from subprocrunner import SubprocessRunner
from subprocrunner._logger._null_logger import NullLogger
+from subprocrunner.error import CalledProcessError
+from subprocrunner.retry import Retry
os_type = platform.system()
@@ -32,6 +34,30 @@
raise NotImplementedError(os_type)
+BACKOFF_FACTOR = 0.01
+JITTER = 0.01
+
+
+class Test_SubprocessRunner_repr:
+ @pytest.mark.parametrize(
+ ["command", "dry_run", "expected"],
+ [
+ [
+ ["ls", "hoge"],
+ False,
+ "SubprocessRunner(command='ls hoge', returncode='not yet
executed')",
+ ],
+ [
+ ["ls", "hoge"],
+ True,
+ "SubprocessRunner(command='ls hoge', returncode='not yet
executed', dry_run=True)",
+ ],
+ ],
+ )
+ def test_normal(self, command, dry_run, expected):
+ assert str(SubprocessRunner(command=["ls", "hoge"], dry_run=dry_run))
== expected
+
+
class Test_SubprocessRunner_run:
@pytest.mark.parametrize(
["command", "dry_run", "expected"],
@@ -62,7 +88,11 @@
assert SubprocessRunner(command).run() == expected
@pytest.mark.parametrize(
- ["command", "expected"], [["echo test", "test"], [["echo", "test"],
"test"]]
+ ["command", "expected"],
+ [
+ ["echo test", "test"],
+ [["echo", "test"], "test"],
+ ],
)
def test_stdout(self, command, expected):
runner = SubprocessRunner(command)
@@ -130,8 +160,18 @@
with pytest.raises(expected):
runner.run(check=True)
+ def test_timeout_kwarg(self, mocker):
+ mocked_communicate = mocker.patch("subprocess.Popen.communicate")
+ mocked_communicate.return_value = ("", "")
+
+ mocker.patch("subprocrunner.Which.verify")
+ runner = SubprocessRunner("dummy")
+ runner.run(timeout=1)
+
+ mocked_communicate.assert_called_with(timeout=1)
+
def test_unicode(self, monkeypatch):
- def monkey_communicate(input=None):
+ def monkey_communicate(input=None, timeout=None):
return ("", "'dummy'
??????????????????????????????????????????????????????"
"????????????????????????????????????????????????
????????????????????????????????????????????????")
monkeypatch.setattr(subprocess.Popen, "communicate",
monkey_communicate)
@@ -139,6 +179,81 @@
runner = SubprocessRunner(list_command)
runner.run()
+ def test_retry(self, mocker):
+ mocker.patch("subprocrunner.Which.verify")
+
+ runner = SubprocessRunner("always-failed-command")
+ retry_ct = 3
+
+ # w/ retry: check=False
+ mocked_run = mocker.patch("subprocrunner.SubprocessRunner._run")
+ mocked_run.return_value = 1
+ runner.run(
+ check=False,
+ retry=Retry(total=retry_ct, backoff_factor=BACKOFF_FACTOR,
jitter=JITTER),
+ )
+ assert mocked_run.call_count == retry_ct + 1
+
+ # w/ retry: check=True
+ mocked_run = mocker.patch("subprocrunner.SubprocessRunner._run")
+ mocked_run.return_value = 1
+ try:
+ runner.run(
+ check=True,
+ retry=Retry(total=retry_ct, backoff_factor=BACKOFF_FACTOR,
jitter=JITTER),
+ )
+ except CalledProcessError:
+ pass
+ assert mocked_run.call_count == retry_ct + 1
+
+ # w/o retry: check=False
+ mocked_run = mocker.patch("subprocrunner.SubprocessRunner._run")
+ mocked_run.return_value = 1
+ runner.run(check=False, retry=None)
+ assert mocked_run.call_count == 1
+
+ # w/o retry: check=True
+ mocked_run = mocker.patch("subprocrunner.SubprocessRunner._run")
+ mocked_run.return_value = 1
+ try:
+ runner.run(check=True, retry=None)
+ except CalledProcessError:
+ pass
+ assert mocked_run.call_count == 1
+
+ def test_no_retry_returncodes(self, mocker):
+ mocker.patch("subprocrunner.Which.verify")
+
+ runner = SubprocessRunner("always-failed-command")
+ mocked_run = mocker.patch("subprocrunner.SubprocessRunner._run")
+ mocked_run.return_value = 2
+ runner.run(
+ check=True,
+ retry=Retry(
+ total=3,
+ backoff_factor=BACKOFF_FACTOR,
+ jitter=JITTER,
+ no_retry_returncodes=[2],
+ ),
+ )
+ assert mocked_run.call_count == 1
+
+ def test_retry_success_ater_failed(self, mocker):
+ mocker.patch("subprocrunner.Which.verify")
+
+ def failed_first_call(*args, **kwargs):
+ attempt = kwargs.get(SubprocessRunner._RETRY_ATTEMPT_KEY)
+ if attempt is None:
+ return 1
+
+ return 0
+
+ runner = SubprocessRunner("always-failed-command")
+ mocked_run = mocker.patch("subprocrunner.SubprocessRunner._run")
+ mocked_run.side_effect = failed_first_call
+ runner.run(check=True, retry=Retry(total=3,
backoff_factor=BACKOFF_FACTOR, jitter=JITTER))
+ assert mocked_run.call_count == 2
+
class Test_SubprocessRunner_popen:
@pytest.mark.parametrize(
@@ -165,18 +280,33 @@
class Test_SubprocessRunner_command_history:
@pytest.mark.parametrize(
- ["command", "dry_run", "expected"], [[list_command, False, 0],
[list_command, True, 0]]
+ ["command", "dry_run"],
+ [
+ [list_command, False],
+ [list_command, True],
+ ],
)
- def test_normal(self, command, dry_run, expected):
+ def test_normal(self, command, dry_run):
+ loop_count = 3
+
SubprocessRunner.is_save_history = False
SubprocessRunner.clear_history()
-
- loop_count = 3
for _i in range(loop_count):
SubprocessRunner(command, dry_run=dry_run).run()
- assert len(SubprocessRunner.get_history()) == 0
+ assert SubprocessRunner.get_history() == []
SubprocessRunner.is_save_history = True
+ SubprocessRunner.clear_history()
for _i in range(loop_count):
SubprocessRunner(command, dry_run=dry_run).run()
- assert len(SubprocessRunner.get_history()) == loop_count
+ assert SubprocessRunner.get_history() == [list_command] * loop_count
+
+ def test_normal_retry(self, mocker):
+ SubprocessRunner.clear_history()
+ SubprocessRunner.is_save_history = True
+ command = [list_command, "not_exist_dir"]
+ runner = SubprocessRunner(command)
+ retry_ct = 3
+
+ runner.run(retry=Retry(total=retry_ct, backoff_factor=BACKOFF_FACTOR,
jitter=JITTER))
+ assert runner.get_history() == [" ".join(command)] * (retry_ct + 1)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/subprocrunner-1.2.2/tox.ini
new/subprocrunner-1.6.0/tox.ini
--- old/subprocrunner-1.2.2/tox.ini 2021-05-05 08:29:55.000000000 +0200
+++ new/subprocrunner-1.6.0/tox.ini 2021-06-05 13:14:22.000000000 +0200
@@ -47,7 +47,7 @@
black
isort>=5
commands =
- autoflake --in-place --recursive --remove-all-unused-imports
--ignore-init-module-imports --exclude ".pytype" .
+ autoflake --in-place --recursive --remove-all-unused-imports
--ignore-init-module-imports .
isort .
black setup.py examples test subprocrunner
@@ -58,10 +58,8 @@
codespell
mypy
pylama
- pytype
commands =
python setup.py check
mypy subprocrunner
- pytype --keep-going --jobs 2 --disable import-error subprocrunner
codespell subprocrunner examples test README.rst -q 2 --check-filenames
pylama
