Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package python-yacron for openSUSE:Factory checked in at 2024-01-03 12:24:28 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-yacron (Old) and /work/SRC/openSUSE:Factory/.python-yacron.new.28375 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-yacron" Wed Jan 3 12:24:28 2024 rev:7 rq:1135636 version:0.19.0 Changes: -------- --- /work/SRC/openSUSE:Factory/python-yacron/python-yacron.changes 2022-09-01 22:13:15.044577049 +0200 +++ /work/SRC/openSUSE:Factory/.python-yacron.new.28375/python-yacron.changes 2024-01-03 12:24:34.638576919 +0100 @@ -1,0 +2,13 @@ +Fri Dec 29 10:21:16 UTC 2023 - Dirk Müller <dmuel...@suse.com> + +- update to 0.19.0: + * Add ability to configure yacron's own logging (#81 #82 #83, + * Add config value for SMTP(validate_certs=False) (David + Batley) + * fixes "Job is always executed immediately on yacron start" + (#67) + * add an `enabled` option in jobs (#73) + * give a better error message when no configuration file is + provided or exists (#72) + +------------------------------------------------------------------- Old: ---- yacron-0.17.0.tar.gz New: ---- yacron-0.19.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-yacron.spec ++++++ --- /var/tmp/diff_new_pack.RQdHOa/_old 2024-01-03 12:24:35.326602057 +0100 +++ /var/tmp/diff_new_pack.RQdHOa/_new 2024-01-03 12:24:35.330602203 +0100 @@ -1,7 +1,7 @@ # # spec file for package python-yacron # -# Copyright (c) 2022 SUSE LLC +# Copyright (c) 2023 SUSE LLC # # All modifications and additions to the file contributed by third parties # remain the property of their copyright owners, unless otherwise agreed @@ -20,7 +20,7 @@ %define skip_python2 1 %define skip_python36 1 Name: python-yacron -Version: 0.17.0 +Version: 0.19.0 Release: 0 Summary: Docker-friendly Cron replacement License: MIT ++++++ yacron-0.17.0.tar.gz -> yacron-0.19.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/HISTORY.rst new/yacron-0.19.0/HISTORY.rst --- old/yacron-0.17.0/HISTORY.rst 2022-06-26 15:01:52.000000000 +0200 +++ new/yacron-0.19.0/HISTORY.rst 2023-03-11 14:38:45.000000000 +0100 @@ -2,6 +2,19 @@ History ======= +0.19.0 (2023-03-11) +------------------- + +* Add ability to configure yacron's own logging (#81 #82 #83, gjcarneiro, bdamian) +* Add config value for SMTP(validate_certs=False) (David Batley) + +0.18.0 (2023-01-01) +------------------- + +* fixes "Job is always executed immediately on yacron start" (#67) +* add an `enabled` option in jobs (#73) +* give a better error message when no configuration file is provided or exists (#72) + 0.17.0 (2022-06-26) ------------------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/PKG-INFO new/yacron-0.19.0/PKG-INFO --- old/yacron-0.17.0/PKG-INFO 2022-06-26 15:02:52.030362400 +0200 +++ new/yacron-0.19.0/PKG-INFO 2023-03-11 14:54:57.224146000 +0100 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: yacron -Version: 0.17.0 +Version: 0.19.0 Summary: A modern Cron replacement that is Docker-friendly Home-page: https://github.com/gjcarneiro/yacron Author: Gustavo Carneiro @@ -134,9 +134,9 @@ schedule: "*/5 * * * *" -The `schedule` option can be a string in the traditional crontab format -(including @reboot, which will only run the job when yacron is initially -executed), or can be an object with properties. The following configuration +The `schedule` option can be a string in a crontab format specified by https://github.com/josiahcarlson/parse-crontab (this module is used by yacron). +Additionally @reboot can be included , which will only run the job when yacron is initially +executed. Further `schedule` can be an object with properties. The following configuration runs a command every 5 minutes, but only on the specific date 2017-07-19, and doesn't run it in any other date: @@ -767,11 +767,75 @@ sentry: ... +Custom logging +++++++++++++++ + +It's possible to provide a custom logging configuration, via the ``logging`` +configuration section. For example, the following configuration displays log lines with +an embedded timestamp for each message. + +.. code-block:: yaml + + logging: + # In the format of: + # https://docs.python.org/3/library/logging.config.html#dictionary-schema-details + version: 1 + disable_existing_loggers: false + formatters: + simple: + format: '%(asctime)s [%(processName)s/%(threadName)s] %(levelname)s (%(name)s): %(message)s' + handlers: + console: + class: logging.StreamHandler + level: DEBUG + formatter: simple + stream: ext://sys.stdout + root: + level: INFO + handlers: + - console + +Obscure configuration options ++++++++++++++++++++++++++++++ + +enabled: true|false (default true) +################################## + +(new in yacron 0.18) + +It is possible to disable a specific cron job by adding a `enabled: false` option. Jobs +with `enabled: false` will simply be skipped, as if they aren't there, apart from +validating the configuration. + +.. code-block:: yaml + + jobs: + - name: test-01 + enabled: false # this cron job will not run until you change this to `true` + command: echo "foobar" + shell: /bin/bash + schedule: "* * * * *" + + + ======= History ======= +0.19.0 (2023-03-11) +------------------- + +* Add ability to configure yacron's own logging (#81 #82 #83, gjcarneiro, bdamian) +* Add config value for SMTP(validate_certs=False) (David Batley) + +0.18.0 (2023-01-01) +------------------- + +* fixes "Job is always executed immediately on yacron start" (#67) +* add an `enabled` option in jobs (#73) +* give a better error message when no configuration file is provided or exists (#72) + 0.17.0 (2022-06-26) ------------------- diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/README.rst new/yacron-0.19.0/README.rst --- old/yacron-0.17.0/README.rst 2022-03-30 12:06:01.000000000 +0200 +++ new/yacron-0.19.0/README.rst 2023-03-11 14:54:45.000000000 +0100 @@ -110,9 +110,9 @@ schedule: "*/5 * * * *" -The `schedule` option can be a string in the traditional crontab format -(including @reboot, which will only run the job when yacron is initially -executed), or can be an object with properties. The following configuration +The `schedule` option can be a string in a crontab format specified by https://github.com/josiahcarlson/parse-crontab (this module is used by yacron). +Additionally @reboot can be included , which will only run the job when yacron is initially +executed. Further `schedule` can be an object with properties. The following configuration runs a command every 5 minutes, but only on the specific date 2017-07-19, and doesn't run it in any other date: @@ -742,3 +742,54 @@ report: sentry: ... + +Custom logging +++++++++++++++ + +It's possible to provide a custom logging configuration, via the ``logging`` +configuration section. For example, the following configuration displays log lines with +an embedded timestamp for each message. + +.. code-block:: yaml + + logging: + # In the format of: + # https://docs.python.org/3/library/logging.config.html#dictionary-schema-details + version: 1 + disable_existing_loggers: false + formatters: + simple: + format: '%(asctime)s [%(processName)s/%(threadName)s] %(levelname)s (%(name)s): %(message)s' + handlers: + console: + class: logging.StreamHandler + level: DEBUG + formatter: simple + stream: ext://sys.stdout + root: + level: INFO + handlers: + - console + +Obscure configuration options ++++++++++++++++++++++++++++++ + +enabled: true|false (default true) +################################## + +(new in yacron 0.18) + +It is possible to disable a specific cron job by adding a `enabled: false` option. Jobs +with `enabled: false` will simply be skipped, as if they aren't there, apart from +validating the configuration. + +.. code-block:: yaml + + jobs: + - name: test-01 + enabled: false # this cron job will not run until you change this to `true` + command: echo "foobar" + shell: /bin/bash + schedule: "* * * * *" + + diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/example/adhoc.yacron.d/_inc.yaml new/yacron-0.19.0/example/adhoc.yacron.d/_inc.yaml --- old/yacron-0.17.0/example/adhoc.yacron.d/_inc.yaml 2021-10-01 20:24:06.000000000 +0200 +++ new/yacron-0.19.0/example/adhoc.yacron.d/_inc.yaml 2023-03-11 14:36:36.000000000 +0100 @@ -46,3 +46,21 @@ host: localhost port: 8125 prefix: my.cron.jobs.prefix.test01 + + +logging: + version: 1 + disable_existing_loggers: false + formatters: + simple: + format: '%(asctime)s [%(processName)s/%(threadName)s] %(levelname)s (%(name)s): %(message)s' + handlers: + console: + class: logging.StreamHandler + level: DEBUG + formatter: simple + stream: ext://sys.stdout + root: + level: INFO + handlers: + - console diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/pyinstaller/requirements.txt new/yacron-0.19.0/pyinstaller/requirements.txt --- old/yacron-0.17.0/pyinstaller/requirements.txt 2022-06-26 12:27:14.000000000 +0200 +++ new/yacron-0.19.0/pyinstaller/requirements.txt 2023-03-11 11:29:20.000000000 +0100 @@ -4,7 +4,7 @@ appdirs==1.4.4 async-timeout==4.0.2 attrs==21.2.0 -certifi==2021.5.30 +certifi==2022.12.7 chardet==3.0.4 coverage==5.5 crontab==0.22.8 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/requirements_dev.txt new/yacron-0.19.0/requirements_dev.txt --- old/yacron-0.17.0/requirements_dev.txt 2022-06-26 12:24:45.000000000 +0200 +++ new/yacron-0.19.0/requirements_dev.txt 2023-03-11 11:29:20.000000000 +0100 @@ -1,33 +1,10 @@ -aiohttp==3.8.1 -appdirs==1.4.4 -async-timeout==4.0.2 -attrs==21.2.0 -chardet==3.0.4 -coverage==5.5 -distlib==0.3.2 -filelock==3.0.12 -flake8==3.9.2 -idna==3.2 -iniconfig==1.1.1 -mccabe==0.6.1 -multidict==5.1.0 +aiohttp~=3.8 +flake8 mypy==0.910 -mypy-extensions==0.4.3 -packaging==20.9 -pluggy==0.13.1 -py==1.10.0 -pycodestyle==2.7.0 -pyflakes==2.3.1 -pyparsing==2.4.7 -pytest==7.1.1 -pytest-cov==2.12.1 -pytz==2021.1 -six==1.16.0 -toml==0.10.2 -tomli==2.0.1 -tox==3.23.1 -typing-extensions==3.10.0.0 -virtualenv==20.4.7 -yarl==1.6.3 -types-pytz==2021.1.0 -pytest-asyncio==0.18.3 +mypy-extensions +pytest +pytest-asyncio +pytest-cov +tox +types-pytz +pytest-asyncio~=0.18 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/tests/test_config.py new/yacron-0.19.0/tests/test_config.py --- old/yacron-0.17.0/tests/test_config.py 2021-11-10 11:42:42.000000000 +0100 +++ new/yacron-0.19.0/tests/test_config.py 2023-03-11 14:36:36.000000000 +0100 @@ -31,7 +31,7 @@ def test_simple_config1(): - jobs, web_config, _ = config.parse_config_string( + conf = config.parse_config_string( """ defaults: shell: /bin/bash @@ -51,9 +51,9 @@ """, "", ) - assert web_config is None - assert len(jobs) == 1 - job = jobs[0] + assert conf.web_config is None + assert len(conf.jobs) == 1 + job = conf.jobs[0] assert job.name == "test-03" assert job.command == ( "trap \"echo '(ignoring SIGTERM)'\" TERM\n" @@ -69,7 +69,7 @@ def test_config_default_report(): - jobs, _, _ = config.parse_config_string( + conf = config.parse_config_string( """ defaults: onFailure: @@ -89,8 +89,8 @@ """, "", ) - assert len(jobs) == 1 - job = jobs[0] + assert len(conf.jobs) == 1 + job = conf.jobs[0] assert job.onFailure == ( { "report": { @@ -117,6 +117,7 @@ }, "tls": False, "starttls": False, + "validate_certs": False, "html": False, }, "sentry": ( @@ -137,7 +138,7 @@ def test_config_default_report_override(): # even if the default says send email on error, it should be possible for # specific jobs to override the default and disable sending email. - jobs, _, _ = config.parse_config_string( + conf = config.parse_config_string( """ defaults: onFailure: @@ -162,8 +163,8 @@ """, "", ) - assert len(jobs) == 1 - job = jobs[0] + assert len(conf.jobs) == 1 + job = conf.jobs[0] assert job.onFailure == ( { "report": { @@ -190,6 +191,7 @@ }, "tls": False, "starttls": False, + "validate_certs": False, "html": False, }, "sentry": ( @@ -208,13 +210,13 @@ def test_empty_config1(): - jobs, web_config, _ = config.parse_config_string("", "") - assert len(jobs) == 0 - assert web_config is None + conf = config.parse_config_string("", "") + assert len(conf.jobs) == 0 + assert conf.web_config is None def test_environ_file(): - jobs, _, _ = config.parse_config_string( + conf = config.parse_config_string( """ defaults: shell: /bin/bash @@ -237,7 +239,7 @@ """, "", ) - job = jobs[0] + job = conf.jobs[0] # NOTE: the file format implicitly verifies that the parsing is being # done correctly on these fronts: @@ -261,7 +263,7 @@ def test_invalid_environ_file(): # invalid file (no key-value) with pytest.raises(ConfigError) as exc: - jobs, _, _ = config.parse_config_string( + config.parse_config_string( """ defaults: shell: /bin/bash @@ -289,7 +291,7 @@ # non-existent file should raise ConfigError, not OSError with pytest.raises(ConfigError) as exc: - jobs, _, _ = config.parse_config_string( + config.parse_config_string( """ defaults: shell: /bin/bash @@ -317,12 +319,39 @@ def test_config_include(): - jobs, _ = config.parse_config( + conf = config.parse_config( os.path.join(os.path.dirname(__file__), "test_include_parent.yaml") ) - assert len(jobs) == 2 - job1, job2 = jobs + assert len(conf.jobs) == 2 + job1, job2 = conf.jobs assert job1.name == "common-task" assert job2.name == "test-03" assert job1.shell == "/bin/ksh" assert job2.shell == "/bin/ksh" + + +def test_logging_config(): + conf = config.parse_config_string( + """ +logging: + version: 1 + incremental: false + disable_existing_loggers: false + formatters: one + filters: two + handlers: three + loggers: four + root: five + """, + "", + ) + assert conf.logging_config == { + "version": 1, + "incremental": False, + "disable_existing_loggers": False, + "formatters": "one", + "filters": "two", + "handlers": "three", + "loggers": "four", + "root": "five", + } diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/tests/test_cron.py new/yacron-0.19.0/tests/test_cron.py --- old/yacron-0.17.0/tests/test_cron.py 2022-04-03 17:21:22.000000000 +0200 +++ new/yacron-0.19.0/tests/test_cron.py 2023-03-11 11:29:20.000000000 +0100 @@ -79,7 +79,7 @@ - name: test command: | echo "foobar" - schedule: "* * * * *" + schedule: "@reboot" """ JOB_THAT_FAILS = """ @@ -88,7 +88,7 @@ command: | echo "foobar" exit 2 - schedule: "* * * * *" + schedule: "@reboot" """ @@ -143,7 +143,7 @@ command: | echo "foobar" exit 2 - schedule: "* * * * *" + schedule: "@reboot" onFailure: retry: maximumRetries: 2 @@ -153,6 +153,7 @@ """ +@pytest.mark.asyncio async def test_fail_retry(tracing_running_job): cron = yacron.cron.Cron(None, config_yaml=RETRYING_JOB_THAT_FAILS) @@ -211,13 +212,14 @@ echo "starting..." sleep 10 echo "all done." - schedule: "* * * * *" + schedule: "@reboot" captureStdout: true executionTimeout: 0.25 killTimeout: 0.25 """ +@pytest.mark.asyncio async def test_execution_timeout(tracing_running_job): cron = yacron.cron.Cron(None, config_yaml=JOB_THAT_HANGS) @@ -266,7 +268,7 @@ echo "starting..." sleep 0.5 echo "all done." - schedule: "* * * * *" + schedule: "@reboot" captureStdout: true concurrencyPolicy: {policy} """ @@ -277,6 +279,7 @@ "policy,expected_numjobs,expected_max_running", [("Allow", 2, 2), ("Forbid", 1, 1), ("Replace", 2, 1)], ) +@pytest.mark.asyncio async def test_concurrency_policy( monkeypatch, tracing_running_job, @@ -366,7 +369,7 @@ command: | echo "foobar" exit 2 - schedule: "* * * * *" + schedule: "@reboot" onFailure: retry: maximumRetries: 1 @@ -376,7 +379,7 @@ """ -# @pytest.mark.xfail +@pytest.mark.asyncio async def test_concurrency_and_backoff(monkeypatch, tracing_running_job): START_TIME = datetime.datetime( year=1999, @@ -483,7 +486,7 @@ @pytest.mark.parametrize( - "schedule, timezone, utc, now, reboot, result", + "schedule, timezone, utc, now, startup, enabled, result", [ ( "* * * * *", @@ -491,6 +494,7 @@ "", DT(2020, 7, 20, 14, 59, 1, tzinfo=UTC), False, + "", True, ), ( @@ -499,14 +503,25 @@ "", DT(2020, 7, 20, 14, 59, 1, tzinfo=UTC), False, + "", True, ), ( + "59 14 * * *", + "", + "", + DT(2020, 7, 20, 14, 59, 1, tzinfo=UTC), + True, # startup + "", + False, + ), + ( "49 14 * * *", "", "", DT(2020, 7, 20, 14, 59, 1, tzinfo=UTC), False, + "", False, ), ( @@ -515,6 +530,7 @@ "utc: true", DT(2020, 7, 20, 14, 59, 1, tzinfo=UTC), False, + "", True, ), ( @@ -523,6 +539,7 @@ "utc: true", # London is UTC+1 during DST DT(2020, 7, 20, 14, 59, 1, tzinfo=UTC).astimezone(LONDON), False, + "", True, ), ( @@ -531,6 +548,7 @@ "utc: false", # London is UTC+1 during DST DT(2020, 7, 20, 14, 59, 1, tzinfo=UTC).astimezone(LONDON), False, + "", False, ), ( @@ -539,6 +557,7 @@ "", DT(2020, 7, 20, 15, 1, 1, tzinfo=UTC), False, + "", True, ), ( @@ -547,6 +566,7 @@ "", DT(2020, 7, 20, 15, 1, 1, tzinfo=UTC), False, + "", False, ), ( @@ -555,6 +575,7 @@ "", DT(2020, 7, 20, 15, 1, 1, tzinfo=UTC), False, + "", False, ), ( @@ -563,12 +584,24 @@ "", DT(2020, 7, 20, 15, 1, 1, tzinfo=UTC), True, + "", True, ), + + # enabled: false + ( + "* * * * *", + "", + "", + DT(2020, 7, 20, 14, 59, 1, tzinfo=UTC), + False, + "enabled: false", + False, + ), ], ) def test_job_should_run( - monkeypatch, schedule, timezone, utc, now, reboot, result + monkeypatch, schedule, timezone, utc, now, startup, enabled, result ): def get_now(timezone): print("timezone: ", timezone) @@ -580,7 +613,7 @@ monkeypatch.setattr("yacron.cron.get_now", get_now) - config_yaml = """ + config_yaml = f""" jobs: - name: test command: | @@ -588,10 +621,9 @@ schedule: "{schedule}" {timezone} {utc} - """.format( - schedule=schedule, timezone=timezone, utc=utc - ) + {enabled} + """ print(config_yaml) cron = yacron.cron.Cron(None, config_yaml=config_yaml) job = list(cron.cron_jobs.values())[0] - assert cron.job_should_run(reboot, job) == result + assert cron.job_should_run(startup, job) == result diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/tests/test_job.py new/yacron-0.19.0/tests/test_job.py --- old/yacron-0.17.0/tests/test_job.py 2022-06-26 12:35:27.000000000 +0200 +++ new/yacron-0.19.0/tests/test_job.py 2023-03-11 14:36:36.000000000 +0100 @@ -42,7 +42,7 @@ "cronjob-1", "stderr", fake_stream, "", save_limit ) - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( """ jobs: - name: test @@ -52,7 +52,7 @@ """, "", ) - job_config = config[0] + job_config = conf.jobs[0] job = yacron.job.RunningJob(job_config, None) async def producer(fake_stream): @@ -76,7 +76,7 @@ "cronjob-1", "stderr", fake_stream, "", 500 ) - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( """ jobs: - name: test @@ -86,7 +86,7 @@ """, "", ) - job_config = config[0] + job_config = conf.jobs[0] job = yacron.job.RunningJob(job_config, None) async def producer(fake_stream): @@ -173,8 +173,8 @@ ) @pytest.mark.asyncio async def test_report_mail(success, stdout, stderr, subject, body): - config, _, _ = yacron.config.parse_config_string(A_JOB, "") - job_config = config[0] + conf = yacron.config.parse_config_string(A_JOB, "") + job_config = conf.jobs[0] print(job_config.onSuccess["report"]) job = Mock( config=job_config, @@ -226,7 +226,12 @@ assert smtp_init_args == ( (), - {"hostname": "smtp1", "port": 1025, "use_tls": False}, + { + "hostname": "smtp1", + "port": 1025, + "use_tls": False, + "validate_certs": False, + }, ) assert len(connect_calls) == 1 assert len(start_tls_calls) == 1 @@ -309,8 +314,8 @@ tmpdir, monkeypatch, ): - config, _, _ = yacron.config.parse_config_string(A_JOB, "") - job_config = config[0] + conf = yacron.config.parse_config_string(A_JOB, "") + job_config = conf.jobs[0] p = tmpdir.join("sentry-secret-dsn") p.write("http://xxx:yyy@sentry/2";) @@ -430,7 +435,7 @@ with tempfile.TemporaryDirectory() as tmp: out_file_path = os.path.join(tmp, "unit_test_file") - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( f""" jobs: - name: test @@ -446,7 +451,7 @@ """, "", ) - job_config = config[0] + job_config = conf.jobs[0] job = Mock( config=job_config, @@ -526,7 +531,7 @@ else: command_snippet = " command: " + command - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( """ jobs: - name: test @@ -543,7 +548,7 @@ ), "", ) - job_config = config[0] + job_config = conf.jobs[0] job = yacron.job.RunningJob(job_config, None) @@ -568,7 +573,7 @@ @pytest.mark.asyncio async def test_execution_timeout(): - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( """ jobs: - name: test @@ -583,7 +588,7 @@ """, "", ) - job_config = config[0] + job_config = conf.jobs[0] job = yacron.job.RunningJob(job_config, None) await job.start() await job.wait() @@ -592,7 +597,7 @@ @pytest.mark.asyncio async def test_error1(): - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( """ jobs: - name: test @@ -601,7 +606,7 @@ """, "", ) - job_config = config[0] + job_config = conf.jobs[0] job = yacron.job.RunningJob(job_config, None) await job.start() @@ -612,7 +617,7 @@ @pytest.mark.asyncio async def test_error2(): - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( """ jobs: - name: test @@ -621,7 +626,7 @@ """, "", ) - job_config = config[0] + job_config = conf.jobs[0] job = yacron.job.RunningJob(job_config, None) with pytest.raises(RuntimeError): @@ -630,7 +635,7 @@ @pytest.mark.asyncio async def test_error3(): - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( """ jobs: - name: test @@ -639,7 +644,7 @@ """, "", ) - job_config = config[0] + job_config = conf.jobs[0] job = yacron.job.RunningJob(job_config, None) with pytest.raises(RuntimeError): @@ -673,7 +678,7 @@ host, port = transport.get_extra_info("sockname") print("Listening UDP on %s:%s" % (host, port)) - config, _, _ = yacron.config.parse_config_string( + conf = yacron.config.parse_config_string( """ jobs: - name: test @@ -688,7 +693,7 @@ ), "", ) - job_config = config[0] + job_config = conf.jobs[0] job = yacron.job.RunningJob(job_config, None) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/yacron/__main__.py new/yacron-0.19.0/yacron/__main__.py --- old/yacron-0.17.0/yacron/__main__.py 2022-06-25 16:25:07.000000000 +0200 +++ new/yacron-0.19.0/yacron/__main__.py 2023-03-11 11:29:20.000000000 +0100 @@ -4,18 +4,27 @@ import logging import signal import sys +import os from yacron.cron import Cron, ConfigError import yacron.version +CONFIG_DEFAULT = "/etc/yacron.d" + def main_loop(loop): - parser = argparse.ArgumentParser() + parser = argparse.ArgumentParser(prog="yacron") parser.add_argument( - "-c", "--config", default="/etc/yacron.d", metavar="FILE-OR-DIR" + "-c", + "--config", + default=CONFIG_DEFAULT, + metavar="FILE-OR-DIR", + help="configuration file, or directory containing configuration files", ) parser.add_argument("-l", "--log-level", default="INFO") - parser.add_argument("-v", "--validate-config", default=False, action="store_true") + parser.add_argument( + "-v", "--validate-config", default=False, action="store_true" + ) parser.add_argument("--version", default=False, action="store_true") args = parser.parse_args() @@ -27,6 +36,15 @@ print(yacron.version.version) sys.exit(0) + if args.config == CONFIG_DEFAULT and not os.path.exists(args.config): + print( + "yacron error: configuration file not found, please provide one " + "with the --config option", + file=sys.stderr, + ) + parser.print_help(sys.stderr) + sys.exit(1) + try: cron = Cron(args.config) except ConfigError as err: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/yacron/config.py new/yacron-0.19.0/yacron/config.py --- old/yacron-0.17.0/yacron/config.py 2021-12-05 14:34:57.000000000 +0100 +++ new/yacron-0.19.0/yacron/config.py 2023-03-11 14:36:36.000000000 +0100 @@ -1,14 +1,16 @@ +from dataclasses import dataclass from pwd import getpwnam from grp import getgrnam import logging import os.path from typing import Union # noqa -from typing import List, Optional, Any, Dict, NewType, Tuple +from typing import List, Optional, Any, Dict, NewType import datetime import pytz import strictyaml from strictyaml import Optional as Opt, EmptyDict +from strictyaml import Any as YamlAny from strictyaml import ( Bool, EmptyNone, @@ -27,6 +29,7 @@ logger = logging.getLogger("yacron.config") WebConfig = NewType("WebConfig", Dict[str, Any]) JobDefaults = NewType("JobDefaults", Dict[str, Any]) +LoggingConfig = NewType("LoggingConfig", Dict[str, Any]) class ConfigError(Exception): @@ -73,6 +76,7 @@ "smtpPort": 25, "tls": False, "starttls": False, + "validate_certs": False, "html": False, "subject": DEFAULT_SUBJECT_TEMPLATE, "body": DEFAULT_BODY_TEMPLATE, @@ -92,7 +96,7 @@ "captureStderr": True, "captureStdout": False, "saveLimit": 4096, - "maxLineLength": 16*1024*1024, + "maxLineLength": 16 * 1024 * 1024, "utc": True, "timezone": None, "failsWhen": { @@ -118,6 +122,7 @@ "killTimeout": 30, "statsd": None, "streamPrefix": "[{job_name} {stream_name}] ", + "enabled": True, } @@ -158,6 +163,7 @@ ), Opt("tls"): Bool(), Opt("starttls"): Bool(), + Opt("validate_certs"): Bool(), Opt("html"): Bool(), } ), @@ -210,6 +216,7 @@ Opt("user"): Str() | Int(), Opt("group"): Str() | Int(), Opt("streamPrefix"): Str(), + Opt("enabled"): Bool(), } _job_schema_dict = dict(_job_defaults_common) @@ -237,6 +244,18 @@ Opt("jobs"): Seq(Map(_job_schema_dict)), Opt("web"): Map({"listen": Seq(Str())}), Opt("include"): Seq(Str()), + Opt("logging"): Map( + { + "version": Int(), + Opt("incremental"): Bool(), + Opt("disable_existing_loggers"): Bool(), + Opt("formatters"): YamlAny(), + Opt("filters"): YamlAny(), + Opt("handlers"): YamlAny(), + Opt("loggers"): YamlAny(), + Opt("root"): YamlAny(), + } + ), } ) @@ -292,6 +311,7 @@ self.saveLimit = config.pop("saveLimit") self.maxLineLength = config.pop("maxLineLength") self.utc = config.pop("utc") + self.enabled: bool = config.pop("enabled") self.timezone = None # type: Optional[datetime.tzinfo] if config["timezone"] is not None: try: @@ -397,9 +417,15 @@ return environ -def parse_config_string( - data: str, path: str -) -> Tuple[List[JobConfig], Optional[WebConfig], JobDefaults]: +@dataclass +class YacronConfig: + jobs: List[JobConfig] + web_config: Optional[WebConfig] + job_defaults: JobDefaults + logging_config: Optional[LoggingConfig] + + +def parse_config_string(data: str, path: str) -> YacronConfig: try: doc = strictyaml.load(data, CONFIG_SCHEMA, label=path).data except YAMLError as ex: @@ -408,36 +434,44 @@ inc_defaults_merged: dict = {} jobs = [] webconf = WebConfig(doc["web"]) if "web" in doc else None + logging_conf = LoggingConfig(doc["logging"]) if "logging" in doc else None for include in doc.get("include", ()): inc_path = os.path.join(os.path.dirname(path), include) - inc_jobs, inc_webconf, inc_defaults = parse_config_file(inc_path) + inc_config = parse_config_file(inc_path) inc_defaults_merged = dict( - mergedicts(inc_defaults_merged, inc_defaults) + mergedicts(inc_defaults_merged, inc_config.job_defaults) ) - jobs.extend(inc_jobs) - if inc_webconf: + jobs.extend(inc_config.jobs) + if inc_config.web_config: if webconf: - raise ConfigError("multiple web config") - webconf = inc_webconf + raise ConfigError("multiple web configs") + webconf = inc_config.web_config + if inc_config.logging_config: + if logging_conf: + raise ConfigError("multiple logging configs") + logging_conf = inc_config.logging_config defaults = dict(mergedicts(DEFAULT_CONFIG, inc_defaults_merged)) defaults = dict(mergedicts(defaults, doc.get("defaults", {}))) for config_job in doc.get("jobs", []): job_dict = dict(mergedicts(defaults, config_job)) jobs.append(JobConfig(job_dict)) - return jobs, webconf, JobDefaults(defaults) + return YacronConfig( + jobs=jobs, + web_config=webconf, + job_defaults=JobDefaults(defaults), + logging_config=logging_conf, + ) def parse_config_file( path: str, -) -> Tuple[List[JobConfig], Optional[WebConfig], JobDefaults]: +) -> YacronConfig: with open(path, "rt", encoding="utf-8") as stream: data = stream.read() return parse_config_string(data, path) -def parse_config( - config_arg: str, -) -> Tuple[List[JobConfig], Optional[WebConfig]]: +def parse_config(config_arg: str) -> YacronConfig: jobs = [] config_errors = {} web_config = None @@ -449,16 +483,16 @@ continue if ext in {".yml", ".yaml"}: try: - config, webconf, _ = parse_config_file(direntry.path) + config = parse_config_file(direntry.path) except ConfigError as err: config_errors[direntry.path] = str(err) except OSError as ex: config_errors[config_arg] = str(ex) else: - jobs.extend(config) - if webconf is not None: + jobs.extend(config.jobs) + if config.web_config is not None: if web_config is None: - web_config = webconf + web_config = config.web_config web_config_source_fname = direntry.path else: raise ConfigError( @@ -469,11 +503,16 @@ ) else: try: - config, web_config, _ = parse_config_file(config_arg) + config = parse_config_file(config_arg) except OSError as ex: config_errors[config_arg] = str(ex) else: - jobs.extend(config) + jobs.extend(config.jobs) if config_errors: raise ConfigError("\n---".join(config_errors.values())) - return jobs, web_config + return YacronConfig( + jobs=jobs, + web_config=config.web_config, + job_defaults=config.job_defaults, + logging_config=config.logging_config, + ) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/yacron/cron.py new/yacron-0.19.0/yacron/cron.py --- old/yacron-0.17.0/yacron/cron.py 2022-06-26 11:35:49.000000000 +0200 +++ new/yacron-0.19.0/yacron/cron.py 2023-03-11 14:36:36.000000000 +0100 @@ -1,3 +1,4 @@ +import logging.config import asyncio import asyncio.subprocess import datetime @@ -13,6 +14,8 @@ ConfigError, parse_config_string, WebConfig, + YacronConfig, + JobDefaults, ) from yacron.job import RunningJob, JobRetryState from crontab import CronTab # noqa @@ -82,8 +85,10 @@ self.update_config() if config_yaml is not None: # config_yaml is for unit testing - config, _, _ = parse_config_string(config_yaml, "") - self.cron_jobs = OrderedDict((job.name, job) for job in config) + config = parse_config_string(config_yaml, "") + self.cron_jobs = OrderedDict( + (job.name, job) for job in config.jobs + ) self._wait_for_running_jobs_task = None # type: Optional[asyncio.Task] self._stop_event = asyncio.Event() @@ -98,10 +103,11 @@ ) startup = True + logging_configured = False while not self._stop_event.is_set(): try: - web_config = self.update_config() - await self.start_stop_web_app(web_config) + config = self.update_config() + await self.start_stop_web_app(config.web_config) except ConfigError as err: logger.error( "Error in configuration file(s), so not updating " @@ -110,6 +116,19 @@ ) except Exception: # pragma: nocover logger.exception("please report this as a bug (1)") + if config.logging_config is not None and not logging_configured: + logging_configured = True + try: + logging.config.dictConfig(config.logging_config) + except Exception as ex: + logger.error( + "Error while configuring logging: %s\n" + "Check for correct format at " + "https://docs.python.org/3/library/logging.config.html"; + "#dictionary-schema-details\n%s", + ex, + config.logging_config, + ) await self.spawn_jobs(startup) startup = False sleep_interval = next_sleep_interval() @@ -135,12 +154,17 @@ logger.debug("Signalling shutdown") self._stop_event.set() - def update_config(self) -> Optional[WebConfig]: + def update_config(self) -> YacronConfig: if self.config_arg is None: - return None - config, web_config = parse_config(self.config_arg) - self.cron_jobs = OrderedDict((job.name, job) for job in config) - return web_config + return YacronConfig( + jobs=[], + web_config=None, + job_defaults=JobDefaults({}), + logging_config=None, + ) + config = parse_config(self.config_arg) + self.cron_jobs = OrderedDict((job.name, job) for job in config.jobs) + return config async def _web_get_version(self, request: web.Request) -> web.Response: return web.Response(text=yacron.version.version) @@ -250,18 +274,24 @@ @staticmethod def job_should_run(startup: bool, job: JobConfig) -> bool: - if ( - startup - and isinstance(job.schedule, str) - and job.schedule == "@reboot" - ): + if not job.enabled: logger.debug( - "Job %s (%s) is scheduled for startup (@reboot)", + "Job %s (%s) is disabled in the config", job.name, job.schedule_unparsed, ) - return True - elif isinstance(job.schedule, CronTab): + return False + if startup: + if isinstance(job.schedule, str) and job.schedule == "@reboot": + logger.debug( + "Job %s (%s) is scheduled for startup (@reboot)", + job.name, + job.schedule_unparsed, + ) + return True + else: + return False + if isinstance(job.schedule, CronTab): crontab = job.schedule # type: CronTab if crontab.test(get_now(job.timezone).replace(second=0)): logger.debug( diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/yacron/job.py new/yacron-0.19.0/yacron/job.py --- old/yacron-0.17.0/yacron/job.py 2022-06-26 12:35:04.000000000 +0200 +++ new/yacron-0.19.0/yacron/job.py 2023-03-11 11:29:20.000000000 +0100 @@ -43,8 +43,8 @@ stream_prefix: str, save_limit: int, ) -> None: - self.save_top = [] # type: List[str] - self.save_bottom = [] # type: List[str] + self.save_top: List[str] = [] + self.save_bottom: List[str] = [] self.job_name = job_name self.save_limit = save_limit self.stream_name = stream_name @@ -207,7 +207,10 @@ else: message.set_content(body) smtp = aiosmtplib.SMTP( - hostname=smtp_host, port=smtp_port, use_tls=mail["tls"] + hostname=smtp_host, + port=smtp_port, + use_tls=mail["tls"], + validate_certs=mail["validate_certs"], ) await smtp.connect() if mail["starttls"]: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/yacron/version.py new/yacron-0.19.0/yacron/version.py --- old/yacron-0.17.0/yacron/version.py 2022-06-26 15:02:51.000000000 +0200 +++ new/yacron-0.19.0/yacron/version.py 2023-03-11 14:54:57.000000000 +0100 @@ -1,5 +1,5 @@ # coding: utf-8 # file generated by setuptools_scm # don't change, don't track in version control -__version__ = version = '0.17.0' -__version_tuple__ = version_tuple = (0, 17, 0) +__version__ = version = '0.19.0' +__version_tuple__ = version_tuple = (0, 19, 0) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/yacron-0.17.0/yacron.egg-info/PKG-INFO new/yacron-0.19.0/yacron.egg-info/PKG-INFO --- old/yacron-0.17.0/yacron.egg-info/PKG-INFO 2022-06-26 15:02:52.000000000 +0200 +++ new/yacron-0.19.0/yacron.egg-info/PKG-INFO 2023-03-11 14:54:57.000000000 +0100 @@ -1,6 +1,6 @@ Metadata-Version: 2.1 Name: yacron -Version: 0.17.0 +Version: 0.19.0 Summary: A modern Cron replacement that is Docker-friendly Home-page: https://github.com/gjcarneiro/yacron Author: Gustavo Carneiro @@ -134,9 +134,9 @@ schedule: "*/5 * * * *" -The `schedule` option can be a string in the traditional crontab format -(including @reboot, which will only run the job when yacron is initially -executed), or can be an object with properties. The following configuration +The `schedule` option can be a string in a crontab format specified by https://github.com/josiahcarlson/parse-crontab (this module is used by yacron). +Additionally @reboot can be included , which will only run the job when yacron is initially +executed. Further `schedule` can be an object with properties. The following configuration runs a command every 5 minutes, but only on the specific date 2017-07-19, and doesn't run it in any other date: @@ -767,11 +767,75 @@ sentry: ... +Custom logging +++++++++++++++ + +It's possible to provide a custom logging configuration, via the ``logging`` +configuration section. For example, the following configuration displays log lines with +an embedded timestamp for each message. + +.. code-block:: yaml + + logging: + # In the format of: + # https://docs.python.org/3/library/logging.config.html#dictionary-schema-details + version: 1 + disable_existing_loggers: false + formatters: + simple: + format: '%(asctime)s [%(processName)s/%(threadName)s] %(levelname)s (%(name)s): %(message)s' + handlers: + console: + class: logging.StreamHandler + level: DEBUG + formatter: simple + stream: ext://sys.stdout + root: + level: INFO + handlers: + - console + +Obscure configuration options ++++++++++++++++++++++++++++++ + +enabled: true|false (default true) +################################## + +(new in yacron 0.18) + +It is possible to disable a specific cron job by adding a `enabled: false` option. Jobs +with `enabled: false` will simply be skipped, as if they aren't there, apart from +validating the configuration. + +.. code-block:: yaml + + jobs: + - name: test-01 + enabled: false # this cron job will not run until you change this to `true` + command: echo "foobar" + shell: /bin/bash + schedule: "* * * * *" + + + ======= History ======= +0.19.0 (2023-03-11) +------------------- + +* Add ability to configure yacron's own logging (#81 #82 #83, gjcarneiro, bdamian) +* Add config value for SMTP(validate_certs=False) (David Batley) + +0.18.0 (2023-01-01) +------------------- + +* fixes "Job is always executed immediately on yacron start" (#67) +* add an `enabled` option in jobs (#73) +* give a better error message when no configuration file is provided or exists (#72) + 0.17.0 (2022-06-26) -------------------
