Great ideas! Thanks for your thoughts.
Try this version of weeutil/logger.py. It includes a shim that does the
proper type conversion before passing on arguments to
logging.handlers.RotatingFileHandler.
To use should be as simple as adding this to weewx.conf (alas, NOT TESTED):
[Logging]
[[loggers]]
[[[root]]]
handlers = syslog, rotate
[[handlers]]
[[[rotate]]]
filename = /tmp/weewx.log
By default, it logs to /var/log/weewx.log. Hence, the need to override that
option if you want to log to /tmp/weewx.log.
-tk
On Sat, May 2, 2020 at 8:05 AM Graham Eddy <[email protected]> wrote:
> i think we all agree that syslog on macos is a lost cause.
> the corollary is that pointing python logging at syslog (via
> logging.handlers.SysLogHandler) is doomed
>
> i accept that syslog works fine on most platforms, but it is not feasible
> for macos
> but there is the simple alternative for macos users of using a different
> handler:
>
> i patched log_dict in bin/weeutil/logger.py to add a
> logging.handlers.RotatingFileHandler and it worked fine.
> adding it via weewx.conf gives a type conversion error that seems simple
> to fix.
> with the fix, i would suggest this be added to the macos installation
> instructions for user to add to weewx.conf (or make it part of the
> installation process)
>
> details follow...
>
> weewx.conf (yes, using that ‘root’ hack introduced to work around a
> ConfigObj limitation):
>
> [Logging]
> [[loggers]]
> [[[root]]]
> level = DEBUG
> propogate = True
> handlers = syslog, rotate
> [[handlers]]
> [[[rotate]]]
> level = DEBUG
> formatter = standard
> class = logging.handlers.RotatingFileHandler
> filename = /tmp/weewx.log
> maxBytes = 10000000
> backupCount = 2
>
>
> gives the following error, which on the face of it would take a simple fix
> of type conversions:
>
> Traceback (most recent call last):
> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
> line 563, in configure
> handler = self.configure_handler(handlers[name])
> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
> line 736, in configure_handler
> result = factory(**kwargs)
> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/handlers.py",
> line 146, in __init__
> if maxBytes > 0:
> TypeError: '>' not supported between instances of 'str' and ‘int’ < - - -
> SEE HER*E*
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
> File "bin/weewxd", line 261, in <module>
> main()
> File "bin/weewxd", line 136, in main
> weeutil.logger.setup(options.log_label, config_dict)
> File "/opt/weewx/bin/weeutil/logger.py", line 203, in setup
> logging.config.dictConfig(log_dict)
> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
> line 800, in dictConfig
> dictConfigClass(config).configure()
> File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
> line 571, in configure
> '%r' % name) from e
> ValueError: Unable to configure handler 'rotate'
>
>
> the log_dict it produces before crashing seems correct:
>
> {'version': 1, 'disable_existing_loggers': False, 'loggers': {'':
> {'level': 'DEBUG', 'propagate': True, 'handlers': ['syslog'], 'propogate':
> 'True'}}, 'handlers': {'syslog': {'level': 'DEBUG', 'formatter':
> 'standard', 'class': 'logging.handlers.SysLogHandler', 'address':
> '/var/run/syslog', 'facility': 'local1'}, 'console': {'level': 'DEBUG',
> 'formatter': 'verbose', 'class': 'logging.StreamHandler', 'stream': '
> ext://sys.stdout'}, 'rotate': {'level': 'DEBUG', 'formatter': 'standard',
> 'class': 'logging.handlers.RotatingFileHandler', 'filename':
> '/tmp/weewx.log', 'maxBytes': '10000000', 'backupCount': '2'}},
> 'formatters': {'simple': {'format': '%(levelname)s %(message)s'},
> 'standard': {'format': 'weewx[%(process)d] %(levelname)s %(name)s:
> %(message)s'}, 'verbose': {'format': '%(asctime)s weewx[%(process)d]
> %(levelname)s %(name)s: %(message)s', 'datefmt': '%Y-%m-%d %H:%M:%S'}}}
>
>
> cheers
>
> On 2 May 2020, at 10:42 pm, Tom Keffer <[email protected]> wrote:
>
> Logging from Python under MacOS High Sierra and later is a complete
> mystery. I have been unable to get it to work and, it seems, neither have
> other
> <https://apple.stackexchange.com/questions/256769/how-to-use-logger-command-on-sierra>
> users
> <https://stackoverflow.com/questions/49805750/macos-high-sierra-syslog-does-not-work/51052538>
> .
>
> Wish I could be of more help...
>
> -tk
>
> On Sat, May 2, 2020 at 5:26 AM Graham Eddy <[email protected]> wrote:
>
>> i reported this incorrectly. instead of giveing me an unchanged logger
>> format, it gives me no lines at all (from the point the weewx.conf log
>> format is parsed). even the following does the same (no logger output).
>> commenting out the “format” line reverts to lines being produced, in
>> original format of course
>>
>> [Logging]
>> [[formatters]]
>> [[[standard]]]
>> format = '%(message)s'
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/85C23BE1-912C-4EAA-80AC-F7E81F8EE021%40gmail.com
> <https://groups.google.com/d/msgid/weewx-user/85C23BE1-912C-4EAA-80AC-F7E81F8EE021%40gmail.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/weewx-user/CAPq0zEACT7zvUHsqwj0TODGn2fWqC_Yd6QYSifZcXuOYQmWXYQ%40mail.gmail.com.
#
# Copyright (c) 2019 Tom Keffer <[email protected]>
#
# See the file LICENSE.txt for your full rights.
#
"""WeeWX logging facility"""
from __future__ import absolute_import
import sys
import logging.config
from six.moves import StringIO
import configobj
import weewx
from weeutil.weeutil import to_int, to_bool
# These values are known only at runtime
if sys.platform == "darwin":
address = '/var/run/syslog'
facility = 'local1'
elif sys.platform.startswith('linux'):
address = '/dev/log'
facility = 'user'
elif sys.platform.startswith('freebsd'):
address = '/var/run/log'
facility = 'user'
elif sys.platform.startswith('netbsd'):
address = '/var/run/log'
facility = 'user'
elif sys.platform.startswith('openbsd'):
address = '/dev/log'
facility = 'user'
else:
address = ('localhost', 514)
facility = 'user'
# The logging defaults. Note that two kinds of placeholders are used:
#
# {value}: these are plugged in by the function setup().
# %(value)s: these are plugged in by the Python logging module.
#
LOGGING_STR = """[Logging]
version = 1
disable_existing_loggers = False
[[loggers]]
# Root logger
[[[root]]]
level = {log_level}
propagate = 1
handlers = syslog,
# Definitions of possible logging destinations
[[handlers]]
# System logger
[[[syslog]]]
level = DEBUG
formatter = standard
class = logging.handlers.SysLogHandler
address = {address}
facility = {facility}
# Log to console
[[[console]]]
level = DEBUG
formatter = verbose
class = logging.StreamHandler
# Alternate choice is 'ext://sys.stderr'
stream = ext://sys.stdout
# Log to a set of rotating files
[[[rotate]]]
level = DEBUG
formatter = standard
class = weeutil.logger.RotatingFileHandler
filename = /var/log/weewx.log
maxBytes = 10000000
backupCount = 4
# How to format log messages
[[formatters]]
[[[simple]]]
format = "%(levelname)s %(message)s"
[[[standard]]]
format = "{process_name}[%(process)d] %(levelname)s %(name)s: %(message)s"
[[[verbose]]]
format = "%(asctime)s {process_name}[%(process)d] %(levelname)s %(name)s: %(message)s"
# Format to use for dates and times:
datefmt = %Y-%m-%d %H:%M:%S
"""
def setup(process_name, user_log_dict):
"""Set up the weewx logging facility"""
# Create a ConfigObj from the default string. No interpolation (it interferes with the
# interpolation directives embedded in the string).
log_config = configobj.ConfigObj(StringIO(LOGGING_STR), interpolation=False, encoding='utf-8')
# Turn off interpolation in the incoming dictionary. First save the old
# value, then restore later. However, the incoming dictionary may be a simple
# Python dictionary and not have interpolation. Hence the try block.
try:
old_interpolation = user_log_dict.interpolation
user_log_dict.interpolation = False
except AttributeError:
old_interpolation = None
# Merge in the user additions / changes:
log_config.merge(user_log_dict)
# Restore the old interpolation value
if old_interpolation is not None:
user_log_dict.interpolation = old_interpolation
# Adjust the logging level in accordance with whether or not the 'debug' flag is on
log_level = 'DEBUG' if weewx.debug else 'INFO'
# Now we need to walk the structure, plugging in the values we know.
# First, we need a function to do this:
def _fix(section, key):
if isinstance(section[key], (list, tuple)):
# The value is a list or tuple
section[key] = [item.format(log_level=log_level,
address=address,
facility=facility,
process_name=process_name) for item in section[key]]
else:
# The value is a string
section[key] = section[key].format(log_level=log_level,
address=address,
facility=facility,
process_name=process_name)
# Using the function, walk the 'Logging' part of the structure
log_config['Logging'].walk(_fix)
# Extract just the part used by Python's logging facility
log_dict = log_config.dict().get('Logging', {})
# The root logger is denoted by an empty string by the logging facility. Unfortunately,
# ConfigObj does not accept an empty string as a key. So, instead, we use this hack:
try:
log_dict['loggers'][''] = log_dict['loggers']['root']
del log_dict['loggers']['root']
except KeyError:
pass
# Make sure values are of the right type
if 'version' in log_dict:
log_dict['version'] = to_int(log_dict['version'])
if 'disable_existing_loggers' in log_dict:
log_dict['disable_existing_loggers'] = to_bool(log_dict['disable_existing_loggers'])
if 'loggers' in log_dict:
for logger in log_dict['loggers']:
if 'propagate' in log_dict['loggers'][logger]:
log_dict['loggers'][logger]['propagate'] = to_bool(log_dict['loggers'][logger]['propagate'])
# Finally! The dictionary is ready. Set the defaults.
logging.config.dictConfig(log_dict)
def log_traceback(log_fn, prefix=''):
"""Log the stack traceback into a logger.
log_fn: One of the logging.Logger logging functions, such as logging.Logger.warning.
prefix: A string, which will be put in front of each log entry. Default is no string.
"""
import traceback
sfd = StringIO()
traceback.print_exc(file=sfd)
sfd.seek(0)
for line in sfd:
log_fn("%s%s", prefix, line)
class RotatingFileHandler(logging.handlers.RotatingFileHandler):
"""Shim for logging.handlers.RotatingFileHandler that does type conversions before passing on
to superclass."""
def __init__(self, filename, mode='a', maxBytes=0, backupCount=0, encoding=None, delay=False):
"Initializer. Perform the necessary type conversions."
super(RotatingFileHandler, self).__init__(filename,
mode=mode,
maxBytes=to_int(maxBytes),
backupCount=to_int(backupCount),
encoding=encoding,
delay=to_bool(delay))
