Hi, Doug I was able to replicate this. The problem is that ConfigObj, the utility that we use to read the configuration files, is trying to interpolate your 'format', rather than simply pass it on to the logging facility.
Replace your version of logger.py with the attached. You should find it in /usr/share/weewx/weeutil/logger.py Let me know if that helps. -tk On Wed, Sep 22, 2021 at 3:50 AM Doug Jenkins <[email protected]> wrote: > Good morning Tom: > > sorry about the late reply on this thread. I did try this on my setup. and > I got the following error when it tried to report: > > # ==== FOR REFERENCE : weewx.conf logging setup ===== > # Do not modify this. It is used when installing and updating weewx. > version = 4.5.1 > > [Logging] > [[formatters]] > [[[ddj]]] > format = '%(asctime)s [%(levelname)s] [%(threadName)s] [%(name)s : > %(funcName)s] %(message)s' > [[root]] > handlers = rotate, > [[loggers]] > [[handlers]] > [[[rotate]]] > level = INFO > formatter = ddj > class = logging.handlers.RotatingFileHandler > filename = /var/log/weewx/weewx.log > maxBytes = 1000000 > backupCount = 10 > # ===== END : weewx.conf logging setup ===== > > > # ===== ERROR message when I execute wee_reports. ===== > Using configuration file /etc/weewx/weewx.conf > Traceback (most recent call last): > > File "/usr/share/weewx/wee_reports", line 103, in <module> > main() > File "/usr/share/weewx/wee_reports", line 63, in main > weeutil.logger.setup('wee_reports', config_dict) > File "/usr/share/weewx/weeutil/logger.py", line 178, in setup > log_config['Logging'].walk(_fix) > File "/usr/lib/python3/dist-packages/configobj.py", line 919, in walk > out[entry] = self[entry].walk( > File "/usr/lib/python3/dist-packages/configobj.py", line 919, in walk > out[entry] = self[entry].walk( > File "/usr/lib/python3/dist-packages/configobj.py", line 894, in walk > val = function(self, entry, **keywargs) > File "/usr/share/weewx/weeutil/logger.py", line 164, in _fix > if isinstance(section[key], (list, tuple)): > File "/usr/lib/python3/dist-packages/configobj.py", line 557, in > __getitem__ > return self._interpolate(key, val) > File "/usr/lib/python3/dist-packages/configobj.py", line 549, in > _interpolate > return engine.interpolate(key, value) > File "/usr/lib/python3/dist-packages/configobj.py", line 352, in > interpolate > value = recursive_interpolate(key, value, self.section, {}) > File "/usr/lib/python3/dist-packages/configobj.py", line 330, in > recursive_interpolate > k, v, s = self._parse_match(match) > File "/usr/lib/python3/dist-packages/configobj.py", line 417, in > _parse_match > value, section = self._fetch(key) > File "/usr/lib/python3/dist-packages/configobj.py", line 386, in _fetch > raise MissingInterpolationOption(key) > configobj.MissingInterpolationOption: missing option "asctime" in > interpolation. > > # ==== END ERROR MESSAGE ===== > > any help is always appreciated! > > DDJ > > On Monday, September 6, 2021 at 8:43:21 AM UTC-4 [email protected] wrote: > >> It should be possible. Did you try it? What happened? >> >> On Sat, Sep 4, 2021 at 8:50 AM Doug Jenkins <[email protected]> >> wrote: >> >>> Not to bring up old issues but, I am trying to format my logs. I read >>> the wiki attached to this thread and I have this configuration working in >>> my weewx.conf: >>> >>> # ===== LOGGING SETUP (Configured 09/04/2021) ===== >>> [Logging]] >>> [[root]] >>> handlers = rotate, >>> [[loggers]] >>> [[handlers]] >>> [[[rotate]]] >>> level = INFO >>> formatter = standard >>> class = logging.handlers.RotatingFileHandler >>> filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log >>> maxBytes = 1000000 >>> backupCount = 10 >>> # ===== END LOGGING SETUP ===== >>> >>> But I want to format the logging out using a custom formatter as such: >>> # ==== PROPOSED CHANGE TO MY WEEWX.CONF ===== >>> [Logging] >>> [[formatters]] >>> [[[ddj]]] >>> format = >>> '%(asctime)s [%(levelname)s] [%(threadName)s] [%(name)s : %(funcName)s] >>> %(message)s' >>> [[root]] >>> handlers = rotate, >>> [[loggers]] >>> [[handlers]] >>> [[[rotate]]] >>> level = INFO >>> formatter = ddj >>> class = logging.handlers.RotatingFileHandler >>> filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log >>> maxBytes = 1000000 >>> backupCount = 10 >>> # ===== END LOGGING SETUP ===== >>> >>> >>> I am running Weewx 4.5.1 on a Raspberry Pi using Raspbian O/S (32bit) >>> which is essentially Debian buster. >>> >>> Is there a way to express a formatter in the weewx.conf file so the >>> logging object will format the results for me? >>> >>> On Friday, August 21, 2020 at 3:27:49 PM UTC-4 [email protected] wrote: >>> >>>> Just like to say I updated to 4.x and just added >>>> >>>> [Logging] >>>> [[handlers]] >>>> [[[rotate]]] >>>> filename = /tmp/weewx.log >>>> >>>> To the top section of my weewx.conf file (after version) and it is >>>> WONDERFUL to finally have real log output from weewx. It's beautiful! I >>>> might cry for not having to stare at Console.app and hope for something. >>>> :) >>>> Thank you all for your continued efforts. >>>> On Sunday, May 3, 2020 at 4:54:13 AM UTC-7 [email protected] wrote: >>>> >>>>> I like your fix better. Added in commit 6348741 >>>>> <https://github.com/weewx/weewx/commit/6348741c64f55c5b2d34d9b9c70a5ed0b6d55c7f> >>>>> . >>>>> >>>>> I also made the rotating log handler the default for MacOS. >>>>> >>>>> One problem I've discovered: if you 'declare' a handler, then the >>>>> logging facility goes ahead and instantiates it, whether or not you >>>>> actually use it. That means the program crashes when it tries to create >>>>> /var/log/weewx.log unless its run with sudo privileges. >>>>> >>>>> So, for non-Mac systems, I took the handler back out, and put >>>>> instructions on how to declare it in the wiki documentation >>>>> <https://bit.ly/2StYSHb>. >>>>> >>>>> -tk >>>>> >>>>> >>>>> On Sun, May 3, 2020 at 12:11 AM Graham Eddy <[email protected]> >>>>> wrote: >>>>> >>>>>> woops! i missed your reference to replacement weeutil/logger.py… >>>>>> putting that in place, it works fine on macos. >>>>>> it achieves the two things i was after: logs entries in a file (a >>>>>> basic function missing from weewx on macos for such a long time!), and >>>>>> allows customising message formats >>>>>> >>>>>> i have two critiques about this fix: >>>>>> >>>>>> (1) linux would want [syslog] handlers only but macos would want >>>>>> [rotate] for default. easiest is to default correctly for one in the code >>>>>> and override it in weewx.conf for the other (applied either by user or >>>>>> installation script). if both enabled, harmless (if not duplicating >>>>>> entries >>>>>> in one logfile on linux) but wasteful >>>>>> >>>>>> (2) shim approach is per- logging handler class only. would be >>>>>> pretty easy for python user to add another shim in user/extensions.py >>>>>> but… >>>>>> alternatively, do the string->int conversions e.g. add after line 130 in >>>>>> weeutil/logger.py “log_config['Logging'].walk(_fix)” something like: >>>>>> >>>>>> def _intify_string_leaf(section, key): >>>>>> >>>>>> if isinstance(section[key], str): >>>>>> # The value is a string... >>>>>> try: >>>>>> value = int(section[key]) >>>>>> except ValueError: >>>>>> pass >>>>>> else: >>>>>> # ... looks like an int, so make it one >>>>>> section[key] = value >>>>>> log_config['Logging'].walk(_intify_string_leaf) >>>>>> >>>>>> (this works.) conversely, intifying all leaves could have unintended >>>>>> consequences... >>>>>> >>>>>> thanks for all that! cheers >>>>>> >>>>>> On 3 May 2020, at 5:07 am, Tom Keffer <[email protected]> wrote: >>>>>> >>>>>> Try this version of weeutil/logger.py. It includes a shim that does >>>>>> the proper type conversion before passing on arguments to >>>>>> logging.handlers.RotatingFileHandler. >>>>>> >>>>>> To use should be as simple as adding this to weewx.conf (alas, NOT >>>>>> TESTED): >>>>>> >>>>>> [Logging] >>>>>> [[loggers]] >>>>>> [[[root]]] >>>>>> handlers = syslog, rotate >>>>>> [[handlers]] >>>>>> [[[rotate]]] >>>>>> filename = /tmp/weewx.log >>>>>> >>>>>> By default, it logs to /var/log/weewx.log. Hence, the need to >>>>>> override that option if you want to log to /tmp/weewx.log. >>>>>> >>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "weewx-user" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> >>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/weewx-user/AB7F41D1-FD08-400D-B80A-01E53BD42D5A%40gmail.com >>>>>> <https://groups.google.com/d/msgid/weewx-user/AB7F41D1-FD08-400D-B80A-01E53BD42D5A%40gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "weewx-user" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> >> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/weewx-user/819f1a6c-2ae0-4861-a5c1-726512243c5an%40googlegroups.com >>> <https://groups.google.com/d/msgid/weewx-user/819f1a6c-2ae0-4861-a5c1-726512243c5an%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "weewx-user" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/weewx-user/41b027e6-29a4-45c6-8bdb-8cb93a0a3640n%40googlegroups.com > <https://groups.google.com/d/msgid/weewx-user/41b027e6-29a4-45c6-8bdb-8cb93a0a3640n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "weewx-user" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/weewx-user/CAPq0zEArqj%3DiPjB1DFpjL_7_YRL4ARnj81eBv3yoa9ZO%2BU1ziQ%40mail.gmail.com.
# # Copyright (c) 2020 Tom Keffer <[email protected]> # # See the file LICENSE.txt for your full rights. # """WeeWX logging facility""" from __future__ import absolute_import import sys import logging.config import six from six.moves import StringIO import configobj import weewx # The logging defaults. Note that two kinds of placeholders are used: # # {value}: these are plugged in by the function setup(). # %(value)s: these are plugged in by the Python logging module. # LOGGING_STR = """[Logging] version = 1 disable_existing_loggers = False # Root logger [[root]] level = {log_level} handlers = syslog, # Additional loggers would go in the following section. This is useful for tailoring logging # for individual modules. [[loggers]] # Definitions of possible logging destinations [[handlers]] # System logger [[[syslog]]] level = DEBUG formatter = standard class = logging.handlers.SysLogHandler address = {address} facility = {facility} # Log to console [[[console]]] level = DEBUG formatter = verbose class = logging.StreamHandler # Alternate choice is 'ext://sys.stderr' stream = ext://sys.stdout # How to format log messages [[formatters]] [[[simple]]] format = "%(levelname)s %(message)s" [[[standard]]] format = "{process_name}[%(process)d] %(levelname)s %(name)s: %(message)s" [[[verbose]]] format = "%(asctime)s {process_name}[%(process)d] %(levelname)s %(name)s: %(message)s" # Format to use for dates and times: datefmt = %Y-%m-%d %H:%M:%S """ # These values are known only at runtime if sys.platform == "darwin": address = '/var/run/syslog' facility = 'local1' # Mac uses slightly different logging setup LOGGING_STR = """[Logging] version = 1 disable_existing_loggers = False # Root logger [[root]] level = {log_level} handlers = rotate, # Additional loggers would go in the following section. This is useful for tailoring logging # for individual modules. [[loggers]] # Definitions of possible logging destinations [[handlers]] # Log to a set of rotating files [[[rotate]]] level = DEBUG formatter = standard class = logging.handlers.RotatingFileHandler filename = /var/log/weewx.log maxBytes = 10000000 backupCount = 4 # Log to console [[[console]]] level = DEBUG formatter = verbose class = logging.StreamHandler # Alternate choice is 'ext://sys.stderr' stream = ext://sys.stdout # How to format log messages [[formatters]] [[[simple]]] format = "%(levelname)s %(message)s" [[[standard]]] format = "{process_name}[%(process)d] %(levelname)s %(name)s: %(message)s" [[[verbose]]] format = "%(asctime)s {process_name}[%(process)d] %(levelname)s %(name)s: %(message)s" # Format to use for dates and times: datefmt = %Y-%m-%d %H:%M:%S """ elif sys.platform.startswith('linux'): address = '/dev/log' facility = 'user' elif sys.platform.startswith('freebsd'): address = '/var/run/log' facility = 'user' elif sys.platform.startswith('netbsd'): address = '/var/run/log' facility = 'user' elif sys.platform.startswith('openbsd'): address = '/dev/log' facility = 'user' else: address = ('localhost', 514) facility = 'user' def setup(process_name, user_log_dict): """Set up the weewx logging facility""" # Create a ConfigObj from the default string. No interpolation (it interferes with the # interpolation directives embedded in the string). log_config = configobj.ConfigObj(StringIO(LOGGING_STR), interpolation=False, encoding='utf-8') # Turn off interpolation in the incoming dictionary. First save the old # value, then restore later. However, the incoming dictionary may be a simple # Python dictionary and not have interpolation. Hence the try block. try: old_interpolation = user_log_dict.interpolation user_log_dict.interpolation = False except AttributeError: old_interpolation = None # Merge in the user additions / changes: log_config.merge(user_log_dict) # Adjust the logging level in accordance with whether or not the 'debug' flag is on log_level = 'DEBUG' if weewx.debug else 'INFO' # Now we need to walk the structure, plugging in the values we know. # First, we need a function to do this: def _fix(section, key): if isinstance(section[key], (list, tuple)): # The value is a list or tuple section[key] = [item.format(log_level=log_level, address=address, facility=facility, process_name=process_name) for item in section[key]] else: # The value is a string section[key] = section[key].format(log_level=log_level, address=address, facility=facility, process_name=process_name) # Using the function, walk the 'Logging' part of the structure log_config['Logging'].walk(_fix) # Now walk the structure again, this time converting any strings to an appropriate type: log_config['Logging'].walk(_convert_from_string) # Extract just the part used by Python's logging facility log_dict = log_config.dict().get('Logging', {}) # Finally! The dictionary is ready. Set the defaults. logging.config.dictConfig(log_dict) # Restore the old interpolation value if old_interpolation is not None: user_log_dict.interpolation = old_interpolation def log_traceback(log_fn, prefix=''): """Log the stack traceback into a logger. log_fn: One of the logging.Logger logging functions, such as logging.Logger.warning. prefix: A string, which will be put in front of each log entry. Default is no string. """ import traceback sfd = StringIO() traceback.print_exc(file=sfd) sfd.seek(0) for line in sfd: log_fn("%s%s", prefix, line) def _convert_from_string(section, key): """If possible, convert any strings to an appropriate type.""" # Check to make sure it is a string if isinstance(section[key], six.string_types): if section[key].lower() == 'false': # It's boolean False section[key] = False elif section[key].lower() == 'true': # It's boolean True section[key] = True elif section[key].count('.') == 1: # Contains a decimal point. Could be float try: section[key] = float(section[key]) except ValueError: pass else: # Try integer? try: section[key] = int(section[key]) except ValueError: pass
