subject:"\[weewx\-user\] Re\: 4.x logging format"

Re: [weewx-user] Re: 4.x logging format

2021-09-22 Thread Doug Jenkins

Tom:

Thanks for the hotfix! I do appreciate it such a quick turnaround on a 
small item like this! :)

I switched the logging.py file at /usr/share/weewx/weeutil/ directory and 
tested it with both wee_reports and the standard weewx service. In both 
cases, Weewx formatted the log using my formatter expression. I made some 
changes to the formatter in my weewx.conf to ensure it was responding to 
the changes. The log reported the changes as expected.

I did not test wee_database, wee_import, or wee_device, but they probably 
will utilize this logging.py change and passthrough the formatter as 
expected.

Please let me know if you need any additional testing for this file or a 
issue/PR logged for the item. 

Again thanks for the great solution and continuous development of weewx! 

DDJ

On Wednesday, September 22, 2021 at 12:16:41 PM UTC-4 tke...@gmail.com 
wrote:

> Hi, Doug
>
> I was able to replicate this. The problem is that ConfigObj, the utility 
> that we use to read the configuration files, is trying to interpolate your 
> 'format', rather than simply pass it on to the logging facility.
>
> Replace your version of logger.py with the attached. You should find it 
> in /usr/share/weewx/weeutil/logger.py
>
> Let me know if that helps.
>
> -tk
>
>
>
> On Wed, Sep 22, 2021 at 3:50 AM Doug Jenkins  
> wrote:
>
>> Good morning Tom:
>>
>> sorry about the late reply on this thread. I did try this on my setup. 
>> and I got the following error when it tried to report:
>>
>> #  FOR REFERENCE : weewx.conf logging setup =
>> # Do not modify this. It is used when installing and updating weewx.
>> version = 4.5.1
>>
>> [Logging]
>>   [[formatters]]
>>   [[[ddj]]]
>> format = '%(asctime)s [%(levelname)s] [%(threadName)s] [%(name)s 
>> : %(funcName)s] %(message)s'
>>   [[root]]
>> handlers = rotate,
>>   [[loggers]]
>> [[handlers]]
>>   [[[rotate]]]
>> level = INFO
>> formatter = ddj
>> class = logging.handlers.RotatingFileHandler
>> filename = /var/log/weewx/weewx.log
>> maxBytes = 100
>> backupCount = 10
>> # = END : weewx.conf logging setup =
>>
>>
>> # = ERROR message when I execute wee_reports. =
>> Using configuration file /etc/weewx/weewx.conf
>> Traceback (most recent call last):
>>
>>   File "/usr/share/weewx/wee_reports", line 103, in 
>> main()
>>   File "/usr/share/weewx/wee_reports", line 63, in main
>> weeutil.logger.setup('wee_reports', config_dict)
>>   File "/usr/share/weewx/weeutil/logger.py", line 178, in setup
>> log_config['Logging'].walk(_fix)
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 919, in walk
>> out[entry] = self[entry].walk(
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 919, in walk
>> out[entry] = self[entry].walk(
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 894, in walk
>> val = function(self, entry, **keywargs)
>>   File "/usr/share/weewx/weeutil/logger.py", line 164, in _fix
>> if isinstance(section[key], (list, tuple)):
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 557, in 
>> __getitem__
>> return self._interpolate(key, val)
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 549, in 
>> _interpolate
>> return engine.interpolate(key, value)
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 352, in 
>> interpolate
>> value = recursive_interpolate(key, value, self.section, {})
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 330, in 
>> recursive_interpolate
>> k, v, s = self._parse_match(match)
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 417, in 
>> _parse_match
>> value, section = self._fetch(key)
>>   File "/usr/lib/python3/dist-packages/configobj.py", line 386, in _fetch
>> raise MissingInterpolationOption(key)
>> configobj.MissingInterpolationOption: missing option "asctime" in 
>> interpolation.
>>
>> #  END ERROR MESSAGE =
>>
>> any help is always appreciated!
>>
>> DDJ
>>
>> On Monday, September 6, 2021 at 8:43:21 AM UTC-4 tke...@gmail.com wrote:
>>
>>> It should be possible. Did you try it? What happened?
>>>
>>> On Sat, Sep 4, 2021 at 8:50 AM Doug Jenkins  
>>> wrote:
>>>
 Not to bring up old issues but, I am trying to format my logs. I read 
 the wiki attached to this thread and I have this configuration working in 
 my weewx.conf:

 # = LOGGING SETUP (Configured 09/04/2021) =
 [Logging]]
   [[root]]
 handlers = rotate,
   [[loggers]]
 [[handlers]]
   [[[rotate]]]
 level = INFO
 formatter = standard
 class = logging.handlers.RotatingFileHandler
 filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log
 maxBytes = 100
 backupCount = 10
 # = END LOGGING SETUP =

 But I want to format the logging out using a custom formatter as such:
 #

Re: [weewx-user] Re: 4.x logging format

2021-09-22 Thread Tom Keffer

Hi, Doug

I was able to replicate this. The problem is that ConfigObj, the utility
that we use to read the configuration files, is trying to interpolate your
'format', rather than simply pass it on to the logging facility.

Replace your version of logger.py with the attached. You should find it
in /usr/share/weewx/weeutil/logger.py

Let me know if that helps.

-tk



On Wed, Sep 22, 2021 at 3:50 AM Doug Jenkins  wrote:

> Good morning Tom:
>
> sorry about the late reply on this thread. I did try this on my setup. and
> I got the following error when it tried to report:
>
> #  FOR REFERENCE : weewx.conf logging setup =
> # Do not modify this. It is used when installing and updating weewx.
> version = 4.5.1
>
> [Logging]
>   [[formatters]]
>   [[[ddj]]]
> format = '%(asctime)s [%(levelname)s] [%(threadName)s] [%(name)s :
> %(funcName)s] %(message)s'
>   [[root]]
> handlers = rotate,
>   [[loggers]]
> [[handlers]]
>   [[[rotate]]]
> level = INFO
> formatter = ddj
> class = logging.handlers.RotatingFileHandler
> filename = /var/log/weewx/weewx.log
> maxBytes = 100
> backupCount = 10
> # = END : weewx.conf logging setup =
>
>
> # = ERROR message when I execute wee_reports. =
> Using configuration file /etc/weewx/weewx.conf
> Traceback (most recent call last):
>
>   File "/usr/share/weewx/wee_reports", line 103, in 
> main()
>   File "/usr/share/weewx/wee_reports", line 63, in main
> weeutil.logger.setup('wee_reports', config_dict)
>   File "/usr/share/weewx/weeutil/logger.py", line 178, in setup
> log_config['Logging'].walk(_fix)
>   File "/usr/lib/python3/dist-packages/configobj.py", line 919, in walk
> out[entry] = self[entry].walk(
>   File "/usr/lib/python3/dist-packages/configobj.py", line 919, in walk
> out[entry] = self[entry].walk(
>   File "/usr/lib/python3/dist-packages/configobj.py", line 894, in walk
> val = function(self, entry, **keywargs)
>   File "/usr/share/weewx/weeutil/logger.py", line 164, in _fix
> if isinstance(section[key], (list, tuple)):
>   File "/usr/lib/python3/dist-packages/configobj.py", line 557, in
> __getitem__
> return self._interpolate(key, val)
>   File "/usr/lib/python3/dist-packages/configobj.py", line 549, in
> _interpolate
> return engine.interpolate(key, value)
>   File "/usr/lib/python3/dist-packages/configobj.py", line 352, in
> interpolate
> value = recursive_interpolate(key, value, self.section, {})
>   File "/usr/lib/python3/dist-packages/configobj.py", line 330, in
> recursive_interpolate
> k, v, s = self._parse_match(match)
>   File "/usr/lib/python3/dist-packages/configobj.py", line 417, in
> _parse_match
> value, section = self._fetch(key)
>   File "/usr/lib/python3/dist-packages/configobj.py", line 386, in _fetch
> raise MissingInterpolationOption(key)
> configobj.MissingInterpolationOption: missing option "asctime" in
> interpolation.
>
> #  END ERROR MESSAGE =
>
> any help is always appreciated!
>
> DDJ
>
> On Monday, September 6, 2021 at 8:43:21 AM UTC-4 tke...@gmail.com wrote:
>
>> It should be possible. Did you try it? What happened?
>>
>> On Sat, Sep 4, 2021 at 8:50 AM Doug Jenkins 
>> wrote:
>>
>>> Not to bring up old issues but, I am trying to format my logs. I read
>>> the wiki attached to this thread and I have this configuration working in
>>> my weewx.conf:
>>>
>>> # = LOGGING SETUP (Configured 09/04/2021) =
>>> [Logging]]
>>>   [[root]]
>>> handlers = rotate,
>>>   [[loggers]]
>>> [[handlers]]
>>>   [[[rotate]]]
>>> level = INFO
>>> formatter = standard
>>> class = logging.handlers.RotatingFileHandler
>>> filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log
>>> maxBytes = 100
>>> backupCount = 10
>>> # = END LOGGING SETUP =
>>>
>>> But I want to format the logging out using a custom formatter as such:
>>> #  PROPOSED CHANGE TO MY WEEWX.CONF =
>>> [Logging]
>>>   [[formatters]]
>>> [[[ddj]]]
>>>   format =
>>> '%(asctime)s [%(levelname)s] [%(threadName)s] [%(name)s : %(funcName)s] 
>>> %(message)s'
>>>   [[root]]
>>> handlers = rotate,
>>>   [[loggers]]
>>> [[handlers]]
>>>   [[[rotate]]]
>>> level = INFO
>>> formatter = ddj
>>> class = logging.handlers.RotatingFileHandler
>>> filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log
>>> maxBytes = 100
>>> backupCount = 10
>>> # = END LOGGING SETUP =
>>>
>>>
>>> I am running Weewx 4.5.1 on a Raspberry Pi using Raspbian O/S (32bit)
>>> which is essentially Debian buster.
>>>
>>> Is there a way to express a formatter in the weewx.conf file so the
>>> logging object will format the results for me?
>>>
>>> On Friday, August 21, 2020 at 3:27:49 PM UTC-4 chri...@gmail.com wrote:
>>>
 Just like to say I updated to 4.x and just added

 [Logging]
   [[handlers]]

Re: [weewx-user] Re: 4.x logging format

2021-09-06 Thread Tom Keffer

It should be possible. Did you try it? What happened?

On Sat, Sep 4, 2021 at 8:50 AM Doug Jenkins  wrote:

> Not to bring up old issues but, I am trying to format my logs. I read the
> wiki attached to this thread and I have this configuration working in my
> weewx.conf:
>
> # = LOGGING SETUP (Configured 09/04/2021) =
> [Logging]]
>   [[root]]
> handlers = rotate,
>   [[loggers]]
> [[handlers]]
>   [[[rotate]]]
> level = INFO
> formatter = standard
> class = logging.handlers.RotatingFileHandler
> filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log
> maxBytes = 100
> backupCount = 10
> # = END LOGGING SETUP =
>
> But I want to format the logging out using a custom formatter as such:
> #  PROPOSED CHANGE TO MY WEEWX.CONF =
> [Logging]
>   [[formatters]]
> [[[ddj]]]
>   format =
> '%(asctime)s [%(levelname)s] [%(threadName)s] [%(name)s : %(funcName)s] 
> %(message)s'
>   [[root]]
> handlers = rotate,
>   [[loggers]]
> [[handlers]]
>   [[[rotate]]]
> level = INFO
> formatter = ddj
> class = logging.handlers.RotatingFileHandler
> filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log
> maxBytes = 100
> backupCount = 10
> # = END LOGGING SETUP =
>
>
> I am running Weewx 4.5.1 on a Raspberry Pi using Raspbian O/S (32bit)
> which is essentially Debian buster.
>
> Is there a way to express a formatter in the weewx.conf file so the
> logging object will format the results for me?
>
> On Friday, August 21, 2020 at 3:27:49 PM UTC-4 chri...@gmail.com wrote:
>
>> Just like to say I updated to 4.x and just added
>>
>> [Logging]
>>   [[handlers]]
>> [[[rotate]]]
>>   filename = /tmp/weewx.log
>>
>> To the top section of my weewx.conf file (after version) and it is
>> WONDERFUL to finally have real log output from weewx.  It's beautiful! I
>> might cry for not having to stare at Console.app and hope for something.  :)
>> Thank you all for your continued efforts.
>> On Sunday, May 3, 2020 at 4:54:13 AM UTC-7 tke...@gmail.com wrote:
>>
>>> I like your fix better. Added in commit 6348741
>>> 
>>> .
>>>
>>> I also made the rotating log handler the default for MacOS.
>>>
>>> One problem I've discovered: if you 'declare' a handler, then the
>>> logging facility goes ahead and instantiates it, whether or not you
>>> actually use it. That means the program crashes when it tries to create
>>> /var/log/weewx.log unless its run with sudo privileges.
>>>
>>> So, for non-Mac systems, I took the handler back out, and put
>>> instructions on how to declare it in the wiki documentation
>>> .
>>>
>>> -tk
>>>
>>>
>>> On Sun, May 3, 2020 at 12:11 AM Graham Eddy  wrote:
>>>
 woops! i missed your reference to replacement weeutil/logger.py…
 putting that in place, it works fine on macos.
 it achieves the two things i was after: logs entries in a file (a basic
 function missing from weewx on macos for such a long time!), and allows
 customising message formats

 i have two critiques about this fix:

   (1) linux would want [syslog] handlers only but macos would want
 [rotate] for default. easiest is to default correctly for one in the code
 and override it in weewx.conf for the other (applied either by user or
 installation script). if both enabled, harmless (if not duplicating entries
 in one logfile on linux) but wasteful

   (2) shim approach is per- logging handler class only. would be pretty
 easy for python user to add another shim in user/extensions.py but…
 alternatively, do the string->int conversions e.g. add after line 130 in
 weeutil/logger.py “log_config['Logging'].walk(_fix)” something like:

 def _intify_string_leaf(section, key):

 if isinstance(section[key], str):
 # The value is a string...
 try:
 value = int(section[key])
 except ValueError:
 pass
 else:
 # ... looks like an int, so make it one
 section[key] = value
 log_config['Logging'].walk(_intify_string_leaf)

 (this works.) conversely, intifying all leaves could have unintended
 consequences...

 thanks for all that! cheers

 On 3 May 2020, at 5:07 am, Tom Keffer  wrote:

 Try this version of weeutil/logger.py. It includes a shim that does the
 proper type conversion before passing on arguments to
 logging.handlers.RotatingFileHandler.

 To use should be as simple as adding this to weewx.conf (alas, NOT
 TESTED):

 [Logging]
   [[loggers]]
 [[[root]]]
   handlers = syslog, rotate
   [[handlers]]
 [[[rotate]]]
   filename = /tmp/weewx.log

Re: [weewx-user] Re: 4.x logging format

2021-09-04 Thread Doug Jenkins

Not to bring up old issues but, I am trying to format my logs. I read the 
wiki attached to this thread and I have this configuration working in my 
weewx.conf:

# = LOGGING SETUP (Configured 09/04/2021) =
[Logging]]
  [[root]]
handlers = rotate,
  [[loggers]]
[[handlers]]
  [[[rotate]]]
level = INFO
formatter = standard
class = logging.handlers.RotatingFileHandler
filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log
maxBytes = 100
backupCount = 10
# = END LOGGING SETUP =

But I want to format the logging out using a custom formatter as such:
#  PROPOSED CHANGE TO MY WEEWX.CONF =
[Logging]
  [[formatters]]
[[[ddj]]]
  format = 
'%(asctime)s [%(levelname)s] [%(threadName)s] [%(name)s : %(funcName)s] 
%(message)s'
  [[root]]
handlers = rotate,
  [[loggers]]
[[handlers]]
  [[[rotate]]]
level = INFO
formatter = ddj
class = logging.handlers.RotatingFileHandler
filename = /mnt/sandisk_512_ssd/weewx/logs/weewx.log
maxBytes = 100
backupCount = 10
# = END LOGGING SETUP =


I am running Weewx 4.5.1 on a Raspberry Pi using Raspbian O/S (32bit) which 
is essentially Debian buster.

Is there a way to express a formatter in the weewx.conf file so the logging 
object will format the results for me?

On Friday, August 21, 2020 at 3:27:49 PM UTC-4 chri...@gmail.com wrote:

> Just like to say I updated to 4.x and just added
>  
> [Logging]
>   [[handlers]]
> [[[rotate]]]
>   filename = /tmp/weewx.log
>
> To the top section of my weewx.conf file (after version) and it is 
> WONDERFUL to finally have real log output from weewx.  It's beautiful! I 
> might cry for not having to stare at Console.app and hope for something.  :)
> Thank you all for your continued efforts. 
> On Sunday, May 3, 2020 at 4:54:13 AM UTC-7 tke...@gmail.com wrote:
>
>> I like your fix better. Added in commit 6348741 
>> 
>> . 
>>
>> I also made the rotating log handler the default for MacOS.
>>
>> One problem I've discovered: if you 'declare' a handler, then the logging 
>> facility goes ahead and instantiates it, whether or not you actually use 
>> it. That means the program crashes when it tries to create 
>> /var/log/weewx.log unless its run with sudo privileges.
>>
>> So, for non-Mac systems, I took the handler back out, and put 
>> instructions on how to declare it in the wiki documentation 
>> .
>>
>> -tk
>>
>>
>> On Sun, May 3, 2020 at 12:11 AM Graham Eddy  wrote:
>>
>>> woops! i missed your reference to replacement weeutil/logger.py…
>>> putting that in place, it works fine on macos.
>>> it achieves the two things i was after: logs entries in a file (a basic 
>>> function missing from weewx on macos for such a long time!), and allows 
>>> customising message formats
>>>
>>> i have two critiques about this fix:
>>>
>>>   (1) linux would want [syslog] handlers only but macos would want 
>>> [rotate] for default. easiest is to default correctly for one in the code 
>>> and override it in weewx.conf for the other (applied either by user or 
>>> installation script). if both enabled, harmless (if not duplicating entries 
>>> in one logfile on linux) but wasteful
>>>
>>>   (2) shim approach is per- logging handler class only. would be pretty 
>>> easy for python user to add another shim in user/extensions.py but… 
>>> alternatively, do the string->int conversions e.g. add after line 130 in 
>>> weeutil/logger.py “log_config['Logging'].walk(_fix)” something like:
>>>
>>> def _intify_string_leaf(section, key):
>>>
>>> if isinstance(section[key], str):
>>> # The value is a string...
>>> try:
>>> value = int(section[key])
>>> except ValueError:
>>> pass
>>> else:
>>> # ... looks like an int, so make it one
>>> section[key] = value
>>> log_config['Logging'].walk(_intify_string_leaf)
>>>
>>> (this works.) conversely, intifying all leaves could have unintended 
>>> consequences...
>>>
>>> thanks for all that! cheers
>>>
>>> On 3 May 2020, at 5:07 am, Tom Keffer  wrote:
>>>
>>> Try this version of weeutil/logger.py. It includes a shim that does the 
>>> proper type conversion before passing on arguments to 
>>> logging.handlers.RotatingFileHandler.
>>>
>>> To use should be as simple as adding this to weewx.conf (alas, NOT 
>>> TESTED):
>>>
>>> [Logging]
>>>   [[loggers]]
>>> [[[root]]]
>>>   handlers = syslog, rotate
>>>   [[handlers]]
>>> [[[rotate]]]
>>>   filename = /tmp/weewx.log
>>>
>>> By default, it logs to /var/log/weewx.log. Hence, the need to override 
>>> that option if you want to log to /tmp/weewx.log.
>>>
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "weewx-user" group.
>>> To

Re: [weewx-user] Re: 4.x logging format

2020-05-03 Thread Tom Keffer

I like your fix better. Added in commit 6348741

.

I also made the rotating log handler the default for MacOS.

One problem I've discovered: if you 'declare' a handler, then the logging
facility goes ahead and instantiates it, whether or not you actually use
it. That means the program crashes when it tries to create
/var/log/weewx.log unless its run with sudo privileges.

So, for non-Mac systems, I took the handler back out, and put instructions
on how to declare it in the wiki documentation .

-tk


On Sun, May 3, 2020 at 12:11 AM Graham Eddy  wrote:

> woops! i missed your reference to replacement weeutil/logger.py…
> putting that in place, it works fine on macos.
> it achieves the two things i was after: logs entries in a file (a basic
> function missing from weewx on macos for such a long time!), and allows
> customising message formats
>
> i have two critiques about this fix:
>
>   (1) linux would want [syslog] handlers only but macos would want
> [rotate] for default. easiest is to default correctly for one in the code
> and override it in weewx.conf for the other (applied either by user or
> installation script). if both enabled, harmless (if not duplicating entries
> in one logfile on linux) but wasteful
>
>   (2) shim approach is per- logging handler class only. would be pretty
> easy for python user to add another shim in user/extensions.py but…
> alternatively, do the string->int conversions e.g. add after line 130 in
> weeutil/logger.py “log_config['Logging'].walk(_fix)” something like:
>
> def _intify_string_leaf(section, key):
>
> if isinstance(section[key], str):
> # The value is a string...
> try:
> value = int(section[key])
> except ValueError:
> pass
> else:
> # ... looks like an int, so make it one
> section[key] = value
> log_config['Logging'].walk(_intify_string_leaf)
>
> (this works.) conversely, intifying all leaves could have unintended
> consequences...
>
> thanks for all that! cheers
>
> On 3 May 2020, at 5:07 am, Tom Keffer  wrote:
>
> Try this version of weeutil/logger.py. It includes a shim that does the
> proper type conversion before passing on arguments to
> logging.handlers.RotatingFileHandler.
>
> To use should be as simple as adding this to weewx.conf (alas, NOT TESTED):
>
> [Logging]
>   [[loggers]]
> [[[root]]]
>   handlers = syslog, rotate
>   [[handlers]]
> [[[rotate]]]
>   filename = /tmp/weewx.log
>
> By default, it logs to /var/log/weewx.log. Hence, the need to override
> that option if you want to log to /tmp/weewx.log.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/AB7F41D1-FD08-400D-B80A-01E53BD42D5A%40gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zEAunFHwK8QTVrsDhbPONif7Z_8i5L4jWn8O1g52dtW7rA%40mail.gmail.com.

Re: [weewx-user] Re: 4.x logging format

2020-05-03 Thread Graham Eddy

woops! i missed your reference to replacement weeutil/logger.py…
putting that in place, it works fine on macos.
it achieves the two things i was after: logs entries in a file (a basic 
function missing from weewx on macos for such a long time!), and allows 
customising message formats

i have two critiques about this fix:

  (1) linux would want [syslog] handlers only but macos would want [rotate] for 
default. easiest is to default correctly for one in the code and override it in 
weewx.conf for the other (applied either by user or installation script). if 
both enabled, harmless (if not duplicating entries in one logfile on linux) but 
wasteful

  (2) shim approach is per- logging handler class only. would be pretty easy 
for python user to add another shim in user/extensions.py but… alternatively, 
do the string->int conversions e.g. add after line 130 in weeutil/logger.py 
“log_config['Logging'].walk(_fix)” something like:
def _intify_string_leaf(section, key):
if isinstance(section[key], str):
# The value is a string...
try:
value = int(section[key])
except ValueError:
pass
else:
# ... looks like an int, so make it one
section[key] = value
log_config['Logging'].walk(_intify_string_leaf)
(this works.) conversely, intifying all leaves could have unintended 
consequences...

thanks for all that! cheers

> On 3 May 2020, at 5:07 am, Tom Keffer  wrote:
> 
> Try this version of weeutil/logger.py. It includes a shim that does the 
> proper type conversion before passing on arguments to 
> logging.handlers.RotatingFileHandler.
> 
> To use should be as simple as adding this to weewx.conf (alas, NOT TESTED):
> 
> [Logging]
>   [[loggers]]
> [[[root]]]
>   handlers = syslog, rotate
>   [[handlers]]
> [[[rotate]]]
>   filename = /tmp/weewx.log
> 
> By default, it logs to /var/log/weewx.log. Hence, the need to override that 
> option if you want to log to /tmp/weewx.log.

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/AB7F41D1-FD08-400D-B80A-01E53BD42D5A%40gmail.com.

Re: [weewx-user] Re: 4.x logging format

2020-05-02 Thread Graham Eddy

simplified logging stanza doesn’t work and shows type conversion fix needed.
details follow...

using that simplified stanza in weewx.conf gives runtime error:

  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
 line 381, in resolve
name = s.split('.')
AttributeError: 'NoneType' object has no attribute 'split'

adding class line back into config viz:

[Logging]
#  [[loggers]]
#[[[root]]]
#  level = DEBUG
#  propogate = True
#  handlers = syslog, rotate
  [[handlers]]
[[[rotate]]]
#  level = DEBUG
#  formatter = standard
  class = logging.handlers.RotatingFileHandler
  filename = /tmp/weewx.log
#  maxBytes = 1000
#  backupCount = 2

does not give immediate runtime error
but also adds no logging messages to /tmp/weewx.log

RotatingFileHandler documentation says defaults for maxBytes and backupCount 
are zero
and either of these being zero means logging is not done.
experimentation shows documentation is correct i.e. this config does not work
therefore both maxBytes and backupCount must be definable to be non-zero
so we need the type conversion fix

note: i am using /tmp/weewx.log here because i am presently fiddling with new 
version on test system and don’t want weewxd running rampant with root 
privilege. at least, not yet :-)


> On 3 May 2020, at 5:07 am, Tom Keffer  wrote:
> 
> Great ideas! Thanks for your thoughts.
> 
> Try this version of weeutil/logger.py. It includes a shim that does the 
> proper type conversion before passing on arguments to 
> logging.handlers.RotatingFileHandler.
> 
> To use should be as simple as adding this to weewx.conf (alas, NOT TESTED):
> 
> [Logging]
>   [[loggers]]
> [[[root]]]
>   handlers = syslog, rotate
>   [[handlers]]
> [[[rotate]]]
>   filename = /tmp/weewx.log
> 
> By default, it logs to /var/log/weewx.log. Hence, the need to override that 
> option if you want to log to /tmp/weewx.log.
> 
> -tk
> 
> 
> On Sat, May 2, 2020 at 8:05 AM Graham Eddy  > wrote:
> i think we all agree that syslog on macos is a lost cause.
> the corollary is that pointing python logging at syslog (via 
> logging.handlers.SysLogHandler) is doomed
> 
> i accept that syslog works fine on most platforms, but it is not feasible for 
> macos
> but there is the simple alternative for macos users of using a different 
> handler:
> 
> i patched log_dict in bin/weeutil/logger.py to add a 
> logging.handlers.RotatingFileHandler and it worked fine.
> adding it via weewx.conf gives a type conversion error that seems simple to 
> fix.
> with the fix, i would suggest this be added to the macos installation 
> instructions for user to add to weewx.conf (or make it part of the 
> installation process)
> 
> details follow...
> 
> weewx.conf (yes, using that ‘root’ hack introduced to work around a ConfigObj 
> limitation):
> 
> [Logging]
>   [[loggers]]
> [[[root]]]
>   level = DEBUG
>   propogate = True
>   handlers = syslog, rotate
>   [[handlers]]
> [[[rotate]]]
>   level = DEBUG
>   formatter = standard
>   class = logging.handlers.RotatingFileHandler
>   filename = /tmp/weewx.log
>   maxBytes = 1000
>   backupCount = 2

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/DB9C3A14-40D2-4B9A-9604-19ED16BC9BAB%40gmail.com.

Re: [weewx-user] Re: 4.x logging format

2020-05-02 Thread Tom Keffer

Great ideas! Thanks for your thoughts.

Try this version of weeutil/logger.py. It includes a shim that does the
proper type conversion before passing on arguments to
logging.handlers.RotatingFileHandler.

To use should be as simple as adding this to weewx.conf (alas, NOT TESTED):

[Logging]
  [[loggers]]
[[[root]]]
  handlers = syslog, rotate
  [[handlers]]
[[[rotate]]]
  filename = /tmp/weewx.log

By default, it logs to /var/log/weewx.log. Hence, the need to override that
option if you want to log to /tmp/weewx.log.

-tk


On Sat, May 2, 2020 at 8:05 AM Graham Eddy  wrote:

> i think we all agree that syslog on macos is a lost cause.
> the corollary is that pointing python logging at syslog (via
> logging.handlers.SysLogHandler) is doomed
>
> i accept that syslog works fine on most platforms, but it is not feasible
> for macos
> but there is the simple alternative for macos users of using a different
> handler:
>
> i patched log_dict in bin/weeutil/logger.py to add a
> logging.handlers.RotatingFileHandler and it worked fine.
> adding it via weewx.conf gives a type conversion error that seems simple
> to fix.
> with the fix, i would suggest this be added to the macos installation
> instructions for user to add to weewx.conf (or make it part of the
> installation process)
>
> details follow...
>
> weewx.conf (yes, using that ‘root’ hack introduced to work around a
> ConfigObj limitation):
>
> [Logging]
>   [[loggers]]
> [[[root]]]
>   level = DEBUG
>   propogate = True
>   handlers = syslog, rotate
>   [[handlers]]
> [[[rotate]]]
>   level = DEBUG
>   formatter = standard
>   class = logging.handlers.RotatingFileHandler
>   filename = /tmp/weewx.log
>   maxBytes = 1000
>   backupCount = 2
>
>
> gives the following error, which on the face of it would take a simple fix
> of type conversions:
>
> Traceback (most recent call last):
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
> line 563, in configure
> handler = self.configure_handler(handlers[name])
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
> line 736, in configure_handler
> result = factory(**kwargs)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/handlers.py",
> line 146, in __init__
> if maxBytes > 0:
> TypeError: '>' not supported between instances of 'str' and ‘int’ < - - -
> SEE HER*E*
>
> The above exception was the direct cause of the following exception:
>
> Traceback (most recent call last):
>   File "bin/weewxd", line 261, in 
> main()
>   File "bin/weewxd", line 136, in main
> weeutil.logger.setup(options.log_label, config_dict)
>   File "/opt/weewx/bin/weeutil/logger.py", line 203, in setup
> logging.config.dictConfig(log_dict)
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
> line 800, in dictConfig
> dictConfigClass(config).configure()
>   File
> "/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
> line 571, in configure
> '%r' % name) from e
> ValueError: Unable to configure handler 'rotate'
>
>
> the log_dict it produces before crashing seems correct:
>
> {'version': 1, 'disable_existing_loggers': False, 'loggers': {'':
> {'level': 'DEBUG', 'propagate': True, 'handlers': ['syslog'], 'propogate':
> 'True'}}, 'handlers': {'syslog': {'level': 'DEBUG', 'formatter':
> 'standard', 'class': 'logging.handlers.SysLogHandler', 'address':
> '/var/run/syslog', 'facility': 'local1'}, 'console': {'level': 'DEBUG',
> 'formatter': 'verbose', 'class': 'logging.StreamHandler', 'stream': '
> ext://sys.stdout'}, 'rotate': {'level': 'DEBUG', 'formatter': 'standard',
> 'class': 'logging.handlers.RotatingFileHandler', 'filename':
> '/tmp/weewx.log', 'maxBytes': '1000', 'backupCount': '2'}},
> 'formatters': {'simple': {'format': '%(levelname)s %(message)s'},
> 'standard': {'format': 'weewx[%(process)d] %(levelname)s %(name)s:
> %(message)s'}, 'verbose': {'format': '%(asctime)s  weewx[%(process)d]
> %(levelname)s %(name)s: %(message)s', 'datefmt': '%Y-%m-%d %H:%M:%S'}}}
>
>
> cheers
>
> On 2 May 2020, at 10:42 pm, Tom Keffer  wrote:
>
> Logging from Python under MacOS High Sierra and later is a complete
> mystery. I have been unable to get it to work and, it seems, neither have
> other
> 
>  users
> 
> .
>
> Wish I could be of more help...
>
> -tk
>
> On Sat, May 2, 2020 at 5:26 AM Graham Eddy  wrote:
>
>> i reported this incorrectly. instead of giveing me an unchanged logger
>> format, it gives me no lines at all (from the point the weewx.conf log
>> format is parsed).  even the following does the same (no logger output).

Re: [weewx-user] Re: 4.x logging format

2020-05-02 Thread Graham Eddy

i think we all agree that syslog on macos is a lost cause.
the corollary is that pointing python logging at syslog (via 
logging.handlers.SysLogHandler) is doomed

i accept that syslog works fine on most platforms, but it is not feasible for 
macos
but there is the simple alternative for macos users of using a different 
handler:

i patched log_dict in bin/weeutil/logger.py to add a 
logging.handlers.RotatingFileHandler and it worked fine.
adding it via weewx.conf gives a type conversion error that seems simple to fix.
with the fix, i would suggest this be added to the macos installation 
instructions for user to add to weewx.conf (or make it part of the installation 
process)

details follow...

weewx.conf (yes, using that ‘root’ hack introduced to work around a ConfigObj 
limitation):

[Logging]
  [[loggers]]
[[[root]]]
  level = DEBUG
  propogate = True
  handlers = syslog, rotate
  [[handlers]]
[[[rotate]]]
  level = DEBUG
  formatter = standard
  class = logging.handlers.RotatingFileHandler
  filename = /tmp/weewx.log
  maxBytes = 1000
  backupCount = 2

gives the following error, which on the face of it would take a simple fix of 
type conversions:

Traceback (most recent call last):
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
 line 563, in configure
handler = self.configure_handler(handlers[name])
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
 line 736, in configure_handler
result = factory(**kwargs)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/handlers.py",
 line 146, in __init__
if maxBytes > 0:
TypeError: '>' not supported between instances of 'str' and ‘int’ < - - - SEE 
HERE

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "bin/weewxd", line 261, in 
main()
  File "bin/weewxd", line 136, in main
weeutil.logger.setup(options.log_label, config_dict)
  File "/opt/weewx/bin/weeutil/logger.py", line 203, in setup
logging.config.dictConfig(log_dict)
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
 line 800, in dictConfig
dictConfigClass(config).configure()
  File 
"/opt/local/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/config.py",
 line 571, in configure
'%r' % name) from e
ValueError: Unable to configure handler 'rotate'

the log_dict it produces before crashing seems correct:

{'version': 1, 'disable_existing_loggers': False, 'loggers': {'': {'level': 
'DEBUG', 'propagate': True, 'handlers': ['syslog'], 'propogate': 'True'}}, 
'handlers': {'syslog': {'level': 'DEBUG', 'formatter': 'standard', 'class': 
'logging.handlers.SysLogHandler', 'address': '/var/run/syslog', 'facility': 
'local1'}, 'console': {'level': 'DEBUG', 'formatter': 'verbose', 'class': 
'logging.StreamHandler', 'stream': 'ext://sys.stdout'}, 'rotate': {'level': 
'DEBUG', 'formatter': 'standard', 'class': 
'logging.handlers.RotatingFileHandler', 'filename': '/tmp/weewx.log', 
'maxBytes': '1000', 'backupCount': '2'}}, 'formatters': {'simple': 
{'format': '%(levelname)s %(message)s'}, 'standard': {'format': 
'weewx[%(process)d] %(levelname)s %(name)s: %(message)s'}, 'verbose': 
{'format': '%(asctime)s  weewx[%(process)d] %(levelname)s %(name)s: 
%(message)s', 'datefmt': '%Y-%m-%d %H:%M:%S'}}}

cheers

> On 2 May 2020, at 10:42 pm, Tom Keffer  wrote:
> 
> Logging from Python under MacOS High Sierra and later is a complete mystery. 
> I have been unable to get it to work and, it seems, neither have other 
> 
>  users 
> .
> 
> Wish I could be of more help...
> 
> -tk
> 
> On Sat, May 2, 2020 at 5:26 AM Graham Eddy  > wrote:
> i reported this incorrectly. instead of giveing me an unchanged logger 
> format, it gives me no lines at all (from the point the weewx.conf log format 
> is parsed).  even the following does the same (no logger output). commenting 
> out the “format” line reverts to lines being produced, in original format of 
> course
> 
> [Logging]
>   [[formatters]]
> [[[standard]]]
>   format = '%(message)s'

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/85C23BE1-912C-4EAA-80AC-F7E81F8EE021%40gmail.com.

Re: [weewx-user] Re: 4.x logging format

2020-05-02 Thread Tom Keffer

Logging from Python under MacOS High Sierra and later is a complete
mystery. I have been unable to get it to work and, it seems, neither have
other

 users

.

Wish I could be of more help...

-tk

On Sat, May 2, 2020 at 5:26 AM Graham Eddy  wrote:

> i reported this incorrectly. instead of giveing me an unchanged logger
> format, it gives me no lines at all (from the point the weewx.conf log
> format is parsed).  even the following does the same (no logger output).
> commenting out the “format” line reverts to lines being produced, in
> original format of course
>
> [Logging]
>   [[formatters]]
> [[[standard]]]
>   format = '%(message)s'
>
>
> On 2 May 2020, at 10:02 pm, Graham Eddy  wrote:
>
> i am trying to change the weewx log format. i have added the following to
> my weewx.conf, but it has no effect. how should i do it? (weewx 4.0.0 on
> macos 10.13.6 high sierra). and can i use { style?
>
> [Logging]
>   [[formatters]]
> [[[verbose]]]
>   format = '%(asctime)s %(levelname)s %(name)s|%(message)s'
>   datefmt = %Y-%m-%d %H:%M:%S
> [[[standard]]]
>   format = '%(asctime)s %(levelname)s %(name)s|%(message)s'
>   datefmt = %Y-%m-%d %H:%M:%S
> [[[simple]]]
>   format = '%(asctime)s %(levelname)s %(name)s|%(message)s'
>   datefmt = %Y-%m-%d %H:%M:%S
>
>
> but it gives me the weewx default
>
> May  2 21:59:44 pebble weewx[1663] INFO __main_] : Platform
> Darwin-17.7.0-x86_64-i386-64bit
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "weewx-user" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to weewx-user+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/weewx-user/55F10FED-0D65-4010-9D74-D61203848291%40gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/CAPq0zEDxB6XB6HRk56EnZw6ONyt_r9T6SMEhCFFfxe4Kv1PD7A%40mail.gmail.com.

[weewx-user] Re: 4.x logging format

2020-05-02 Thread Graham Eddy

i reported this incorrectly. instead of giveing me an unchanged logger format, 
it gives me no lines at all (from the point the weewx.conf log format is 
parsed).  even the following does the same (no logger output). commenting out 
the “format” line reverts to lines being produced, in original format of course

[Logging]
  [[formatters]]
[[[standard]]]
  format = '%(message)s'


> On 2 May 2020, at 10:02 pm, Graham Eddy  wrote:
> 
> i am trying to change the weewx log format. i have added the following to my 
> weewx.conf, but it has no effect. how should i do it? (weewx 4.0.0 on macos 
> 10.13.6 high sierra). and can i use { style?
> 
> [Logging]
>   [[formatters]]
> [[[verbose]]]
>   format = '%(asctime)s %(levelname)s %(name)s|%(message)s'
>   datefmt = %Y-%m-%d %H:%M:%S
> [[[standard]]]
>   format = '%(asctime)s %(levelname)s %(name)s|%(message)s'
>   datefmt = %Y-%m-%d %H:%M:%S
> [[[simple]]]
>   format = '%(asctime)s %(levelname)s %(name)s|%(message)s'
>   datefmt = %Y-%m-%d %H:%M:%S
> 
> but it gives me the weewx default
> May  2 21:59:44 pebble weewx[1663] INFO __main_] : Platform 
> Darwin-17.7.0-x86_64-i386-64bit
> 

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/weewx-user/55F10FED-0D65-4010-9D74-D61203848291%40gmail.com.

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

Re: [weewx-user] Re: 4.x logging format

[weewx-user] Re: 4.x logging format

11 matches

Site Navigation

Mail list logo

Footer information