Usual caveat, I might be talking through my hat, but...
I've never been able to get csv working in a logical manner either,
but I was able to work around it as Mario and John did in this thread.
ie: I could only get the headers in overwrite mode.
This has prompted me to revisit the issue and I think there is
something off with the logic in cvs.py
It should always default to headers being True as that's the default
in the dictionary lookup in cvs.py so that's part of the puzzle.
The write header conditional in cvs.py acts after the file is opened
so I'm not sure that it has any affect as it will always return
not(true) for the path.exists? So we are down to the headers being
True or the overwrite flag being True.
> https://docs.python.org/3/library/os.path.html
[...]
> os.path.exists(path)
> Return True if path refers to an existing path or an open file descriptor.
That may have been the intent but I think the header flag should write
once only to the start of either file type (append or overwrite) if
it's set to True (the default).
If it's False then data only.
To that end, if that conditional is moved up, before the file is
opened it writes a single header line to each type. I've tested the
following change and it does what I describe.
Original...
def write_data(self, data):
flag = "a" if self.mode == 'append' else "w"
with open(self.filename, flag) as f:
if self.header and (not os.path.exists(self.filename) or
flag == "w"):
f.write('%s\n' % ','.join(self.sort_keys(data)))
f.write('%s\n' % ','.join(self.sort_data(data)))
Modified...
def write_data(self, data):
flag = "a" if self.mode == 'append' else "w"
h_eader = ''
if self.header and (not os.path.exists(self.filename) or flag == "w"):
h_eader = ('%s\n' % ','.join(self.sort_keys(data)))
with open(self.filename, flag) as f:
f.write(h_eader)
f.write('%s\n' % ','.join(self.sort_data(data)))
And because this was written as much in response to Jacques-Oliver's
query (and as we appear to have paused in this thread :-)) I'll
include the very useful date addition that mwall originally posted
way back in another thread, which needed a slight correction.
That's all in the attachment and should run as is.
If I'm correct in all the above, I'll add the date addition with a
suitable config option, test it and then post it as an issue to weewx
on github? Or add it to my github account to be picked up from there?
Cheers
Glenn
rorpi - read only raspberry pi & various weewx addons
https://github.com/glennmckechnie
On 6 October 2017 at 10:06, Thomas Keffer <[email protected]> wrote:
> Did you set the option header to true?
>
> [CSV]
> header = True
> .
> .
> .
>
> -tk
>
> On Thu, Oct 5, 2017 at 12:49 AM, Jacques-Olivier Farcy
> <[email protected]> wrote:
>>
>> Hi,
>>
>> I have exactly the same situation.
[...]
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
# $Id: csv.py 1319 2015-05-10 08:27:43Z mwall $
# Copyright 2015 Matthew Wall
import os
import os.path
import time
import weewx
import weewx.engine
import weeutil.weeutil
class CSV(weewx.engine.StdService):
def __init__(self, engine, config_dict):
super(CSV, self).__init__(engine, config_dict)
d = config_dict.get('CSV', {})
self.filename = d.get('filename', '/var/tmp/data.csv')
self.header = weeutil.weeutil.tobool(d.get('header', True))
self.mode = d.get('mode', 'append')
self.binding = d.get('binding', 'loop')
if self.binding == 'loop':
self.bind(weewx.NEW_LOOP_PACKET, self.handle_new_loop)
else:
self.bind(weewx.NEW_ARCHIVE_RECORD, self.handle_new_archive)
def handle_new_loop(self, event):
self.write_data(event.packet)
def handle_new_archive(self, event):
self.write_data(event.record)
def write_data(self, data):
flag = "a" if self.mode == 'append' else "w"
basename = self.filename[:-4]
tstr = time.strftime('%Y-%m', time.gmtime(data['dateTime']))
filename = "%s-%s.csv" % (basename, tstr)
h_eader = ''
if self.header and (not os.path.exists(filename) or flag == "w"):
h_eader = ('%s\n' % ','.join(self.sort_keys(data)))
with open(filename, flag) as f:
f.write(h_eader)
f.write('%s\n' % ','.join(self.sort_data(data)))
def sort_keys(self, record):
fields = ['dateTime']
for k in sorted(record):
if k != 'dateTime':
fields.append(k)
return fields
def sort_data(self, record):
fields = [str(record['dateTime'])]
for k in sorted(record):
if k != 'dateTime':
fields.append(str(record[k]))
return fields
