Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Andrew Milner]

I am sure that using a Pi for VP stations will be a better answer.  VP's 
output loop data at I believe 2 second intervals - even if record interval 
is 5 minutes.  Fine offset on the other hand will only output fresh data at 
a minimum of every 54 seconds so is usually configured to be polled every 
minute - a big difference in method of operation and load on the processor.



On Thursday, 24 May 2018 04:41:13 UTC+3, Hrmeteohub Pljusak wrote:
>
> Dear Vince, 
>
> I believe you are right on the money with your last remark. Dead 
> software works as long as you have hardware, so I tried with cramming 
> weewx in router, and strange things happened. 
> Creating embedded firmware with Python inside for 24/7 task is 
> problematic at best. Low RAM, slow processor and so on is all hitting me 
> in the head. Check the capture, it runs with only 20MB of free RAM. I 
> have to move database out of RAM to USB stic/thumbdrive/disk. 
> By the way this router is only $17. ...not so important in grand scheme 
> of things. 
>
> There is no skin (generating) at all. Only CVS is rewritten to one file 
> (in RAM). There is no room/processor power for graphing. Even with such 
> "naked" weewx the processor routinely runs at more than 30%. 
>
> So, let me recap. Syncing archiving interval with WMR and WH stations on 
> 5 minutes helped, and it works ok now. My friends are testing other 
> stations. 
>
> However, if moving database to USB disk does not help, there are only 
> two way out of this - using embedded device with more RAM, or clearing 
> measurement on VP periodically to prevent this in the first place. I am 
> not sure the first one will help, and latter feels bad on so many 
> levels. If router with more RAM does not help, Raspberry pi, here I come 
> fo VP stations. 
>
> Thanx! 
>
> Nick 
>
> On 22.5.2018. 19:43, vince wrote: 
> > On Tuesday, May 22, 2018 at 5:27:23 AM UTC-7, Andrew Milner wrote: 
> > 
> > As has been said many times in this forum comments like 'VP 
> > struggles' do not tell us anything - we (and you) need to look at 
> > the log, maybe run with debug enabled and so forth in order to 
> > understand and more importantly see what is going on within weewx - 
> > any error messages and so forth. 
> > 
> > 
> > On a 60MB RAM system it is possible the computer simply is out of memory 
> > and oddities are happening. 
> > Turn off the CSV extension and see if that helps. 
> > Alternately turn off WU and run only the CSV extension and see if 'that' 
> > helps. 
> > 
> > But ideally you'd have some data you could tell us about the ram/cpu 
> > usage on the system. 
> > 
> > You haven't shown us your custom weewx skin either.  If that's too 
> > complicated the computer will work too hard when the output is 
> > generated.  We have had numerous occcasions where people trying to run a 
> > complicated skin makes the system unstable or unpredictable (my example 
> > - running Saratoga templates on a 128MB RAM Seagate Dockstar (pogoplug) 
> > here). 
> > 
> > Ideally, set debug=1 and provide the syslog around the time(s) you see 
> > whatever 'struggles' you are experiencing. 
> > 
> > I still think the long-term solution is to just tell people to buy a 
> > Raspberry Pi if they want to run weewx.   You're talking well under $50 
> > US and it is a very known quantity.  You're doing all kinds of things 
> > like writing custom firmware for embedded routers and trying to fit too 
> > much stuff in too small a capacity computer.  That's guaranteed pain. 
> > 
> > Or just keep running (unsupported/dead) wview on your 
> > (unsupported/hacked/legacy) old routers 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to a topic in the 
> > Google Groups "weewx-user" group. 
> > To unsubscribe from this topic, visit 
> > https://groups.google.com/d/topic/weewx-user/HXHjLqxOJxE/unsubscribe. 
> > To unsubscribe from this group and all its topics, send an email to 
> > weewx-user+...@googlegroups.com  
> > . 
> > For more options, visit https://groups.google.com/d/optout. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[HrMeteoHub]

Dear Vince,

I believe you are right on the money with your last remark. Dead 
software works as long as you have hardware, so I tried with cramming 
weewx in router, and strange things happened.
Creating embedded firmware with Python inside for 24/7 task is 
problematic at best. Low RAM, slow processor and so on is all hitting me 
in the head. Check the capture, it runs with only 20MB of free RAM. I 
have to move database out of RAM to USB stic/thumbdrive/disk.
By the way this router is only $17. ...not so important in grand scheme 
of things.


There is no skin (generating) at all. Only CVS is rewritten to one file 
(in RAM). There is no room/processor power for graphing. Even with such 
"naked" weewx the processor routinely runs at more than 30%.


So, let me recap. Syncing archiving interval with WMR and WH stations on 
5 minutes helped, and it works ok now. My friends are testing other 
stations.


However, if moving database to USB disk does not help, there are only 
two way out of this - using embedded device with more RAM, or clearing 
measurement on VP periodically to prevent this in the first place. I am 
not sure the first one will help, and latter feels bad on so many 
levels. If router with more RAM does not help, Raspberry pi, here I come 
fo VP stations.


Thanx!

Nick

On 22.5.2018. 19:43, vince wrote:

On Tuesday, May 22, 2018 at 5:27:23 AM UTC-7, Andrew Milner wrote:

As has been said many times in this forum comments like 'VP
struggles' do not tell us anything - we (and you) need to look at
the log, maybe run with debug enabled and so forth in order to
understand and more importantly see what is going on within weewx -
any error messages and so forth.


On a 60MB RAM system it is possible the computer simply is out of memory 
and oddities are happening.

Turn off the CSV extension and see if that helps.
Alternately turn off WU and run only the CSV extension and see if 'that' 
helps.


But ideally you'd have some data you could tell us about the ram/cpu 
usage on the system.


You haven't shown us your custom weewx skin either.  If that's too 
complicated the computer will work too hard when the output is 
generated.  We have had numerous occcasions where people trying to run a 
complicated skin makes the system unstable or unpredictable (my example 
- running Saratoga templates on a 128MB RAM Seagate Dockstar (pogoplug) 
here).


Ideally, set debug=1 and provide the syslog around the time(s) you see 
whatever 'struggles' you are experiencing.


I still think the long-term solution is to just tell people to buy a 
Raspberry Pi if they want to run weewx.   You're talking well under $50 
US and it is a very known quantity.  You're doing all kinds of things 
like writing custom firmware for embedded routers and trying to fit too 
much stuff in too small a capacity computer.  That's guaranteed pain.


Or just keep running (unsupported/dead) wview on your 
(unsupported/hacked/legacy) old routers



--
You received this message because you are subscribed to a topic in the 
Google Groups "weewx-user" group.
To unsubscribe from this topic, visit 
https://groups.google.com/d/topic/weewx-user/HXHjLqxOJxE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to 
weewx-user+unsubscr...@googlegroups.com 
.

For more options, visit https://groups.google.com/d/optout.


--
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[vince]

On Tuesday, May 22, 2018 at 5:27:23 AM UTC-7, Andrew Milner wrote:
>
> As has been said many times in this forum comments like 'VP struggles' do 
> not tell us anything - we (and you) need to look at the log, maybe run with 
> debug enabled and so forth in order to understand and more importantly see 
> what is going on within weewx - any error messages and so forth.
>

On a 60MB RAM system it is possible the computer simply is out of memory 
and oddities are happening.
Turn off the CSV extension and see if that helps.
Alternately turn off WU and run only the CSV extension and see if 'that' 
helps.

But ideally you'd have some data you could tell us about the ram/cpu usage 
on the system.

You haven't shown us your custom weewx skin either.  If that's too 
complicated the computer will work too hard when the output is generated. 
 We have had numerous occcasions where people trying to run a complicated 
skin makes the system unstable or unpredictable (my example - running 
Saratoga templates on a 128MB RAM Seagate Dockstar (pogoplug) here).

Ideally, set debug=1 and provide the syslog around the time(s) you see 
whatever 'struggles' you are experiencing.

I still think the long-term solution is to just tell people to buy a 
Raspberry Pi if they want to run weewx.   You're talking well under $50 US 
and it is a very known quantity.  You're doing all kinds of things like 
writing custom firmware for embedded routers and trying to fit too much 
stuff in too small a capacity computer.  That's guaranteed pain.

Or just keep running (unsupported/dead) wview on your 
(unsupported/hacked/legacy) old routers


-- 
You received this message because you are subscribed to the Google Groups 
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to weewx-user+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Andrew Milner]

I did not quite follow your posting.  I do not see how the buffer size in 
the VP can overload weewx!!  Weewx is designed to run 24/7 and only do 
catchup processing/recovery in the event of power failures, so under normal 
usage I would not have thought there would be a problem.  Have you trierd 
ther VP with software record generation or only hardware?  When you change 
the archive interval in weewx did you also change it in the VP station.  
What interval is weewx using (see log at startup of weewx)

If lengthening the archive interval solved the wmr problem then as I see it 
you have a simple choice - use the larger archive interval or don't use wmr 
hardware!!

I would have thought that only the stations which experience problems and 
issues would need to use the larger interval - not necessarily every 
station in your network.

As has been said many times in this forum comments like 'VP struggles' do 
not tell us anything - we (and you) need to look at the log, maybe run with 
debug enabled and so forth in order to understand and more importantly see 
what is going on within weewx - any error messages and so forth.



On Tuesday, 22 May 2018 15:14:13 UTC+3, Hrmeteohub Pljusak wrote:
>
> Huh, I have waited two days to see how my WMR and Davis are going with 
> aforemntioned change. While WMR performs butifully, with not a single 
> drop-out, Davis VP still struggles. I have to dig into that next. 
> As far as lengthening the archive interval, this is not solely my 
> decision. There are many stations involved, and I have to see if others 
> agree with it. I'll try, and then the we'll make a decision. 
>
> One thing that bothers me are constraints of the router that runs the 
> installed software. Python is a beast, and after everything is set up, 
> there is not much memory left on the RAM. Think 400MHz processor, 60MB 
> total RAM. I suspect that is why archive interval is having a strong 
> impact on performance in Davis stations as they have large archive 
> capacity and may overload the router, either on the startup or collide 
> with some other housekeeping that OS is doing. The situation is very 
> different from e.g. Raspberry Pi with +10x RAM. 
> Also, I think I have to dig deeper in the process how I am reporting the 
> info to our page, as you have suggested. I am thin on Python knowledge 
> so I will need help from someone for that. 
>
> I'll keep you posted, and thank you very much! 
>
>
> On 17.5.2018. 15:34, Andrew Milner wrote: 
> > try 10 minutes even as the interval.  Be honest - since weewx is storing 
> > AVERAGES anyway, there is probably little difference between a 5 minute 
> > interval and a 10 minute interval except that your servers will have a 
> > fraction of the workload and the stations will suffer fewer dropouts and 
> > losses of data which to me would be a win-win situation!!  You may as 
> > well try it and I think you will be very pleasantly sur[rised at the 
> > result of using either 10 or even 15 minute archive intervals!! 
> > 
> > On Thursday, 17 May 2018 15:18:06 UTC+3, Hrmeteohub Pljusak wrote: 
> > 
> > Setting 5-min archiving interval made great difference on WMR88 - I 
> > had only one drop-out in last four hours. Already I have made it a 
> > default in the firmware, for all stations. I'll test this on WMR88 
> > for several days and then see how it goes. 
> > I the mean time I will focus probably on Davis stations. It is hard 
> > to say if archiving interval of 5 minutes made a difference so far, 
> > since the errors did happen only 1-2 times each day. 
> > 
> > I would really like to work with just one type of station, but our 
> > volunteers have all kind of stations, from very old to brand new, 
> > and expect us to support them all. 
> > You are right on the money with a need for new API. 
> > 
> > Thank you for the advice. 
> > 
> > On Thu, May 17, 2018 at 1:16 PM, Andrew Milner 
> >  wrote: 
> > 
> > I seriously recommend that you focus on one particular model of 
> > weather station first - eg FineOffset (use software records 
> > rather than hardware - see weewx hardware manual for more on the 
> > options) and maybe Davis - use hardware records and once again 
> > see weewx hardware manual for options).  In both cases I would 
> > suggest use a 5 minute archive interval.  For WU do not use 
> > rapid fire until you are happy things are ok. 
> > 
> > For the rest of your setup it would probably be simplest for you 
> > to have arranged for your server to have accepted input via a 
> > restless API and built a restless service to upload directly to 
> > your site - like weewx does to WU and other hosts - but that is 
> > far more suitable for the development forum rather than this 
> > forum as it is outside the scope of current weewx. 
> > 
> > 
> > 
> > On 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[HrMeteoHub]

Huh, I have waited two days to see how my WMR and Davis are going with 
aforemntioned change. While WMR performs butifully, with not a single 
drop-out, Davis VP still struggles. I have to dig into that next.
As far as lengthening the archive interval, this is not solely my 
decision. There are many stations involved, and I have to see if others 
agree with it. I'll try, and then the we'll make a decision.


One thing that bothers me are constraints of the router that runs the 
installed software. Python is a beast, and after everything is set up, 
there is not much memory left on the RAM. Think 400MHz processor, 60MB 
total RAM. I suspect that is why archive interval is having a strong 
impact on performance in Davis stations as they have large archive 
capacity and may overload the router, either on the startup or collide 
with some other housekeeping that OS is doing. The situation is very 
different from e.g. Raspberry Pi with +10x RAM.
Also, I think I have to dig deeper in the process how I am reporting the 
info to our page, as you have suggested. I am thin on Python knowledge 
so I will need help from someone for that.


I'll keep you posted, and thank you very much!


On 17.5.2018. 15:34, Andrew Milner wrote:
try 10 minutes even as the interval.  Be honest - since weewx is storing 
AVERAGES anyway, there is probably little difference between a 5 minute 
interval and a 10 minute interval except that your servers will have a 
fraction of the workload and the stations will suffer fewer dropouts and 
losses of data which to me would be a win-win situation!!  You may as 
well try it and I think you will be very pleasantly sur[rised at the 
result of using either 10 or even 15 minute archive intervals!!


On Thursday, 17 May 2018 15:18:06 UTC+3, Hrmeteohub Pljusak wrote:

Setting 5-min archiving interval made great difference on WMR88 - I
had only one drop-out in last four hours. Already I have made it a
default in the firmware, for all stations. I'll test this on WMR88
for several days and then see how it goes.
I the mean time I will focus probably on Davis stations. It is hard
to say if archiving interval of 5 minutes made a difference so far,
since the errors did happen only 1-2 times each day.

I would really like to work with just one type of station, but our
volunteers have all kind of stations, from very old to brand new,
and expect us to support them all.
You are right on the money with a need for new API.

Thank you for the advice.

On Thu, May 17, 2018 at 1:16 PM, Andrew Milner
 wrote:

I seriously recommend that you focus on one particular model of
weather station first - eg FineOffset (use software records
rather than hardware - see weewx hardware manual for more on the
options) and maybe Davis - use hardware records and once again
see weewx hardware manual for options).  In both cases I would
suggest use a 5 minute archive interval.  For WU do not use
rapid fire until you are happy things are ok.

For the rest of your setup it would probably be simplest for you
to have arranged for your server to have accepted input via a
restless API and built a restless service to upload directly to
your site - like weewx does to WU and other hosts - but that is
far more suitable for the development forum rather than this
forum as it is outside the scope of current weewx.



On Thursday, 17 May 2018 10:49:39 UTC+3, Hrmeteohub Pljusak wrote:

Dear Andrew,

First, thank you and thank Thomas and everybody else for
producing weewx. I apologize once more for coming harsh in
my first post.

It appears that you are quite right that I am forcing weewx
into something it was not made to do. Perhaps maybe not all
of it, but I suspect it is the way I am doing thins more to
be blame in some instances.
Also, we are doing that with several weather stations, at
locations apart, with so many different co-factors. Seems
like a good recipe for headache.
As luck will have it, weewx is now my tool of choice. You
are very much correct when you say that I have many problems
on a number of fronts simultaneously. Unfortunately. It
looks a bit chaotic when I have presented it all at once.
Please accept my exclamations as expression of anger onto
my/our choices and problems.

Thank you for giving me info on WU. I did not know that.
If nothing goes through the database, than the problem must
be on "my side", CSV reporting and sending to pljusak.com
 server.

Some of the problems were expected, like intermittent
sending of data from WMR stations to weewx. 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Andrew Milner]

try 10 minutes even as the interval.  Be honest - since weewx is storing 
AVERAGES anyway, there is probably little difference between a 5 minute 
interval and a 10 minute interval except that your servers will have a 
fraction of the workload and the stations will suffer fewer dropouts and 
losses of data which to me would be a win-win situation!!  You may as well 
try it and I think you will be very pleasantly sur[rised at the result of 
using either 10 or even 15 minute archive intervals!!

On Thursday, 17 May 2018 15:18:06 UTC+3, Hrmeteohub Pljusak wrote:
>
> Setting 5-min archiving interval made great difference on WMR88 - I had 
> only one drop-out in last four hours. Already I have made it a default in 
> the firmware, for all stations. I'll test this on WMR88 for several days 
> and then see how it goes.
> I the mean time I will focus probably on Davis stations. It is hard to say 
> if archiving interval of 5 minutes made a difference so far, since the 
> errors did happen only 1-2 times each day.
>
> I would really like to work with just one type of station, but our 
> volunteers have all kind of stations, from very old to brand new, and 
> expect us to support them all. 
> You are right on the money with a need for new API.
>
> Thank you for the advice.
>
> On Thu, May 17, 2018 at 1:16 PM, Andrew Milner  > wrote:
>
>> I seriously recommend that you focus on one particular model of weather 
>> station first - eg FineOffset (use software records rather than hardware - 
>> see weewx hardware manual for more on the options) and maybe Davis - use 
>> hardware records and once again see weewx hardware manual for options).  In 
>> both cases I would suggest use a 5 minute archive interval.  For WU do not 
>> use rapid fire until you are happy things are ok.
>>
>> For the rest of your setup it would probably be simplest for you to have 
>> arranged for your server to have accepted input via a restless API and 
>> built a restless service to upload directly to your site - like weewx does 
>> to WU and other hosts - but that is far more suitable for the development 
>> forum rather than this forum as it is outside the scope of current weewx.
>>
>>
>>
>> On Thursday, 17 May 2018 10:49:39 UTC+3, Hrmeteohub Pljusak wrote:
>>>
>>> Dear Andrew,
>>>
>>> First, thank you and thank Thomas and everybody else for producing 
>>> weewx. I apologize once more for coming harsh in my first post.
>>>
>>> It appears that you are quite right that I am forcing weewx into 
>>> something it was not made to do. Perhaps maybe not all of it, but I suspect 
>>> it is the way I am doing thins more to be blame in some instances.
>>> Also, we are doing that with several weather stations, at locations 
>>> apart, with so many different co-factors. Seems like a good recipe for 
>>> headache.
>>> As luck will have it, weewx is now my tool of choice. You are very much 
>>> correct when you say that I have many problems on a number of fronts 
>>> simultaneously. Unfortunately. It looks a bit chaotic when I have presented 
>>> it all at once. Please accept my exclamations as expression of anger onto 
>>> my/our choices and problems. 
>>>
>>> Thank you for giving me info on WU. I did not know that.
>>> If nothing goes through the database, than the problem must be on "my 
>>> side", CSV reporting and sending to pljusak.com server.
>>>
>>> Some of the problems were expected, like intermittent sending of data 
>>> from WMR stations to weewx. I was combing through data and testing parts of 
>>> the process for Davis and FineOffset stations for weeks now, and simply can 
>>> not put my finger on the problem. No such event was seen in weeks for 
>>> WS2300, WMR100, WMR200 stations in weeks of testing.
>>>
>>> Please do not read the rest of my post if you do not have too much time 
>>> on your hands. As Thomas wrote, I tend to rant.
>>>
>>>
>>> Let me give some perspective to HRmeteohub project. 
>>>
>>> The goal of the project is to send data from weather stations to crowd 
>>> sourcing web page pljusak.com, as well as to WU, AWEKAS, and so on. The 
>>> project was started by amateur meteorologists and enthusiasts in Croatia, 
>>> and it runs for about ten years now. At first phase, all users have used 
>>> some kind of windows based software. In that (first) phase, our job was to 
>>> hand out the templates for PC software that users were using and users 
>>> would use FTP to send the graphs and data to server. It worked, but what a 
>>> mess (and security headaches). 
>>> In second phase we started using Open2300 in small devices such as NSLU 
>>> and wifi routers. About six years ago this has transformed in customized 
>>> OpenWRT firmware for 5-6 different weather station types. Hence my 
>>> references to wview. At that time our team stabilized on one sysadmin, one 
>>> programmer, and me in charge of building the firmware, testing, documening 
>>> as well as supporting users together with sysadmin.
>>> However, 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Hrmeteohub Pljusak]

Setting 5-min archiving interval made great difference on WMR88 - I had
only one drop-out in last four hours. Already I have made it a default in
the firmware, for all stations. I'll test this on WMR88 for several days
and then see how it goes.
I the mean time I will focus probably on Davis stations. It is hard to say
if archiving interval of 5 minutes made a difference so far, since the
errors did happen only 1-2 times each day.

I would really like to work with just one type of station, but our
volunteers have all kind of stations, from very old to brand new, and
expect us to support them all.
You are right on the money with a need for new API.

Thank you for the advice.

On Thu, May 17, 2018 at 1:16 PM, Andrew Milner 
wrote:

> I seriously recommend that you focus on one particular model of weather
> station first - eg FineOffset (use software records rather than hardware -
> see weewx hardware manual for more on the options) and maybe Davis - use
> hardware records and once again see weewx hardware manual for options).  In
> both cases I would suggest use a 5 minute archive interval.  For WU do not
> use rapid fire until you are happy things are ok.
>
> For the rest of your setup it would probably be simplest for you to have
> arranged for your server to have accepted input via a restless API and
> built a restless service to upload directly to your site - like weewx does
> to WU and other hosts - but that is far more suitable for the development
> forum rather than this forum as it is outside the scope of current weewx.
>
>
>
> On Thursday, 17 May 2018 10:49:39 UTC+3, Hrmeteohub Pljusak wrote:
>>
>> Dear Andrew,
>>
>> First, thank you and thank Thomas and everybody else for producing weewx.
>> I apologize once more for coming harsh in my first post.
>>
>> It appears that you are quite right that I am forcing weewx into
>> something it was not made to do. Perhaps maybe not all of it, but I suspect
>> it is the way I am doing thins more to be blame in some instances.
>> Also, we are doing that with several weather stations, at locations
>> apart, with so many different co-factors. Seems like a good recipe for
>> headache.
>> As luck will have it, weewx is now my tool of choice. You are very much
>> correct when you say that I have many problems on a number of fronts
>> simultaneously. Unfortunately. It looks a bit chaotic when I have presented
>> it all at once. Please accept my exclamations as expression of anger onto
>> my/our choices and problems.
>>
>> Thank you for giving me info on WU. I did not know that.
>> If nothing goes through the database, than the problem must be on "my
>> side", CSV reporting and sending to pljusak.com server.
>>
>> Some of the problems were expected, like intermittent sending of data
>> from WMR stations to weewx. I was combing through data and testing parts of
>> the process for Davis and FineOffset stations for weeks now, and simply can
>> not put my finger on the problem. No such event was seen in weeks for
>> WS2300, WMR100, WMR200 stations in weeks of testing.
>>
>> Please do not read the rest of my post if you do not have too much time
>> on your hands. As Thomas wrote, I tend to rant.
>>
>>
>> Let me give some perspective to HRmeteohub project.
>>
>> The goal of the project is to send data from weather stations to crowd
>> sourcing web page pljusak.com, as well as to WU, AWEKAS, and so on. The
>> project was started by amateur meteorologists and enthusiasts in Croatia,
>> and it runs for about ten years now. At first phase, all users have used
>> some kind of windows based software. In that (first) phase, our job was to
>> hand out the templates for PC software that users were using and users
>> would use FTP to send the graphs and data to server. It worked, but what a
>> mess (and security headaches).
>> In second phase we started using Open2300 in small devices such as NSLU
>> and wifi routers. About six years ago this has transformed in customized
>> OpenWRT firmware for 5-6 different weather station types. Hence my
>> references to wview. At that time our team stabilized on one sysadmin, one
>> programmer, and me in charge of building the firmware, testing, documening
>> as well as supporting users together with sysadmin.
>> However, hardware producers have been changing their routers and weather
>> stations, combined with changes in OpenWRT - making our task almost
>> impossible.
>>
>> At this time about 40 users use HRmeteohub firmware, and some would like
>> to, but supported wifi routers are not on the market anymore. Pljusak.com
>> is NGO run by volunteers, all the software, firmware and services are
>> provided for free and all income from advertising is used to maintain the
>> server and keep NGO functioning. The web page is now about 5-6 years old
>> and ready for overhaul. Basically the server produces something like this:
>>
>> And each station has got it's page:
>>
>> This is where weewx comes on stage. We have been 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Andrew Milner]

I seriously recommend that you focus on one particular model of weather 
station first - eg FineOffset (use software records rather than hardware - 
see weewx hardware manual for more on the options) and maybe Davis - use 
hardware records and once again see weewx hardware manual for options).  In 
both cases I would suggest use a 5 minute archive interval.  For WU do not 
use rapid fire until you are happy things are ok.

For the rest of your setup it would probably be simplest for you to have 
arranged for your server to have accepted input via a restless API and 
built a restless service to upload directly to your site - like weewx does 
to WU and other hosts - but that is far more suitable for the development 
forum rather than this forum as it is outside the scope of current weewx.



On Thursday, 17 May 2018 10:49:39 UTC+3, Hrmeteohub Pljusak wrote:
>
> Dear Andrew,
>
> First, thank you and thank Thomas and everybody else for producing weewx. 
> I apologize once more for coming harsh in my first post.
>
> It appears that you are quite right that I am forcing weewx into something 
> it was not made to do. Perhaps maybe not all of it, but I suspect it is the 
> way I am doing thins more to be blame in some instances.
> Also, we are doing that with several weather stations, at locations apart, 
> with so many different co-factors. Seems like a good recipe for headache.
> As luck will have it, weewx is now my tool of choice. You are very much 
> correct when you say that I have many problems on a number of fronts 
> simultaneously. Unfortunately. It looks a bit chaotic when I have presented 
> it all at once. Please accept my exclamations as expression of anger onto 
> my/our choices and problems. 
>
> Thank you for giving me info on WU. I did not know that.
> If nothing goes through the database, than the problem must be on "my 
> side", CSV reporting and sending to pljusak.com server.
>
> Some of the problems were expected, like intermittent sending of data from 
> WMR stations to weewx. I was combing through data and testing parts of the 
> process for Davis and FineOffset stations for weeks now, and simply can not 
> put my finger on the problem. No such event was seen in weeks for WS2300, 
> WMR100, WMR200 stations in weeks of testing.
>
> Please do not read the rest of my post if you do not have too much time on 
> your hands. As Thomas wrote, I tend to rant.
>
>
> Let me give some perspective to HRmeteohub project. 
>
> The goal of the project is to send data from weather stations to crowd 
> sourcing web page pljusak.com, as well as to WU, AWEKAS, and so on. The 
> project was started by amateur meteorologists and enthusiasts in Croatia, 
> and it runs for about ten years now. At first phase, all users have used 
> some kind of windows based software. In that (first) phase, our job was to 
> hand out the templates for PC software that users were using and users 
> would use FTP to send the graphs and data to server. It worked, but what a 
> mess (and security headaches). 
> In second phase we started using Open2300 in small devices such as NSLU 
> and wifi routers. About six years ago this has transformed in customized 
> OpenWRT firmware for 5-6 different weather station types. Hence my 
> references to wview. At that time our team stabilized on one sysadmin, one 
> programmer, and me in charge of building the firmware, testing, documening 
> as well as supporting users together with sysadmin.
> However, hardware producers have been changing their routers and weather 
> stations, combined with changes in OpenWRT - making our task almost 
> impossible.
>
> At this time about 40 users use HRmeteohub firmware, and some would like 
> to, but supported wifi routers are not on the market anymore. Pljusak.com 
> is NGO run by volunteers, all the software, firmware and services are 
> provided for free and all income from advertising is used to maintain the 
> server and keep NGO functioning. The web page is now about 5-6 years old 
> and ready for overhaul. Basically the server produces something like this:
>
> And each station has got it's page:
>
> This is where weewx comes on stage. We have been testing it on RaspberryPi 
> with good results, but it is rather too complicated for our average 
> meteo-enthusiast to install and setup R-pi with weewx and our scripts for 
> sending the data to pljusak.com. Plus, R-pi (version 2B at that time) 
> wirth power source, wifi card, box was a bit more expensive than wifi 
> router. Our users could can buy second-hand PC with old windows on it for 
> the same price. It (r-pi) also did not work as reliably as wifi routers. At 
> that time, putting Python and weewx in small, cheap wifi router was simply 
> not possible, but in time, OpenWRT merged with LEDE, routers got faster 
> with more Flash and RAM, and suddenly it become a viable option.
> After about six months of tries and errors, the firmware proved to be 
> stable. I can not recall single crash of weewx 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Andrew Milner]

Nothing goes through the data and changes the data contained in the archive.

The plotting engine when drawing graphplots (creating the .png files) does, 
I believe, try and smooth (and scale) the data for the plots.

Have you compared your graphs with the standard weewx report graphs/plots??



On Thursday, 17 May 2018 08:21:39 UTC+3, Hrmeteohub Pljusak wrote:
>
> Thank you.
> I will try extending the archive interval for stations other that Davis 
> Vantage series. However, I believe that weewx uses archive interval from VP 
> station, as it is set to "record_generation = hardware".
>
> Is there some function that goes through archive data in weewx and does 
> "the smoothing"? I ask that because I have compared local data with data 
> that are on WU and our remote server, and the values with errors did not 
> match on all three locations. At the end of the day, I could not see errors 
> on WU while database dumps to CSV file contained 1-2 error per day. The csv 
> file was done with extenstion set up:
> [CSV]
> filename = /tmp/data.txt
> binding = archive
>
> At first I thought that these errors are perhaps only in loop data, or 
> errors on USB communication, but after several weeks I could not catch them 
> in logs with matching timestamps.
>
>
>
> On Thursday, May 17, 2018 at 7:02:27 AM UTC+2, Andrew Milner wrote:
>>
>> …. the choices seem to include:
>> 1.  Use weewx database and make your graphing package do the smoothing 
>> (in the way I presume wview does)
>> 2.  Use weewx database and weewx graphs - rather than making your own 
>> graphs
>> 3.  Increase archive interval to ensure every archive record contains at 
>> least one reading for each sensor - I think some sensors only output every 
>> 15 minutes (=900 seconds)
>> 4.  Explain what you feel to be wrong
>> or
>> 5.  Return to wview
>>
>> As a further point you seem to be jumping around many different station 
>> types and drivers.  Stations of different types emit different data and 
>> weewx will cope with each.  However the config file needs to be set up for 
>> each station type with regard to hardware/software preferences, archive 
>> interval, hardware/software record creation etc.  HOWEVER - it is 
>> overriding within weewx that an archive record contains the average value 
>> for a reading during an archive period.  If there is no reading there is no 
>> value to store.  In the hgardware manual there is information regarding 
>> each station type - and for example the config options for a fine offset 
>> station would involve periodic polling at about 60 second intervals (no 
>> point in more frequently as station does not emit any faster) and software 
>> record creation rather than hardware …. and so on.  The options to use for 
>> fine offset are not the same options you would use with a davis station, so 
>> it is not possible to just change drivers when changing station types.
>>
>>
>>
>> My FineOffset has run for several years without issues.  Using QC values 
>> in your config helps reduce spikes
>> On Thursday, 17 May 2018 07:39:05 UTC+3, Thomas Keffer wrote:
>>>
>>> ​Your rant would be a lot more useful if it offered ​some constructive 
>>> examples of where you think weewx is failing. Instead, there's just vague 
>>> generalities like "erratic measurements from Davis VantagePro2..." 
>>>
>>> I flatly do not believe this. The VP2 driver is nearly 10 years old, has 
>>> been extremely well vetted, and used by thousands without any problems. 
>>> Most likely the problem is in your configuration. 
>>>
>>> The WMR100 driver is not as mature, but is still used by many people. I 
>>> would not be surprised if it had some problems, but your inchoate rant 
>>> isn't going to put us any closer to solving them.
>>>
>>> I am sorry you are disappointed. If you had paid anything for Weewx, I 
>>> would be happy to refund your money. But you didn't. 
>>>
>>> Stop whining and offer a pull request if you think there's a problem.
>>>
>>> -tk
>>>
>>>
>>> On Wed, May 16, 2018, 2:38 PM Hrmeteohub Pljusak  
>>> wrote:
>>>
 I am not positive that my questions have not been answered. If so, 
 please point me to location where I can read how to mitigate the problems. 
 Thank you!

 Recently I have switched from wview to weewx 3.7.1. I must say I got 
 rather disappointed with problems I have encountered.

 First, Oregon Scientific WMR88/100/200 stations started to produce 
 erratic graphs on WU and other web pages, with number of incomplete 
 measurements. The weewx documentation gently describes that as "station 
 problem". That is not acceptable, or nice, but just a dirty 
 walk-around-the-problem.
 Yes, I am aware of the way that stations "blasts data" from sensors to 
 station, but wview did away with it quite gently. Also, other (read 
 windows-based) software works nicely with OS stations. Weewx simply gives 
 us nothing (null), and lets the user 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Andrew Milner]

WU is a world of its own.  I know that it reports readings for times when 
none have been submitted (eg I have been down) and so on - so matching 
times is probably impossible,  It certainly seems to always adjust times to 
5 minute boundaries.



On Thursday, 17 May 2018 08:27:01 UTC+3, Andrew Milner wrote:
>
> You are probably bashing your head against a brick wall because you are 
> possibly trying to do something weewx is not designed to do.  It is not a 
> real time reporting system.
>
> Try starting with an archive interval of 300 seconds and avoid things like 
> 60 second archive intervals.
>
> Seek help for getting one station type to work correctly before moving on 
> to others - it may not be just the interval that needs 'tweaking' for 
> differing station types.
>
> weewx will use the vantage interval, but the interval should be ideally 
> set to say 300 seconds (use wee_device to set hardware interval)
>
> weewx should never invent data for an archive record, and should always 
> report none when there has been no input during the archive interval.
>
>
>
> On Thursday, 17 May 2018 08:02:27 UTC+3, Andrew Milner wrote:
>>
>> …. the choices seem to include:
>> 1.  Use weewx database and make your graphing package do the smoothing 
>> (in the way I presume wview does)
>> 2.  Use weewx database and weewx graphs - rather than making your own 
>> graphs
>> 3.  Increase archive interval to ensure every archive record contains at 
>> least one reading for each sensor - I think some sensors only output every 
>> 15 minutes (=900 seconds)
>> 4.  Explain what you feel to be wrong
>> or
>> 5.  Return to wview
>>
>> As a further point you seem to be jumping around many different station 
>> types and drivers.  Stations of different types emit different data and 
>> weewx will cope with each.  However the config file needs to be set up for 
>> each station type with regard to hardware/software preferences, archive 
>> interval, hardware/software record creation etc.  HOWEVER - it is 
>> overriding within weewx that an archive record contains the average value 
>> for a reading during an archive period.  If there is no reading there is no 
>> value to store.  In the hgardware manual there is information regarding 
>> each station type - and for example the config options for a fine offset 
>> station would involve periodic polling at about 60 second intervals (no 
>> point in more frequently as station does not emit any faster) and software 
>> record creation rather than hardware …. and so on.  The options to use for 
>> fine offset are not the same options you would use with a davis station, so 
>> it is not possible to just change drivers when changing station types.
>>
>>
>>
>> My FineOffset has run for several years without issues.  Using QC values 
>> in your config helps reduce spikes
>> On Thursday, 17 May 2018 07:39:05 UTC+3, Thomas Keffer wrote:
>>>
>>> ​Your rant would be a lot more useful if it offered ​some constructive 
>>> examples of where you think weewx is failing. Instead, there's just vague 
>>> generalities like "erratic measurements from Davis VantagePro2..." 
>>>
>>> I flatly do not believe this. The VP2 driver is nearly 10 years old, has 
>>> been extremely well vetted, and used by thousands without any problems. 
>>> Most likely the problem is in your configuration. 
>>>
>>> The WMR100 driver is not as mature, but is still used by many people. I 
>>> would not be surprised if it had some problems, but your inchoate rant 
>>> isn't going to put us any closer to solving them.
>>>
>>> I am sorry you are disappointed. If you had paid anything for Weewx, I 
>>> would be happy to refund your money. But you didn't. 
>>>
>>> Stop whining and offer a pull request if you think there's a problem.
>>>
>>> -tk
>>>
>>>
>>> On Wed, May 16, 2018, 2:38 PM Hrmeteohub Pljusak  
>>> wrote:
>>>
 I am not positive that my questions have not been answered. If so, 
 please point me to location where I can read how to mitigate the problems. 
 Thank you!

 Recently I have switched from wview to weewx 3.7.1. I must say I got 
 rather disappointed with problems I have encountered.

 First, Oregon Scientific WMR88/100/200 stations started to produce 
 erratic graphs on WU and other web pages, with number of incomplete 
 measurements. The weewx documentation gently describes that as "station 
 problem". That is not acceptable, or nice, but just a dirty 
 walk-around-the-problem.
 Yes, I am aware of the way that stations "blasts data" from sensors to 
 station, but wview did away with it quite gently. Also, other (read 
 windows-based) software works nicely with OS stations. Weewx simply gives 
 us nothing (null), and lets the user deal with it. Well guess what! 
 This should be corrected.

 The graph below is from WU.
 Just to ilustrate, this is an example of the data from weewx.sdb:

 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Andrew Milner]

You are probably bashing your head against a brick wall because you are 
possibly trying to do something weewx is not designed to do.  It is not a 
real time reporting system.

Try starting with an archive interval of 300 seconds and avoid things like 
60 second archive intervals.

Seek help for getting one station type to work correctly before moving on 
to others - it may not be just the interval that needs 'tweaking' for 
differing station types.

weewx will use the vantage interval, but the interval should be ideally set 
to say 300 seconds (use wee_device to set hardware interval)

weewx should never invent data for an archive record, and should always 
report none when there has been no input during the archive interval.



On Thursday, 17 May 2018 08:02:27 UTC+3, Andrew Milner wrote:
>
> …. the choices seem to include:
> 1.  Use weewx database and make your graphing package do the smoothing (in 
> the way I presume wview does)
> 2.  Use weewx database and weewx graphs - rather than making your own 
> graphs
> 3.  Increase archive interval to ensure every archive record contains at 
> least one reading for each sensor - I think some sensors only output every 
> 15 minutes (=900 seconds)
> 4.  Explain what you feel to be wrong
> or
> 5.  Return to wview
>
> As a further point you seem to be jumping around many different station 
> types and drivers.  Stations of different types emit different data and 
> weewx will cope with each.  However the config file needs to be set up for 
> each station type with regard to hardware/software preferences, archive 
> interval, hardware/software record creation etc.  HOWEVER - it is 
> overriding within weewx that an archive record contains the average value 
> for a reading during an archive period.  If there is no reading there is no 
> value to store.  In the hgardware manual there is information regarding 
> each station type - and for example the config options for a fine offset 
> station would involve periodic polling at about 60 second intervals (no 
> point in more frequently as station does not emit any faster) and software 
> record creation rather than hardware …. and so on.  The options to use for 
> fine offset are not the same options you would use with a davis station, so 
> it is not possible to just change drivers when changing station types.
>
>
>
> My FineOffset has run for several years without issues.  Using QC values 
> in your config helps reduce spikes
> On Thursday, 17 May 2018 07:39:05 UTC+3, Thomas Keffer wrote:
>>
>> ​Your rant would be a lot more useful if it offered ​some constructive 
>> examples of where you think weewx is failing. Instead, there's just vague 
>> generalities like "erratic measurements from Davis VantagePro2..." 
>>
>> I flatly do not believe this. The VP2 driver is nearly 10 years old, has 
>> been extremely well vetted, and used by thousands without any problems. 
>> Most likely the problem is in your configuration. 
>>
>> The WMR100 driver is not as mature, but is still used by many people. I 
>> would not be surprised if it had some problems, but your inchoate rant 
>> isn't going to put us any closer to solving them.
>>
>> I am sorry you are disappointed. If you had paid anything for Weewx, I 
>> would be happy to refund your money. But you didn't. 
>>
>> Stop whining and offer a pull request if you think there's a problem.
>>
>> -tk
>>
>>
>> On Wed, May 16, 2018, 2:38 PM Hrmeteohub Pljusak  
>> wrote:
>>
>>> I am not positive that my questions have not been answered. If so, 
>>> please point me to location where I can read how to mitigate the problems. 
>>> Thank you!
>>>
>>> Recently I have switched from wview to weewx 3.7.1. I must say I got 
>>> rather disappointed with problems I have encountered.
>>>
>>> First, Oregon Scientific WMR88/100/200 stations started to produce 
>>> erratic graphs on WU and other web pages, with number of incomplete 
>>> measurements. The weewx documentation gently describes that as "station 
>>> problem". That is not acceptable, or nice, but just a dirty 
>>> walk-around-the-problem.
>>> Yes, I am aware of the way that stations "blasts data" from sensors to 
>>> station, but wview did away with it quite gently. Also, other (read 
>>> windows-based) software works nicely with OS stations. Weewx simply gives 
>>> us nothing (null), and lets the user deal with it. Well guess what! 
>>> This should be corrected.
>>>
>>> The graph below is from WU.
>>> Just to ilustrate, this is an example of the data from weewx.sdb:
>>>
>>> dateTime,ET,altimeter,appTemp,barometer,cloudbase,dewpoint,hourRain,humidex,inDewpoint,inHumidity,inTemp,inTempBatteryStatus,interval,maxSolarRad,outHumidity,outTemp,outTempBatteryStatus,pressure,rain,rain24,rainBatteryStatus,rainRate,rainTotal,usUnits,windBatteryStatus,windDir,windGust,windGustDir,windSpeed,windrun
>>>
>>> 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Hrmeteohub Pljusak]

Thank you.
I will try extending the archive interval for stations other that Davis 
Vantage series. However, I believe that weewx uses archive interval from VP 
station, as it is set to "record_generation = hardware".

Is there some function that goes through archive data in weewx and does 
"the smoothing"? I ask that because I have compared local data with data 
that are on WU and our remote server, and the values with errors did not 
match on all three locations. At the end of the day, I could not see errors 
on WU while database dumps to CSV file contained 1-2 error per day. The csv 
file was done with extenstion set up:
[CSV]
filename = /tmp/data.txt
binding = archive

At first I thought that these errors are perhaps only in loop data, or 
errors on USB communication, but after several weeks I could not catch them 
in logs with matching timestamps.



On Thursday, May 17, 2018 at 7:02:27 AM UTC+2, Andrew Milner wrote:
>
> …. the choices seem to include:
> 1.  Use weewx database and make your graphing package do the smoothing (in 
> the way I presume wview does)
> 2.  Use weewx database and weewx graphs - rather than making your own 
> graphs
> 3.  Increase archive interval to ensure every archive record contains at 
> least one reading for each sensor - I think some sensors only output every 
> 15 minutes (=900 seconds)
> 4.  Explain what you feel to be wrong
> or
> 5.  Return to wview
>
> As a further point you seem to be jumping around many different station 
> types and drivers.  Stations of different types emit different data and 
> weewx will cope with each.  However the config file needs to be set up for 
> each station type with regard to hardware/software preferences, archive 
> interval, hardware/software record creation etc.  HOWEVER - it is 
> overriding within weewx that an archive record contains the average value 
> for a reading during an archive period.  If there is no reading there is no 
> value to store.  In the hgardware manual there is information regarding 
> each station type - and for example the config options for a fine offset 
> station would involve periodic polling at about 60 second intervals (no 
> point in more frequently as station does not emit any faster) and software 
> record creation rather than hardware …. and so on.  The options to use for 
> fine offset are not the same options you would use with a davis station, so 
> it is not possible to just change drivers when changing station types.
>
>
>
> My FineOffset has run for several years without issues.  Using QC values 
> in your config helps reduce spikes
> On Thursday, 17 May 2018 07:39:05 UTC+3, Thomas Keffer wrote:
>>
>> ​Your rant would be a lot more useful if it offered ​some constructive 
>> examples of where you think weewx is failing. Instead, there's just vague 
>> generalities like "erratic measurements from Davis VantagePro2..." 
>>
>> I flatly do not believe this. The VP2 driver is nearly 10 years old, has 
>> been extremely well vetted, and used by thousands without any problems. 
>> Most likely the problem is in your configuration. 
>>
>> The WMR100 driver is not as mature, but is still used by many people. I 
>> would not be surprised if it had some problems, but your inchoate rant 
>> isn't going to put us any closer to solving them.
>>
>> I am sorry you are disappointed. If you had paid anything for Weewx, I 
>> would be happy to refund your money. But you didn't. 
>>
>> Stop whining and offer a pull request if you think there's a problem.
>>
>> -tk
>>
>>
>> On Wed, May 16, 2018, 2:38 PM Hrmeteohub Pljusak  
>> wrote:
>>
>>> I am not positive that my questions have not been answered. If so, 
>>> please point me to location where I can read how to mitigate the problems. 
>>> Thank you!
>>>
>>> Recently I have switched from wview to weewx 3.7.1. I must say I got 
>>> rather disappointed with problems I have encountered.
>>>
>>> First, Oregon Scientific WMR88/100/200 stations started to produce 
>>> erratic graphs on WU and other web pages, with number of incomplete 
>>> measurements. The weewx documentation gently describes that as "station 
>>> problem". That is not acceptable, or nice, but just a dirty 
>>> walk-around-the-problem.
>>> Yes, I am aware of the way that stations "blasts data" from sensors to 
>>> station, but wview did away with it quite gently. Also, other (read 
>>> windows-based) software works nicely with OS stations. Weewx simply gives 
>>> us nothing (null), and lets the user deal with it. Well guess what! 
>>> This should be corrected.
>>>
>>> The graph below is from WU.
>>> Just to ilustrate, this is an example of the data from weewx.sdb:
>>>
>>> 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Hrmeteohub Pljusak]

Sorry. I am not a native speaker. I see that my wording is not quite as it 
should be.
I believe that I am setting up something wrong. I would like to believe 
this is just a fluke, but after five months of testing, trying on six 
locations with four different stations, with lots of variations of 
settings, cabling, locations, I am going a bit off. 

However, I am not ranting about money, or support, I am just hitting a wall 
because the problems are showing where they should not, and I see no 
logical connection why archive tables have errors. 
But, if its just me, than I guess, than I am sorry to rant on you.

On Thursday, May 17, 2018 at 6:39:05 AM UTC+2, Thomas Keffer wrote:
>
> ​Your rant would be a lot more useful if it offered ​some constructive 
> examples of where you think weewx is failing. Instead, there's just vague 
> generalities like "erratic measurements from Davis VantagePro2..." 
>
> I flatly do not believe this. The VP2 driver is nearly 10 years old, has 
> been extremely well vetted, and used by thousands without any problems. 
> Most likely the problem is in your configuration. 
>
> The WMR100 driver is not as mature, but is still used by many people. I 
> would not be surprised if it had some problems, but your inchoate rant 
> isn't going to put us any closer to solving them.
>
> I am sorry you are disappointed. If you had paid anything for Weewx, I 
> would be happy to refund your money. But you didn't. 
>
> Stop whining and offer a pull request if you think there's a problem.
>
> -tk
>
>
> On Wed, May 16, 2018, 2:38 PM Hrmeteohub Pljusak  > wrote:
>
>> I am not positive that my questions have not been answered. If so, please 
>> point me to location where I can read how to mitigate the problems. Thank 
>> you!
>>
>> Recently I have switched from wview to weewx 3.7.1. I must say I got 
>> rather disappointed with problems I have encountered.
>>
>> First, Oregon Scientific WMR88/100/200 stations started to produce 
>> erratic graphs on WU and other web pages, with number of incomplete 
>> measurements. The weewx documentation gently describes that as "station 
>> problem". That is not acceptable, or nice, but just a dirty 
>> walk-around-the-problem.
>> Yes, I am aware of the way that stations "blasts data" from sensors to 
>> station, but wview did away with it quite gently. Also, other (read 
>> windows-based) software works nicely with OS stations. Weewx simply gives 
>> us nothing (null), and lets the user deal with it. Well guess what! 
>> This should be corrected.
>>
>> The graph below is from WU.
>> Just to ilustrate, this is an example of the data from weewx.sdb:
>>
>> dateTime,ET,altimeter,appTemp,barometer,cloudbase,dewpoint,hourRain,humidex,inDewpoint,inHumidity,inTemp,inTempBatteryStatus,interval,maxSolarRad,outHumidity,outTemp,outTempBatteryStatus,pressure,rain,rain24,rainBatteryStatus,rainRate,rainTotal,usUnits,windBatteryStatus,windDir,windGust,windGustDir,windSpeed,windrun
>>
>> 1526378490.0,None,1008.47595669,18.7382035985,1008.41843898,1330.2378329,9.93516495581,0.0,20.4483894915,51.3461294561,46.0,23.0,0.0,1,None,55.0,19.2,0.0,988.0,0.0,6.35,0.0,0.0,2.41,17,0.0,337.5,0.73479692,None,0.73479692,0.0
>>
>> 1526378610.0,None,None,18.174962293,None,1364.37850406,9.66136186735,20.3234925844,51.3461294561,46.0,23.0,0.0,1,None,54.0,19.2,0.0,None,17,0.0,225.0,1.4695938,None,1.4695938,0.0420002087815
>>
>> 1526378670.0,None,1008.47595669,18.384962815,1008.41843898,1364.37850406,9.66136186735,20.3234925844,51.3461294561,46.0,23.0,0.0,1,None,54.0,19.2,0.0,988.0,17,0.0,315.0,1.1546809,None,1.1546809,0.126000626344
>>
>> 1526378730.0,None,1008.47595669,None,None,None,None,None,None,1,None,988.0,17,0.0,225.0,0.94473889,None,0.94473889,0.19200095443
>>
>> 1526378850.0,None,1008.47595669,None,None,None,None,0.0,None,51.3461294561,46.0,23.0,0.0,1,None,988.0,0.0,6.35,0.0,0.0,2.41,17,0.0,315.0,1.1546809,None,1.1546809,0.246001222863
>>
>> 1526378910.0,None,1008.47595669,18.6649635109,1008.41843898,1364.37850406,9.66136186735,20.3234925844,None,1,None,54.0,19.2,0.0,988.0,17,0.0,225.0,0.73479692,None,0.73479692,0.312001550948
>>
>> 1526379030.0,None,1008.47595669,None,None,None,None,0.0,None,51.3461294561,46.0,23.0,0.0,1,None,988.0,0.0,6.35,0.0,0.0,2.41,17,0.35400175973
>>
>> 1526379150.0,None,1008.47595669,None,None,None,None,0.0,None,51.3461294561,46.0,23.0,0.0,1,None,988.0,0.0,6.35,0.0,0.0,2.41,17,0.35400175973
>>
>> 1526379210.0,None,1008.47595669,None,None,None,None,None,51.3461294561,46.0,23.0,0.0,1,None,988.0,17,0.0,250.773234946,0.94473889,None,0.750003728241,0.35400175973
>>
>> 1526379330.0,None,None,18.2996293732,None,1365.28742772,9.75407243522,0.0,20.4655511807,51.3461294561,46.0,23.0,0.0,1,None,54.0,19.3,0.0,None,0.0,6.35,0.0,0.0,2.41,17,0.0,292.5,1.4695938,None,1.4695938,0.399001983424
>>
>> 

Re: [weewx-user] How to avoid problems with errors in weewx archive?

[Andrew Milner]

…. the choices seem to include:
1.  Use weewx database and make your graphing package do the smoothing (in 
the way I presume wview does)
2.  Use weewx database and weewx graphs - rather than making your own graphs
3.  Increase archive interval to ensure every archive record contains at 
least one reading for each sensor - I think some sensors only output every 
15 minutes (=900 seconds)
4.  Explain what you feel to be wrong
or
5.  Return to wview

As a further point you seem to be jumping around many different station 
types and drivers.  Stations of different types emit different data and 
weewx will cope with each.  However the config file needs to be set up for 
each station type with regard to hardware/software preferences, archive 
interval, hardware/software record creation etc.  HOWEVER - it is 
overriding within weewx that an archive record contains the average value 
for a reading during an archive period.  If there is no reading there is no 
value to store.  In the hgardware manual there is information regarding 
each station type - and for example the config options for a fine offset 
station would involve periodic polling at about 60 second intervals (no 
point in more frequently as station does not emit any faster) and software 
record creation rather than hardware …. and so on.  The options to use for 
fine offset are not the same options you would use with a davis station, so 
it is not possible to just change drivers when changing station types.



My FineOffset has run for several years without issues.  Using QC values in 
your config helps reduce spikes
On Thursday, 17 May 2018 07:39:05 UTC+3, Thomas Keffer wrote:
>
> ​Your rant would be a lot more useful if it offered ​some constructive 
> examples of where you think weewx is failing. Instead, there's just vague 
> generalities like "erratic measurements from Davis VantagePro2..." 
>
> I flatly do not believe this. The VP2 driver is nearly 10 years old, has 
> been extremely well vetted, and used by thousands without any problems. 
> Most likely the problem is in your configuration. 
>
> The WMR100 driver is not as mature, but is still used by many people. I 
> would not be surprised if it had some problems, but your inchoate rant 
> isn't going to put us any closer to solving them.
>
> I am sorry you are disappointed. If you had paid anything for Weewx, I 
> would be happy to refund your money. But you didn't. 
>
> Stop whining and offer a pull request if you think there's a problem.
>
> -tk
>
>
> On Wed, May 16, 2018, 2:38 PM Hrmeteohub Pljusak  > wrote:
>
>> I am not positive that my questions have not been answered. If so, please 
>> point me to location where I can read how to mitigate the problems. Thank 
>> you!
>>
>> Recently I have switched from wview to weewx 3.7.1. I must say I got 
>> rather disappointed with problems I have encountered.
>>
>> First, Oregon Scientific WMR88/100/200 stations started to produce 
>> erratic graphs on WU and other web pages, with number of incomplete 
>> measurements. The weewx documentation gently describes that as "station 
>> problem". That is not acceptable, or nice, but just a dirty 
>> walk-around-the-problem.
>> Yes, I am aware of the way that stations "blasts data" from sensors to 
>> station, but wview did away with it quite gently. Also, other (read 
>> windows-based) software works nicely with OS stations. Weewx simply gives 
>> us nothing (null), and lets the user deal with it. Well guess what! 
>> This should be corrected.
>>
>> The graph below is from WU.
>> Just to ilustrate, this is an example of the data from weewx.sdb:
>>
>> dateTime,ET,altimeter,appTemp,barometer,cloudbase,dewpoint,hourRain,humidex,inDewpoint,inHumidity,inTemp,inTempBatteryStatus,interval,maxSolarRad,outHumidity,outTemp,outTempBatteryStatus,pressure,rain,rain24,rainBatteryStatus,rainRate,rainTotal,usUnits,windBatteryStatus,windDir,windGust,windGustDir,windSpeed,windrun
>>
>> 1526378490.0,None,1008.47595669,18.7382035985,1008.41843898,1330.2378329,9.93516495581,0.0,20.4483894915,51.3461294561,46.0,23.0,0.0,1,None,55.0,19.2,0.0,988.0,0.0,6.35,0.0,0.0,2.41,17,0.0,337.5,0.73479692,None,0.73479692,0.0
>>
>> 1526378610.0,None,None,18.174962293,None,1364.37850406,9.66136186735,20.3234925844,51.3461294561,46.0,23.0,0.0,1,None,54.0,19.2,0.0,None,17,0.0,225.0,1.4695938,None,1.4695938,0.0420002087815
>>
>> 1526378670.0,None,1008.47595669,18.384962815,1008.41843898,1364.37850406,9.66136186735,20.3234925844,51.3461294561,46.0,23.0,0.0,1,None,54.0,19.2,0.0,988.0,17,0.0,315.0,1.1546809,None,1.1546809,0.126000626344
>>
>> 1526378730.0,None,1008.47595669,None,None,None,None,None,None,1,None,988.0,17,0.0,225.0,0.94473889,None,0.94473889,0.19200095443
>>
>> 1526378850.0,None,1008.47595669,None,None,None,None,0.0,None,51.3461294561,46.0,23.0,0.0,1,None,988.0,0.0,6.35,0.0,0.0,2.41,17,0.0,315.0,1.1546809,None,1.1546809,0.246001222863
>>
>>