On 29 July 2016 at 00:46, vince <[email protected]> wrote:
> Interesting stuff and surprisingly hard to get done completely....
[...]
> but having logs up to a minute max
> beforehand was helpful in 95% of the post-mortem cases
Ah, duly noted. More on that later...
[...]
> result was 'really' small in ramdisk - I was far smaller than a 64 MB
> partition for the whole thing, although Redhat/Fedora/Centos got fatter over
> time.
Redhat/Fedora - Yes, gone are the days of a tight, minimal Fedora
install. I remember the satisfaction of using RH6.2 to build up a
network of 486 slaves to act as floppy disk duplicators - but I had
the luxury of hard drives!
> But that's a heck of a lot of work. A 'lot' of work. And it doesn't help
> the pi problem where the SD might/can/will eventually corrupt on you.
> Lastly it makes you really custom, so it's harder to get support.
What I liked about Pebble, and led me to tack this together, was the
debian base. Adding/upgrading packages is a simple and known process.
Besides being easier for users that want the benefits but don't have
the time or knowledge, it should minimize support issues too; as you
identify above, those can be huge especially as you go further down
the rabbit hole.
It's tempting to start squeezing the kernel... but I have a lie down
when that feeling occurs, those moments are far fewer now :) Plus we
get back to simplicity for that almost mythical, general user; and
rabbit holes!
There is a script named remove.docs in /usr/local/sbin that sweeps
through and removes a lot of the easy stuff, but given that drive
space isn't really an issue anymore I've plonked an optional message
and an exit 0 to pause anyone that blindly uses it. (The number of
times I've gone to open a non-existent man file means I include myself
in that statement.)
> Re: Raspbian, my belief is that you don't have to be read-only, you have to
> be read 'mostly', concentrating on where the os writes 'often' (hint:
> /var/log) and doing everything possible to not write to the SD...
Agreed.
This is labelled as a read-only system but, as I try to convey in the
notes, it's really a read-only system which you then wind back or
modify to suit your needs. The SDcard is locked down nicely so we know
its minimally touched, and if we then use USBsticks/drives for the
chatty files, and understand just how chatty they are, we're less
likely to be surprised when something does fall over.
It was an interesting exercise going through it all and I was
surprised at just how much the ext4 journaling system was involved in
disk writes, they almost make syslog seem silent.
> Looking at one pi model-B that's been up 184 days running nginx, weewx
> simulator, some cron jobs querying ds18b20 temperature sensors, and driving
> a nokia 5110 lcd screen, I see the following files modified in the last 16
> months:
>
> the weewx public_html tree - easy to put in ramdisk
> the weewx archive tree - easy to put in ramdisk (but save to physical media
> occasionally and 'check' it was saved !)
check?... got it :-)
> /var/lib/{apt,dpkg,cache} - for debian packages, written to rarely
> /var/log - of course this is the big one........
>
> Some thoughts:
>
> raspbian logrotate saves too much. Save less versions. Rotate less often.
> Maybe don't save most of them (logrotate.d edits required)
Good point, I'm not sure where to draw that line but it's probably
safest to limit it to 'rotate 1' and use the 'mail' option to move
them offsite.
> raspbian syslog is too chatty. There are options to quiet it down a bit
> you might be able to move syslog to ramdisk, but it would probably involve
> some gymnastics, again, you'll want to save syslog 'someplace' occasionally
> to give you forensics when/if you get an unexpected happening
I use the *.* @example.com option to push log messages to a remote
machine, and now that you've mention it I notice that I left that out
of the tips page. I'll add that in as while it's not a solution for
everyone, it's an easy one if you do have a second daemon running
remotely.
> moving your public_html tree to ramdisk means weewx will regenerate the noaa
> files every boot. That's very slow. You could probably fake that with
> symlinks so that those written-rarely files are on SD card without too much
> risk.
Ah, excellent point. I hadn't thought of those with large archives and
symlinks would be the simplest solution. WeeWx obviously performs the
rollover monthly at midnight and that's nothing to sweat about.
> Assuming you could handle the /var/log tree,
I haven't gotten around to exploring the busybox-syslogd version yet
but when I get the time it's at the top of the ToDo list. It would
seem to be the perfect replacement but it may add a layer of
complexity for weewx support if users have to do battle when getting
any log extracts for this list. The last thing I want to do is hinder
that process.
That said, I've started and I can see some further in-depth notes in
my immediate future ;-)
> I could see a little
> fix-up-my-pi script folks could run once to do the fun after installing
> Raspbian and before doing weewx perhaps. That would be pretty slick.
I originally had thoughts the files could be packaged up as a .deb and
installed via apt-get. That would save a huge amount of fiddly edits,
but could also leave a new user a bit perplexed as it does change the
user experience somewhat.
For those more advanced it would certainly be a slick ( yes, I like
that term ;-) solution, and would also get around the owner /
permission issue that github has introduced to the files. Definitely
would be a nice option.
Thanks for your thoughts and input vince. Much appreciated.
--
Cheers
Glenn
--
You received this message because you are subscribed to the Google Groups
"weewx-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
