Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Anno domini 2019 Tue, 12 Nov 18:14:36 -0800 tom scripsit: > On Tue, 12 Nov 2019 16:51:25 -0700 > Gregory Nowak wrote: > > > On Tue, Nov 12, 2019 at 11:58:06AM -0800, Bruce Ferrell wrote: > > > There are actually a couple of ways around the SD wear issue, even > > > though people seem to dearly LOVE SSDs with the exact same issue; > > > > I haven't seen anyone mention even once in this entire thread so far > > that the rpi supports trimming on SD cards. I have a rpi2b, and rpi3b > > the first for three years now, and the second for two. Both are up > > 24/7 running off SD cards, and both have a weekly cron job that runs > > fstrim -a. They're still doing fine on the original SD cards. > > > > Greg > > > > > > Wait, TRIM works on SD cards? I thought that was a feature that > firmware had to implement. > It's in the SD-cards firmware. Some support it, some don't. Nik -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
There is another option I think is worth looking into. There's been word on Samsung adding another filesystem to the Linux kernel called the Flash Friendly FileSystem (F2FS). Perhaps instead of debating disabling the ext4 journal we could just replace it with F2FS? -- __ / "We have the right to survive!" \ | | | "Not by killing others." | | | | -- Deela and Kirk, "Wink of An Eye", | \ stardate 5710.5 / -- \ \ /\ /\ //\\_//\\ \_ _// / / * * \/^^^] \_\O/_/[ ] / \_[ / \ \_ / / [ [ / \/ _/ _[ [ \ /_/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019 10:31:27 +0100 Edward Bartolo via Dng wrote: > Hi all, > > The Raspberry Pi is very frequency used with an SD Card which is > highly intolerant of frequent writes as these are limited. My first SD > Card became read only after about six weeks with Devuan running. Using > Raspbian, this issue did not repeat itself. > > Needless to state, although it seems, it is actually needed for some > people, the Raspberry Pi is not a full blown server, although it can > be used by the hobbyist adolescent who wants to experiment and learn. > > The suggested defaults in this thread will make Devuan even more > unuseable for the vast majority of use cases concerning the Raspberry > Pi. > > For those who cannot affort brand new hardware, they can always opt to > use second hand hardware. If one wants a cheap computer/server, there > is absolutely no need to buy new or to buy the best of brands. > > In short, trying defaults which assume an infinite number of disk > writes, is contrary to what a Raspberry Pi is. > > Please, promote defaults that respect what a Raspberry Pi is. > > Finally, Devuan's Image for Raspberry Pi did NOT DETECT my sound card > by IQaudIO atlhough the kernel modules were included in the image. > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng I'm not constantly writing to the SD card so I really don't understand how having the journal enabled is going to cause the thing to burn out significantly sooner that with it off. I am not expecting to use the thing like an FTP server, but I do expect to be able to boot off it and store basic system stuff like a last-known-time since the thing lacks an RTC and time can't be acquired until after network init. I do not think it's appropriate to discount the Pis or any other SBC and a toy for children. Sure it's used that way but so are pencils, books, and anything else. Some things just don't require a full blown server and are served just fine by the pi. For example, I have an old venerable UPS and power distribution unit I have taken apart, serviced, and modified over the years. I can get data from it and control it via a custom driver shim for Nut Daemon. Now I want all the servers, routers, and switches in the rack to be aware of the UPS's state so that if the power is out past a certain battery threshold, various devices will start to power down so that only essentials are left. I also want to be able to programaticly energize each inverter inside the thing as more or less load is needed to help achieve maximum efficiency. No point in having both inverters active when at less than 40% load. I also want to record line conditions like input freq, voltage, etc for historical analysis on the monitoring server. The Pi is what I had laying around the seemed suitable for the task. It's got GPIO pins and a network port. Sure there are other SBCs I could have used but a lot of them are still going to have SD cards or some kind of write-limited flash storage for their rom. It's not as good as spending the time to actually write some firmware based on OpenWRT but for 30 dollars it's well worth it's weight. -- __ / "We have the right to survive!" \ | | | "Not by killing others." | | | | -- Deela and Kirk, "Wink of An Eye", | \ stardate 5710.5 / -- \ \ /\ /\ //\\_//\\ \_ _// / / * * \/^^^] \_\O/_/[ ] / \_[ / \ \_ / / [ [ / \/ _/ _[ [ \ /_/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019 16:51:25 -0700 Gregory Nowak wrote: > On Tue, Nov 12, 2019 at 11:58:06AM -0800, Bruce Ferrell wrote: > > There are actually a couple of ways around the SD wear issue, even > > though people seem to dearly LOVE SSDs with the exact same issue; > > I haven't seen anyone mention even once in this entire thread so far > that the rpi supports trimming on SD cards. I have a rpi2b, and rpi3b > the first for three years now, and the second for two. Both are up > 24/7 running off SD cards, and both have a weekly cron job that runs > fstrim -a. They're still doing fine on the original SD cards. > > Greg > > Wait, TRIM works on SD cards? I thought that was a feature that firmware had to implement. -- __ / "We have the right to survive!" \ | | | "Not by killing others." | | | | -- Deela and Kirk, "Wink of An Eye", | \ stardate 5710.5 / -- \ \ /\ /\ //\\_//\\ \_ _// / / * * \/^^^] \_\O/_/[ ] / \_[ / \ \_ / / [ [ / \/ _/ _[ [ \ /_/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, Nov 12, 2019 at 11:58:06AM -0800, Bruce Ferrell wrote: > There are actually a couple of ways around the SD wear issue, even though > people seem to dearly LOVE SSDs with the exact same issue; I haven't seen anyone mention even once in this entire thread so far that the rpi supports trimming on SD cards. I have a rpi2b, and rpi3b the first for three years now, and the second for two. Both are up 24/7 running off SD cards, and both have a weekly cron job that runs fstrim -a. They're still doing fine on the original SD cards. Greg -- web site: http://www.gregn.net gpg public key: http://www.gregn.net/pubkey.asc skype: gregn1 (authorization required, add me to your contacts list first) If we haven't been in touch before, e-mail me before adding me to your contacts. -- Free domains: http://www.eu.org/ or mail dns-mana...@eu.org ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019, Simon Hobson wrote: > Ah yes, to think that many of us routinely carry around in our pockets > more storage, RAM, and CPU capacity than we could have dreamed of having > access to back when I got into IT. Cue obligatory Four Yorkshiremen > sketch :D > https://www.youtube.com/watch?v=VKHFZBUTA4k > > If you haven't seen this before, it's worth waiting for the punchline ... Very apropos ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Jim Jackson wrote: > (*) These pi's are a lot more powerfull than the Sun Sparc servers we had > NFS serving user data to 60+ workstations back in the 00's :-) Ah yes, to think that many of us routinely carry around in our pockets more storage, RAM, and CPU capacity than we could have dreamed of having access to back when I got into IT. Cue obligatory Four Yorkshiremen sketch :D https://www.youtube.com/watch?v=VKHFZBUTA4k If you haven't seen this before, it's worth waiting for the punchline ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019, Edward Bartolo via Dng wrote: > Needless to state, although it seems, it is actually needed for some > people, the Raspberry Pi is not a full blown server, although it can > be used by the hobbyist adolescent who wants to experiment and learn. :-) Like everything, it depends. As long as you understand the limitations, work within those contraints and don't have unreasonable expectations it is amazing what can be done.(*) This 67 year adolescent runs 2 RPI household servers. One provides DNS, imap, smtp, dhcp, ntp, syslog and some limited nfs and web services for the household LAN machines. However, it does that from a USB attached harddrive and only boots off the SD card /boot partition, which is ordinarily read-only - no other partitions used. It currently has been up over 400 days. The other is a backup server. Raspberry PI's CAN be made to do all sorts - as can any of the other look-alike SBC's out there now. I am though about to upgrade to a Raspberry Pi 4B - the extra memory and USB/LAN throughput will be nice to have, especially for the backup server. Jim retired Sys & Network Admin (*) These pi's are a lot more powerfull than the Sun Sparc servers we had NFS serving user data to 60+ workstations back in the 00's :-) ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On 11/11/19 5:06 PM, Steve Litt wrote: On Sat, 9 Nov 2019 12:53:26 -0800 tom wrote: On Fri, 8 Nov 2019 16:55:34 +0100 "Dr. Nikolaus Klepp" wrote: Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 Joril via Dng scripsit: On 08/11/19 16:21, g4sra via Dng wrote: FYI: ext4 filesystem journaling (and swap) *should* be disabled by default on SD\SDHC media. To reduce wear? Yes. I really don't think data-loss is an acceptable compromise just to reduce wear. Lack of a journal doesn't necessarily mean data loss. It just means that you might need to run fsck.ext4 on the drive after unmounting. Occasionally you do lose data: I think it has to do with crash shutdowns twice in a row without an intervening fsck.ext4. SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng There are actually a couple of ways around the SD wear issue, even though people seem to dearly LOVE SSDs with the exact same issue; 1.) Use a USB drive. 2.) Somewhat more esoteric, PXE boot and run from an NFS image. The how-tos for the later seem to avoid talking about how to set up a proper PXE boot environment using ISC DHCP, but I'll crack that nut shortly. I love PXE and use it to install windows 7, freebsd, VMWare and a whole bunch of Linux distros in my home lab ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Why is spamassasin compiling here?
On November 8, 2019 8:26:54 PM CST, tom wrote: :: On Wed, 06 Nov 2019 07:45:56 -0600 :: hal wrote: :: :: > The past few days I have been getting this message from my Devuan :: > ASCII mail host. Anyone know what's up with this? :: > :: > I've installed the libc6-dev-i386 package to see if it solves it :: but :: > then I wondered what is spamassasin doing compiling things anyway? :: > :: > :: > :: > :: > /etc/cron.daily/spamassassin: :: > In file included from body_0.xs:2:0: :: > /usr/lib/x86_64-linux-gnu/perl/5.24/CORE/perl.h:689:23: fatal :: error: :: > sys/types.h: No such file or directory #include :: >^ :: > compilation terminated. :: > make: *** [body_0.o] Error 1 :: > command 'make PREFIX=/tmp/.spamassassin25658u4PidNtmp/ignored :: > INSTALLSITEARCH=/var/lib/spamassassin/compiled/5.024/3.004002 :: > >>/tmp/.spamassassin25658u4PidNtmp/log' failed: exit 2 :: > :: > :: > ___ :: > Dng mailing list :: > Dng@lists.dyne.org :: > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng :: :: SpamAss can compile the filtering rules into a machine readable :: binary :: format which offers higher performance than parsing the rules. :: :: -- :: :: / Reality -- what a concept! \ :: || :: \ -- Robin Williams / :: :: \ :: \ ::/\ /\ :: //\\_//\\ :: \_ _// / ::/ * * \/^^^] ::\_\O/_/[ ] :: / \_[ / :: \ \_ / / :: [ [ / \/ _/ :: _[ [ \ /_/ :: ___ :: Dng mailing list :: Dng@lists.dyne.org :: https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng Thank you for the clarification. I get an little concerned when programs start compiling themselves :j ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
[DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Hi all, The Raspberry Pi is very frequency used with an SD Card which is highly intolerant of frequent writes as these are limited. My first SD Card became read only after about six weeks with Devuan running. Using Raspbian, this issue did not repeat itself. Needless to state, although it seems, it is actually needed for some people, the Raspberry Pi is not a full blown server, although it can be used by the hobbyist adolescent who wants to experiment and learn. The suggested defaults in this thread will make Devuan even more unuseable for the vast majority of use cases concerning the Raspberry Pi. For those who cannot affort brand new hardware, they can always opt to use second hand hardware. If one wants a cheap computer/server, there is absolutely no need to buy new or to buy the best of brands. In short, trying defaults which assume an infinite number of disk writes, is contrary to what a Raspberry Pi is. Please, promote defaults that respect what a Raspberry Pi is. Finally, Devuan's Image for Raspberry Pi did NOT DETECT my sound card by IQaudIO atlhough the kernel modules were included in the image. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Anno domini 2019 Tue, 12 Nov 09:05:30 +0100 Adam Borowski scripsit: > On Mon, Nov 11, 2019 at 08:06:26PM -0500, Steve Litt wrote: > > > I really don't think data-loss is an acceptable compromise just to > > > reduce wear. > > > > Lack of a journal doesn't necessarily mean data loss. It just means > > that you might need to run fsck.ext4 on the drive after unmounting. > > Occasionally you do lose data: I think it has to do with crash > > shutdowns twice in a row without an intervening fsck.ext4. > > No: it's not just "might need to run fsck". It's data corruption of > unknown parts of the filesystem. Not on a pair of crashes: a single > crash is enough. If the disk was quiescent the damage might be negligible > or non-existant, but in the normal case, I'd be looking at recovery of > newest writes and restore the rest from backups. I just wonder how we surved in the last millenia with ext2. What you describe was the "default" behaviour of reiserfs. It had the additional feature of truncating all files that were opened for writing to size=0 in case of a crash - no matter if an actual write operation was pending when the crash accured. Don't know if that was ever fixed. Nik > > > Meow! -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Mon, Nov 11, 2019 at 08:06:26PM -0500, Steve Litt wrote: > > I really don't think data-loss is an acceptable compromise just to > > reduce wear. > > Lack of a journal doesn't necessarily mean data loss. It just means > that you might need to run fsck.ext4 on the drive after unmounting. > Occasionally you do lose data: I think it has to do with crash > shutdowns twice in a row without an intervening fsck.ext4. No: it's not just "might need to run fsck". It's data corruption of unknown parts of the filesystem. Not on a pair of crashes: a single crash is enough. If the disk was quiescent the damage might be negligible or non-existant, but in the normal case, I'd be looking at recovery of newest writes and restore the rest from backups. Meow! -- ⢀⣴⠾⠻⢶⣦⠀ A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol, ⣾⠁⢠⠒⠀⣿⡁ 1kg raspberries, 0.4kg sugar; put into a big jar for 1 month. ⢿⡄⠘⠷⠚⠋⠀ Filter out and throw away the fruits (can dump them into a cake, ⠈⠳⣄ etc), let the drink age at least 3-6 months. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
