Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Thu, 28 Nov 2019, Adrian Zaugg wrote: > > On 08.11.19 17:13, g4sra via Dng wrote: > > PS: Being Raspberry Pi specific, I do not know why Raspbian does not use > > F2FS, but that does not exclude Devuan from using it. > > There are some alternatives to F2FS like UBIFS and a few others. As it > is said thas FS that do wear levelling on their own infringe with the > firmware wear levelling of cards, it would be crucial to know if an SD > card has built in wear levelling or not to chose the right FS. Is there > a way to tell from software, i.e. using hdparm or so? AFAIK they all do. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On 08.11.19 17:13, g4sra via Dng wrote: > PS: Being Raspberry Pi specific, I do not know why Raspbian does not use > F2FS, but that does not exclude Devuan from using it. There are some alternatives to F2FS like UBIFS and a few others. As it is said thas FS that do wear levelling on their own infringe with the firmware wear levelling of cards, it would be crucial to know if an SD card has built in wear levelling or not to chose the right FS. Is there a way to tell from software, i.e. using hdparm or so? Regards, Adrian. signature.asc Description: OpenPGP digital signature ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Fri, Nov 15, 2019 at 04:52:50PM -0600, John Morris wrote: > Can't use fstrim in the VMs because qemu / libvirt doesn't support it yet. It does if you use virtio-scsi instead of virtio-blk; same for scsi passthrough or even fake-hardware emulation of regular scsi. Meow! -- ⢀⣴⠾⠻⢶⣦⠀ A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol, ⣾⠁⢠⠒⠀⣿⡁ 1kg raspberries, 0.4kg sugar; put into a big jar for 1 month. ⢿⡄⠘⠷⠚⠋⠀ Filter out and throw away the fruits (can dump them into a cake, ⠈⠳⣄ etc), let the drink age at least 3-6 months. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Thu, 2019-11-14 at 22:03 -0500, Steve Litt wrote: > The piece of information I couldn't find in your (John Morris') data > is > how much of the 160GiB is consumed with data. It makes a big > difference. They have had most of their space written to at this point. The setup is three physical machines that host two virtual machines (web frontend and a postgresql database) running by themselves on two of them and another pair of development VMs on the third or the third is powered down as a spare. The VMs rotate between the three physical hosts on every upgrade cycle to spread the wear on drives and the servers. The production images are about 32GB, between 50% and 80% utilized currently, but the remainder of the drives are unused, but dirty, between moves. Can't use fstrim in the VMs because qemu / libvirt doesn't support it yet. signature.asc Description: This is a digitally signed message part ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
The piece of information I couldn't find in your (John Morris') data is how much of the 160GiB is consumed with data. It makes a big difference. Consider my SSD hosted root partition, which is 4 or 5 years old: [slitt@mydesk ~]$ df -h / Filesystem Size Used Avail Use% Mounted on /dev/sda1 220G 23G 187G 11% / [slitt@mydesk ~]$ I fstrim / about once a week, delete obviously gratuitous files, and 89% of the drive is still available. New writes are scattered across that huge area. Now imagine my SSD were 90% occupied by needed data: A perfectly useable situation with spinning rust drives. New writes would be confined to the other 10%, which quickly gets written to death. By the way, since 2014 I've been using this setup, with a 256G SSD as root, with /home, /var, /tmp, and various data directories mounted to spinning rust. The result is a fast computer, with access to everything under /usr and /etc being SSD speed, while sparing the SSD a lot of repeated writes. And because of the spinning rust mounts, the SSD can be small (read that as cheap). I'd recommend this setup for anyone with a desktop or server requiring more than a few GB of disk space, or doing a lot of writes. SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr On Thu, 14 Nov 2019 15:20:11 -0600 John Morris wrote: > On Wed, 2019-11-13 at 04:06 -0800, Bruce Ferrell wrote: > > Well, I was thinking more along the lines of the "early" failure > > rate for SSD and not so much the convenience of a thing as small as > > my baby finger nail with insane amounts of > > storage. I have active and still in use rotational media from the > > 90's. SSD just can't do that and flash... We don't need to go into > > it. That's what started this thread. > > There is a big difference between SD cards, USB sticks and real SSDs > too. And there is another big difference between consumer SSD and > Enterprise gear. Here is some real world data. Drive has been in > pretty much constant use in production at a public library running the > online catalog and in house cataloging / automation / etc. since 2011. > > === START OF INFORMATION SECTION === > Model Family: Intel X25-M SSD > Device Model: INTEL SSDSA2M160G2GN > Serial Number:CVPO0510036E160AGN > Firmware Version: 2CV102HD > User Capacity:160,041,885,696 bytes > Device is:In smartctl database [for details use: -P show] > ATA Version is: 7 > ATA Standard is: ATA/ATAPI-7 T13 1532D revision 1 > Local Time is:Thu Nov 14 15:09:55 2019 CST > SMART support is: Available - device has SMART capability. > SMART support is: Enabled > > Vendor Specific SMART Attributes with Thresholds: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE > UPDATED WHEN_FAILED RAW_VALUE 3 Spin_Up_Time0x0020 100 > 100 000Old_age Offline - 0 4 Start_Stop_Count > 0x0030 100 100 000Old_age Offline - 0 5 > Reallocated_Sector_Ct 0x0032 100 100 000Old_age Always > - 21 9 Power_On_Hours 0x0032 100 100 000 > Old_age Always - 71816 12 Power_Cycle_Count > 0x0032 100 100 000Old_age Always - 148 192 > Power-Off_Retract_Count 0x0032 100 100 000Old_age Always > - 101 225 Host_Writes_Count 0x0030 200 200 000 > Old_age Offline - 414459 226 Load-in_Time > 0x0032 100 100 000Old_age Always - 184 227 > Torq-amp_Count 0x0032 100 100 000Old_age Always > - 0 228 Power-off_Retract_Count 0x0032 100 100 000 > Old_age Always - 3027407118 232 > Available_Reservd_Space 0x0033 099 099 010Pre-fail Always > - 0 233 Media_Wearout_Indicator 0x0032 096 096 000 > Old_age Always - 0 184 End-to-End_Error0x0033 > 100 100 099Pre-fail Always - 0 > > SMART Error Log Version: 1 > No Errors Logged > > So yeah I trust SSDs now in production workloads. It is in a RAID1 > though so trust but verify is still the watchword. There are six of > these drives in the three servers making up our Evergreen install, all > bought at the same time and all still going strong. Unless something > unusual happens they are more likely to be taken out of service for > being too small than becoming unreliable. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Wed, 2019-11-13 at 04:06 -0800, Bruce Ferrell wrote: > Well, I was thinking more along the lines of the "early" failure rate > for SSD and not so much the convenience of a thing as small as my baby > finger nail with insane amounts of > storage. I have active and still in use rotational media from the > 90's. SSD just can't do that and flash... We don't need to go into > it. That's what started this thread. There is a big difference between SD cards, USB sticks and real SSDs too. And there is another big difference between consumer SSD and Enterprise gear. Here is some real world data. Drive has been in pretty much constant use in production at a public library running the online catalog and in house cataloging / automation / etc. since 2011. === START OF INFORMATION SECTION === Model Family: Intel X25-M SSD Device Model: INTEL SSDSA2M160G2GN Serial Number:CVPO0510036E160AGN Firmware Version: 2CV102HD User Capacity:160,041,885,696 bytes Device is:In smartctl database [for details use: -P show] ATA Version is: 7 ATA Standard is: ATA/ATAPI-7 T13 1532D revision 1 Local Time is:Thu Nov 14 15:09:55 2019 CST SMART support is: Available - device has SMART capability. SMART support is: Enabled Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 3 Spin_Up_Time0x0020 100 100 000Old_age Offline - 0 4 Start_Stop_Count0x0030 100 100 000Old_age Offline - 0 5 Reallocated_Sector_Ct 0x0032 100 100 000Old_age Always - 21 9 Power_On_Hours 0x0032 100 100 000Old_age Always - 71816 12 Power_Cycle_Count 0x0032 100 100 000Old_age Always - 148 192 Power-Off_Retract_Count 0x0032 100 100 000Old_age Always - 101 225 Host_Writes_Count 0x0030 200 200 000Old_age Offline - 414459 226 Load-in_Time0x0032 100 100 000Old_age Always - 184 227 Torq-amp_Count 0x0032 100 100 000Old_age Always - 0 228 Power-off_Retract_Count 0x0032 100 100 000Old_age Always - 3027407118 232 Available_Reservd_Space 0x0033 099 099 010Pre-fail Always - 0 233 Media_Wearout_Indicator 0x0032 096 096 000Old_age Always - 0 184 End-to-End_Error0x0033 100 100 099Pre-fail Always - 0 SMART Error Log Version: 1 No Errors Logged So yeah I trust SSDs now in production workloads. It is in a RAID1 though so trust but verify is still the watchword. There are six of these drives in the three servers making up our Evergreen install, all bought at the same time and all still going strong. Unless something unusual happens they are more likely to be taken out of service for being too small than becoming unreliable. signature.asc Description: This is a digitally signed message part ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Wed, Nov 13, 2019 at 07:27:03PM -0500, Steve Litt wrote: > On Tue, 12 Nov 2019 10:31:27 +0100 > > Regarding eliminating the journal, you bring up a good point. But so > did some other people arguing the opposite. I suggest an installation > that gives the following choices: > > * Don't use a journal > * Use a journal but keep it on an always-connected spinning rust drive > * Use a journal on the SSD or SD card > > My suggestion is that the installer be clear about the tradeoffs when > SSD or SD card are involved, and also ask you whether you want to > fstrim manually or by cron. From what I understand, putting fstrim in > /etc/fstab is always a bad idea. Also, the installer could remind the > user to delete or archive to spinning rust files not needed, to > preserve free space on the SSD or SD card. > > I'm thinking of using an Rpi as a poor man's laptop, because I've had > too many laptops go bad from spilled drinks and other keyboard > destroying mistakes. So I'd have an attached 2.5 inch USB spinning > rust. So I could bind mount (I love bind mounts) part of my spinning > rust to /var very early in the boot. > > But then I might use another Rpi as an experimental thing, and perhaps > shut off journaling to save the memory card. Or perhaps install a big > honking memory card, log rotate ruthlessly, and fstrim every day. Memory cards aren't what they used to be. I remember adding a memory card to an ancient PC to upgrade it from 128K to 649K -- hendrik ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019 10:31:27 +0100 Edward Bartolo via Dng wrote: > Hi all, > > The Raspberry Pi is very frequency used with an SD Card which is > highly intolerant of frequent writes as these are limited. My first SD > Card became read only after about six weeks with Devuan running. Using > Raspbian, this issue did not repeat itself. > > Needless to state, although it seems, it is actually needed for some > people, the Raspberry Pi is not a full blown server, although it can > be used by the hobbyist adolescent who wants to experiment and learn. Regarding eliminating the journal, you bring up a good point. But so did some other people arguing the opposite. I suggest an installation that gives the following choices: * Don't use a journal * Use a journal but keep it on an always-connected spinning rust drive * Use a journal on the SSD or SD card My suggestion is that the installer be clear about the tradeoffs when SSD or SD card are involved, and also ask you whether you want to fstrim manually or by cron. From what I understand, putting fstrim in /etc/fstab is always a bad idea. Also, the installer could remind the user to delete or archive to spinning rust files not needed, to preserve free space on the SSD or SD card. I'm thinking of using an Rpi as a poor man's laptop, because I've had too many laptops go bad from spilled drinks and other keyboard destroying mistakes. So I'd have an attached 2.5 inch USB spinning rust. So I could bind mount (I love bind mounts) part of my spinning rust to /var very early in the boot. But then I might use another Rpi as an experimental thing, and perhaps shut off journaling to save the memory card. Or perhaps install a big honking memory card, log rotate ruthlessly, and fstrim every day. Anyway, Edward's got a point, those with the opposite viewpoint have a point, so maybe the right thing is to empower the (perhaps not too knowledgeable) user to do the most advantageous thing on install. SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On 11/13/19 12:26 AM, Edward Bartolo via Dng wrote: Quote from Bruce Ferrel: "There are actually a couple of ways around the SD wear issue, even though people seem to dearly LOVE SSDs with the exact same issue; 1.) Use a USB drive. 2.) Somewhat more esoteric, PXE boot and run from an NFS image." [/Quote] You are right, people love SD Cards, not for their limitations, but for their great convenience. A tiny SD Card can hold gigabytes of files and it fits neatly in the the Raspberry Pi itself. No need of combersome USB external drives dangling around, and more importantly, no need of extra expenses. Remember, money makes a lot of sense to many people, and I dare say, a lot of people do not have much to spare. Quote from Simon Hobson: Ah yes, to think that many of us routinely carry around in our pockets more storage, RAM, and CPU capacity than we could have dreamed of having access to back when I got into IT. Cue obligatory Four Yorkshiremen sketch :D https://www.youtube.com/watch?v=VKHFZBUTA4k [/Quote] In the old days people used to die from bacterial infections often after a long painful illness. Today, thanks to great people working in medicine, we have no idea of these terrible experiences. Progress is nice when done properly. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng Well, I was thinking more along the lines of the "early" failure rate for SSD and not so much the convenience of a thing as small as my baby finger nail with insane amounts of storage. I have active and still in use rotational media from the 90's. SSD just can't do that and flash... We don't need to go into it. That's what started this thread. "dangling" usb drive? Yeah, it's a pain. But I do have a Pi here at home with a 4TB USB drive and 10 inch hdmi monitor rigged to run off of a smallish solar panel (100w) so it has a LOT of stuff dangling. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
[DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Quote from Bruce Ferrel: "There are actually a couple of ways around the SD wear issue, even though people seem to dearly LOVE SSDs with the exact same issue; 1.) Use a USB drive. 2.) Somewhat more esoteric, PXE boot and run from an NFS image." [/Quote] You are right, people love SD Cards, not for their limitations, but for their great convenience. A tiny SD Card can hold gigabytes of files and it fits neatly in the the Raspberry Pi itself. No need of combersome USB external drives dangling around, and more importantly, no need of extra expenses. Remember, money makes a lot of sense to many people, and I dare say, a lot of people do not have much to spare. Quote from Simon Hobson: Ah yes, to think that many of us routinely carry around in our pockets more storage, RAM, and CPU capacity than we could have dreamed of having access to back when I got into IT. Cue obligatory Four Yorkshiremen sketch :D https://www.youtube.com/watch?v=VKHFZBUTA4k [/Quote] In the old days people used to die from bacterial infections often after a long painful illness. Today, thanks to great people working in medicine, we have no idea of these terrible experiences. Progress is nice when done properly. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Anno domini 2019 Tue, 12 Nov 18:14:36 -0800 tom scripsit: > On Tue, 12 Nov 2019 16:51:25 -0700 > Gregory Nowak wrote: > > > On Tue, Nov 12, 2019 at 11:58:06AM -0800, Bruce Ferrell wrote: > > > There are actually a couple of ways around the SD wear issue, even > > > though people seem to dearly LOVE SSDs with the exact same issue; > > > > I haven't seen anyone mention even once in this entire thread so far > > that the rpi supports trimming on SD cards. I have a rpi2b, and rpi3b > > the first for three years now, and the second for two. Both are up > > 24/7 running off SD cards, and both have a weekly cron job that runs > > fstrim -a. They're still doing fine on the original SD cards. > > > > Greg > > > > > > Wait, TRIM works on SD cards? I thought that was a feature that > firmware had to implement. > It's in the SD-cards firmware. Some support it, some don't. Nik -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
There is another option I think is worth looking into. There's been word on Samsung adding another filesystem to the Linux kernel called the Flash Friendly FileSystem (F2FS). Perhaps instead of debating disabling the ext4 journal we could just replace it with F2FS? -- __ / "We have the right to survive!" \ | | | "Not by killing others." | | | | -- Deela and Kirk, "Wink of An Eye", | \ stardate 5710.5 / -- \ \ /\ /\ //\\_//\\ \_ _// / / * * \/^^^] \_\O/_/[ ] / \_[ / \ \_ / / [ [ / \/ _/ _[ [ \ /_/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019 10:31:27 +0100 Edward Bartolo via Dng wrote: > Hi all, > > The Raspberry Pi is very frequency used with an SD Card which is > highly intolerant of frequent writes as these are limited. My first SD > Card became read only after about six weeks with Devuan running. Using > Raspbian, this issue did not repeat itself. > > Needless to state, although it seems, it is actually needed for some > people, the Raspberry Pi is not a full blown server, although it can > be used by the hobbyist adolescent who wants to experiment and learn. > > The suggested defaults in this thread will make Devuan even more > unuseable for the vast majority of use cases concerning the Raspberry > Pi. > > For those who cannot affort brand new hardware, they can always opt to > use second hand hardware. If one wants a cheap computer/server, there > is absolutely no need to buy new or to buy the best of brands. > > In short, trying defaults which assume an infinite number of disk > writes, is contrary to what a Raspberry Pi is. > > Please, promote defaults that respect what a Raspberry Pi is. > > Finally, Devuan's Image for Raspberry Pi did NOT DETECT my sound card > by IQaudIO atlhough the kernel modules were included in the image. > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng I'm not constantly writing to the SD card so I really don't understand how having the journal enabled is going to cause the thing to burn out significantly sooner that with it off. I am not expecting to use the thing like an FTP server, but I do expect to be able to boot off it and store basic system stuff like a last-known-time since the thing lacks an RTC and time can't be acquired until after network init. I do not think it's appropriate to discount the Pis or any other SBC and a toy for children. Sure it's used that way but so are pencils, books, and anything else. Some things just don't require a full blown server and are served just fine by the pi. For example, I have an old venerable UPS and power distribution unit I have taken apart, serviced, and modified over the years. I can get data from it and control it via a custom driver shim for Nut Daemon. Now I want all the servers, routers, and switches in the rack to be aware of the UPS's state so that if the power is out past a certain battery threshold, various devices will start to power down so that only essentials are left. I also want to be able to programaticly energize each inverter inside the thing as more or less load is needed to help achieve maximum efficiency. No point in having both inverters active when at less than 40% load. I also want to record line conditions like input freq, voltage, etc for historical analysis on the monitoring server. The Pi is what I had laying around the seemed suitable for the task. It's got GPIO pins and a network port. Sure there are other SBCs I could have used but a lot of them are still going to have SD cards or some kind of write-limited flash storage for their rom. It's not as good as spending the time to actually write some firmware based on OpenWRT but for 30 dollars it's well worth it's weight. -- __ / "We have the right to survive!" \ | | | "Not by killing others." | | | | -- Deela and Kirk, "Wink of An Eye", | \ stardate 5710.5 / -- \ \ /\ /\ //\\_//\\ \_ _// / / * * \/^^^] \_\O/_/[ ] / \_[ / \ \_ / / [ [ / \/ _/ _[ [ \ /_/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019 16:51:25 -0700 Gregory Nowak wrote: > On Tue, Nov 12, 2019 at 11:58:06AM -0800, Bruce Ferrell wrote: > > There are actually a couple of ways around the SD wear issue, even > > though people seem to dearly LOVE SSDs with the exact same issue; > > I haven't seen anyone mention even once in this entire thread so far > that the rpi supports trimming on SD cards. I have a rpi2b, and rpi3b > the first for three years now, and the second for two. Both are up > 24/7 running off SD cards, and both have a weekly cron job that runs > fstrim -a. They're still doing fine on the original SD cards. > > Greg > > Wait, TRIM works on SD cards? I thought that was a feature that firmware had to implement. -- __ / "We have the right to survive!" \ | | | "Not by killing others." | | | | -- Deela and Kirk, "Wink of An Eye", | \ stardate 5710.5 / -- \ \ /\ /\ //\\_//\\ \_ _// / / * * \/^^^] \_\O/_/[ ] / \_[ / \ \_ / / [ [ / \/ _/ _[ [ \ /_/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, Nov 12, 2019 at 11:58:06AM -0800, Bruce Ferrell wrote: > There are actually a couple of ways around the SD wear issue, even though > people seem to dearly LOVE SSDs with the exact same issue; I haven't seen anyone mention even once in this entire thread so far that the rpi supports trimming on SD cards. I have a rpi2b, and rpi3b the first for three years now, and the second for two. Both are up 24/7 running off SD cards, and both have a weekly cron job that runs fstrim -a. They're still doing fine on the original SD cards. Greg -- web site: http://www.gregn.net gpg public key: http://www.gregn.net/pubkey.asc skype: gregn1 (authorization required, add me to your contacts list first) If we haven't been in touch before, e-mail me before adding me to your contacts. -- Free domains: http://www.eu.org/ or mail dns-mana...@eu.org ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019, Simon Hobson wrote: > Ah yes, to think that many of us routinely carry around in our pockets > more storage, RAM, and CPU capacity than we could have dreamed of having > access to back when I got into IT. Cue obligatory Four Yorkshiremen > sketch :D > https://www.youtube.com/watch?v=VKHFZBUTA4k > > If you haven't seen this before, it's worth waiting for the punchline ... Very apropos ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Jim Jackson wrote: > (*) These pi's are a lot more powerfull than the Sun Sparc servers we had > NFS serving user data to 60+ workstations back in the 00's :-) Ah yes, to think that many of us routinely carry around in our pockets more storage, RAM, and CPU capacity than we could have dreamed of having access to back when I got into IT. Cue obligatory Four Yorkshiremen sketch :D https://www.youtube.com/watch?v=VKHFZBUTA4k If you haven't seen this before, it's worth waiting for the punchline ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Tue, 12 Nov 2019, Edward Bartolo via Dng wrote: > Needless to state, although it seems, it is actually needed for some > people, the Raspberry Pi is not a full blown server, although it can > be used by the hobbyist adolescent who wants to experiment and learn. :-) Like everything, it depends. As long as you understand the limitations, work within those contraints and don't have unreasonable expectations it is amazing what can be done.(*) This 67 year adolescent runs 2 RPI household servers. One provides DNS, imap, smtp, dhcp, ntp, syslog and some limited nfs and web services for the household LAN machines. However, it does that from a USB attached harddrive and only boots off the SD card /boot partition, which is ordinarily read-only - no other partitions used. It currently has been up over 400 days. The other is a backup server. Raspberry PI's CAN be made to do all sorts - as can any of the other look-alike SBC's out there now. I am though about to upgrade to a Raspberry Pi 4B - the extra memory and USB/LAN throughput will be nice to have, especially for the backup server. Jim retired Sys & Network Admin (*) These pi's are a lot more powerfull than the Sun Sparc servers we had NFS serving user data to 60+ workstations back in the 00's :-) ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On 11/11/19 5:06 PM, Steve Litt wrote: On Sat, 9 Nov 2019 12:53:26 -0800 tom wrote: On Fri, 8 Nov 2019 16:55:34 +0100 "Dr. Nikolaus Klepp" wrote: Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 Joril via Dng scripsit: On 08/11/19 16:21, g4sra via Dng wrote: FYI: ext4 filesystem journaling (and swap) *should* be disabled by default on SD\SDHC media. To reduce wear? Yes. I really don't think data-loss is an acceptable compromise just to reduce wear. Lack of a journal doesn't necessarily mean data loss. It just means that you might need to run fsck.ext4 on the drive after unmounting. Occasionally you do lose data: I think it has to do with crash shutdowns twice in a row without an intervening fsck.ext4. SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng There are actually a couple of ways around the SD wear issue, even though people seem to dearly LOVE SSDs with the exact same issue; 1.) Use a USB drive. 2.) Somewhat more esoteric, PXE boot and run from an NFS image. The how-tos for the later seem to avoid talking about how to set up a proper PXE boot environment using ISC DHCP, but I'll crack that nut shortly. I love PXE and use it to install windows 7, freebsd, VMWare and a whole bunch of Linux distros in my home lab ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
[DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Hi all, The Raspberry Pi is very frequency used with an SD Card which is highly intolerant of frequent writes as these are limited. My first SD Card became read only after about six weeks with Devuan running. Using Raspbian, this issue did not repeat itself. Needless to state, although it seems, it is actually needed for some people, the Raspberry Pi is not a full blown server, although it can be used by the hobbyist adolescent who wants to experiment and learn. The suggested defaults in this thread will make Devuan even more unuseable for the vast majority of use cases concerning the Raspberry Pi. For those who cannot affort brand new hardware, they can always opt to use second hand hardware. If one wants a cheap computer/server, there is absolutely no need to buy new or to buy the best of brands. In short, trying defaults which assume an infinite number of disk writes, is contrary to what a Raspberry Pi is. Please, promote defaults that respect what a Raspberry Pi is. Finally, Devuan's Image for Raspberry Pi did NOT DETECT my sound card by IQaudIO atlhough the kernel modules were included in the image. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Anno domini 2019 Tue, 12 Nov 09:05:30 +0100 Adam Borowski scripsit: > On Mon, Nov 11, 2019 at 08:06:26PM -0500, Steve Litt wrote: > > > I really don't think data-loss is an acceptable compromise just to > > > reduce wear. > > > > Lack of a journal doesn't necessarily mean data loss. It just means > > that you might need to run fsck.ext4 on the drive after unmounting. > > Occasionally you do lose data: I think it has to do with crash > > shutdowns twice in a row without an intervening fsck.ext4. > > No: it's not just "might need to run fsck". It's data corruption of > unknown parts of the filesystem. Not on a pair of crashes: a single > crash is enough. If the disk was quiescent the damage might be negligible > or non-existant, but in the normal case, I'd be looking at recovery of > newest writes and restore the rest from backups. I just wonder how we surved in the last millenia with ext2. What you describe was the "default" behaviour of reiserfs. It had the additional feature of truncating all files that were opened for writing to size=0 in case of a crash - no matter if an actual write operation was pending when the crash accured. Don't know if that was ever fixed. Nik > > > Meow! -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Mon, Nov 11, 2019 at 08:06:26PM -0500, Steve Litt wrote: > > I really don't think data-loss is an acceptable compromise just to > > reduce wear. > > Lack of a journal doesn't necessarily mean data loss. It just means > that you might need to run fsck.ext4 on the drive after unmounting. > Occasionally you do lose data: I think it has to do with crash > shutdowns twice in a row without an intervening fsck.ext4. No: it's not just "might need to run fsck". It's data corruption of unknown parts of the filesystem. Not on a pair of crashes: a single crash is enough. If the disk was quiescent the damage might be negligible or non-existant, but in the normal case, I'd be looking at recovery of newest writes and restore the rest from backups. Meow! -- ⢀⣴⠾⠻⢶⣦⠀ A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol, ⣾⠁⢠⠒⠀⣿⡁ 1kg raspberries, 0.4kg sugar; put into a big jar for 1 month. ⢿⡄⠘⠷⠚⠋⠀ Filter out and throw away the fruits (can dump them into a cake, ⠈⠳⣄ etc), let the drink age at least 3-6 months. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Sat, 9 Nov 2019 12:53:26 -0800 tom wrote: > On Fri, 8 Nov 2019 16:55:34 +0100 > "Dr. Nikolaus Klepp" wrote: > > > Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 > > Joril via Dng scripsit: > > > On 08/11/19 16:21, g4sra via Dng wrote: > > > > FYI: ext4 filesystem journaling (and swap) *should* be disabled > > > > by default on SD\SDHC media. > > > > > > To reduce wear? > > > > Yes. > > I really don't think data-loss is an acceptable compromise just to > reduce wear. Lack of a journal doesn't necessarily mean data loss. It just means that you might need to run fsck.ext4 on the drive after unmounting. Occasionally you do lose data: I think it has to do with crash shutdowns twice in a row without an intervening fsck.ext4. SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Dr. Nikolaus Klepp writes: > Anno domini 2019 Sat, 9 Nov 20:16:40 -0800 > tom scripsit: > [snip] >> Because the whole point of my starting this thread was to provide >> instructions on how to recover a system after the dataloss occurred. If >> you would take the time to read the original post. This is not even a >> question. I would not have posted this or found out about these crazy >> defaults if my data was never lost in the first place. > > Maybe I am a bit biased here, but putting important data on 30.- > hardware like the RPi might raise the question of commensurability. # Guessing you imply EUR as units (seeing an .at domain) but Definitly so if Zimbabwean dollar, LOL. Even JPY, although these don't have fractional bits, anymore. Just my 2 JPY, -- Olaf Meeuwissen, LPIC-2FSF Associate Member since 2004-01-27 GnuPG key: F84A2DD9/B3C0 2F47 EA19 64F4 9F13 F43E B8A4 A88A F84A 2DD9 Support Free Softwarehttps://my.fsf.org/donate Join the Free Software Foundation https://my.fsf.org/join ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Anno domini 2019 Sat, 9 Nov 20:16:40 -0800 tom scripsit: > On Sat, 9 Nov 2019 22:09:13 + > g4sra via Dng wrote: > > > On 09/11/2019 20:53, tom wrote: > > > On Fri, 8 Nov 2019 16:55:34 +0100 > > > "Dr. Nikolaus Klepp" wrote: > > > > > >> Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 > > >> Joril via Dng scripsit: > > >>> On 08/11/19 16:21, g4sra via Dng wrote: > > FYI: ext4 filesystem journaling (and swap) *should* be disabled > > by default on SD\SDHC media. > > >>> > > >>> To reduce wear? > > >> > > >> Yes. > > > > > > I really don't think data-loss is an acceptable compromise just to > > > reduce wear. > > > > > > > Why do you believe turning off the journaling causes data loss ? > > Journaling is not an acceptable method to prevent data loss - it > > doesn't. ___ > > Because the whole point of my starting this thread was to provide > instructions on how to recover a system after the dataloss occurred. If > you would take the time to read the original post. This is not even a > question. I would not have posted this or found out about these crazy > defaults if my data was never lost in the first place. > > Maybe I am a bit biased here, but putting important data on 30.- hardware like the RPi might raise the question of commensurability. Nik -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Sat, 9 Nov 2019 22:09:13 + g4sra via Dng wrote: > On 09/11/2019 20:53, tom wrote: > > On Fri, 8 Nov 2019 16:55:34 +0100 > > "Dr. Nikolaus Klepp" wrote: > > > >> Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 > >> Joril via Dng scripsit: > >>> On 08/11/19 16:21, g4sra via Dng wrote: > FYI: ext4 filesystem journaling (and swap) *should* be disabled > by default on SD\SDHC media. > >>> > >>> To reduce wear? > >> > >> Yes. > > > > I really don't think data-loss is an acceptable compromise just to > > reduce wear. > > > > Why do you believe turning off the journaling causes data loss ? > Journaling is not an acceptable method to prevent data loss - it > doesn't. ___ Because the whole point of my starting this thread was to provide instructions on how to recover a system after the dataloss occurred. If you would take the time to read the original post. This is not even a question. I would not have posted this or found out about these crazy defaults if my data was never lost in the first place. -- / Into love and out again, \ || | Thus I went and thus I go. Spare your | | voice, and hold your pen: | || | Well and bitterly I know All the songs | | were ever sung,| || | All the words were ever said; Could it | | be, when I was young, | || | Someone dropped me on my head? | || \ -- Dorothy Parker, "Theory"/ \ \ /\ /\ //\\_//\\ \_ _// / / * * \/^^^] \_\O/_/[ ] / \_[ / \ \_ / / [ [ / \/ _/ _[ [ \ /_/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On 09/11/2019 20:53, tom wrote: > On Fri, 8 Nov 2019 16:55:34 +0100 > "Dr. Nikolaus Klepp" wrote: > >> Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 >> Joril via Dng scripsit: >>> On 08/11/19 16:21, g4sra via Dng wrote: FYI: ext4 filesystem journaling (and swap) *should* be disabled by default on SD\SDHC media. >>> >>> To reduce wear? >> >> Yes. > > I really don't think data-loss is an acceptable compromise just to > reduce wear. > Why do you believe turning off the journaling causes data loss ? Journaling is not an acceptable method to prevent data loss - it doesn't. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Anno domini 2019 Sat, 9 Nov 12:53:26 -0800 tom scripsit: > On Fri, 8 Nov 2019 16:55:34 +0100 > "Dr. Nikolaus Klepp" wrote: > > > Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 > > Joril via Dng scripsit: > > > On 08/11/19 16:21, g4sra via Dng wrote: > > > > FYI: ext4 filesystem journaling (and swap) *should* be disabled > > > > by default on SD\SDHC media. > > > > > > To reduce wear? > > > > Yes. > > I really don't think data-loss is an acceptable compromise just to > reduce wear. > > Wear is the ultimate data loss. -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Fri, 8 Nov 2019 16:55:34 +0100 "Dr. Nikolaus Klepp" wrote: > Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 > Joril via Dng scripsit: > > On 08/11/19 16:21, g4sra via Dng wrote: > > > FYI: ext4 filesystem journaling (and swap) *should* be disabled > > > by default on SD\SDHC media. > > > > To reduce wear? > > Yes. I really don't think data-loss is an acceptable compromise just to reduce wear. -- ___ / Cold, adj.: \ | | | When the politicians walk around with | \ their hands in their own pockets. / --- \ \ /\ /\ //\\_//\\ \_ _// / / * * \/^^^] \_\O/_/[ ] / \_[ / \ \_ / / [ [ / \/ _/ _[ [ \ /_/ ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Am 2019-11-08 20:28, schrieb Adam Borowski: I've had an eMMC card that survived 4 years of a constant I/O-bound load with more writes than reads, before starting to fail. On the other hand, you can kill a SD card within a day. A nice example what you should NOT do with flash memory ;-) https://www.vice.com/en_us/article/qvgxqp/worn-out-flash-memory-is-suddenly-bricking-tesla-cars ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On Fri, Nov 08, 2019 at 04:13:58PM +, g4sra via Dng wrote: > On 08/11/2019 15:36, Joril via Dng wrote: > > On 08/11/19 16:21, g4sra via Dng wrote: > >> FYI: ext4 filesystem journaling (and swap) *should* be disabled by default > >> on SD\SDHC media. > > > > To reduce wear? > There is good documentation 'out there' about how SD\SDHC Flash and > Journaling works. > The journal repeatedly written to a confined section of the filesystem. > > Not all SD's are equal, either in the quality of the memory cells or in the > quality of the wear-leveling algorithm. > NOTE: these in-built wear-leveling algorithms can be incompatible with flash > filesystems. > Personal experience has demonstrated once it is 'burned' the whole SD becomes > read-only. I've had an eMMC card that survived 4 years of a constant I/O-bound load with more writes than reads, before starting to fail. On the other hand, you can kill a SD card within a day. > It should be the Users choice to enable journaling and risk SD destruction in > preference to shutting down properly. > I recommend filesystem check at boot should be enabled with journaling off > and disabled with journaling on. > > PS: Being Raspberry Pi specific, I do not know why Raspbian does not use > F2FS, but that does not exclude Devuan from using it. Or btrfs. Both of these have a massive performance win over ext4 or XFS. But btrfs is... well, btrfs: it has both features you can't live without and WTF-level caveats, while f2fs is a regular filesystem without surprises. Basically: * ext4 and XFS are old-style filesystems with a journal attached. Both metadata (inodes, extent lists) and the journal get rewritten in place all the time. * f2fs is a journal someone forgot to attach a filesystem to. It does all writes in a whole-disk append-only log that matches the needs of low-quality flash perfectly (and also works well on shingled HDDs). The downside is that from time to time it has to take an erase block that has a mix of live and deleted/obsoleted data, and rewrite the live pieces at the head of the journal. * btrfs likewise does everything CoW: every update instead of being done in-place goes to an unused part of the disk, then pieces of metadata trees that point to those new writes get also written to an unused part of the disk, free space being garbage collected later. It avoids repeated rewrites of f2fs, but those metadata tree updates tend to get really nasty. Btrfs can also win big with compression: new ARMs and lowest-end netbooks have quite adequate CPUs but abysmally slow storage. But in the general, f2fs is a safer choice to unleash onto an unsuspecting user. Meow! -- ⢀⣴⠾⠻⢶⣦⠀ A MAP07 (Dead Simple) raspberry tincture recipe: 0.5l 95% alcohol, ⣾⠁⢠⠒⠀⣿⡁ 1kg raspberries, 0.4kg sugar; put into a big jar for 1 month. ⢿⡄⠘⠷⠚⠋⠀ Filter out and throw away the fruits (can dump them into a cake, ⠈⠳⣄ etc), let the drink age at least 3-6 months. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On 08/11/2019 15:36, Joril via Dng wrote: > On 08/11/19 16:21, g4sra via Dng wrote: >> FYI: ext4 filesystem journaling (and swap) *should* be disabled by default >> on SD\SDHC media. > > To reduce wear? There is good documentation 'out there' about how SD\SDHC Flash and Journaling works. The journal repeatedly written to a confined section of the filesystem. Not all SD's are equal, either in the quality of the memory cells or in the quality of the wear-leveling algorithm. NOTE: these in-built wear-leveling algorithms can be incompatible with flash filesystems. Personal experience has demonstrated once it is 'burned' the whole SD becomes read-only. It should be the Users choice to enable journaling and risk SD destruction in preference to shutting down properly. I recommend filesystem check at boot should be enabled with journaling off and disabled with journaling on. PS: Being Raspberry Pi specific, I do not know why Raspbian does not use F2FS, but that does not exclude Devuan from using it. > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Anno domini 2019 Fri, 8 Nov 16:36:24 +0100 Joril via Dng scripsit: > On 08/11/19 16:21, g4sra via Dng wrote: > > FYI: ext4 filesystem journaling (and swap) *should* be disabled by default > > on SD\SDHC media. > > To reduce wear? Yes. > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng > -- Please do not email me anything that you are not comfortable also sharing with the NSA, CIA ... ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On 08/11/19 16:21, g4sra via Dng wrote: FYI: ext4 filesystem journaling (and swap) *should* be disabled by default on SD\SDHC media. To reduce wear? ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
FYI: ext4 filesystem journaling (and swap) *should* be disabled by default on SD\SDHC media. On 08/11/2019 14:46, Joril via Dng wrote: > On 27/10/19 03:32, tom wrote: > >> The defaults on the linux kernel flags have the options >> rootflags=noload. This has the effect of disabling ext4 filesystem >> journaling, checksumming, and all other safeguards. >> >> In addition to that the root filesystem's parameters are set to always >> disable filesystem checks. > > Thanks for pointing this out :) > It looks like "noload" is useful only for recovery purposes > ___ > Dng mailing list > Dng@lists.dyne.org > https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
Re: [DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
On 27/10/19 03:32, tom wrote: The defaults on the linux kernel flags have the options rootflags=noload. This has the effect of disabling ext4 filesystem journaling, checksumming, and all other safeguards. In addition to that the root filesystem's parameters are set to always disable filesystem checks. Thanks for pointing this out :) It looks like "noload" is useful only for recovery purposes ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
[DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Hi, I have been using the Raspberry Pi for a music player with an IQaudIO DAC PRO card. First, I used Devuan, which very sadly, did not recognize the DAC. After about 2 months of using Devuan, my SDCard, was burnt: no more write cycles were allowed and it become read-only. To correct this, and at the same time, to continue using Devuan, I used 'noatime', and this resulted in the SDCard remaining useable without issues. However, not being able to use the DAC, and after being advised from the DAC's manufacturer, I had to switch to Raspbian. The latter, readily recognised the DAC and I could use it. The audio difference betweent the Pi's built in sound chip and the DAC, is outstanding, to say the least. So, needless to state, I am now on Raspbian, but with an init other than systemd. Pi users normally use an SDCard for their OS. I would have imagined a read-only bootable installation with file writes restricted to when an OS is shutdown, or better, on user requests. This to protect the SDCard. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
[DNG] Insane defaults on Raspberry Pi images - How to fix corruption/dataloss
Hello, I wanted to warn those of you using the Raspberry Pi or other aarch64 Devuan ASCII installs. The defaults on the linux kernel flags have the options rootflags=noload. This has the effect of disabling ext4 filesystem journaling, checksumming, and all other safeguards. In addition to that the root filesystem's parameters are set to always disable filesystem checks. To be clear, all filesystem safeguards have been turned off AND all filesystem sanity checking has been disabled. This will inevitably lead to major filesystem corruption and data loss as has happened to me. There is more. The vfat partition that stores the system's blobs, kernel, hardware configuration, and boot partition is not able to be checked because the userspace tools required for doing so are missing from the default install. If you have ever wondered why your RPIs have stopped working properly this is probably why. I have also developed a recovery process. First you will need to remove the insane kernel flag from your cmd line. edit /boot/cmdline.txt with a text editor and remove the section 'rootflags=noload'. It is between rootwait and net.ifnames=0. Next your going to install some utilities. apt install dosfstools debsums the dosfstools package contains the fsck.vfat utility. The debsums utility will allow us to detect some corrupted and missing system files later ounce we get the system into a more consistent state. Now that we have removed the insane kernelflag you will want to edit your /etc/fstab to look like this: /dev/mmcblk0p1/boot vfat defaults 02 /dev/mmcblk0p2 / ext4 rw,data=ordered,relatime,block_validity,delalloc,journal_checksum,barrier,user_xattr,acl 0 1 This will enable the proper filesystem safeguards for the Pi's finicky power connector and purpose as a development device that may get powered on and off at any time. The 5 volt rail is also poorly designed which without a specially offset power supply, can drop more than 0.45 volts from the input and bring the SoC out of spec as it only has a tolerance of +/- 0.25V. This is due to the highly resistive self-resetting fuse on the 5V power input. Next we will want to properly adjust the root filesystem's parameters. tune2fs -C 1 /dev/mmcblk0p2 # Check filesystem on every boot/mount tune2fs -i 1m /dev/mmcblk0p2# Check filesystem monthly tune2fs -e panic /dev/mmcblk0p2 # trigger kernel panic on fatal disk error If you would rather not check the filesystem on every bootup you can omit the tune2fs -C line. However keep in mind that this only takes about 2 seconds to check an ext4 filesystem of 64GB on a aarch64 cpu @200Mhz. Filesystem checking time is really a thing of the past and only a problem with ext3 filesystem not ext4 thanks to ext4 features like uninit_bg,extent, and other improvements. next, create a new text file at /etc/sysctl.d/local.conf containing: # Reboot on kpanic after 10 seconds kernel.panic = 10 now, use the following command: touch /forcefsck reboot This will reboot your Pi and check all filesystems. Ounce your Pi comes back up it may come up in a degraded state as mine did. Some system files may be missing such as /etc/hostname. Simply re-recreate those. Now will will want to run debsums -c. This will check all installed packages for corrupted/missing files. It should tell you what packages these are from. If not you can use dpkg to query for them. Ounce you have determined which packages are corrupted you can reinstall them from the repositories with like like so. In this example Vim, eudev, and tzdata had missing and corrupted files on my pi. apt-get install --reinstall eudev vim-common tzdata It's a good idea to run debsums -c one more time to make sure you got everything. Ounce you've reinstalled all your corrupted packages reboot again and watch your bootup process for any more errors you may need to resolve by hand. If not double check all your mounts for insane parameters using the 'mount' command. If everything looks perfect you should be safe from now on. I am currently studying how the Raspberry Pi Devuan image is made so that I may supply patches to fix this by default for the next point release or hotfix. Till then this manual procedure should suffice. My guess is that RPI support was just copy-pasted from rasbian without a whole lot of checking. ___ Dng mailing list Dng@lists.dyne.org https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
