Re: BTRFS partition corrupted after deleting files in /home
On Wed, Jan 13, 2021 at 1:09 PM Sreyan Chakravarty wrote: > > On Tue, Jan 12, 2021 at 9:04 AM Chris Murphy wrote: > > > > Because of the --init-csum-tree failure i'm a bit concerned that it's > > possible the --repair step didn't completely repair things. So... just > > keep important things backed up. Top priority. If you told me you hate > > your data then I wouldn't worry about it. > > > > Just letting you know, your suspicions were right. The fsck was not > fully successful. > > I have lost all my previous snapshots, one of them is an empty > directory and one of them, while I can see files in it, can't be used > for restoration as the system won't boot. > > My system is now a ticking time bomb for another crash. At this point I suggest not booting from it. Boot the live USB, mount the file system, and copy the important files from the "home" subvolume (or any of its snapshots) onto some other drive. Using the file manager is sufficient. -- Chris Murphy ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Wed, Jan 13, 2021 at 6:02 AM George N. White III wrote: > Your drive has been used for 1671 hours, and > 1491 power-on cycles. Mechanical device wear is often highest at startup, > so this is probably getting close to the design lifetime of a consumer laptop > HDD. I missed that this is a 2.5 inch drive. Probably time for it to have a dirt nap. Or use it for torture testing file systems! -- Chris Murphy ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Wed, Jan 13, 2021 at 2:41 AM Sreyan Chakravarty wrote: > > On Tue, Jan 12, 2021 at 9:16 AM Chris Murphy wrote: > > > > > > -x has more information that might be relevant including firmware > > revision and some additional logs for recent drive reported errors > > which usually are benign. But might be clues. > > > > These two attributes I'm not familiar with > > 187 Reported_Uncorrect 0x0032 100 096 000Old_age > > Always - 4294967301 > > 188 Command_Timeout 0x0032 100 100 000Old_age > > Always - 98785820672 > > > > But the value is well above threshold for both so I'm not worried about it. > > > > > > Here is the output of: > > # smartctl -Ax /dev/sda > > https://pastebin.com/raw/GrgrQrSf > > I have no idea what it means. 9 Power_On_Hours -O--CK 097 097 000-1671 There's a bunch of write errors at 548 hours. And more recently read errors followed by: 10 -- 41 05 40 00 00 60 d9 6e 08 00 00 Error: IDNF at LBA = 0x60d96e08 = 1624862216 548 hours is not an old drive. It shouldn't have any write errors. But as a drive ages there might be some and they should be handled transparently and not affect any other operation. Something is definitely wrong that there are write errors followed by read errors followed by this IDNF error which suggests the drive is having problems read its own data written to sectors reserved for its own use, for example its firmware is often on the drive for some make/models, and so is the bad blocks map. If this sector isn't reading correctly *who knows* what weird things that could trigger. I know who, the person who writes drive firmware. I don't, so I can only speculate that this looks pretty much like a drive that needs to be used for some other purpose or disposed of. And also? Maybe it's getting worse. Maybe it wasn't exhibiting any of these problems yet when you were using NTFS and ext4. Or wasn't ever detected. The first thing that did detect it was LVM thin's metadata checksumming. And the most recent is btrfs. If it were in warranty I'd say make it the manufacturers problem. If it's not, well you could take a chance with it in a Btrfs raid1 with some other cheap drive. Whatever problems they have will be different, and Btrfs will automatically detect and repair them as long as they don't happen at the same time in the same place (astronomically unlikely). So that'd be a good use for it. I personally wouldn't be any other file system on it except maybe ZFS, because with this behavior, you don't want to trust any data to it without full data checksumming. Let alone important data. > This is the problem with SMART tests, they are so esoteric that it is > difficult for a common user to make sense of it. All low level stuff is like this, yeah. -- Chris Murphy ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Wed, Jan 13, 2021 at 2:36 AM Sreyan Chakravarty wrote: > > 1) Is it possible there is nothing wrong with my drive, but there is > something with my BIOS/HDD Firmware ? May be my firmware is not > capable of BTRFS's stringent write requirements ? The sample size is too small to know for sure what kind of HDD defect it is. If it were an actuator or read/write head, it would happen more often. If it were a localized media surface defect, it's unlikely two copies of metadata would be affected. Btrfs dup profile metadata chunks aren't colocated. They aren't very far apart but far enough I'd expect a lot more metadata and/or data corruption than just one commit. If it were defective memory (used as cache) in the drive, I'd expect it'd happen more often. I have discussed with folks who know way more about myriad drive failures that they've seen cases where a write failure results in all queued (cached) writes being dropped. Is that what happened? *shrug* Speculation. And is it a firmware bug, or is it some other transient problem with the drive? *shrug* I don't think it has anything to do with BIOS, or logic board related including memory. And I don't think it has anything to do with Btrfs write patterns. Btrfs write pattern isn't that variable, so whatever pattern triggers a problem is going to happen more often than once every month or two. Probably hundreds of times per day or more. And yet, this happened once with Btrfs in a bit over a month. Pretty weird. Another possibility is power supply. Either brown outs or noisy incoming power. Or even made noisy by a power supply. I had a student a while back with a high end imaging setup, all brand new equipment. Constant crashes. Replaced memory first. Then other hardware. And got to adding a UPS. Voila. All problems went away. Unfortunately if you're in such a situation it is process of elimination. And if it only reproduces every couple of months, that could take a while. > I say this because I have used Windows with NTFS on this machine, I > have used Ubuntu with EXT4, and Fedora with thick-LVM with EXT4. None > of these configurations gave me any such problems. Yeah it's a fair point. But you did have a problem with LVM thin provisioning which is not Btrfs. But does use checksums for at least some (maybe all, not sure) of its metadata. NTFS doesn't checksum anything. ext4 checksums its own metadata but not data. A lost metadata write on either will be immediately detected on ext4 before it causes too much confusion where NTFS will need to get confused before it realizes something is wrong. A lost data write on NTFS or ext4 means the next time that data is read, it's just not there. It's garbage. So the OS won't even care, it'll just hand over what it finds to the application, and it'd be up to the application to handle the fact it got back garbage. It could manifest in all kinds of ways or not even at all. So they are sufficiently different in this area that they're not that comparable. The most comparable would be OpenZFS. It also checksums all metadata and data, but it's not a supported file system in Fedora. So you're kind on your own, but there are Fedora users using OpenZFS for sysroot (maybe even /boot, GRUB supports it). > 2) Since there is a high likelihood that my filesystem is not > completely fixed, then when I take a backup using partclone, dd or > clonezilla won't those errors be carried over ? Yes. I recommend a Pika Backup for a simple GUI solution to back things up. It doesn't have any file system specific dependencies. I'm sure if you look through the list archive for backups or start a new thread with your requirements you'll get more suggestions. > > Even if I buy a new drive and restore the backup, I still might get crashes. You definitely want a backup with its own independent file system. A dd/ddrescue/clone is mainly for troubleshooting and disaster recovery. It's not a great backup because a backup you want easy to keep up to date. Daily or weekly, depending on your tolerance for loss. > > 3) This is a weird question but can you recommend me a HDD that I can > buy which can handle BTRFS ? Or even which features I might look for > while buying (not a SSD but a HDD) All the drive manufacturers have played enough musical chairs, I can't keep track of who makes or made what. Every drive fails eventually. HDD follow the bell curve, so they tend to either fail early or fail late in their lifespan. You can't really game the system. The odds of picking something that exhibits this same behavior is astronomical. Except for the NVMe drive, which came in the laptop I'm using, I pretty much use a mix of warranty and price. It's cheaper to mitigate risk with a backup, which you need anyway even if you get an expensive drive. So I just ignore all claims of reliability and I don't even care about 5+ year warranties. No 90 day warranties. 1 year if it's dirt cheap. Otherwise 3 years. And never buy an extended warranty. But I gotta say
Re: Any recent progress toward Fedora support for Raspberry Pi 4?
On Tue, 2021-01-12 at 09:51 -0500, Jonathan Billings wrote: > On Mon, Jan 11, 2021 at 03:46:12PM -0600, Robert G. (Doc) Savage via > users wrote: > > With 8 GB the Pi4 finally has enough memory to be a fairly > > satisfactory > > workstation. By "headless server" do I understand you to mean you > > have > > no desktop installed? If so, I need more. > > I suggest following up on the thread I mentioned, then. They have a > UEFI boot and anaconda working. I just chose the simplest method of > copying the exisitng Fedora aarch64 image to an SD card, since I have > a more limited requirements than you. > > Even with 8GB of RAM, the system isn't exactly fast, particularly > with > slow media. Unless they can get an NVME connector, it's going to be > a > slow workstation. Jonathan, Besides offering far more RAM than its predecessors, the Pi 4 can boot and run from a thumb drive which is substantially faster than an old microSD card. Not a gaming machine, but a good (and tiny) alternative to desktop PCs. --Doc ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Tue, Jan 12, 2021 at 9:04 AM Chris Murphy wrote: > > Because of the --init-csum-tree failure i'm a bit concerned that it's > possible the --repair step didn't completely repair things. So... just > keep important things backed up. Top priority. If you told me you hate > your data then I wouldn't worry about it. > Just letting you know, your suspicions were right. The fsck was not fully successful. I have lost all my previous snapshots, one of them is an empty directory and one of them, while I can see files in it, can't be used for restoration as the system won't boot. My system is now a ticking time bomb for another crash. -- Regards, Sreyan Chakravarty ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On 1/13/21 12:36 AM, Sreyan Chakravarty wrote: I am fully open to the fact that there may be something wrong with my HDD. But it is difficult to believe that, since this laptop is from 2016 and I had been using Windows 10 on it for a long time and saw no problems. None that you know of. Windows storage doesn't checksum its content, so if any writes were missed, there's a fairly good chance the computer would never tell you that something was wrong. But, setting that aside, you have to bear in mind that hard drives wear out. Power supplies wear out. Cables come loose. Even in mostly solid-state computers, systems wear out over time. "It worked in the past" is never evidence that a computer is working *now*. Every computer component that fails was working fine before it failed. There's nothing particularly surprising about the idea that a five year old hard drive isn't working any more. Having said all that, I don't believe there is something wrong with the hardware per se, but I do believe the firmware is not capable for BTRFS. You're drawing an entirely unsupported conclusion from the evidence in front of you. btrfs doesn't demand something from your drive that other filesystems don't, it's just better at telling you that the storage isn't reliable. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
[389-users] Re: ERR - _entryrdn_insert_key - Same DN (dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=cesnet,dc=cz) is already in the ,entryrdn file with different ID 10458. Expected ID is 10459
Hi Jan, This is definitely an older version of the server, I would highly suggest to get onto the latest 1.4.x version that you can. 1.4.0 has not been maintained in a very long time, and is missing important fixes. As for this error, I have not seen this before. The first thing to try would have been to restart the server which promises to refresh the RUV entry & in-memory. If that does not help maybe try to reindex the entryrdn index once more to see if it helps. If that still does not help, then you can export the database (using the replication data option) and reimport it (this will not break replication or require any further action): dsconf -D "cn=Directory Manager" -w "$pswd" ldap://localhost backend export cesnet_cz --replication --ldif repl_export.ldif dsconf -D "cn=Directory Manager" -w "$pswd" ldap://localhost backend import cesnet_cz repl_export.ldif HTH, Mark On 1/13/21 11:22 AM, Jan Tomasek wrote: Hello, I've 389DS 1.4.0.21-1 on Debian/Buster in configuration with one master two consumers and several suffixes. After running dsconf -D "cn=Directory Manager" -w "$pswd" ldap://localhost backend index reindex cesnet_cz and completing indexing, err logfile on supplier server start show: ERR - _entryrdn_insert_key - Same DN (dn: nsuniqueid=---,dc=cesnet,dc=cz) is already in the ,entryrdn file with different ID 10458. Expected ID is 10459. Complete log of that indexing: [13/Jan/2021:16:43:10.896747048 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: aci [13/Jan/2021:16:43:10.900306795 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: cesnetemplid [13/Jan/2021:16:43:10.901040884 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: cn [13/Jan/2021:16:43:10.902098982 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: dc [13/Jan/2021:16:43:10.902728398 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: displayname [13/Jan/2021:16:43:10.903230082 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: entityidofidp [13/Jan/2021:16:43:10.903811474 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing entryrdn [13/Jan/2021:16:43:10.906245372 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: entrystatus [13/Jan/2021:16:43:10.906993328 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: entryusn [13/Jan/2021:16:43:10.907625883 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: givenName [13/Jan/2021:16:43:10.909503905 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: iphostnumber [13/Jan/2021:16:43:10.910181673 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: mail [13/Jan/2021:16:43:10.911199555 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: mailAlternateAddress [13/Jan/2021:16:43:10.911712896 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: mailHost [13/Jan/2021:16:43:10.912187870 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: member [13/Jan/2021:16:43:10.912657039 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: memberOf [13/Jan/2021:16:43:10.913311121 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nsCertSubjectDN [13/Jan/2021:16:43:10.913817035 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nscpEntryDN [13/Jan/2021:16:43:10.915613931 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nsds5ReplConflict [13/Jan/2021:16:43:10.916112906 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nsTombstoneCSN [13/Jan/2021:16:43:10.916587945 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nsuniqueid [13/Jan/2021:16:43:10.918419748 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: ntUniqueId [13/Jan/2021:16:43:10.918898277 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: ntUserDomainId [13/Jan/2021:16:43:10.919347819 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: numsubordinates [13/Jan/2021:16:43:10.919940562 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: objectclass [13/Jan/2021:16:43:10.921719909 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: owner [13/Jan/2021:16:43:10.922432531 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: parentid [13/Jan/2021:16:43:10.923072797 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: seeAlso [13/Jan/2021:16:43:10.923580070 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: sn [13/Jan/2021:16:43:10.924288238 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: sponsor [13/Jan/2021:16:43:10.924959286 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: taccmd
[389-users] ERR - _entryrdn_insert_key - Same DN (dn: nsuniqueid=ffffffff-ffffffff-ffffffff-ffffffff,dc=cesnet,dc=cz) is already in the ,entryrdn file with different ID 10458. Expected ID is 10459.
Hello, I've 389DS 1.4.0.21-1 on Debian/Buster in configuration with one master two consumers and several suffixes. After running dsconf -D "cn=Directory Manager" -w "$pswd" ldap://localhost backend index reindex cesnet_cz and completing indexing, err logfile on supplier server start show: ERR - _entryrdn_insert_key - Same DN (dn: nsuniqueid=---,dc=cesnet,dc=cz) is already in the ,entryrdn file with different ID 10458. Expected ID is 10459. Complete log of that indexing: [13/Jan/2021:16:43:10.896747048 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: aci [13/Jan/2021:16:43:10.900306795 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: cesnetemplid [13/Jan/2021:16:43:10.901040884 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: cn [13/Jan/2021:16:43:10.902098982 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: dc [13/Jan/2021:16:43:10.902728398 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: displayname [13/Jan/2021:16:43:10.903230082 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: entityidofidp [13/Jan/2021:16:43:10.903811474 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing entryrdn [13/Jan/2021:16:43:10.906245372 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: entrystatus [13/Jan/2021:16:43:10.906993328 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: entryusn [13/Jan/2021:16:43:10.907625883 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: givenName [13/Jan/2021:16:43:10.909503905 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: iphostnumber [13/Jan/2021:16:43:10.910181673 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: mail [13/Jan/2021:16:43:10.911199555 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: mailAlternateAddress [13/Jan/2021:16:43:10.911712896 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: mailHost [13/Jan/2021:16:43:10.912187870 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: member [13/Jan/2021:16:43:10.912657039 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: memberOf [13/Jan/2021:16:43:10.913311121 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nsCertSubjectDN [13/Jan/2021:16:43:10.913817035 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nscpEntryDN [13/Jan/2021:16:43:10.915613931 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nsds5ReplConflict [13/Jan/2021:16:43:10.916112906 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nsTombstoneCSN [13/Jan/2021:16:43:10.916587945 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: nsuniqueid [13/Jan/2021:16:43:10.918419748 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: ntUniqueId [13/Jan/2021:16:43:10.918898277 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: ntUserDomainId [13/Jan/2021:16:43:10.919347819 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: numsubordinates [13/Jan/2021:16:43:10.919940562 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: objectclass [13/Jan/2021:16:43:10.921719909 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: owner [13/Jan/2021:16:43:10.922432531 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: parentid [13/Jan/2021:16:43:10.923072797 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: seeAlso [13/Jan/2021:16:43:10.923580070 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: sn [13/Jan/2021:16:43:10.924288238 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: sponsor [13/Jan/2021:16:43:10.924959286 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: taccmd [13/Jan/2021:16:43:10.925596618 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: tacnaspointer [13/Jan/2021:16:43:10.926234600 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: tacprofilepointer [13/Jan/2021:16:43:10.926857374 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: tacuserpointer [13/Jan/2021:16:43:10.927401468 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: targetuniqueid [13/Jan/2021:16:43:10.927833486 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: telephoneNumber [13/Jan/2021:16:43:10.928480717 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: uid [13/Jan/2021:16:43:10.929047096 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexing attribute: uniquemember [13/Jan/2021:16:43:11.845074815 +0100] - INFO - ldbm_back_ldbm2index - cesnet_cz: Indexed 1000 entries (9%). [13/Jan/2021:16:43:12.658177768 +0100] - INFO -
Re: BTRFS partition corrupted after deleting files in /home
On Wed, 2021-01-13 at 14:06 +0530, Sreyan Chakravarty wrote: > I am fully open to the fact that there may be something wrong with my > HDD. But it is difficult to believe that, since this laptop is from > 2016 and I had been using Windows 10 on it for a long time and saw no > problems. Faults can suddenly appear. Or there could have been a long standing problem on a part of the drive that you weren't using until now. And you may not realise there's been drive faults with Windows. When I used to use Windows in the past, you kind of got used to its flakiness, and glossed over various quirks as Windows being Windows. And going back even further I never got any warnings from Windows that my drive was bad. Yet when I installed Linux on it, it immediately complained about faults. I sorted out the drive, and Windows general flakiness improved, too. -- uname -rsvp Linux 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 Boilerplate: All unexpected mail to my mailbox is automatically deleted. I will only get to see the messages that are posted to the mailing list. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Wed, 13 Jan 2021 at 05:41, Sreyan Chakravarty wrote: > On Tue, Jan 12, 2021 at 9:16 AM Chris Murphy > wrote: > > > > > > -x has more information that might be relevant including firmware > > revision and some additional logs for recent drive reported errors > > which usually are benign. But might be clues. > > > > These two attributes I'm not familiar with > > 187 Reported_Uncorrect 0x0032 100 096 000Old_age > > Always - 4294967301 > > 188 Command_Timeout 0x0032 100 100 000Old_age > > Always - 98785820672 > > > > But the value is well above threshold for both so I'm not worried about > it. > > > > > > Here is the output of: > > # smartctl -Ax /dev/sda > > https://pastebin.com/raw/GrgrQrSf > > I have no idea what it means. > You are not alone.Most people stop reading at the line: SMART overall-health self-assessment test result: PASSED Before retiring I worked in remote sensing, which is a data-intensive activity. HDD failures were a major issue. One sure way to kill a drive was to start a batch job that filled a disk and then kept hammering the drive over a long weekend when I was off somewhere without network access. I could usually get warranty replacements for failed drives by submitting the smartctrl reports. We use XFS starting on SGI IRIX and then on linux when it became available, with striped arrays for thruput with I/O bound processes. XFS was designed to avoid lengthy filesystem repair times, so getting a system back after a drive failure just meant waiting for the tape robot to find and restore the backup tapes. HDD's are mechanical so subject to wear. With heavy use they tend to die shortly after end-or-warranty.I started replacing drives at end-or-warranty which, along with measures to reduce runaway batch jobs, greatly reduced the number of failures. Your drive has been used for 1671 hours, and 1491 power-on cycles. Mechanical device wear is often highest at startup, so this is probably getting close to the design lifetime of a consumer laptop HDD. There are workloads (image processing, numerical modelling) where recovering the work done since the last backup just means restarting a batch job and is generally easier than trying to repair a filesystem with a bunch of partially written HDF5 files. Given the age of your HDD, I would replace it. If your laptop came with Windows, you should be able to install Windows 10 on a small partition in order to upgrade the BIOS and maybe run the drive vendor's diagnostics. You may want to revisit your choices of drive technology, filesystem, backup and recovery strategy, etc. with your use case in mind. > This is the problem with SMART tests, they are so esoteric that it is > difficult for a common user to make sense of it. > > Let me know what you think, if you see any glaring faults. > > You are to be commended for helping the btrfs developers investigate one of the rare situations that make filesystems such a hard problem. My experience indicates your HDD is involved, either by old age or some BIOS or drive firmware glitch, so your best way forward is to make sure your BIOS is current and replace the drive with one suited to your use case. -- George N. White III ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: btrfs: What to do with large growing files?
On Wed, 2021-01-13 at 19:21 +1030, Tim via users wrote: > On Tue, 2021-01-12 at 23:01 +, Patrick O'Callaghan wrote: > > but from what I understand of how blockchain works the database is > > append-only, as each new block includes a cryptographically strong > > hash of the previous one. That's what makes it a chain. > > An infinitely expanding file on a consumer computer sounds like a > recipe for disaster. AFAIK it doesn't grow very quickly, but yes. However IIUC this is only applicable to bitcoin mining, which probably isn't suitable for a consumer desktop in any case. poc ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: Calibre on Fedora 33
On Tue, 2021-01-12 at 17:37 -0800, Clifford Snow wrote: > Patrick O'Callaghan you said DeDRM worked for you. Did that include > epubs? I've only used it on AZW3 (Kindle) books, converting them to MOBI. poc ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Tue, Jan 12, 2021 at 9:16 AM Chris Murphy wrote: > > > -x has more information that might be relevant including firmware > revision and some additional logs for recent drive reported errors > which usually are benign. But might be clues. > > These two attributes I'm not familiar with > 187 Reported_Uncorrect 0x0032 100 096 000Old_age > Always - 4294967301 > 188 Command_Timeout 0x0032 100 100 000Old_age > Always - 98785820672 > > But the value is well above threshold for both so I'm not worried about it. > > Here is the output of: # smartctl -Ax /dev/sda https://pastebin.com/raw/GrgrQrSf I have no idea what it means. This is the problem with SMART tests, they are so esoteric that it is difficult for a common user to make sense of it. Let me know what you think, if you see any glaring faults. -- Regards, Sreyan Chakravarty ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Tue, Jan 12, 2021 at 11:05 PM Chris Murphy wrote: > > > Short version: Josef (Btrfs dev) and I agree there's probably > something wrong with the drive. The advice is to replace it, just > because trying to investigate further isn't worth your time, and also > necessarily puts your data at risk. > > Longer version: > > LVM thinp uses device-mapper as its backend to do all the work, and we > see checksum errors in the months old report. Where LVM thick has a > simpler on-disk format, so it's not as likely to discover such a > problem. And LUKS/dm-crypt is also using device-mapper as its backend > to do all work. So the two problems have two things in common: the > drive, and device-mapper. It's more probable there's an issue with the > drive, just because if there was a problem with device-mapper, we'd > have dozens of reports of it at the rate you're seeing this problem > once every couple of months (if that trend holds). > > Is it possible LVM+ext4 on this same drive is more reliable? I think > that's specious. The problem can be masked due to much less stringent > integrity guarantees, i.e. there are no data checksums. Since the data > is overwhelmingly the largest target for corruption, just being a much > bigger volume compared to file system metadata. All things being > equal, there's a greater chance the problem affects data. On the other > hand, if it adversely impacts metadata, it could be true that e2fsck > has a better chance of fixing the damage than btrfsck right now. Of > course no fsck fixes your data. > > So if you keep using the drive, you're in a catch-22. Btrfs is more > sensitive because everything is checksummed, so the good news is > you'll be informed about it fairly quickly, the bad news is that it's > harder to repair in this case. If you revert to LVM+ext4 the automatic > fsck at startup might fix up these problems as they occur, but it's > possible undetected data corruption shows up later or replicates into > your backups. > > Regardless of what you decide to do, definitely keep frequent backups > of anything important. > Ok first I don't mean to imply that I don't believe you or Josef when you say there is something wrong with my HDD or that you are wrong. But I have a lot of questions that I want to discuss : 1) Is it possible there is nothing wrong with my drive, but there is something with my BIOS/HDD Firmware ? May be my firmware is not capable of BTRFS's stringent write requirements ? I say this because I have used Windows with NTFS on this machine, I have used Ubuntu with EXT4, and Fedora with thick-LVM with EXT4. None of these configurations gave me any such problems. 2) Since there is a high likelihood that my filesystem is not completely fixed, then when I take a backup using partclone, dd or clonezilla won't those errors be carried over ? Even if I buy a new drive and restore the backup, I still might get crashes. 3) This is a weird question but can you recommend me a HDD that I can buy which can handle BTRFS ? Or even which features I might look for while buying (not a SSD but a HDD) 4) My manufacturer HP, does not make firmware updates for Linux, only for Windows. So is there a way to update the firmware(if available) without being on Windows ? Any ideas? Would a Windows PXE help ? 5) When you say "checksum errors in the month's old report" - which report are you referring to ? The thin-LVM crash or the smartctl crash ? -- Regards, Sreyan Chakravarty ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: btrfs: What to do with large growing files?
On Tue, 2021-01-12 at 23:01 +, Patrick O'Callaghan wrote: > but from what I understand of how blockchain works the database is > append-only, as each new block includes a cryptographically strong > hash of the previous one. That's what makes it a chain. An infinitely expanding file on a consumer computer sounds like a recipe for disaster. -- uname -rsvp Linux 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 Boilerplate: All unexpected mail to my mailbox is automatically deleted. I will only get to see the messages that are posted to the mailing list. ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Tue, Jan 12, 2021 at 8:52 AM Tom Horsley wrote: > > Not to worry! I read that bcachefs is finally making it into > the linux kernel, so real soon now it will be the new shiny object > and btrfs will be obsolete :-). > ___ Let's hope it does a better job of preventing fragmentation on larger files. Large files are a pain in BTRFS to manage. -- Regards, Sreyan Chakravarty ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Tue, Jan 12, 2021 at 9:22 AM Gordon Messmer wrote: > > https://listi.jpberlin.de/pipermail/smartmontools-support/2020-November/000559.html > > Your message indicated a sequence of errors that look like write > failures, which sounds a lot like the problem that Chris described with > your btrfs, "multiple writes from one commit are missing" > > With all respect due to Carlos E. R., given that you were seeing read > and write errors then, and you appear to be seeing write errors still, > perhaps the conclusion that your hardware is OK was premature. You > should remain open the possibility that there is something wrong with > one piece of hardware or another, particularly as you continue to gather > new evidence. I am fully open to the fact that there may be something wrong with my HDD. But it is difficult to believe that, since this laptop is from 2016 and I had been using Windows 10 on it for a long time and saw no problems. Then I used Ubuntu with EXT4, then migrated to Fedora with Old-LVM with EXT4 and had been using it without any problems. I mean I restored LVM snapshots as old as 3 months, it took time but it never failed on me. The reason I switched to thin-pools and BTRFS is because older LVM snapshots are activated and boot times and depending on the size of the snapshot slows down boot exponentially. Having said all that, I don't believe there is something wrong with the hardware per se, but I do believe the firmware is not capable for BTRFS. I will need further investigation for this, lets see what I can find. -- Regards, Sreyan Chakravarty ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: Is it possible to create an email-only user in fedora linux?
> Am 13.01.2021 um 00:46 schrieb Bill Oliver : > > On Mon, 2021-01-11 at 21:47 +0100, Peter Boy wrote: >>> Am 11.01.2021 um 19:24 schrieb Bill Oliver : >>> >> >> I hope I do not misunderstand your requirements. You need not set up >> a system user to allow mail usage. You just create all your mail >> users in dovecot (/etc/dovecot/users) using doveadm and configure >> postfix to authenticate against dovecot (virtual mailboxes in postfix >> speech). The mail ist stored at e.g. /home/vmail// and >> everything is managed by dovecot (vmail being the dovecot process >> owner). >> >> [snip] > > Thanks. I had no idea. Dovecot seems a bit opaque to me. I'll read up > on it. I found the following quite helpful: Thomas Leistner: https://thomas-leister.de/mailserver-debian-stretch/ (Debian Stretch 2019) Thomas Leistner: https://thomas-leister.de/mailserver-unter-ubuntu-16.04/ (older Version) Nausch (CentOS 7): https://dokuwiki.nausch.org/doku.php/centos:mail_c7:start Ivan Tomica (CentOS 8): https://www.tomica.net/blog/2019/10/self-hosted-email-server (2019) If you are a bit fluent in German, I could send you my instructions for our local installations. I plan to contribute this as part of the Fedora Server documentation / howtos. But that will take some time. Best Peter ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
Re: BTRFS partition corrupted after deleting files in /home
On Tue, Jan 12, 2021 at 9:04 AM Chris Murphy wrote: > > > We should talk about a backup plan in another thread :D In my own > case, I never need them, but I have 1/2 dozen completely independent > backups, because I hate my data and I'm constantly trying to destroy > it. (This is turning into a really bad joke but hopefully it's weirdly > amusing. If you like your data you *definitely* should be backing up! > Doesn't matter what the file system is, or if it's on raid, or "safe" > in the cloud, etc.) > Now that the filesystem is working should I take a partclone backup or a dd backup ? Partclone is small but since as you say the filesystem is not completely fixed, my backup may be messed up. While a dd backup keeps things the way they are. > Because of the --init-csum-tree failure i'm a bit concerned that it's > possible the --repair step didn't completely repair things. So... just > keep important things backed up. Top priority. If you told me you hate > your data then I wouldn't worry about it. > I too believe that the repair is not complete. I say this because when I started my system my swap file could not be activated and System Logging Service failed to start. I have disabled my swap file and will recreate it and the System Logging Service is now working after a reboot. > Can you tell me what you get for: > > free -m > btrfs fi us / > > It'll give an idea what the memory requirement is for the > init-csum-tree. It really should not be a lot, it's less complicated > than the repair by a lot. Glad that worked though, phew! > # free -m totalusedfree shared buff/cache available Mem: 785210455079 39317276163 Swap: 3925 03925 # sudo btrfs fi us / Overall: Device size: 930.00GiB Device allocated:115.02GiB Device unallocated: 814.97GiB Device missing: 0.00B Used: 96.11GiB Free (estimated):831.30GiB (min: 423.81GiB) Data ratio: 1.00 Metadata ratio: 2.00 Global reserve: 224.05MiB (used: 0.00B) Multiple profiles: no Data,single: Size:111.01GiB, Used:94.68GiB (85.29%) /dev/mapper/luks-1136a62b-955b-4391-b9a4-b48ab11a862d 111.01GiB Metadata,DUP: Size:2.00GiB, Used:733.83MiB (35.83%) /dev/mapper/luks-1136a62b-955b-4391-b9a4-b48ab11a862d 4.00GiB System,DUP: Size:8.00MiB, Used:16.00KiB (0.20%) /dev/mapper/luks-1136a62b-955b-4391-b9a4-b48ab11a862d 16.00MiB Unallocated: /dev/mapper/luks-1136a62b-955b-4391-b9a4-b48ab11a862d 814.97GiB -- Regards, Sreyan Chakravarty ___ users mailing list -- users@lists.fedoraproject.org To unsubscribe send an email to users-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/users@lists.fedoraproject.org
