Re: [gentoo-user] Getting output of a program running in background after a crash
Jude DaShiell wrote: > Linux is being targeted by ransomware and other forms of malware so it may > be worthwhile to run ferensics on your backup and find what ferensics have > to tell you. After check and see if any of what you found out were false > positives. > > > -- Jude "There are four boxes to be used in > defense of liberty: soap, ballot, jury, and ammo. Please use in that > order." Ed Howdershelt 1940. > > It only does this when I'm copying files over. Right now I'm copying about 26TBs of data over ethernet and it is taking a while. Once I stop it or it finishes the copy, the CPU goes to about nothing, unless I'm doing something else. So it has something to do with the copy process. Since the LVs are encrypted, I think a good bit of it is related to that. I read somewhere that new CPUs have a instruction set that makes encryption really efficient. Mine doesn't have that, that I'm aware of anyway. It's yet another reason for me building a newer rig. One thing I've noticed, as my drive space increases, this seems to use more CPU. One thing I hope to do, have a system that does nothing but torrent stuff. Right now, I have a lot of stuff running on this rig. For its age, it's a good size load at times. I'd like to put some of that load on another machine. I'm looking at that AMD Ryzen 9 5900X which I've been told will run circles around this old FX-8350. I also need more memory too. I sometimes have to close Firefox to free up some memory especially during some compiles. I'm working on it. Takes time tho. Dale :-) :-)
Re: [gentoo-user] Getting output of a program running in background after a crash
Linux is being targeted by ransomware and other forms of malware so it may be worthwhile to run ferensics on your backup and find what ferensics have to tell you. After check and see if any of what you found out were false positives. -- Jude "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." Ed Howdershelt 1940. On Wed, 11 Oct 2023, Dale wrote: > Frank Steinmetzger wrote: > > Am Sun, Oct 08, 2023 at 07:44:06PM -0500 schrieb Dale: > > > >> Just as a update. The file system I was trying to do a file system > >> check on was my large one, about 40TBs worth. While running the file > >> system check, it started using HUGE amounts of memory. It used almost > >> all my 32GBs and most of swap as well. It couldn't finish due to not > >> enough memory, it literally crashed itself. So, I don't know if this is > >> because of some huge problem or what but if this is expected behavior, > >> don't try to do a file system check on devices that large unless you > >> have a LOT of memory. > > Or use a different filesystem. O:-) > > I'm using ext4 which is said to be one of the most reliable and widely > used file systems. I do wonder tho, am I creating file systems that may > be to large or that it just has trouble with??? I doubt that but I'm up > to about 40TBs now. I just can't figure out a way to split that data > up, yet. > > > >> I ended up recreating the LVM devices from scratch and redoing the > >> encryption as well. I have backups tho. This all started when using > >> pvmove to replace a hard drive with a larger drive. I guess pvmove > >> isn't always safe. > > I think that may be a far-fetched conclusion. If it weren’t safe, it > > wouldn’t be in the software – or at least not advertised as safe. > > > > Well, something went sideways. Honestly, I think it might not be pvmove > but something happened with the file system itself. After all, LVM > wasn't complaining at all and everything showed the move completed with > no errors. I guess it is possible pvmove had a problem but given it was > the file system that complained so loudly, I'm leaning to it having a > issue. > > > >> P. S. I currently have my backup system on my old Gigabyte 770T mobo > >> and friends. It is still a bit slower than copying when no encryption > >> is used so I guess encryption does slow things down a bit. That said, > >> the CPU does hang around 50% most of the time. htop doesn't show what > >> is using that so it must be IO or encryption. > > You can add more widgets (“meters”) to htop, one of them shows disk > > throughput. But there is none for I/O wait. One tool that does show that is > > glances. And also dstat which I mentioned a few days ago. Not only can dstat > > tell you the total percentage, but also which process is the most expensive > > one. > > > > I set up bash aliases for different use cases of dstat: > > alias ,d='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? > > /dev/mmcblk? 2>/dev/null | tr "\n" ,) --net --mem --swap' > > alias ,dd='dstat --time --cpu --disk --disk-util -D $(ls /dev/sd? > > /dev/nvme?n? /dev/mmcblk? 2>/dev/null | tr "\n" ,) --mem-adv' > > alias ,dm='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? > > /dev/mmcblk? 2>/dev/null | tr "\n" ,) --net --mem-adv --swap' > > alias ,dt='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? > > /dev/mmcblk? 2>/dev/null | tr "\n" ,) --net --mem --swap --top-cpu > > --top-bio --top-io --top-mem' > > > > Because I attach external storage once in a while, I use a dynamic list of > > devices to watch that is passed to the -D argument. If I don’t use -D, dstat > > will only show a total for all drives. > > > > The first is a simple overview (d = dstat). > > > > The second is the same but only for disk statistics (dd = dstat disks). I > > use it mostly on my NAS (five SATA drives in total, which creates a very > > wide table). > > > > The third shows more memory details like dirty cache (dm = dstat memory), > > which is interesting when copying large files. > > > > And the last one shows the top “pigs”, i.e. expensive processes in terms of > > CPU, IO and memory (dt = dstat top). > > > >> Or something kernel > >> related that htop doesn't show. No idea. > > Perhaps my tool tips give you ideas. :) > > > > -- Grüße | Greetings | Salut | Qapla’ Please do not share anything > > from, with or about me on any social network. What is the difference > > between two flutes? – A semitone. > > > Dang, I have a lot of drives here to add to all that. Bad thing is, > every time I reboot, all but two I think tend to move around, even tho I > haven't moved anything. This is why I use either labels or UUIDs by the > way. Once ages ago, I saw a way to make commands/scripts see all drives > on a system with some sort of inclusive trick. I think it used brackets > but not sure. I can't find that trick anymore. I should have saved > that thing. > > I used some command,
Re: [gentoo-user] world updates blocked by Qt
On Wed, Oct 11, 2023 at 4:49 PM Michael Cook wrote: > On 10/11/23 09:43, Alan McKinnon wrote: > > Hi all, > > Today a sync and emerge world produces a huge list of blockers. > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and > being blocked. > All the visible blockers are Qt itself so --verbose-conflicts is needed. > > That will be an exceptionally long list and today I feel really lazy and > unwilling :-D > Has anyone seen what the actual blockers are? > I spotted okular, I suspect there are more. > > > -- > Alan McKinnon > alan dot mckinnon at gmail dot com > > I just --backtrack=100 and walked away, seemed to have figured something > out for my system and updated normally. > This is the one that solved it. Been away too long, forgot all about backtrack -- Alan McKinnon alan dot mckinnon at gmail dot com
Re: [gentoo-user] Getting output of a program running in background after a crash
Frank Steinmetzger wrote: > Am Sun, Oct 08, 2023 at 07:44:06PM -0500 schrieb Dale: > >> Just as a update. The file system I was trying to do a file system >> check on was my large one, about 40TBs worth. While running the file >> system check, it started using HUGE amounts of memory. It used almost >> all my 32GBs and most of swap as well. It couldn't finish due to not >> enough memory, it literally crashed itself. So, I don't know if this is >> because of some huge problem or what but if this is expected behavior, >> don't try to do a file system check on devices that large unless you >> have a LOT of memory. > Or use a different filesystem. O:-) I'm using ext4 which is said to be one of the most reliable and widely used file systems. I do wonder tho, am I creating file systems that may be to large or that it just has trouble with??? I doubt that but I'm up to about 40TBs now. I just can't figure out a way to split that data up, yet. >> I ended up recreating the LVM devices from scratch and redoing the >> encryption as well. I have backups tho. This all started when using >> pvmove to replace a hard drive with a larger drive. I guess pvmove >> isn't always safe. > I think that may be a far-fetched conclusion. If it weren’t safe, it > wouldn’t be in the software – or at least not advertised as safe. > Well, something went sideways. Honestly, I think it might not be pvmove but something happened with the file system itself. After all, LVM wasn't complaining at all and everything showed the move completed with no errors. I guess it is possible pvmove had a problem but given it was the file system that complained so loudly, I'm leaning to it having a issue. >> P. S. I currently have my backup system on my old Gigabyte 770T mobo >> and friends. It is still a bit slower than copying when no encryption >> is used so I guess encryption does slow things down a bit. That said, >> the CPU does hang around 50% most of the time. htop doesn't show what >> is using that so it must be IO or encryption. > You can add more widgets (“meters”) to htop, one of them shows disk > throughput. But there is none for I/O wait. One tool that does show that is > glances. And also dstat which I mentioned a few days ago. Not only can dstat > tell you the total percentage, but also which process is the most expensive > one. > > I set up bash aliases for different use cases of dstat: > alias ,d='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? > /dev/mmcblk? 2>/dev/null | tr "\n" ,) --net --mem --swap' > alias ,dd='dstat --time --cpu --disk --disk-util -D $(ls /dev/sd? > /dev/nvme?n? /dev/mmcblk? 2>/dev/null | tr "\n" ,) --mem-adv' > alias ,dm='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? > /dev/mmcblk? 2>/dev/null | tr "\n" ,) --net --mem-adv --swap' > alias ,dt='dstat --time --cpu --disk -D $(ls /dev/sd? /dev/nvme?n? > /dev/mmcblk? 2>/dev/null | tr "\n" ,) --net --mem --swap --top-cpu --top-bio > --top-io --top-mem' > > Because I attach external storage once in a while, I use a dynamic list of > devices to watch that is passed to the -D argument. If I don’t use -D, dstat > will only show a total for all drives. > > The first is a simple overview (d = dstat). > > The second is the same but only for disk statistics (dd = dstat disks). I > use it mostly on my NAS (five SATA drives in total, which creates a very > wide table). > > The third shows more memory details like dirty cache (dm = dstat memory), > which is interesting when copying large files. > > And the last one shows the top “pigs”, i.e. expensive processes in terms of > CPU, IO and memory (dt = dstat top). > >> Or something kernel >> related that htop doesn't show. No idea. > Perhaps my tool tips give you ideas. :) > > -- Grüße | Greetings | Salut | Qapla’ Please do not share anything > from, with or about me on any social network. What is the difference > between two flutes? – A semitone. Dang, I have a lot of drives here to add to all that. Bad thing is, every time I reboot, all but two I think tend to move around, even tho I haven't moved anything. This is why I use either labels or UUIDs by the way. Once ages ago, I saw a way to make commands/scripts see all drives on a system with some sort of inclusive trick. I think it used brackets but not sure. I can't find that trick anymore. I should have saved that thing. I used some command, can't recall which it was, and I think it is the kernel itself using so much CPU time. Given when it does it, I think it is either processing the encryption or working to send the data to the disks, or both. I'd suspect both but I dunno. Anyway, I'm restoring from a fresh LVM rebuild now. No way to test anything to see what the problem was now. Dale :-) :-)
Re: [gentoo-user] world updates blocked by Qt
On Wed, 11 Oct 2023 16:04:08 +0200, Alan McKinnon wrote: > > On Wed, 11 Oct 2023 15:43:46 +0200, Alan McKinnon wrote: > > > > > Today a sync and emerge world produces a huge list of blockers. > > > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree > > > and being blocked. > > > All the visible blockers are Qt itself so --verbose-conflicts is > > > needed. > > > > No such issues here, on ~amd64. > > > > Hmmm. When did you last sync? Forgot to mention I am ~amd too A few hours earliers, but I synced again before replying. As posted elsewhere, try cranking up --backtrack. I have it set to 20, which avoids almost all issues without hammering dependency calculation times. -- Neil Bothwick I am Zaphod of Borg. Now, where's the coolest place to be assimilated... pgpG0cifn5AWL.pgp Description: OpenPGP digital signature
Re: [gentoo-user] world updates blocked by Qt
Philip Webb wrote: > 231011 Alan McKinnon wrote: >> Today a sync and emerge world produces a huge list of blockers. >> qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and >> being blocked. >> All the visible blockers are Qt itself so --verbose-conflicts is needed. > My experience for some time has been that Qt pkgs block one another, > st the only way out is to unmerge them all, then remerge them all. > If anyone knows a better method, please let us know > I haven't ran into anything like that in ages and I have all KDE and qt packages as unstable. I wonder why yours does that and mine doesn't? I'm the one who usually runs into weird problems around here. What did I miss? LOL Dale :-) :-) P. S. I'm having that problem again with missing replies. I'm still missing one of Neil's from several hours ago. If someone replies and I don't, I may not have it yet. :-/
Re: [gentoo-user] world updates blocked by Qt
Michael Cook wrote: > On 10/11/23 09:43, Alan McKinnon wrote: >> Hi all, >> >> Today a sync and emerge world produces a huge list of blockers. >> qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree >> and being blocked. >> All the visible blockers are Qt itself so --verbose-conflicts is needed. >> >> That will be an exceptionally long list and today I feel really lazy >> and unwilling :-D >> Has anyone seen what the actual blockers are? >> I spotted okular, I suspect there are more. >> >> >> -- >> Alan McKinnon >> alan dot mckinnon at gmail dot com > > I just --backtrack=100 and walked away, seemed to have figured > something out for my system and updated normally. > I have backtrack set to that by default. I did that a long time ago because sometimes it doesn't work out a solution otherwise. It takes longer to build the upgrade list sometimes but it figures out the solution which tends to save time in the long run. You may want to add that to emerge defaults in make.conf. Oh, don't forget to add -1 to so that the world file stays clean. ;-) Dale :-) :-)
RE: [gentoo-user] world updates blocked by Qt
> -Original Message- > From: Philip Webb > Sent: Wednesday, October 11, 2023 9:44 AM > To: Gentoo User > Subject: [gentoo-user] world updates blocked by Qt > > CAUTION: This is an EXTERNAL email. Do not click links or open attachments > unless you recognize the sender and know the content is safe. > > 231011 Alan McKinnon wrote: > > Today a sync and emerge world produces a huge list of blockers. > > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree > > and being blocked. > > All the visible blockers are Qt itself so --verbose-conflicts is needed. > > My experience for some time has been that Qt pkgs block one another, st the > only way out is to unmerge them all, then remerge them all. > If anyone knows a better method, please let us know > If you don't want to do that, then adding all the currently installed qt packages to the merge list explicitly works as well. If you're good with eix's search and output settings it can be done without too much agony. LMP
[gentoo-user] world updates blocked by Qt
231011 Alan McKinnon wrote: > Today a sync and emerge world produces a huge list of blockers. > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and > being blocked. > All the visible blockers are Qt itself so --verbose-conflicts is needed. My experience for some time has been that Qt pkgs block one another, st the only way out is to unmerge them all, then remerge them all. If anyone knows a better method, please let us know -- ,, SUPPORT ___//___, Philip Webb ELECTRIC /] [] [] [] [] []| Cities Centre, University of Toronto TRANSIT`-O--O---' purslowatcadotinterdotnet
Re: [gentoo-user] world updates blocked by Qt
On 10/11/23 09:43, Alan McKinnon wrote: Hi all, Today a sync and emerge world produces a huge list of blockers. qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and being blocked. All the visible blockers are Qt itself so --verbose-conflicts is needed. That will be an exceptionally long list and today I feel really lazy and unwilling :-D Has anyone seen what the actual blockers are? I spotted okular, I suspect there are more. -- Alan McKinnon alan dot mckinnon at gmail dot com I just --backtrack=100 and walked away, seemed to have figured something out for my system and updated normally.
Re: [gentoo-user] world updates blocked by Qt
On 10/11/23 10:14, Alan McKinnon wrote: Strange, eix says no such package as opensoundmeter and libreoffice-bin is not installed here media-sound/opensoundmeter is in ::guru. Those are just the blockers I had, definitely more to it than that! -- cara
Re: [gentoo-user] world updates blocked by Qt
On Wed, Oct 11, 2023 at 4:14 PM Philip Webb wrote: > 231011 Alan McKinnon wrote: > > Today a sync and emerge world produces a huge list of blockers. > > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and > > being blocked. > > All the visible blockers are Qt itself so --verbose-conflicts is needed. > > My experience for some time has been that Qt pkgs block one another, > st the only way out is to unmerge them all, then remerge them all. > If anyone knows a better method, please let us know. > My experience is 4 years out of date, I don't recall ever having to do that. Alan -- Alan McKinnon alan dot mckinnon at gmail dot com
Re: [gentoo-user] world updates blocked by Qt
On Wed, Oct 11, 2023 at 4:07 PM Cara Salter wrote: > On 10/11/23 09:43, Alan McKinnon wrote: > > Hi all, > > > > Today a sync and emerge world produces a huge list of blockers. > > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and > > being blocked. > > All the visible blockers are Qt itself so --verbose-conflicts is needed. > I had a similar issue -- I resolved it by uninstalling opensoundmeter > and libreoffice-bin temporarily. > > Today's sync/merge is going smoothly so far. > Strange, eix says no such package as opensoundmeter and libreoffice-bin is not installed here -- Alan McKinnon alan dot mckinnon at gmail dot com
Re: [gentoo-user] world updates blocked by Qt
231011 Alan McKinnon wrote: > Today a sync and emerge world produces a huge list of blockers. > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and > being blocked. > All the visible blockers are Qt itself so --verbose-conflicts is needed. My experience for some time has been that Qt pkgs block one another, st the only way out is to unmerge them all, then remerge them all. If anyone knows a better method, please let us know. -- ,, SUPPORT ___//___, Philip Webb ELECTRIC /] [] [] [] [] []| Cities Centre, University of Toronto TRANSIT`-O--O---' purslowatcadotinterdotnet
Re: [gentoo-user] world updates blocked by Qt
On 10/11/23 09:43, Alan McKinnon wrote: Hi all, Today a sync and emerge world produces a huge list of blockers. qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and being blocked. All the visible blockers are Qt itself so --verbose-conflicts is needed. I had a similar issue -- I resolved it by uninstalling opensoundmeter and libreoffice-bin temporarily. Today's sync/merge is going smoothly so far. -- cara
Re: [gentoo-user] world updates blocked by Qt
On Wed, Oct 11, 2023 at 3:59 PM Neil Bothwick wrote: > On Wed, 11 Oct 2023 15:43:46 +0200, Alan McKinnon wrote: > > > Today a sync and emerge world produces a huge list of blockers. > > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and > > being blocked. > > All the visible blockers are Qt itself so --verbose-conflicts is needed. > > No such issues here, on ~amd64. > Hmmm. When did you last sync? Forgot to mention I am ~amd too > > > That will be an exceptionally long list and today I feel really lazy and > > unwilling :-D > > Has anyone seen what the actual blockers are? > > I spotted okular, I suspect there are more. > > Have you tried syncing again, just in case you caught it partway through > a stack of Qt packages being updated? > Yes, synced twice today, 2 hours apart. No change -- Alan McKinnon alan dot mckinnon at gmail dot com
Re: [gentoo-user] world updates blocked by Qt
On Wed, 11 Oct 2023 15:43:46 +0200, Alan McKinnon wrote: > Today a sync and emerge world produces a huge list of blockers. > qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and > being blocked. > All the visible blockers are Qt itself so --verbose-conflicts is needed. No such issues here, on ~amd64. > That will be an exceptionally long list and today I feel really lazy and > unwilling :-D > Has anyone seen what the actual blockers are? > I spotted okular, I suspect there are more. Have you tried syncing again, just in case you caught it partway through a stack of Qt packages being updated? -- Neil Bothwick If at first you don't succeed, skydiving is not for you. pgpeAsjv07peE.pgp Description: OpenPGP digital signature
[gentoo-user] world updates blocked by Qt
Hi all, Today a sync and emerge world produces a huge list of blockers. qt 5.15.10 is currently installed and qt 5.15.11 is new in the tree and being blocked. All the visible blockers are Qt itself so --verbose-conflicts is needed. That will be an exceptionally long list and today I feel really lazy and unwilling :-D Has anyone seen what the actual blockers are? I spotted okular, I suspect there are more. -- Alan McKinnon alan dot mckinnon at gmail dot com
