Re: [gentoo-user] Testing a used hard drive to make SURE it is good.
I just tested a couple of 4TB drives, the hgst 7k400 series (7200 rpm), it takes about a day to fill one of those, copying from another one of those. I prefer to use "real" data, rather than a test pattern. My procedure is to first run the short SMART test, format the drive (which also takes awhile..) and fill it with data. I like to copy to the test drive from 2 drives at the same time, producing fragmentation, and then copy what's on the test drive already into a new folder on the test drive and repeat, it makes it a brutal mechanical test. After that I fsck it, power cycle the computer/drive, run fsck again, and then run the drives long SMART test, which does take a looong time, and then run badblocks which should find nothing. I've had a drive that tested fine but would corrupt data (I don't know if the drive electronics started doing random writes or what, this wasn't an '80's drive), often within minutes, which is part of why I power cycle, and why I use fsck after putting data on the drive. I of course I then reformat the drive. It does bloody take days, plan to not be watching it. I've done this on my main machine as it doesn't hurt anything else. These are Refurbished drives in this case, going into a raid 6 to try and end bitrot, which I"ve seen too much of using similiar drives, it tends to slowly delete scenes from video when this happens without any indication (other than not showing things you know are in the video from watching before). It's a major pain, but so is watching stuff get corrupted! Note that a failing power supply can make a drive look like it's failed, seen this a couple of months ago. As soon as I put the drive in question on an external supply it magically worked fine. Did this right after another drive appeared to fail the same way (different brand). Apparently one drive was more sensitive to bad power then the other, both drives are in fact fine! I've also seen ram repeatedly test bad with a failing power supply. Not really surprising that a failing power supply can make things look bad. What was unexpected was the subtlety of the failure over time. Lesson being to always test drives that just failed on another machine, or with external supply and usb adapter after the machine boots to an OS (also great drives that have failed so that they delay/block even the BIOS from coming up, which failing drives can do). Obviously checking the power supply is always a good idea, though substituting another supply works. Power supply failures are not always apparent with a voltmeter as they can be noisy and your' meter won't tell you that generally. Also of course, ALL of this can be temperature sensitive and hence intermittent, I've done repair work, intermittent failures can take you all over trying to find them, random correlation abounds and misleads! So if the drive does appear to fail, it may or may not be the drive. I mention this as it's really annoying to find the real issue after replacing parts that weren't bad and possibly throwing out perfectly good hardware. I did get a refurbished drive that was invisible to the bios, and wouldn't work externally on a different supply either. Can't be sure until you try it on a different machine or external. Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On Monday, June 15, 2020 10:07 AM, Dale wrote: > Howdy, > > I finally bought a 8TB drive. It is used but they claim only a short > duration. Still, I want to test it to be sure it is in grade A shape before > putting a lot of data on it and depending on it. I am familiar with some > tools already. I know about SMART but it is not always 100%. It seems to > catch most problems but not all. I'm familiar with dd and writing all zeores > or random to it to see if it can in fact write to all the parts of the drive > but it is slow. It can take a long time to write and fill up a 8TB drive. > Days maybe?? I googled and found a new tool but not sure how accurate it is > since I've never used it before. The command is badblocks. It is installed on > my system so I'm just curious as to what it will catch that others won't. Is > it fast or slow like dd? > > I plan to run the SMART test anyway. It'll take several hours but I'd like to > run some other test to catch errors that SMART may miss. If there is such a > tool that does that. If you bought a used drive, what would you run other > than the long version of SMART and its test? Would you spend the time to dd > the whole drive? Would badblocks be a better tool? Is there another better > tool for this? > > While I'm at it, when running dd, I have zero and random in /dev. Where does > a person obtain a one? In other words, I can write all zeros, I can write all > random but I can't write all ones since it isn't in /dev. Does that even > exist? Can I create it myself somehow? Can I download it or install it > somehow? I been curious about
Re: [gentoo-user] new genkernel problem
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Thursday, June 11, 2020 8:18 AM, antlists wrote: > On 06/06/2020 23:11, Neil Bothwick wrote: > > > You don't boot from an encrypted drive (yet) or use unusual hardware, > > that's what I meant by a plain system. Dracut handles booting from a a > > btrfs root on a LUKS encrypted block device here with no fancy > > configuration. It really is impressive the way it figures so much out for > > itself. > > All I need is for it to figure out dm-integrity, and it'll boot my setup > fine ... hard-drive -> dm-integrity -> md-raid -> lvm -> filesystem > > I'm trying to get round the fact that a damaged disk will mess up raid, > so I want to add just that little bit more robustness :-) > > Cheers, > Wol Are you just using striping? A failed drive would be bad then, X times (where X is the number of drives) worse than a normal single drive failure. It's important to remember that drives fail completely and suddenly sometimes in any case. I'm building a file server for my house and plan to use raid6 to mitigate that issue, It also allows me to comfortably use refurbished drives (6 of the 4 tb htsg drives, $60 ea. on ebay). I am keeping a spare drive just in case, and I used multiple sources so the drives will hopefully be less likely to fail in a group over a short time. It'll still be very fast and near the limit of my Gb/s network.
Re: [gentoo-user] new genkernel problem
hey, when you come over bring your' sam's club card, so I can get a pallet of coke. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Thursday, June 11, 2020 8:18 AM, antlists wrote: > On 06/06/2020 23:11, Neil Bothwick wrote: > > > You don't boot from an encrypted drive (yet) or use unusual hardware, > > that's what I meant by a plain system. Dracut handles booting from a a > > btrfs root on a LUKS encrypted block device here with no fancy > > configuration. It really is impressive the way it figures so much out for > > itself. > > All I need is for it to figure out dm-integrity, and it'll boot my setup > fine ... hard-drive -> dm-integrity -> md-raid -> lvm -> filesystem > > I'm trying to get round the fact that a damaged disk will mess up raid, > so I want to add just that little bit more robustness :-) > > Cheers, > Wol
Re: [gentoo-user] no ebuilds for telegram
Just talked to doc. If you are willing to risk it come on over and lets shop! Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Thursday, May 14, 2020 3:08 PM, n952162 wrote: > On 05/14/20 23:03, Matt Connell (Gmail) wrote: > > > On 2020-05-14 15:29, n952162 wrote: > > > > > $ lf /usr/portage/net-im/telegram-desktop > > > Manifest telegram-desktop-2.1.1.ebuild > > > files/ telegram-desktop-2.1.2.ebuild > > > metadata.xml telegram-desktop-2.1.3.ebuild > > > telegram-desktop-2.0.1-r1.ebuild telegram-desktop-2.1.4.ebuild > > > telegram-desktop-2.1.0.ebuild telegram-desktop-2.1.6.ebuild > > > > Hmm. Seems you have the ebuild for the latest (as of today) version, > > as I do, but when you try to use the ebuild, emerge isn't finding it. > > Do you have an alternative PORTDIR defined in /etc/portage/make.conf > > or anywhere else? The default is /usr/portage, according to both of > > my systems. > > $ grep PORT /etc/portage/make.conf > PORTDIR="/usr/portage"
Re: [gentoo-user] a day of PAIN.
I'm using a nearly 10 year old server, bought specifically so I can compile faster. None of my machines is newer than that. None of my machines support UEFI other than an older imac. The server, which was inexpensive ($500) only has 48 cores, 64 as soon as I update the processors (2 generations newer and more cores!). I love server grade hardware! Servers lose value quickly, they are a great bargain if you don't mind the sound of a wind tunnel or have another room to put it in. This machine was probably $20-30K new, no slouch, it was previously used for high frequency trading. Sadly the seller does seem to have emptied their' warehouse. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Thursday, May 14, 2020 11:50 AM, Andrew Udvare wrote: > > On 2020-05-12, at 18:45, Alan Grimes alonz...@verizon.net wrote: > > Why is this not a forced-on setting for any machine with UEFI enabled? I > > can't imagine that this would be unacceptable for more than 0.001% of > > the install base. > > Because not every user has UEFI and many just use the BIOS compatibility > layer (CSM) if they don't care about UEFI and Secure Boot. The CSM is going > to be there for a very long time. > > If you don't need UEFI features, then there's nothing really wrong with using > CSM. I am using UEFI because I like having a signed kernel and signed boot > loader (systemd-boot). > > -- > > Andrew
Re: [gentoo-user] /proc/meminfo vs lshw -c memory
If the bios sees it, an lshw sees it it's not the ram, I seriously doubt new ram would help. I've seen a failing power supply make ram look flaky before, possibly the drive activity during booting is loading down the power supply enough to confuse your' system. In any case, you should run memtest, boot into it (obviously try to install if necessary). Also, how dusty is your' cpu? A hot cpu can do truly crazy things, like cause an install to fail at exactly the same point 3 times in a row, seen it. Also, don't desperately reseat the ram anymore, it's not the problem and those connectors don't last for ever, a few dozen times can wear them out though it should be closer to 50-100 minimum. In any case, It's very unlikely to be the ram, the bios is seeing it, lshw is seeing it, for some reason the os isn't. What command/program is telling you there is only 8G? Is it system monitor, about this computer or what? Sent with [ProtonMail](https://protonmail.com) Secure Email. ‐‐‐ Original Message ‐‐‐ On Friday, April 17, 2020 7:00 PM, Adam Carter wrote: > The BIOS and lshw -c memory shows 2x8Gig but the OS (/proc/meminfo) only > shows 8Gig. I've re-seated the RAM, tried different slots, booted from CD, > but no change. Last time i looked the OS showed 16Gig. > > Is there anything else i can try or is it new RAM time?
Re: [gentoo-user] repair uefi vfat /boot?
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Sunday, March 22, 2020 5:54 PM, Wols Lists wrote: > On 22/03/20 22:40, Caveman Al Toraboran wrote: > > > any idea why 1 partition (uefi vfat) is suffering errors, but the other > > ext4 isn't? > > Simple. If the surface is decaying, it will be localised. It's decided > to hit the locality of the uefi partition. > > This is what would, in the old days, lead to a head crash. Once the > magnetic layer started to physically deform (flake), the head would hit > it which would start a cascade of debris scouring the disk surface and > trashing the heads. > > Cheers, > Wol Drives may have failed that way at one time (and still can on some drives if they are thumped hard enough), the main issue, according to drive makers is that the coating starts to crystallize, which changes it's magnetic properties (heads and how they are driven are pretty tightly optimized now). These crystals grow over time, affecting more and more of the platter they are on. This is why altitude and temperature cycles (well within operating specs.) will accelerate this failure, both tend to cause crystal growth. In any case, once the drive is out of spare sectors your' data is going to disappear. On all modern drives, running out of spare sectors means back it up quick, preferably immediately without a shutdown or reboot. I've seen drives dying this way, until they run out of sectors (quietly) you get data changes as it miss copies the sectors involved to backup sectors (they re-read until the CRC is right, in this case it's often until there are multiple errors that make it appear to read correctly). If you can hear your drive, and it's constantly seeking when it should be idle, or if you hear it repeatedly load and unload the head it's time to get a spare quickly, if you care about your' data but don't have it all backed up. High operating temperature also accelerates this. I've found keeping them under 100 Deg F. makes them last a long time, even drives I've pulled out of Tivo units have lasted me years this way, and in cable boxes or Tivo's the drive run criminally hot (at least the older ones, don't know about newer ones). Bus for now, just get a new drive, don't waste effort beating a dying horse. It will only frustrate you. I've even seen a drive that passed the "long" smart test when it was failing, the data would change within a few hours! When a drive fails me, after any possible back up, I usually toss them across the room so I know not to trust them. If drives do get particulate matter in them (sometimes the seals fail, or are just bad, or the air filter get poked etc.) I pulled a drive out of a laptop once, took off the cover (because it was already dead) and found about a quarter teaspoon of black power that was supposed to be on the platters. If this happens the heads get ground and more and more coating gets ground off, it accelerates very quickly. I've taken apart dozens of failed drives, but only found this once on a failed drive. Many drives are rated for 20G of shock while operating (200G when not spinning). I wouldn't take those number too seriously though. Hope you didn't lose valuable data.
Re: [gentoo-user] How to fix license error during install?
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Friday, March 20, 2020 8:18 PM, Jeffrey Walton wrote: > On Fri, Mar 20, 2020 at 10:16 PM John Covici cov...@ccs.covici.com wrote: > > > On Fri, 20 Mar 2020 21:16:10 -0400, > > Jeffrey Walton wrote: > > > > > Hi Everyone, > > > I'm having trouble installing Gentoo in a Virtual Box VM for testing. > > > It is a x86_64 guest. I selected a hardened profile to test PaX, which > > > means I selected 18 in 'eselect profile'. > > > I'm at "Configuring the Linux kernel" in the Handbook > > > (https://wiki.gentoo.org/wiki/Handbook:AMD64/Installation/Kernel#Alternative:_Using_genkernel). > > > The part "emerge --ask sys-kernel/gentoo-sources" seems OK and does > > > not report errors. The genkernel part fails. > > > The specific error is: > > > $ LICENSE_ACCEPT="*" emerge --ask --autounmask-write > > > sys-kernel/genkernel 2>&1 | tee kernel.txt > > > $ cat kernel.txt > > > > > > - IMPORTANT: 6 config files in '/etc/portage' need updating. > > > Calculating dependencies * See the CONFIGURATION FILES and > > > CONFIGURATION FILES UPDATE TOOLS > > > > > > - sections of the emerge man page to learn how to update config files. > > > ... done! > > > [ebuild N ] app-arch/cpio-2.12-r1 USE="nls" > > > [ebuild N ] sys-kernel/linux-firmware-20200316 > > > USE="redistributable -initramfs -savedconfig (-unknown-license)" > > > [ebuild N ] sys-kernel/genkernel-4.0.4 USE="firmware (-ibm)" > > > > > > > > > The following license changes are necessary to proceed: > > > (see "package.license" in the portage(5) man page for more details) > > > > > > required by sys-kernel/genkernel-4.0.4::gentoo[firmware] > > > > > > = > > > > > > required by genkernel (argument) > > > > > > = > > > > > > =sys-kernel/linux-firmware-20200316 linux-fw-redistributable > > > no-source-code > > > Autounmask changes successfully written. > > > > > > - IMPORTANT: 7 config files in '/etc/portage' need updating. > > > - See the CONFIGURATION FILES and CONFIGURATION FILES UPDATE TOOLS > > > - sections of the emerge man page to learn how to update config > > > files.Here is the > > > > > > > > > Here is the portage(5) man page: > > > https://dev.gentoo.org/~zmedico/portage/doc/man/portage.5.html. Here > > > is the part about package.license: > > > > > > This will allow ACCEPT_LICENSE (see make.conf(5)) to be augmented > > > > > > > > > for a single package. > > > > > > Format: > > > > > > - comment lines begin with # (no inline comments) > > > - one DEPEND atom per line followed by additional licenses or > > > groups > > > > > > > > > Removing LICENSE_ACCEPT="*" and --autounmask-write does not help. > > > The information provided in portage(5) and package.license leaves a > > > lot to be desired. > > > What is the problem and how do I fix it? > > > > Well, you need to change your config files as portage asked you to do > > before proceeding. There are several utilities to do that, I use > > etc-update, but there are several others. > > Thanks John. > > The Handbook does not say to make any configuration changes. That > seems safe to me since I have no idea what changes to make. > > Jeff The manual is a starting point, not a step by step guide that tells you enough by itself. You have to pay attention to what your' system is telling you and look up the things you don't understand. A lack of configuration of many files will doom later steps. For example, if you lack substantial shell experience, and even if you don't you will have to learn new things, on your' own or with some help from the list. If you just do the basic things in the manual, you won't get anywhere. It's more of an outline with some steps well documented, some over documented, and some not documented properly unless you are smart enough too look up error messages and do what is required. Most if not all the info is on the gentoo site, but it's your' job to find it. It's a learning experience, not a complete and detailed set of instructions. Part of this is because you have options, you have to chose how some things will be done and investigate. I strongly recommend using a "tee" etc. to dump everything into text files you can skim. Often compiling will produce specific suggestions that won't be on the last page of text that scrowls by, you have to read ALL of the output from each step, and learn enough to have some understanding of why and how which will require research on the gentoo site, and often other sites. You have to modify the "USE" files for each application, as your' machine will tell you, that's actually the easy part. The hard part involves dealing with issues you don't understand yet. You have to learn, you have to try and want to learn, and you have to pay attention to what your' computer etc. is trying to tell you. It's
Re: [gentoo-user] SDD strategies...
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Tuesday, March 17, 2020 5:18 AM, Peter Humphrey wrote: > > Hm. My NVMe boot drive doesn't show a lifetime attribute, but my two 1TB SSDs > do, and they both show 100%, which makes me suspicious that either the data > are not being collected or they're being misinterpreted. > > - > > Regards, > Peter. At best that would be how much rewriting the cells still have left. Just as in hard drives there is no way of knowing when a random failure, due to a broken bond wire or a bad solder joint will occur. "Life time" can be calculated for hard drives as well, but that doesn't really mean it won't crap out tomorrow after claiming to have years left. If you really care about your' data, Raid (preferably raid 6 or better) and backups (preferably off site) are the only way to go. The issue is not usually end of trusted life, but rather random failure. I've barely managed to recover failed hard drives, That is less likely on SSD though possibly less likely to happen.
Re: [gentoo-user] CPU speed scaling quirk (Intel; Dell i660)
To reduce problems with emitted Radio Frequency Interference, most processors now use a clock that varies in speed over time. This doesn't really reduce the emitted energy, but because it is always changing frequency interference with other devices tends to be intermittent, and Ideally unnoticeable. Also the oscillators used in computers are not the most precise, they don't need to be and precision cost. The bios may let you toggle this deliberate frequency variation and off, which I suppose could be critical in some real-time cases, or a varying clock may, in some cases cause objectionable interference where as the fixed clock, may not, YMMV. Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Wednesday, March 4, 2020 5:28 PM, Walter Dnes wrote: > I've cobbled together a script to select cpu governors and speeds. > One weird thing I've noticed is that reported cpu speed doesn't quite > match the selected speed. E.g. on my machine (yours will vary)... > > cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_frequencies > > ...shows avalable speeds... > > 3001000 300 290 280 270 260 250 240 230 > 220 210 200 190 180 170 160 > > IMPORTANT "userspace" governor must be present. > > /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq reports speeds > very close to, but slightly lower than the selected values, and they also > seem to jump around a bit. What gives? > > - > > Walter Dnes waltd...@waltdnes.org > I don't run "desktop environments"; I run useful applications
Re: [gentoo-user] Libreoffice and pasting HTML from web pages.
Sent with ProtonMail Secure Email. ‐‐‐ Original Message ‐‐‐ On Monday, March 2, 2020 5:35 PM, Dale wrote: > Michael wrote: > > > On Monday, 2 March 2020 23:07:03 GMT Dale wrote: > > > > > Howdy, > > > I don't use LOo a whole lot but several months ago, I noticed a > > > problem. Sometimes I go to a electronic type website that has info on > > > circuits and how something works. Those pages usually contain text, > > > pics and such. I highlight what contains the info I want, which may > > > include some things I don't, then copy it to my clipboard. I then go to > > > LOo and paste it as HTML since that's what it is. Once pasted, I remove > > > things I don't want, such as social media icons and other unwanted > > > things. I also make some images larger, make text larger and such as > > > well. When I get it right, I print it as a pdf file or save it as a LOo > > > document. That's how I did it in the past. When I would paste the > > > contents, it would include images and all and would be done in seconds, > > > well under a minute for sure. > > > When I paste the content now, it doesn't include pics. The boxes is > > > there and text for the link to the image but no image. It also locks > > > Thanks. > > > Dale > > > :-) :-) > > > I haven't looked into how LO works, but it may have something to do with > > > the > > > tonne of JavaScript HTML pages contain these days and how this is > > > processed/ > > > filtered. When I try it here it works for small amounts of text/pics, but > > > unlike your LO, I do not have java installed as a dependency. > > > > An alternative you could try is to save the page as HTML - File/Save Page > > As. > > Then select one of the options to save it as below, which when you open with > > LO does not take long to process. If you select 'HTML page complete' it will > > save all JS and images in a separate folder, which you could delete > > thereafter, or repurpose as you see fit. > > Now that helps a LOT. It works just like it used to work. Very fast > too. It took maybe one to two seconds and it was done. Everything is > there too. > > It takes a long while to compile LOo but I'm going to try disabling java > just to see if that helps. I'm not sure what all the page contains but > here's a link to the page I just tested this with. It's not like it is > some super secret site or anything. > > https://www.electronics-tutorials.ws/waveforms/555-circuits-part-1.html > > It looks like plain stuff to me but I have no idea what is going on > behind the scene. Maybe some others will enjoy that site. It has some > good info and they add stuff pretty regular. > > Thanks much. It's a workaround but at least I can get them copied and > printed now. ;-) > > Dale > > :-) :-) One caveat when using print to pdf, you should immediately check the pdf, sometimes it only saves part of the page, not sure why, trying it again seems to usually work.
Re: [gentoo-user] External hard drive and idle activity
‐‐‐ Original Message ‐‐‐ On Thursday, January 2, 2020 12:12 PM, Rich Freeman wrote: > On Thu, Jan 2, 2020 at 1:41 PM Dale rdalek1...@gmail.com wrote: > > > Rich Freeman wrote: > > > > > Out of curiosity, what model drive is it? Is it by chance an SMR / > > > archive drive? > > > > Device Model: ST8000AS0003-2HH188 > > I recall reading about SMR but can't recall the details of what it is. > > As far as I know, this is just a basic 8TB drive. > > This is an SMR drive. You should DEFINITELY read up on what they are. > > For reads they're completely normal. For sequential writes to unused > space they're completely normal. For random writes or overwrites they > are significantly different from traditional hard drives. > > They work a bit like an SSD in the sense that blocks are arranged into > larger erase regions. Within a region blocks can only be written > sequentially. If you want to overwrite one block in the middle of a > region, the drive will read the entire region into RAM, then write the > entire region sequentially with the overwritten block to a new spot on > the disk. This is just like in an SSD where if try to overwrite a > block in a region with any unTRIMmed blocks the drive must read the > entire region, erase the region, and write the modified region. > > Except that in an SSD those extra reads/writes operate with SSD access > times. With an SMR drive those extra reads/writes operate with hard > drive latencies, so they're MUCH more costly. > > For backup use they're usually fine, IF you're writing in a sequential > file format that is appended to. If you're using rsync to do your > backups then that isn't what you're doing and you're probably paying a > heavy penalty. If you were doing incremental backups using > tar/duplicity/whatever then you'd probably be fine. > > Some filesystems might be optimized for these drives to reduce the > amount of overwriting in place. I haven't looked into it. I'd expect > a log-based filesystem to work fairly well, though those can have high > levels of fragmentation which is better suited for SSD than SMR. > > These drives all have fairly active firmware that manages this special > overwrite process so that they can be used with operating systems that > are naive to how they work. I wouldn't be surprised if this is what > is causing the drive to be active after you unmount it. In theory it > should be harmless to power it off. However, leaving it powered on > probably will improve its performance as it can take care of any > garbage collection before the next time you use it. If whatever > journal it is using to speed things up gets full then you'll feel the > full brunt of any write penalties until it is flushed. > Rich Thank you for the excellent education! I haven't read the full thread but I'd also suggest that running Wireshark on the USB port would likely help diagnose any other issues. I'm having similiar problems with an external drive "freeezing" and refusing to unmount normally and this will be my next step to diagnose it.
