Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300
On Wed, Apr 17, 2024 at 9:33 AM Dale wrote: > > Rich Freeman wrote: > > > All AM5 CPUs have GPUs, but in general motherboards with video outputs > > do not require the CPU to have a GPU built in. The ports just don't > > do anything if this is lacking, and you would need a dedicated GPU. > > > > OK. I read that a few times. If I want to use the onboard video I have > to have a certain CPU that supports it? Do those have something so I > know which is which? Or do I read that as all the CPUs support onboard > video but if one plugs in a video card, that part of the CPU isn't > used? The last one makes more sense but asking to be sure. To use onboard graphics, you need a motherboard that supports it, and a CPU that supports it. I believe that internal graphics and an external GPU card can both be used at the same time. Note that internal graphics solutions typically steal some RAM from other system use, while an external GPU will have its own dedicated RAM (and those can also make use of internal RAM too). The 7600X has a built-in RDNA2 GPU. All the original Ryzen zen4 CPUs had GPU support, but it looks like they JUST announced a new line of consumer zen4 CPUs that don't have it - they all end in an F right now. In any case, if you google the CPU you're looking at it will tell you if it supports integrated graphics. Most better stores/etc have filters for this feature as well (places like Newegg or PCPartPicker or whatever). If you don't play games, then definitely get integrated graphics. Even if the CPU costs a tiny bit more, it will give you a free empty 16x PCIe slot at whatever speed the CPU supports (v5 in this case - which is as good as you can get right now). > That could mean a slight price drop for the things I'm looking at then. > One can hope. Right??? Everything comes down in price eventually... > > I might add, simply right clicking on the desktop can take sometimes 20 > or 30 seconds for the menu to pop up. Switching from one desktop to > another can take several seconds, sometimes 8 or 10. This rig is > getting slower. Actually, the software is just getting bigger. You get > my meaning tho. I bet the old KDE3 would be blazingly fast compared to > the rig I ran it on originally. That sounds like RAM but I couldn't say for sure. In any case a modern system will definitely help. > Given the new rig can have 128GBs, I assume it comes in 32GB sticks. Consumer DDR5 seems to come as large as 48GB, though that seems like an odd size. > I'd get 32GBs at first. Maybe a month or so later get another 32GB. > That'll get me 64Gbs. Later on, a good sale maybe, buy another 32GB or > a 64GB set and max it out. You definitely want to match the timings, and you probably want to match the sticks themselves. Also, you generally need to be mindful of how many channels you're occupying, though as I understand it DDR5 is essentially natively dual channel. If you just stick one DDR4 stick in a system it will not perform as well as two sticks of half the size. I forget the gory details but I believe it comes down to the timings of switching between two different channels vs moving around within a single one. DDR RAM timings get really confusing, and it comes down to the fact that addresses are basically grouped in various ways and randomly seeking from one address to another can take a different amount of time depending on how the new address is related to the address you last read. The idea of "seeking" with RAM may seem odd, but recent memory technologies are a bit like storage, and they are accessed in a semi-serial manner. Essentially the latencies and transfer rates are such that even dynamic RAM chips are too slow to work in the conventional sense. I'm guessing it gets into a lot of gory details with reactances and so on, and just wiring up every memory cell in parallel like in the old days would slow down all the voltage transitions. > I've looked at server type boards. I'd like to have one. I'd like one > that has SAS ports. So, I don't really spend much time looking at them, but I'm guessing SAS is fairly rare on the motherboards themselves. They probably almost always have an HBA/RAID controller in a PCIe slot. You can put the same cards in any PC, but of course you're just going to struggle to have a slot free. You can always use a riser or something to cram an HBA into a slot that is too small for it, but then you're going to suffer reduced performance. For just a few spinning disks though it probably won't matter. Really though I feel like the trend is towards NVMe and that gets into a whole different world. U.2 allows either SAS or PCIe over the bus, and there are HBAs that will handle both. Or if you only want NVMe it looks like you can use bifurcation-based solutions to more cheaply break slots out. I'm kinda thinking about going that direct
Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300
On Wed, Apr 17, 2024 at 6:33 AM Dale wrote: > > On the AM5 link, I found a mobo that I kinda like. I still wish it had > more PCIe slots tho. AM5 has 28 PCIe lanes. Anything above that comes from a switch on the motherboard. 0.1% of the population cares about having anything on their motherboard besides a 16x slot for the GPU. So, that's what all the cheaper boards deliver these days. The higher end boards often have a switch and will deliver extra lanes, and MAYBE those will go into another PCIe slot (probably not wired for 16x but it might have that form factor), and more often those go into additional M.2 slots and USB3 ports. (USB3 is very high bandwidth, especially later generations, and eats up PCIe lanes as a result.) Keep in mind those 28 v5 lanes have the bandwidth of over 100 v3 lanes, which is part of why the counts are being reduced. The problem is that hardware to do that conversion is kinda niche right now. It is much easier to bifurcate a larger slot, but that doesn't buy you more lanes. > It supports not only the Ryzen 9 > series but also supports Ryzen 5 series. That is because the 9 and 5 are branding and basically convey no information at all besides the price point. The Ryzen 7 1700X has about half the performance of the Ryzen 5 7600X, and that would be because the first chip came out in 2017, and the second came out in 2022 and is three generations newer. Likewise the intel branding of "i3" or "i7" and so on also conveys no information beyond the general price level they were introduced at. You can expect the bigger numbers to offer more performance/features than the smaller ones OF THE SAME GENERATION. The same branding keeps getting re-applied to later generations of chips, and IMO it is intentionally confusing. > I looked up the Ryzen 5 7600X > and 8600G. I think the X has no video and the G has video support. Both have onboard graphics. The G designates zen1-3 chips with a GPU built in, and all zen4 CPUs have this as a standard feature. The 7600X is zen4. See what I mean about the branding getting confusing? > I > haven't researched yet to see if the mobo requires the G since it has > video ports, two to be more precise which is the minimum I need. All AM5 CPUs have GPUs, but in general motherboards with video outputs do not require the CPU to have a GPU built in. The ports just don't do anything if this is lacking, and you would need a dedicated GPU. > Anyway, those two CPUs are cheaper than the Ryzen 9 I was looking at. I > could upgrade later on as prices drop. I'm sure a new Ryzen is lurking > around the corner. Zen5 is supposedly coming out later this year. It will be very expensive. Zen4 is still kinda expensive I believe though I haven't gone looking recently at prices. I have a zen4 system and it was expensive (particularly the motherboard, and the DDR5 is more expensive, and if you want NVMe that does v5 that is more expensive as well). > I have a FX-8350 8 core CPU now. Would the Ryzen 5's mentioned above be > a good bit faster, a lot, a whole lot? So, that very much depends on what you're doing. Single-thread performance of that 7600X is 2-3x faster. Total performance is almost 5x faster. The 7600X will use moderately less power at full load, and I'm guessing WAY less power at less than full load. It will also have much better performance than those numbers reflect for very short bursts of work, since modern CPUs can boost. That's just pure CPU performance. The DDR5 performance of the recent CPU is MUCH better than that of the DDR3 you're using now. Your old motherboard might be PCIe v2 (I think the controller for that was on the motherboard back then?). If so each lane delivers 8x more bandwidth on the recent CPU, which matters a great deal for graphics, or for NVMe performance if you're using an NVMe that supports it and have a workload that benefits from it. Gaming tends to be a workload that benefits the most from all of these factors. If your system is just acting as a NAS and all the storage is on hard drives, I'm guessing you won't see much of a difference at all, except maybe in boot time, especially if you put the OS on an NVMe. If this is just for your NAS I would not drop all that money on zen4, let alone zen5. I'd look for something older, possibly used, that is way cheaper. > Still, I need more memory too. 32GBs just isn't much when running > Seamonkey, three Firefox profiles and torrent software. Ok, if this is for a desktop you'll benefit more from a newer CPU. RAM is really expensive though these days. Getting something off-lease is going to save you a fortune as the RAM is practically free in those. You can get something with 32GB of DDR4 for $150 or less in a SFF PC. > I'm not running > out but at times, it's using a lot of it. I was hoping for a mobo that > would handle more than 128GB but that is a lot of memory. Any recent motherboard will handle 128GB. You'll just need to use large DIMMs as
Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300
On Sat, Apr 13, 2024 at 8:20 AM Dale wrote: > > Right now, I have a three drive setup in a removable cage for the NAS > box. If you only need three drives I'm sure you can find cheap used hardware that will handle that. Odds are it will use way less power and perform better than whatever you're going to upgrade your system to. > I'm not familiar with Ceph but I've seen it mentioned before. Do NOT deploy Ceph with three drives on one host. Ceph is what you think about using when you are tired of stacking HBAs to cram a dozen SATA ports in a single host. It isn't what you'd use for backup/etc storage. Honestly, if you're just looking for backup drives I'd consider USB3 drives you just plug into a host and run in a zpool or whatever. Export the filesystem and unplug the drives and you're done. That is how I backup Ceph right now (k8s job that runs restic against ceph dumping it on a zpool). -- Rich
Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300
On Sat, Apr 13, 2024 at 8:11 AM Dale wrote: > > My biggest thing right now, finding a mobo with plenty of PCIe slots. > They put all this new stuff, wifi and such, but remove things I do need, > PCIe slots. PCIe and memory capacity seem to have become the way the server/workstation and consumer markets are segmented. AM5 gets you 28x v5 lanes. SP5 gets you 128x v5 lanes. The server socket also has way more memory capacity, though I couldn't quickly identify exactly how much more due to the ambiguous way in which DDR5 memory channels are referenced all over the place. Suffice it to say you can put several times as many DIMMs into a typical server motherboard, especially if you have two CPUs on it (two CPUs likewise increases the PCIe capacity). IT would be nice if there were switches out there that would let you take a v5 PCIe slot on newer consumer hardware and break it out into a bunch of v3/4 NVMe adapters (U.2, M.2, PCIe, whatever). -- Rich
Re: [gentoo-user] NAS box and switching from Phenom II X6 1090T to FX-6300
On Sat, Apr 13, 2024 at 3:58 AM Dale wrote: > > Given the FX-6300 has a higher clocks speed, 3.8GHz versus 3.2GHz for > the Phenom, I'd think the FX would be a upgrade, quite a good one at > that. More L2 cache too. Both are 6 cores according to what I found. > Anyone know something I don't that would make switching to the FX-6300 a > bad idea? The most obvious issue is that you're putting money into a very obsolete system. Obviously hardware of this generation is fairly cheap, but it isn't actually the best bang for the buck, ESPECIALLY when you factor in power use. Like most AMD chips of that generation (well, most chips in general when you get that old), that CPU uses quite a bit of power at idle, and so that chip which might cost you $35 even at retail might cost you double that amount per year just in electricity. If your goal is to go cheap you also need to consider alternatives. You can get used hardware from various places, and most of it is 3-5 years old. Even commodity hardware of that age is far more powerful than a 15 year old CPU socket and often it starts at $100 or so - and that is for a complete system. Often you can get stuff that is ex-corporate that has a fair bit of RAM as well, since a lot of companies need to deal with compatibility with office productivity software that might be a little RAM hungry. RAM isn't cheap these days, and they practically give it away when they dispose of old hardware. The biggest issue you're going to have with NAS is finding something with the desired number of drive bays, as a lot of used desktop hardware is SFF (but also super-low-power, which is something companies consider in their purchasing decisions when picking something they're going to be buying thousands of). Right now most of my storage is on Ceph on SFF PCs. I do want to try to get future expansion onto NVMe but even used systems that support much of that are kinda expensive still (mostly servers since desktop CPUs have so few PCIe lanes, and switches aren't that common). One of my constraints using Ceph though is I need a lot of RAM, which is part of why I'm going the SFF route - for $100 you can get one with 32GB of RAM and 2-3 SATA ports, plus USB3 and an unused 4-16x PCIe slot. That is a lot of RAM/IO compared to most options at that price point (ARM in particular tends to lack both - not that it doesn't support it, but rather nobody makes cheap ARM hardware with PCIe+DIMM slots). -- Rich
Re: [gentoo-user] Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo
On Sun, Mar 31, 2024 at 5:36 PM Wol wrote: > > On 31/03/2024 20:38, Håkon Alstadheim wrote: > > For commercial entities, the government could just contact the company > > and apply pressure, no need to sneak the backdoor in. Cf. RSA . > > Serving a "secret compliance" notice on a third party is always fraught > with danger. Okay, I probably can't trust my own government to protect > me, but if the US Government served a compliance notice on me I'd treat > it with the respect it deserved - probably use it as loo paper! I imagine most large companies would just comply with their local government, but there are some major limitations all the same: 1. It isn't necessarily the local government who wants to plant the back door. The FBI can't just call up Huawei and get the same results they would with Google. 2. Even if the company complies, there are going to be more people who are aware of the back door. Some of those could be foreign agents. If you infiltrate the company and obfuscate your code, then only your own agents are aware there is an intrusion. 3. The methods employed in your attack might also be sensitive, and so that's another reason to not want to disclose them. If you have some way of subtly compromising some encryption scheme, you might not want any employees of the company to even know the cryptosystem weakness even exists, let alone the fact that you're exploiting it. When the methods are secret in this way it is that much easier to obfuscate a clandestine attack as well. When you look at engineer salaries against national defense budgets, it wouldn't surprise me if a LOT of FOSS (and other) contributors are being paid to add back doors. On the positive side, that probably also means that they're getting paid to fix a lot of bugs and add features just to give them cover. To bomb a power plant might take the combined efforts of 1-2 dozen military aircraft in various roles, at $100M+ each (granted, that's acquisition cost and not operational cost). Installing a trojan that would cause the plant to blow itself up on command might just require paying a few developers for a few years, for probably less than $1M total, and it isn't even that obvious that you were involved if it gets discovered, or even after the plant blows up. -- Rich
Re: [gentoo-user] Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo
On Sun, Mar 31, 2024 at 10:59 AM Michael wrote: > > On Sunday, 31 March 2024 13:33:20 BST Rich Freeman wrote: > > (moving this to gentoo-user as this is really getting off-topic for -dev) > > Thanks for bringing this to our attention Rich. > > Is downgrading to app-arch/xz-utils-5.4.2 all that is needed for now, or are > we meant to rebuilding any other/all packages, especially if we rebuilt our > @world only a week ago as part of the move to profile 23.0? It is not necessary to rebuild anything, unless you're doing something so unusual that you'd already know the answer to the question. -- Rich
[gentoo-user] Re: [gentoo-dev] Current unavoidable use of xz utils in Gentoo
(moving this to gentoo-user as this is really getting off-topic for -dev) On Sun, Mar 31, 2024 at 7:32 AM stefan1 wrote: > > Had I seen someone say that a bad actor would spend years gaining the > trust of FOSS > project maintainers in order to gain commit access and introduce such > sophisticated > back doors, I would have told them to take their meds. > This is insane. It makes quite a bit of sense though. For a low-activity FOSS project, how much manpower does it take to gain a majority share of the governance? In this case it is one person, but even for a big project (such as Gentoo) I suspect that 3-4 people working full time could probably hit upwards of 50% of the commit volume. That doesn't have to be 3-4 "Gentoo developers." It could be 3-4 human beings with 1 admin assistant who manages 50 email addresses that the commits get spread across, and they sign up as 50 Gentoo developers and get 50 votes for the Council (and probably half the positions there if they want them), the opportunity to peer review "each other's" contributions, and so on. I just use Gentoo as an example as we're all familiar with it and probably assume it couldn't happen here. As you go on, the actual targets are likely to be other projects... > If this happened to something like firefox, I don't think anyone would > have found out. > No one bats an eye if a website loads 0.5s longer. It seems likely that something like this has ALREADY happened to firefox. It might also happen with commercial software, but the challenge there is HR as you can't just pay 1 person to masquerade as 10 when they all need to deal with payroll taxes. We're going on almost 20 years since the Snowden revelations, and back then the NSA was basically doing intrusion on an industrial scale. You'd have dev teams building zero days and rootkits, sysadmin teams who just administrate those back doors to make sure there are always 2-3 ways in just in case one gets closed, SMEs who actually make sense of the stolen data, rooms full of engineers who receive intercepted shipments of hardware and install backdoors on them, and so on. We're looking at what probably only one person can do if they can dedicate full time to something like this. Imagine what a cube farm full of supervised developers with a $50M budget could do, and that is pocket change to most state actors. The US government probably spends more than that in a year on printer paper. -- Rich
Re: [gentoo-user] Re: Suggestions for backup scheme?
First, thanks for the Ars link in the other email. I'll give that a read. On Mon, Feb 5, 2024 at 7:55 AM J. Roeleveld wrote: > > On Wednesday, January 31, 2024 6:56:47 PM CET Rich Freeman wrote: > > The main barrier is that its license isn't GPL-compatible. It is > > FOSS, but the license was basically designed to keep it from being > > incorporated into the mainline kernel. > > Which isn't as much of an issue as it sounds. You can still add it into the > initramfs and can easily load the module. > And the code still works with the functions the kernel devs pushed behind the > GPL-wall if you simply remove that wall from your own kernel. > (Which is advisable as it will improve performance) So, that's great for random individuals, but companies are going to be hesitant to do that, especially for anything they redistribute. This is part of why it isn't mainstream. A big part of the reason that Linux is mainstream is that it doesn't have any legal/license encumbrances. If you have 100 instances of something and want to have 200 instances, you just turn a dial or add hardware. There isn't anybody you need to get permission from or pay. Personally I think the idea that the GPL prevents linking, or that you can have GPL-only APIs is legally ridiculous. Then again, I thought some of the court rulings in the Oracle vs Google Android lawsuits were ridiculous as well around API copyrighting. Dynamic linking is just adding a look up table to your program. It would be like suing me because I advised you to call somebody, arguing that by telling you to call somebody I was violating the copyright of the phone book. Linking is just an instruction to the loader to go find some symbol and substitute its address at this location. However, MANY people would disagree with what I just said, and some might even sue a company that published a large piece of software that failed to comply with the mainstream interpretation of the GPL. A court might agree with them and award damages. I think they and the court would be wrong in that case, but the police don't care what I think, and they do care what the court thinks. The result is that the murky legal situation makes ZFS unattractive. If I were publishing some large commercial software package, I'd personally be hesitant to embrace ZFS on Linux in it for that reason, even though I use it all the time personally. > > > The odd thing is that right now Oracle controls both ZFS and btrfs, > > with the latter doing mostly the same thing and being GPL-compatible, > > but it hasn't tended to be as stable. So we're in a really long > > transitional period to btrfs becoming as reliable. > > After all this time, I have given up on waiting for btrfs. As mentioned in my > other reply, it's still nowhere near reliable. Clearly Oracle likes this state of affairs. Either that, or they are encumbered in some way from just GPLing the ZFS code. Since they on paper own the code for both projects it seems crazy to me that this situation persists. > > To make this easier, there is a compatiblity option when creating a new zpool. > It's also listed in the zfs-kmod ebuild: > - zpool create -o compatibility=*grub*2 ... > - Refer to /usr/share/zfs/compatibility.d/*grub*2 for list of features. Oh, that is VERY helpful. I've found random many-years-old webpages with the appropriate instructions, but something that is part of the maintained project is much more useful. Granted, I think the bottom line is that boot probably shouldn't be on the same filesystem as large volumes of data, as these feature restrictions are going to be cumbersome. I'm guessing you can't shrink vdevs, for example. -- Rich
Re: [gentoo-user] Re: Suggestions for backup scheme?
On Fri, Feb 2, 2024 at 6:39 PM Grant Edwards wrote: > > On 2024-01-31, Rich Freeman wrote: > > > In any case, these COW filesystems, much like git, store data in a > > way that makes it very efficient to diff two snapshots and back up > > only the data that has changed. [...] > > In order to take advantage of this, I assume that the backup > destination and source both have to be ZFS? So, the data needs to be RESTORED to ZFS for this to work. However, the zfs send command serializes the data and so you can just store it in files. Those files can only be read back into zfs. It is probably a bit more typical to just pipe the send command into zfs receive (often over ssh) so that you're just directly mirroring the filesystem, and not storing the intermediate data. > Do backup source and > destination need to be in the same filesystem? Or volume? Or Pool? No on all of these, but they can be. > If you'll forgive the analogy, we'll say the the functionality of > rsync (as used by rsnapshot) is built-in to ZFS. Is there an > application that does with ZFS snapshots what the rsnapshot > application itself does with rsync? There are a few wrappers around zfs send. I'm using sys-fs/zfs-auto-snapshot and what looks like a much older version of: https://github.com/psy0rz/zfs_autobackup > > I googled for ZFS backup applications, but didn't find anything that > seemed to be widespread and "supported" the way that rsnapshot is. They're less popular since many just DIY them, but honestly I think the wrapper is a nicer solution. It will rotate backups, make sure that snapshots aren't needed before deleting them, and so on. In order to do an incremental backup the source/destination systems need to have matching snapshots to base them on, so that is important if backups are sporadic. If you're just saving all the send streams then knowing which ones are obsolete is also important, unless you want to have points in time. -- Rich
Re: [gentoo-user] Re: Suggestions for backup scheme?
On Wed, Jan 31, 2024 at 1:42 PM Wols Lists wrote: > > On 31/01/2024 17:56, Rich Freeman wrote: > > I don't think there are > > any RAID implementations that do full write journaling to protect > > against the write hole problem, but those would obviously underperform > > zfs as well. > > This feature has been added to mdraid, iirc. > Oh, it looks like it has. Kind of annoying that it only works for a separate device. I guess you could create a separate partition on all of the devices, create a mirror across all of those, and then use that as the journal for the real raid. It would be nice if it had an option for an internal journal. I'd expect the performance of btrfs/zfs to be better of course because it just does the write to known-unused blocks, so interrupted writes don't cause any issues. Depending on how far it gets when interrupted the write will be ignored (it is just data written into unused space), or will get completed (the root of the metadata was updated and now points to a consistent new state). -- Rich
Re: [gentoo-user] Re: Suggestions for backup scheme?
On Wed, Jan 31, 2024 at 12:40 PM Thelma wrote: > > If zfs file system is superior to ext4 and it seems to it is. > Why hasn't it been adopted more widely in Linux? > The main barrier is that its license isn't GPL-compatible. It is FOSS, but the license was basically designed to keep it from being incorporated into the mainline kernel. The odd thing is that right now Oracle controls both ZFS and btrfs, with the latter doing mostly the same thing and being GPL-compatible, but it hasn't tended to be as stable. So we're in a really long transitional period to btrfs becoming as reliable. ZFS also cannot be shrunk as easily. I think that is something that has been improved more recently, but I'm not certain of the state of it. Also, bootloaders like grub aren't 100% compatible with all of its later features, and it isn't even clear in the docs which ones are and aren't supported. So it doesn't hurt to keep /boot off of zfs. I'm sure ext4 also performs better. It has to be faster to just overwrite a block in place than to remap the extents around the change, even if the latter is safer. I'd expect zfs to outperform ext4 with full data journaling though, which would have a comparable level of safety, assuming it isn't on RAID. I don't think there are any RAID implementations that do full write journaling to protect against the write hole problem, but those would obviously underperform zfs as well. -- Rich
Re: [gentoo-user] Re: Suggestions for backup scheme?
On Wed, Jan 31, 2024 at 6:45 AM John Covici wrote: > > I know you said you wanted to stay with ext4, but going to zfs reduced > my backup time on my entire system from several hours to just a few > minutes because taking a snapshot is so quick and copying to another > pool is also very quick. > Honestly, at this point I would not run any storage I cared about on anything but zfs. There are just so many benefits. I'd consider btrfs, but I'd have to dig into whether the reliability issues have been solved. I was using that for a while, but I found that even features that were touted as reliable had problems from time to time. That was years ago, however. On paper I think it is the better option, but I just need to confirm whether I can trust it. In any case, these COW filesystems, much like git, store data in a way that makes it very efficient to diff two snapshots and back up only the data that has changed. They are far superior to rsync in this regard, providing all the benefits of rsync --checksum but without even having to stat all of the inodes let alone read file contents. -- Rich
Re: [gentoo-user] Suggestions for backup scheme?
On Tue, Jan 30, 2024 at 3:08 PM Wol wrote: > > On 30/01/2024 19:19, Rich Freeman wrote: > > I'd echo the other advice. It really depends on your goals. > > If you just want a simple backup, I'd use something like rsync onto lvm > or btrfs or something. I've got a little script that sticks today's date > onto the snapshot name So, you've basically described what rsnapshot does, minus half the features. You should consider looking at it. It is basically an rsync wrapper and will automatically rotate multiple snapshots, and when it makes them they're all hard-linked such that they're as close to copy-on-write copies as possible. The result is that all those snapshots don't take up much space, unless your files are constantly changing. -- Rich
Re: [gentoo-user] Suggestions for backup scheme?
On Tue, Jan 30, 2024 at 1:15 PM Grant Edwards wrote: > > Are there other backup solutions that people would like to suggest I > look at to replace rsnapshot? I was happy enough with rsnapshot (when > it was running), but perhaps there's something else I should consider? > I'd echo the other advice. It really depends on your goals. I think the key selling point for rsnapshot is that it can generate a set of clones of the filesystem contents that are directly readable. That isn't as efficient as it can be, but it is very simple to work with, and it is done about as well as can be done with this sort of approach. Restoration basically requires no special tooling this way, so that is great if you want to restore from a generic rescue disk and not have to try to remember what commands to use. send-based tools for filesystems like brtrfs/zfs are SUPER-efficient in execution time/resources as they are filesystem-aware and don't need to stat everything on a filesystem to identify exactly what changed in an incremental backup. However, you're usually limited to restoring to another filesystem of the same type and have to use those tools. There are some scripts out there that automate the process of managing all of this (you need to maintain snapshots/etc to allow the incremental backups). There are a bunch of other tools for backing up specific applications/filesystems/etc as well. (Every database has one, which you should definitely use, and there are tools like volsync for k8s and so on.) Restic seems to be the most popular tool to backup to a small set of files on disk/cloud. I use duplicity for historical reasons, and restic does the same and probably supports more back-ends. These tools are very useful for cloud backups as they're very efficient about separating data/indexes and keeping local copies of the latter so you aren't paying to read back your archive data every time you do a new incremental backup, and they're very IO-efficient. Bacula is probably the best solution for tape backups of large numbers of systems, but it is really crufty and unwieldy. I would NOT use this to backup one host, and especially not to back up the host running bacula. Bootstrapping it is a pain. It is very much designed around a tape paradigm. If you have windows hosts you want to backup then be sure to find a solution that supports volume shadow copy - there aren't many. Bacula is one of them which is the main reason I even use it at this point. If you don't have that feature then you won't back up the registry, and you can imagine how losing that is on a windows machine. If you're just backing up documents though then anything will work, as long as files aren't open, because windows is extra-fussy about that. -- Rich
Re: [gentoo-user] Re: OFF TOPIC Need Ubuntu network help
On Tue, Oct 17, 2023 at 11:15 PM Grant Edwards wrote: > > For example, if one > of the links is down, Ubuntu is really fond of waiting a couple > mintues for it to come up before it finishes booting. [If it doesn't > wait for all the network interfaces, how is it going to do all that > cloudy crap nobody really wants?] I think the intent is to prevent dependency issues, though IMO that would be better avoided by just setting dependencies on the systemd units. However, many distros try to abstract systemd behind a wall of distro configuration in part because they wanted to the original transition to systemd to be seamless. I have a bunch of ubuntu hosts that have dual NICs and they just love to take forever to boot. This is despite having only one entry in /etc/netplan and having it have "optional: true" set. networkctl shows one interface as "configuring" even after the system is up for days. Hmm, might even be a systemd-networkd bug. I see ubuntu created /run/systemd/network/10-netplan-alleths.network and it contains "RequiredForOnline=no". Oh well, I rarely reboot so it just hasn't been on the top of my list of things to fix. Honestly, I'd prefer if it just let me configure networkd directly. I'm sure there is some way to do that, but I feel like if I do then I'll have to read the release notes every time there is a new release to make sure it isn't going to break it. If you're going to run a distro like Ubuntu I've found it is generally best to just figure out the "Ubuntu Way" and do it their way. If that isn't adequate, the easier solution is to just use a more appropriate distro. -- Rich
Re: [gentoo-user] gentoo packages contain binary images?
On Thu, Sep 28, 2023 at 12:24 PM n952162 wrote: > > $ tar -xjvf /var/cache/binpkgs/dev-util/cmake-3.22.2.tbz2 ./usr/bin/cmake > It looks to me that it's in the tarball received from gentoo. Unless you tell portage to fetch binpkgs it won't fetch one from Gentoo. Until very recently Gentoo didn't even offer them. Distfiles are located in DISTDIR, not PKGDIR, as defined in make.conf > Maybe I'm misinterpreting something? > > emerge --getbinpkg n -v --tree --deep --update --noreplace --changed-use > --verbose-conflicts --keep-going --with-bdeps=y --backtrack=100 cmake Most likely you have FEATURES=buildpkg set. Unless you're short on space I recommend leaving it set. Being able to re-install from a binary package can be handy if somehow a package gets corrupted. It doesn't require a working toolchain/etc, and of course is faster, and gives you the exact same result as rebuilding from source assuming you haven't changed the underlying dependencies, USE flags, etc. Portage won't actually use a binary package unless you tell it to with --usepkg(only) (-k/K). Most common use cases for this are when you want to either use upstream binaries, or build your own to stage or deploy en-masse. I build binpkgs with a cron job overnight and then when I review the packages to be installed I can skip most of the build time by using --usepkg - I get the exact same result as building them after I review the packages to be installed, since these are all built with my settings anyway. -- Rich
Re: [gentoo-user] Network throughput from main Gentoo rig to NAS box.
On Sat, Sep 23, 2023 at 8:04 AM Dale wrote: > > I'm expecting more of a consistent throughput instead of all the > idle time. The final throughput is only around 29.32MB/s according to > info from rsync. If it was not stopping all the time and passing data > through all the time, I think that would improve. Might even double. Is anything else reading data from the NAS at the same time? The performance is going to depend on a lot of details you haven't provided, but anything that reads from a hard disk is going to significantly drop throughput - probably to levels around what you're seeing. That seems like the most likely explanation, assuming you don't have some older CPU or a 100Mbps network port, or something else like WiFi in the mix. The bursty behavior is likely due to caching. -- Rich
Re: [gentoo-user] Password questions, looking for opinions. cryptsetup question too.
On Sat, Sep 23, 2023 at 11:05 AM Dale wrote: > > I'm not to clear on this but it looks like it is using 'aes-xts-plain64' > to me. If so, is that good enough? Is there better? You are using the defaults, which is what you should be using, as they're very secure. As far as I'm aware there is no known attack on AES that is faster than a brute force attack, and a brute-force attack on AES itself is not practical. I think it is unlikely that anybody knows of an attack on the cipher, but of course that cannot be ruled out. Current estimates of the time required to brute-force AES are in the billions of years. If somebody wanted to decrypt the drive without your knowledge, the only practical attacks would be to evesdrop on you somehow to capture your passphrase, or to brute force your passphrase. LUKS is designed to make a brute-force attack on a passphrase impractical as long as it is reasonably long. On typical hardware it should take a full second or two to make one decryption attempt on the passphrase - obviously an attacker could have more sophisticated hardware available but to even be able to attempt tens of thousands of guesses per second would require a very large expense, and your passphrase should be long enough to make that search very long. The most likely attack would be evesdropping. Stopping that requires good physical security, and also keeping any malware out of your bootloader. Unfortunately, the latter is generally not something linux distros do much to prevent. Corporate laptops running windows are typically set up to protect against this using a TPM and secure boot. I'm not sure if any linux distros support a fully signed boot processes up to disk decryption - doing that on Gentoo would be tricky since the OS is being modified so often. A release-based distro could do it a bit more easily - just stick the essential bits in a squashfs and sign everything up to that point, and use secure boot. Then of course if an attacker didn't mind you knowing about their intrusion, they could use the rubber hose method. The only way to defeat that sort of thing is to have some trusted external agent involved in the process who could revoke your own access to your device (think TPM and remote attestation to secure the boot chain plus online authentication required for the device to obtain the session key - though at that point you'd probably also just run a thin client and keep the bulk of the data someplace more secure). -- Rich
Re: [gentoo-user] Controlling emerges
On Thu, Sep 21, 2023 at 9:13 PM Dale wrote: > > Sometimes I wish they would announce when they add features. Rich, you > frequent this list. If you hear of something new, could you post it? Sure, if a relevant topic comes up and I'm aware of it. However, I doubt this setting is going to do much that nice doesn't already do. The original focus seemed to be on memory use, and niceness will not have any impact on the memory use of a build. The only thing that will is reducing the number of parallel jobs. There really isn't any way to get portage to regulate memory use short of letting it be killed (which isn't helpful), maybe letting it being stopped when things get out of hand (which will help as the memory could at least be swapped, but the build might not be salvageable without jumping through a lot of hoops), or if the package maintainer provides some kind of hinting to the package manager so that it can anticipate how much memory it will use. Otherwise trying to figure out how much memory a build system will use without just trying it is like solving the halting problem. -- Rich
Re: [gentoo-user] Re: Password questions, looking for opinions. cryptsetup question too.
On Wed, Sep 20, 2023 at 4:22 PM Frank Steinmetzger wrote: > > Am Tue, Sep 19, 2023 at 11:49:24PM -0500 schrieb Dale: > > > Anyway, when I do that and use the new passwords successfully, I make a > > backup copy and on my rig, I can encrypt it with a right click. I then > > shred the original. > > Just on a sidenote, once you’re on an SSD, shredding has no use and is > actually detrimental. > I'm not sure I'd go quite that far, but it certainly isn't as effective. No way to be certain how well it works, but it is certainly worth doing an ATA Secure Erase command on the drive. A good SSD should implement that in a way that ensures all the data is actually unretrievable (probably by implementing full disk encryption and erasing the key). Of course, there is no way to tell if the drive was implemented well. Full-disk encryption at the OS level is of course the best way to protect against recovery of data on a discarded disk. -- Rich
Re: [gentoo-user] Computer case for new build
On Tue, Sep 19, 2023 at 1:05 PM Frank Steinmetzger wrote: > > Am Tue, Sep 19, 2023 at 11:01:48AM -0400 schrieb Rich Freeman: > > No, the chipset downlink is always four lanes wide. The diagram you linked has 8, but I can't vouch for its accuracy. Haven't looked into it for AM4. > > Again, that is AM4 which I haven't looked into as much. AM5 increases > > the v5 lanes and still has some v4 lanes. > > AFAIR, PCIe 5 is only guaranteed for the NVMe slot. The rest is optional or > subject to the chipset. Actually, PCIe v5 isn't guaranteed for the NVMe slot either, or even the first 16x slot. It is all subject to the motherboard design. There are AM5 MBs that don't have any PCIe v5 slots. > > I'm sure PCIe v5 switching is hard/expensive, but they definitely > > could mix things up however they want. The reality is that most IO > > devices aren't going to be busy all the time, so you definitely could > > split 8 lanes up 64 ways, especially if you drop a generation or two > > along the way. > > Unfortunately you can’t put low-speed connectors on a marketing sheet, when > competitors have teh shizz. Well, you can, but they don't fit on a tweet. Just my really long emails... We're not their target demographic in any case. Now, if Dale wanted more RGB lights and transparent water hoses, and not more PCIe slots, the market would be happy to supply... > > > Server hardware definitely avoids many of the limitations, but it just > > tends to be super-expensive. > > Which is funny because with the global cloud trend, you would think that its > supply increases and prices go down. I think the problem is that the buyers are way less price-sensitive. When a medium/large company is buying a server, odds are they're spending at least tens of thousands of dollars on the software/maintenance side of the project, if not hundreds of thousands or more. They also like to standardize on hardware, so they'll pick the one-size-fits-all solution that can work in any situation, even if it is pricey. Paying $5k for a server isn't a big deal, especially if it is reliable/etc so that it can be neglected for 5 years (since touching it involves dragging in the project team again, which involves spending $15k worth of time just getting the project approved). The place where they are price-sensitive is on really large-scale operations, like cloud providers, Google, social media, and so on - where they need tens of thousands of identical servers. These companies would create demand for very efficiently-priced hardware. However, at their scale they can afford to custom develop their own stuff, and they don't sell to the public, so while that cheap server hardware exists, you can't obtain it. Plus it will be very tailored to their specific use case. If Google needs a gazillion workers for their search engine they might have tensor cores and lots of CPU, and maybe almost no RAM/storage. If they need local storage they might have one M.2 slot and no PCIe slots at all, or some other lopsided config. Backblaze has their storage pods that are basically one giant stack of HDD replicators and almost nothing else. They probably don't even have sideband management on their hardware, or if they do it is something integrated with their own custom solutions. Oh, the other big user is the US government, and they're happy to pay for a million of those $5k servers as long as they're assembled in the right congressional districts. Reducing the spending probably reduces the number of jobs, so that is an anti-feature... :) -- Rich
Re: [gentoo-user] Computer case for new build
On Tue, Sep 19, 2023 at 10:35 AM Frank Steinmetzger wrote: > > Am Tue, Sep 19, 2023 at 09:17:45AM -0400 schrieb Rich Freeman: > > > The higher-end motherboards have switches, and not all > > the lanes may be the highest supported generation, but I don't think > > any modern AMD motherboards have any kind of PCIe controller on them. > > Here are the I/O capabilities of the socket: > https://www.reddit.com/r/Amd/comments/bus60i/amd_x570_detailed_block_diagram_pcie_lanes_and_io/ So, that is AM4, not AM5 > A slight problem is that it is connected to the CPU by only 4.0×4. So tough > luck if you want to do parallel high-speed stuff with two PCIe×4 M.2 drives. So, that block diagram is a bit weak. If you look on the left side it clearly shows 20 PCIe lanes, and the GPU only needs 16. So there are 8 lanes for the MB chipset to use. The 4 on the left aren't the same as the 4 on the right I think. Again, that is AM4 which I haven't looked into as much. AM5 increases the v5 lanes and still has some v4 lanes. > > Basically memory, USB, and PCIe are all getting so fast that trying to > > implement a whole bunch of separate controller chips just doesn't make > > sense. > > However, the CPU has a limited number of them, hence there are more in the > chipset. Most notably SATA. Yup, especially since AM5 dropped SATA entirely. The chipset would be using PCIe lanes for SATA. > Those look really weird. “Huge” ATX boards, but all covered up with fancy > gamer-style plastics lids and only two slots poking out. Yeah, that is definitely the trend. Few are using PCIe cards, so they aren't supporting as many. In theory they could take one PCIe v5 lane on the board and run it into a switch and provide 4 more 1x v3 lanes for older expansion cards, and so on. Those v5 lanes can move a lot of data and other than the GPU and maybe NVMe little is using them. All the same desktop CPUs are a bit starved for lanes. > > > Look at the X670 chipset boards as those tend to have PCIe switches which > > give them more lanes. The switched interfaces will generally not support > > PCIe v5. > > The X series are two “B-chipset chips” daisychained together to double the > downstream connections. Meaning one sits behind the other from the POV of > the CPU and they share their uplink. > > Here are some nice block diagrams of the different AM5 chipset families: > https://www.hwcooling.net/en/amd-am5-platform-b650-x670-x670e-chipsets-and-how-they-differ/ Thanks - that site is handy. I'm sure PCIe v5 switching is hard/expensive, but they definitely could mix things up however they want. The reality is that most IO devices aren't going to be busy all the time, so you definitely could split 8 lanes up 64 ways, especially if you drop a generation or two along the way. It is all packet switched so it is really no different than having a 24 port gigabit network switch with a 10Gb uplink - sure in theory the uplink could be saturated but typically it would not be. Ultimately though the problem is supply and demand. There just isn't much demand for consumer boards with stacks of expansion cards, so nobody makes them. They'd rather give you more M.2 slots, USB, or just make the CPU cheaper. That is why I've been trying to change how I design my storage/etc. Rather than trying to find the one motherboard+HBA combo that lets me cram 16 drives into one case, it is WAY easier to get a bunch of $100 used corporate SFF desktops, slap a 10GbE NIC in them, and plug USB3 hard drives into them. The drives still perform about as fast, and it is infinitely expandable. If anything breaks it can be readily replaced by commoditized hardware. Hardest part is just making sure the SFF PC has a 16x slot and integrated graphics, but that isn't too big of an ask. Server hardware definitely avoids many of the limitations, but it just tends to be super-expensive. Granted, I haven't been looking on eBay for used stuff. The used desktop gear at least tends to be reasonably low-power - you have to watch the server gear as the older stuff can tend to guzzle power (newer stuff isn't as bad). Granted, you can definitely find server hardware that can accomodate 12+ drives. -- Rich
Re: [gentoo-user] Computer case for new build
On Tue, Sep 19, 2023 at 8:26 AM Frank Steinmetzger wrote: > > BTW: it’s APU, without the G. Because it is an Accellerated Processing Unit > (i.e. a processor), not a GPU. No real "reason" for it besides branding/naming/etc. They could have called it a MPU for Mixed Processing Unit if they wanted, and no doubt somebody would come up with a nice explanation about how that is the only term that makes sense. The GPU in a processor with an integrated GPU is a GPU like just about any other. It might not have dedicated memory, and it might not be as big, but they do the same thing. > Well they allow you to put larger cards in, but they don’t have the lanes > for it. Somewhere else in the thread was mentioned that the number of lanes > is very limited. Only the main slot (the big one for the GPU) is directly > connected to the CPU. The rest is hooked up to the chipset which itself is > connected to the CPU either via PCIe×4 (AMD) or whatchacallit (DMI?) for > Intel. So, on most AMD boards these days all the PCIe lanes are wired to the CPU I believe. The higher-end motherboards have switches, and not all the lanes may be the highest supported generation, but I don't think any modern AMD motherboards have any kind of PCIe controller on them. Basically memory, USB, and PCIe are all getting so fast that trying to implement a whole bunch of separate controller chips just doesn't make sense. They benefit from higher end fabs, though not necessarily quite as high-end as the processor cores themselves (hence the AMD chiplet design). A single PCIe v5 lane is moving data at 32Gb/s. Plus if you were going to consolidate things at that speed on the motherboard you'd need one heck of a pipe to get it from there to the CPU anyway (something the CPU has internally). It looks like Intel still puts a controller on the motherboard AM5 has 28 PCIe v5 lanes, 4 PCIe v4 lanes, and 4 20Gbps USB3 ports. LGA1700+MB has 16 PCIe v5 lanes, 16 PCIe v4 lanes, 16 PCI v3 lanes, and quite a bit more USB3 (though that is via the MB so I'm not sure it can sustain them all at max). Not meant as an Intel vs AMD comparison as I'm sure there are caveats in the details, and individual motherboards use that IO differently, but just meant to give a sense of what these desktop CPUs typically deliver. In contrast here are the latest server sockets: SP5 (AMD) has 128 PCIe v5 lanes, and 4 20Gbps USB3 ports (and 16 memory channels is of course a big selling point - vs 4 on AM5) LGA 4677 (Intel) has 16 PCIe v4 lanes and 12 PCIe v3 lanes, and again more USB3 (all via the MB). I'm actually kinda surprised how few lanes it has. It also only has 8 memory channels. Seems like PCIe v5 isn't as much of a selling point on servers. If I missed some detail please point it out - I mainly run AMD desktop CPUs so there could be some server/Intel capabilities out there I'm less familiar with. With the Intel approach of putting more on the motherboard I suspect that there might be bottlenecks if all that IO were to be used at once, though that does seem unlikely. > Look for youself and filter what you need, like 1 or 2 HDMI, DP and PCIe: > AM4: https://skinflint.co.uk/?cat=mbam4=18869_4%7E4400_ATX > AM5: https://skinflint.co.uk/?cat=mbam5=18869_4%7E4400_ATX > Interestingly: the filter goes up to 6 PCIe slots for the former, but only to > 4 for the latter. You can definitely get more PCIe slots on AM5, but the trend is to have less in general. Look at the X670 chipset boards as those tend to have PCIe switches which give them more lanes. The switched interfaces will generally not support PCIe v5. That said, the SATA ports tend to take up lanes (AM5 has no SATA support on the CPU), so motherboards that have 4x/2x slots available might disable some SATA ports if they use them. The trend is definitely more towards M.2 and those each eat up 4 lanes. In any case, if what you want is lots of IO, I guess you can shell out for an EPYC... -- Rich
Re: [gentoo-user] Computer case for new build
On Tue, Sep 19, 2023 at 5:43 AM Dale wrote: > > I been on Newegg using their rig builder feature. Just to get rough > ideas, I picked a AMD Ryzen 9 5900X 12-Core 3.7 GHz Socket AM4. Yea, I > did a copy and paste. lol It's a bit pricey but compared to my current > rig, I think it will run circles around it. My current rig has a AMD FX > -8350 Eight-Core Processor running at 4GHz or so. You think I'll see > some speed improvement or am I on the wrong track? Lol - they'd be night and day, and that's just looking at CPU. The RAM is way faster too. CPU mark lists the 5900X as 6-7x faster, and the 7900X as almost 9x faster. > My problem is the mobo. I need a few PCIe slots. Most just don't have > enough. The trend is towards fewer slots. Some of that is driven by the addition of M.2 slots which require 4 lanes each. More of the IO is going to USB compared to PCIe, probably because that is what people tend to use with desktops. > Most have a slot for a video card. Then maybe 2 other slightly > slower ones and maybe one slow one. I can't recall what the names are > at the moment. I know the length of the connector tends to tell what > speed it is, tho some cheat and put long connectors but most of the > faster pins aren't used. This is actually pretty simple. PCIe is measured in lanes. There are no slower/faster pins. There are just lanes. A 4x slot has 4 lanes, and a 1x slot as 1 lane, and a 16x slot has 16 lanes. What you're talking about with "faster pins not being used" is something like a 16x slot with only 4 lanes wired. That behaves like a 4x slot, but lets you plug in physically larger cards. The missing 12 lanes aren't any faster than the 4 lanes that are wired, but obviously 4 lanes don't go as fast as 16 lanes. The other factor is PCIe generation. Each generation doubles the bandwidth, so a 1x PCIe v5 card with a supporting CPU is the same speed as a 16x PCIe v1 card. The interface runs at the maximum generation supported by both the card and the controller (located on the CPU these days). Most cards don't actually support recent generations - GPUs are the main ones that keep pace. I was talking about 10GbE NICs earlier, and if one supported a recent enough PCIe generation it could work fine in a 1x slot, but most use older generations and require a 4x slot or so. PCIe works fine if all the lanes aren't actually connected - you can plug a 16x GPU into a 1x riser, or a 1x slot that has an open notch on the end, and it will work fine. Though, in the latter case it will probably need physical support as the 16x slots have locks for large boards. The GPU will of course perform poorly with any kind of data transfer. > That confuses things. Anyway, mobo, which I > will likely change, CPU and memory is already adding up to about $600. If you're going to be spending THAT much on CPU+MB+RAM then I'd seriously look at how much moving to zen4 / AM5 costs. If you can get something cheap by going AM4 by all means do it, but if you aren't saving significant cash then you're buying into a much older platform. > I don't need much of a video card tho. Freeing up the 16x slot when you're so driven by PCIe requirements is a HUGE consideration here. > If someone knows of a good mobo, Gigabyte, ASUS preferred, that has > several PCIe slots, I'd like to know the model so I can check into it. I think you need to rethink your approach. Look, there is no reason you shouldn't be able to find a reasonably-priced motherboard that has lots of PCIe slots. If nothing else the manufacturer could stick a switch on the board, especially if you don't need PCIe v5 and don't mind the board switching the v5 lanes into a ton of v3-4 ones. However, nobody makes anything like that for consumers. There are chips out there that do some of that, but you'd have to custom-build something to use them. You really need to figure out how to get by with mostly 1x cards, and maybe 1-2 larger ones if you ditch the GPU. That is part of what drove me to distributed storage, and also using USB3 for large numbers of hard drives. PCs tend to have lots of unused USB3 capacity, and that works fine for spinning disks. It just looks ugly. (As a bonus the USB3 disks can often be obtained far cheaper.) -- Rich
Re: [gentoo-user] Controlling emerges
On Tue, Sep 19, 2023 at 5:48 AM Peter Humphrey wrote: > > On Tuesday, 19 September 2023 10:14:42 BST William Kenworthy wrote: > > That is where you set per package compiler parameters by overriding > > make.conf settings. > > And which make.conf setting might achieve what I want? Careful reading of the > make.conf man page hasn't revealed anything relevant. > There isn't one. At best there is -l which regulates jobs by system load, but there is nothing that takes into account RAM use. I just use package.env to limit jobs on packages that I know are RAM-hungry. Right now my list includes: calligra qtwebengine qtwebkit ceph nodejs passwdqc scipy pandas spidermonkey (It has been ages since I've pruned the list, and of course what is "too much RAM" will vary.) The other thing I will tweak is avoiding building in a tmpfs. Obviously anything that is RAM constrained is a good contender for not using a tmpfs, but there are also packages that just have really large build directories that otherwise don't need to much RAM when building. -- Rich
Re: [gentoo-user] Password questions, looking for opinions. cryptsetup question too.
On Tue, Sep 19, 2023 at 4:26 AM Michael wrote: > > On Tuesday, 19 September 2023 06:36:13 BST Dale wrote: > > Howdy, > > > A strong > password, like a strong door lock, buys you time. Hence the general > recommendation to change your passwords frequently. While that can help on websites, it is of no use for full disk encryption passwords - at least not without jumping through some big hoops. In order to crack your LUKS password somebody obviously needs to be able to read the encrypted contents of your disk. They cannot begin cracking it until they have a copy of the LUKS headers. However, once they do have it, they can make a copy and crack it at their leisure. If they manage to crack it, then it will give them the volume key. At that point if they were able to make a full copy of your disk they can read whatever was on it at the time. If they can make a fresh copy of your disk then changing the passphrase will not change the volume key, and so they'll be able to read what is currently on your disk. Changing the volume key would defeat this, but requires running cryptsetup-reencrypt which will take considerable time/CPU, though it sounds like it can be done online. > > In the real world tho, how do people reading this make passwords that no > > one could ever guess? You didn't ask this question, but I'll just note that most organizations don't use human-readable passwords to implement full disk encryption. The most commonly used solution is to use a TPM to measure the boot process and secure the disk encryption keys. If the system is booted normally, the bootloader can read the encryption keys from the TPM and can decrypt the disk without any user interaction (or even awareness it is happening). If the system is booted from alternative media, or the on-disk bootloader is tampered with, or even if the firmware is tampered with, then the TPM measurements will not agree with those used to store the key, and the TPM will not allow the keys to be read. This is how solutions like Bitlocker work. The components for this exist in the Linux world, but I'm not aware of any distro/etc actually implementing this with a pretty front-end, and there are obviously details that need to be carefully handled so that a bootloader or firmware update doesn't render your disk unreadable. Typically software implementations have ways to store "recovery keys" for these situations (just another copy of the disk key stored outside the TPM). > You can use gpg, or openssl, or app-admin/apg, or app-admin/pwgen, to generate > random enough strings to use as passwords. You might want to also consider app-admin/xkcdpass > > Also, I use cryptsetup luksFormat -s 512 ... to encrypt things. Is > > that 512 a good number? Can it be something different? I'd think since > > it is needed as a option, it can have different values and encrypt > > stronger or weaker. Is that the case? I've tried to find out but it > > seems everyone uses 512. If that is the only value, why make it a > > option? I figure it can have other values but how does that work? You can use a different size, but 512b is the recommended value for the default cipher. It is also the default I believe, so there isn't much point in passing it. Actually, I'd consider passing that parameter harmful unless you also specify the cipher. If in the future the default changes to some other cipher, perhaps 512b will no longer be appropriate, and you'll weaken it by specifying one and not the other. If you just want to trust the defaults, then trust the defaults. As to why 512b is the recommendation, that seems like it would require a LOT more reading. Apparently it is in an IEEE standard and I'd need to grok a lot more crypto to appreciate it. -- Rich
Re: [gentoo-user] Computer case for new build
On Mon, Sep 18, 2023 at 9:02 AM Frank Steinmetzger wrote: > > Am Mon, Sep 18, 2023 at 07:16:17AM -0400 schrieb Rich Freeman: > > > On Mon, Sep 18, 2023 at 6:13 AM Frank Steinmetzger wrote: > > > > > > Am Mon, Sep 18, 2023 at 12:17:20AM -0500 schrieb Dale: > > > because a NIC is going to need a 4-8x port > > most likely > > Really? PCIe 3.0 has 1 GB/s/lane, that is 8 Gbps/lane, so almost as much as > 10 GbE. I can't find any 10GbE NICs that use a 1x slot - if you can I'll be impressed. In theory somebody could probably make one that uses PCIe v4/5 or so, but I'm not seeing one today. If it needs more than a 1x slot, then it is all moot after that, as most consumer motherboards tend to have 1x slots, a 16x slot, and MAYBE a 4x slot in a 16x physical form. Oh, and good luck finding boards with an open end on the slot, even if there would be room to let a card dangle. My point with micro ATX was that with consumer CPUs having so few lanes available having room for more slots wouldn't help, as there wouldn't be lanes available to connect to them, unless you added a switch. That's something else which is really rare on motherboards. I don't get why they charge $250 for an AM5 motherboard, and maybe even have a switch on the X series ones, but they can't be bothered to give you larger slots. I can't imagine that all the lanes are busy all the time, so a switch would probably help quite a bit. > this is probably very restricted in > length. Which will also be the case for 10 GbE, so probably no options for > the outhouse. :D With an SFP+ port you can just use fiber and go considerable distances. That's assuming you're running network to your outhouse, and not bothering to put a switch in there (which would be more logical). > I have a four-bay NAS with server board (ASRock Rack E3C224D2I), actually my > last surviving Gentoo system. ;-) With IPMI-Chip (which alone takes several > watts), 16 GiB DDR3-ECC, an i3-4170 and 4×6 TB, it draws around 33..35 W > from the plug at idle — that is after I enabled all powersaving items in > powertop. Without them, it is around 10 W more. It has two gigabit ports > (plus IPMI port) and a 300 W 80+ gold PSU. That's an ITX system though, and a very old one at that. Not sure how useful more PCIe lanes are in a form factor like that. > > The advantage of > > distributed filesystems is that you can build them out of a bunch of > > cheap boxes […] > > For a simple media storage, I personally would find this too cumbersome to > manage. Especially if you stick to Gentoo and don’t have a homogeneous > device pool (not to mention compile times). I don't generally use Gentoo just to run containers. On a k8s box the box itself basically does nothing but run k8s. I probably only run about 5 commands to provision one from bare metal. :) -- Rich
Re: [gentoo-user] Controlling emerges
On Mon, Sep 18, 2023 at 12:13 PM Alan McKinnon wrote: > > Whether you just let emerge do it's thing or try get it to do big packages on > their own, everything is still going to use the same number of cpu cycles > overall and you will save nothing. That is true of CPU, but not RAM. The problem with large parallel builds is that for 95% of packages they're fine, and for a few packages they'll eat up all the RAM in the system until the OOM killer kicks in, or the system just goes into a swap storm (which can cause panics with some less-than-perfect kernel drivers). I'm not aware of any simple solutions. I do have some packages set to just build with a small number of jobs, but that won't prevent other packages from being built alongside them. Usually that is enough though. It is just frustrating to watch a package take all day to build because I can't use more than -j2 or so without running out of RAM, usually just at one step of the build process. I can't see anybody bothering with this, but in theory packages could have a variable to hint at the max RAM consumed per job, and the max number of jobs it will run. Then the package manager could take the lesser of -j and the max jobs the package can run, multiply it by the RAM requirement, and compare that to available memory (or have a setting to limit max RAM). Basically treat RAM as a resource and let the package manager reduce -j to manage it if necessary. Hmm, I guess a workaround would be to set ulimits on the portage user so that emerge is killed before RAM use gets too out of hand. That won't help complete builds, but it would at least keep it from killing the system. -- Rich
Re: [gentoo-user] Computer case for new build
On Mon, Sep 18, 2023 at 6:13 AM Frank Steinmetzger wrote: > > Am Mon, Sep 18, 2023 at 12:17:20AM -0500 schrieb Dale: > > […] > > The downside, only micro ATX and > > mini ITX mobo. This is a serious down vote here. > > Why is that bad? µATX comes with up to four PCIe slots. Even for ten drives, > you only need one SATA expander (with four or six on-board). Perhaps a fast > network card if one is needed, that makes two slots. Tend to agree. The other factor here is that desktop-oriented CPUs tend to not have a large number of PCIe lanes free for expansion slots, especially if you want 1-2 NVMe slots. (You also have to watch out as the lanes for those can be shared with some of the expansion slots so you can't use both.) If you want to consider a 10GbE+ card I'd definitely get something with integrated graphics, because a NIC is going to need a 4-8x port most likely (maybe there are expensive ones that use later generations and fewer lanes). On most motherboards you may only get one slot with that kind of bandwidth. > Speaking of RAM; might I interest you in server-grade hardware? The reason > being that you can then use ECC memory, which is a nice perk for storage. That and way more PCIe lanes. That said, it seems super-expensive, both in terms of dollars, and power use. Is there any entry point into server-grade hardware that is reasonably priced, and which can idle at something reasonable (certainly under 50W)? > > I was hoping to turn > > my current rig into a NAS. The mobo and such parts. This won't be a > > option with this case. Otherwise, it gives ideas on what I'm looking > > for. And not. ;-) > > I was going to upgrade my 9 years old Haswell system at some point to a new > Ryzen build. Have been looking around for parts and configs for perhaps two > years now but I can’t decide (perhaps some remember previous ramblings about > that). The latest zen generation is VERY nice, but also pretty darn expensive. Going back to zen3 might get you more for the money, depending on how big you're scaling up. A big part of the cost of zen4 is the motherboard, so if you're building something very high end where the CPU+RAM dominates, then zen4 may be a better buy. If you just want a low-core system then you're paying a lot just to get started. RE NAS: I used to build big boxes with lots of drives on ZFS. These days I'm using distributed filesystems (I've migrated from MooseFS to Ceph, though both have their advantages). The advantage of distributed filesystems is that you can build them out of a bunch of cheap boxes, vs trying to find one box that you can cram a dozen hard drives into. They're just much easier to expand. Plus you get host-level redundancy. Ceph is better for HA - I can literally reboot every host in my network (one at a time) and all my essential services stay running. MooseFS performs much better at small scale on hard drives, but depends on a master node for the FOSS version, so if that goes down the cluster is down (the locking behavior also seems to have issues - I've had corruption issues with sqllite and such with it). When you start getting up to a dozen drives the cost of getting them to all work on a single host starts going up. You need big cases, expansion cards, etc. Then when something breaks you need to find a replacement quickly from a limited pool of options. If I lose a node on my Rook cluster I can just go to newegg and look at $150 used SFF PCs, then install the OS and join the cluster and edit a few lines of YAML and the disks are getting formatted... > ¹ There was once a time when ECC was supported by all boards and CPUs. But > then someone invented market segmentation to increase profits through > upselling. Yeah, zen1 used to support ECC on most motherboards. Then later motherboards dropped support. Definitely a case of market segmentation. This is part of why I like storage implementations that have more robustness built into the software. Granted, it is still only as good as your clients, but with distributed storage I really don't want to be paying for ECC on all of my nodes. If the client calculates a checksum and it remains independent of the data, then any RAM corruption should be detectable as a mismatch (that of course assumes the checksum is preserved and not re-calculated at any point). -- Rich
Re: [gentoo-user] Re: attic
On Mon, Sep 4, 2023 at 4:38 AM William Kenworthy wrote: > > On 4/9/23 16:04, Nuno Silva wrote: > > > > (But note that Rich was suggesting using the *search* feature of the > > gitweb interface, which, in this case, also finds the same topmost > > commit if I search for "reedsolomon".) > > > tkx, missed that! Note that in terms of indexing git and CVS have their pros and cons, because they use different data structures. I've heard the saying that Git is a data structure masquerading as an SCM, and certainly the inconsistencies in the command line operations bear that out. Git tends to be much more useful in general, but for things like finding deleted files CVS was definitely more time-efficient. The reason for this is that everything in git is reachable via commits, and these are reachable from a head via a linked list. The most recent commit gives access to the current version of the repository, and a pointer to the immediately previous commit(s). To find a deleted file, git must go to the most recent commit in whatever branch you are searching, then descend its tree to look for the file. If it is not found, it then goes to the previous commit and descends that tree. There are 745k commits in the active Gentoo repository. I think there are something like 2M of them in the historical one. Each commit is a random seek, and then each step down the directory tree to find a file is another random seek. In CVS everything is organized first by file, and then each file has its own commit history. So finding a file, deleted or otherwise, just requires a seek for each level in the directory tree. Then you can directly read its history. So finding an old deleted file in the gentoo git repo can require millions of reads, while doing so in CVS only required about 3. It is no surprise that the web interfaces were designed to make that operation much easier - if you do sufficiently complex searches in the git web interface it will time you out to avoid bogging down the server, which is why some searches may require you to clone the repo and do it locally. Now, if you want to find out what changed in a particular commit the situation is reversed. If you identify a commit in git and want to see what changed, it can directly read the commit from disk using its hash. It then looks at the parent commit, then descends both trees doing a diff at each level. Since everything is content-hashed only directory trees that contain differences need to be read. If a commit had changes to 50 files, it might only take 10 reads to figure out which files changed, and then another 100 to compare the contents of each file and generate diffs. If you wanted to do that in CVS you'd have to read every single file in the repository and read the sequential history of each file to find any commits that have the same time/author. CVS commits also aren't atomic so ordering across files might not be the same. Git is a thing of beauty when you think about what it was designed to do and how well-suited to this design its architecture is. The same can be said of several data-driven FOSS applications. The right algorithm can make a huge difference... -- Rich
Re: [gentoo-user] attic
On Sun, Sep 3, 2023 at 4:44 AM Michael wrote: > > On Sunday, 3 September 2023 07:49:36 BST William Kenworthy wrote: > > Hi , I used to be able to get old ebuilds from "the attic" but I cant > > find it on google - is it still around? > > Perhaps have a look here at the archives? > > https://gitweb.gentoo.org/ The archives will only contain data migrated from CVS - so only things from more than a few years ago. You want to look into the main repo for anything recently deleted. > > * gentoo has moved dev-embedded/reedsolomon to dev-embedded/reedsolo > > (then removing the old ebuilds) breaking my homeassistant install > > easiest fix is a local copy until HA catches up. Both CVS and git maintain a record of anything that has been deleted, but they do it differently. The attic directory in CVS contains anything deleted from CVS. In git you need to search the commit history for these files. This can be done via the website, though the search capability is a little limited. I ended up having to search from a local clone because your package name contains an error and the web search found nothing. To find your file, go to: https://gitweb.gentoo.org/repo/gentoo.git/ Go to the search box in the top right and search for: dev-python/reedsolomon (note that the package category is different from what was in your email) Find the commit one commit before the one that removed your package. (ie one that contains your package in its most recent version) If you find the one that deleted your file, then just look at the parent in the commit header and click on that to go back one version where it is still present. Click the tree hash to browse the historical version of the repository that existed before your file was deleted. For example, you can find v1.6.1 of that package at: https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/reedsolomon-1.6.1.ebuild?id=149a131188ebce76a87fd8363fb212f5f1620a02 If the search function is too limiting on the website, here is how to do it from a local checkout. This is what I ended up doing since you had the wrong package name. Note that the first step here requires a few minutes and a few GB of space. My search example is also a bit broader than it would have to be, but you got the package category wrong and searching for "dev-embedded/reedsolomon" turned up nothing. git clone https://anongit.gentoo.org/git/repo/gentoo.git cd gentoo git log --all --full-history --raw --no-merges -- "**/reedsolomon/*" Then browse through the history for the file you're interested in. Suppose you want reedsolomon-1.6.1.ebuild. Easiest way to do that is to find a commit just before it was deleted, so just search the log for that filename. Ignore the first commit where it comes up, which is where the file was deleted (if you examine that commit the file will be missing, since it was deleted). Go to the next one. So reedsolomon-1.6.1.ebuild was deleted in commit beedaf82bd7abd72a654e26627774aef38590149. The next commit in the log search is 149a131188ebce76a87fd8363fb212f5f1620a02, git checkout 149a131188ebce76a87fd8363fb212f5f1620a02 cd dev-python/reedsolomon cat reedsolomon-1.6.1.ebuild You can also see this on the web interface at: https://gitweb.gentoo.org/repo/gentoo.git/tree/dev-python/reedsolomon/reedsolomon-1.6.1.ebuild?id=149a131188ebce76a87fd8363fb212f5f1620a02 The web git interface is capable of displaying past commits. It just can't search for wildcards/etc. -- Rich
Re: [gentoo-user] Migrate install from Intel 6th gen to AMD Zen 4
On Tue, Aug 29, 2023 at 6:22 AM Victor Ivanov wrote: > > My existing make.conf has: > > COMMON_FLAGS="-march=skylake -O2 -pipe" > CPU_FLAGS_X86="aes avx avx2 f16c fma3 mmx mmxext pclmul popcnt sse > sse2 sse3 sse4_1 sse4_2 ssse3" > > 1) Replace "-march=skylake" with "x86_64[-v1|v2|v3|v4]" or just "generic" > 4) Update to "-march=znver4" > 5) Update CPU_FLAGS_X86 with output of "$ cpuid2cpuflags" > > Am I missing anything critical that could break step (8) or any > packages I should include in step (2) in addition to @system to avoid > likelihood of segfaults? Well, you won't get segfaults so much as SIGILL, but I'd probably simplify a bit. I'm running on zen1 and these are my current flags: CFLAGS="-O2 -mtune=znver1 --param l1-cache-line-size=64 --param l1-cache-size=32 -pipe -funit-at-a-time -fstack-protector" I do have CPU_FLAGS_X86 set, but it seems like most of these are used by less critical packages, and I'm not sure how much trouble getting these wrong would be. As you can see I'm not even setting march. Maybe I could set it a little more aggressively and not risk any problems, but I'm concerned about the situation where I might have an unplanned CPU upgrade. My motherboard could die, and if it does I'd rather be able to just replace the motherboard+CPU and not have to fuss around with rebuilding my entire system from a rescue disk. I'm sure march performs better than mtune but I'd be surprised if the difference is more than a percent or two for most general purpose loads. If I had some high performance application that needed every clock cycle then I'd probably just build that one application with more optimizations. In general though your plan seems reasonable. Only thing I would probably consider is at least rebuilding all of @world with the unrestrictive -march setting. I might let it upgrade over time to the newer CPU optimizations, but having random stuff breaking until I rebuild @world seems like unnecessary pain. You say you're "thinking about upgrading" so it sounds like you aren't in a hurry and odds are you don't have the new hardware looking at you and begging you to boot it up. Doing a full @world rebuild is just a few cents worth of electricity and a day or two. That's how I'd look at it personally... -- Rich
Re: [gentoo-user] Encrypted swap with keyfile
On Sat, Jul 8, 2023 at 4:29 AM efeizbudak wrote: > > I use genkernel to make my initramfs and I am passing the crypt_swap, > crypt_swap_keydev and crypt_swap_key options but I'm guessing that since > /dev/sda2 comes before /dev/sda3, the initramfs tries to decrypt that one > first. How can I go around this? Is my only option to try and reorder these > partitions? > So, I've never really used genkernel, but this seems like the sort of thing that wouldn't be hard to do with dracut. If it doesn't already have native support for putting keyfiles in the initramfs, it would be pretty easy to create a module that does. Just loading the keyfile into the initramfs is trivial using install_items in the config file. You'd still need to change the logic to load it, or maybe do it via kernel command line. Perhaps genkernel has some way to add a file to the initramfs as well. -- Rich
Re: [gentoo-user] Systemd query ...
On Wed, May 17, 2023 at 6:18 AM Jacques Montier wrote: > > Well, well, Rich, you are completely right, you've found the key ! > I have that line in make.conf > INSTALL_MASK="/lib/systemd/system /usr/lib/systemd/system" > I now see where it comes from. > On the same machine, I have another OpenRC Gentoo with systemd masqued. > I just copîed the make.conf without uncommenting that line... How silly i am > !!! > So I delete that bl...y line ! > So, I realize this will be controversial, but this is why I don't make super-minimalistic builds. If I were trying to make a Gentoo build to run on a C64 or something and every last inode counted, then sure. However, things like text files simply don't do anything if nothing reads them. These days I also tend to be generous with building kernel modules - it slows down kernel builds, but it has no impact on running kernels if they aren't actually loaded. I also use -mtune these days and not -march. Sure, you lose a little performance, but if I lose a motherboard then I can just build a new PC, stick my hard drive in it, and it will just work. Now, if you're building disposable workers in some cluster that processes lots of jobs, then sure that extra few percent performance might be worth it, but then the individual hosts are all disposable anyway. Otherwise, I've found it is much better to optimize things for MY time than CPU time. -- Rich
Re: [gentoo-user] Systemd query ...
On Wed, May 17, 2023 at 4:43 AM Jacques Montier wrote: > > As I didn't mask anything, I don't understand why this file was not installed > as it was declared in the apache ebuild... You don't have anything set in INSTALL_MASK? Check "emerge --info www-servers/apache" You might want to check the build log for anything. I don't think there is anything conditional about systemd_newunit, and it is supposed to generate a fatal error if it fails. -- Rich
Re: [gentoo-user] Systemd query ...
On Tue, May 16, 2023 at 3:32 PM Jacques Montier wrote: > > After install, apache2.service not found... Have you done something to mask service file installs/etc? The unit file is in the gentoo repo: www-servers/apache/files/apache2.4-hardened.service -- Rich
Re: [gentoo-user] config file '/etc/mtab' needs updating
On Tue, Apr 11, 2023 at 9:47 AM Matt Connell wrote: > > I usually try not to edit any files that are 'managed' by packages, but > sometimes it is unavoidable (eg. no thing.conf.d directory support), so > I wind up having to either accept the change and then re-edit it, or > zap the change and allow the file to get stale as the package is > updated, leading back to the first scenario regardless. > Yeah, conf.d is a much better paradigm, but cfg-update dates back to when that wasn't popular, and of course there are still plenty of packages that don't support it today. Plus when you do need to manually edit a file you can do it in meld with 3-way diffs. -- Rich
Re: [gentoo-user] config file '/etc/mtab' needs updating
On Tue, Apr 11, 2023 at 9:14 AM Matt Connell wrote: > > On Mon, 2023-04-10 at 23:44 -0600, the...@sys-concept.com wrote: > > After update I get: > > * IMPORTANT: config file '/etc/mtab' needs updating. > > > > What is this, don't remember seeing it before. > > > > cfg-update -u > > doesn't give me an option to view it. > > > > > > dispatch-conf will show you what is being changed and give you the > option to use/zap the change. > > I never even knew cfg-update existed (I've always used dispatch-conf). > I'm guessing cfg-update doesn't understand symlinks, and obviously they can't be conventionally edited. cfg-update is a bit crufty, but its main advantage is support for 3-way merges, which are usually automated. So if you change one line in the middle of a config file you won't have to manually go through diffs to re-apply the change every time it is updated. If the section immediately around the line you edited didn't change, then it will just accept the upstream changes while maintaining your customization. It uses RCS, which is obviously dated. I'm maintaining it, so patches are welcome, but I'm not really putting any effort into it. As others have pointed out, it should be a symlink to /proc/self/mounts, as with the increasingly more popular use of mount namespaces there is no system-wide concept of what is/isn't mounted where. With the symlink each process will see what is actually mounted in its own namespace, avoiding leaking info from the host namespace, and also avoiding providing information that is incorrect from the process's perspective. -- Rich
Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card
On Mon, Mar 27, 2023 at 5:30 AM Wols Lists wrote: > > On 27/03/2023 01:18, Dale wrote: > > Thanks for any light you can shed on this. Googling just leads to a ton > > of confusion. What's true 6 months ago is wrong today. :/ It's hard > > to tell what still applies. > > Well, back in the days of the megahurtz wars, a higher clock speed > allegedly meant a faster CPU. Now they all run about 5GHz, and anything > faster would break the speed of light ... so how they do it nowadays I > really don't know ... Effective instructions per clock in general, and increasing core count are some of the big ones. I say effective because I think IPC is somewhat synthetic and idealized, and there are MANY bottlenecks in a CPU. Efficiency improvements that allow a CPU to boost for longer or increase sustained clock speed also help. Getting more done in a clock cycle can happen in many ways: 1. Actually reducing the number of cycles needed to complete an instruction at the most elementary level. 2. Using speculative execution to increase the number of instructions run in parallel. 3. Improving branch prediction to maximize your speculative execution budget. 4. Reducing the cost of prediction errors by reducing pipelines/etc. 5. Better cache to reduce wait time. 6. Better internal IO to reduce wait time. 7. Better external IO (esp RAM) to reduce wait time. Those are just ones I've thought of offhand. I'm sure there are tons of info on which ones matter the most in practice, and things I haven't thought of. -- Rich
Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card
On Mon, Mar 27, 2023 at 6:37 AM Frank Steinmetzger wrote: > > Bogomips seems to be vry simple, because it takes the current frequency > into account. So the number will be low when your PC idles and very high > when you compile something. The “bogo” stands for bogus for a reason. > Just to add to this, you need to also keep in mind its purpose. The kernel needs to be able to measure timings that wouldn't make sense to measure using a timer chip (lots of reasons for this). So it uses a timer chip to calibrate a delay loop. I don't know what instructions are being executed in the delay loop but the obvious design goal with a delay loop is maximum consistency with a short enough time per iteration that you have sufficient resolution. The BogoMIPS output is just telling you what the calibration factor was for each cycle of the loop. It is about as synthetic a benchmark as you can get, and it measures how quickly your CPU can execute code designed to do nothing more than waste time. > > Of course, only you can answer that in the end. Write down what you need and > what you care about. Weigh those factors. Then decide. Raw CPU power, > electricity bill, heat budget (cooling, noise, dust), the “new and shiny” > factor (like DDR5), and price. As I mentioned earlier, the 7xxx-X series are > hotheads. But when run with a lower power budget, they are very efficient > (which is basically what the non-X do). Are they actually hotheads on an energy consumed per unit of work basis? As you say, they're efficient. If the CPU has 2x the power draw, but does 2.5x as much work in a unit of time than the "cooler" CPU you're comparing it to, then actually doing any job is going to consume less electricity and produce less heat - it is just doing it faster. Max sustained power draw matters for cooling and electrical design (the latter being something users typically don't try to change). It isn't really a measure of thermal efficiency since that requires incorporating some measure of work getting done. A recent trend is upping the power draw of CPUs/GPUs to increase their throughput, but as long as efficiency remains the same, it creates some thermal headaches, but doesn't actually make the systems use more energy for a given amount of work. Of course if you throw more work at them then they use more energy. -- Rich
Re: [gentoo-user]Computer build, was PCIe x1 or PCIe x4 SATA controller card
On Thu, Mar 16, 2023 at 6:01 AM Frank Steinmetzger wrote: > > Gamer boards tend to skimp on ports, because those people generally care > mostly for their GPU (plus design and RGB). Well, that, and the CPU only has so many PCIe lanes and adding ports beyond that requires a switch. Also, if they have two 16x slots to allow for dual graphics cards those eat up quite a few of the lanes (even if one isn't actually 16x). > > Here, Linus is showcasing an 8-drives storage machine in a Fractal Define R4: > https://www.youtube.com/watch?v=DpJViwtct5g > And here a system with 18 drives 浪 in a Fractal Define 7XL: > https://www.youtube.com/watch?v=FAy9N1vX76o I think a lot of the consumer cases have been moving away from accommodating hard drives and making more room for gigantic GPUs. All that said, I have largely abandoned the crusade of trying to squeeze a dozen hard drives into one host, in favor of distributed filesystems. If you're only putting a few drives per host and having more hosts then it becomes pretty easy to find hardware that works. Finally, for any system that will be running 24x7 I'd suggest optimizing for power use per unit of computation (which is a hard figure to find), and idle power use (unless you actually do something that keeps the server busy 24x7). Usually newer processors will do better here. The up-front costs of a CPU are likely to be dwarfed by the cost of powering it. ARM is of course advantageous if you don't need too much horsepower or RAM. Unfortunately ARM boards with lots of RAM are crazy-expensive so it isn't a great option if you need more than a few GB. There has been interest in using mini PCs from corporate used sales as servers, and I'm thinking about building storage around a solution like this. The drives would need to be external of course, but USB3 is plenty fast for hard drives. However, it is hard to find easy to lookup metrics on power use and stats like USB3/etc - most filters on used product sale sites just have filters for RAM and maybe CPU. You do need to be careful as some of those systems could have high power draw or lack USB3 or even gigabit LAN, making them unsuitable for 24x7 storage. The price and form factor can be very attractive though, and power use still tends to be low since large companies do think about those costs. -- Rich -- Rich
Re: [gentoo-user] PCIe x1 or PCIe x4 SATA controller card
On Mon, Mar 13, 2023 at 8:24 AM Dale wrote: > > According to my google searches, PCIe x4 is faster > than PCIe x1. It's why some cards are PCIe x8 or x16. I think video > cards are usually x16. My question is, given the PCIe x4 card costs > more, is it that much faster than a PCIe x1? It could be slower than PCIe x1, because you didn't specify the version. PCIe uses lanes. Each lane provides a certain amount of bandwidth depending on the version in use. For example, a v1 4x card has 1 GB/s of bandwidth. A v4 1x card has 2GB/s of bandwidth. Note that slot size is only loosely coupled with the number of lanes. Lots of motherboards have a second 16x slot that only provides 4-8 lanes to save on the cost of a PCIe swich. You can also use adapters to connect a 16x card to a 1x slot, or you might find a motherboard that has an open-ended slot so that you can just fit a 16x card onto the 1x slot. It will of course only use a single lane that way. So what you need to do is consider the following: 1. How much bandwidth do you actually need? If you're using spinning disks you aren't going to sustain more than 200MB/s to a single drive, and the odds of having 10 drives using all that bandwidth are pretty low. If you're using SSDs then you're more likely to max them out since the seek cost is much lower. 2. What PCIe version does your motherboard support? Sticking a v4 card on an old motherboard that only supports v2 is going to result in it running at v2 speeds, so don't pay a premium for something you won't use. Likewise, if they cut down on the number of lanes assuming they'll have more bandwidth you might have less than you expected to have. Then look up the number of lanes and the PCIe version and see what you can expect: https://en.wikipedia.org/wiki/PCI_Express#History_and_revisions I think odds are you aren't going to want to pay a premium if you're just using spinning disks. If you actually wanted solid state storage then I'd also be avoiding SATA and trying to use NVMe, though doing that at scale requires a lot of IO, and that will cost you quite a bit. There is a reason your motherboard has mostly 1x slots - PCIe lanes are expensive to support. On most consumer motherboards they're only handled by the CPU, and consumer CPUs are very limited in how many they offer. Higher end motherboards may have a switch and offer more lanes, but they'll still bottleneck if they're all maxed out getting into the CPU. If you buy a server CPU for several thousand dollars one of the main features they offer is a LOT more PCIe lanes, so you can load up on NVMes and have them running at v4-5. (Typical NVMe uses a 4x M.2 slot, and of course you can have 16x cards offering multiples of those.) The whole setup is pretty analogous to networking. If you have a computer with 4 network ports you can bond them together and run them to a switch that supports this with 4 cables, and get 4x the bandwidth. However, you can also get a single connection to run at higher speeds (1Gb, 2.5Gb, 10Gb, etc), and you can do both. PCIe lanes are just like bonded network cables - they are just pairs of signal wires that use differential signaling, just like twisted pairs in an ethernet cable. Longer slots just add more of them. Everything is packet switched, so if there are more lanes it just spreads the packets across them. Higher versions mean higher speeds in each lane. -- Rich
Re: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting
On Thu, Feb 16, 2023 at 9:08 AM John Covici wrote: > > hmmm, but what should I use for the source ip, I only assign those > when I bring the interface up when I start the interface -- I have > something like this: > [Unit] > Description=Network Connectivity for %i > ... > So, before I run this, I don't think the card has any ip address, does > it? So, "cards" don't have an IP address. The kernel assigns an IP address to an interface, which is entirely a software construct. It happens to be a software construct that the network console feature largely ignores anyway. I didn't go reading the source code, but I'm guessing it is just constructing raw UDP packets and it will happily set the IP to whatever you want it to be. After all, it is just a field on the packet. So you can make the source IP whatever you want it to be. Just expect the packets to show up with the IP you set on them. There is no connection, so the IP doesn't need to be reachable by anything else. You could stick literally anything in there as long as some firewall isn't going to object and drop the packet. The destination IP matters because that is where it is going to go, and the interface matters because if it gets sent out on the wrong interface then obviously it won't make it there. I have no idea if the netconsole packets get seen by netfilter, but if this is on some kind of router that might be something you need to check, because if netfilter is configured to drop unassociated UDP from the firewall to the LAN that could be an issue. However, it is possible this just bypasses netfilter entirely. If you have the dynamic netconsole option enabled you could have a script update the settings after your network is up to set the source IP to the one assigned by DHCP and make sure it is on the right interface. As you point out though at boot time the interface won't have an IP. It won't even be "up," not that this is likely to bother the kernel. -- Rich
Re: [gentoo-user] Jobs and load-average
On Thu, Feb 16, 2023 at 8:39 AM Peter Humphrey wrote: > > I've just looked at 'man make', from which it's clear that -j = --jobs, and > that both those and --load-average are passed to /usr/bin/make, presumably > untouched unless portage itself has identically named variables. So I wonder > how feasible it might be for make to incorporate its own checks to ensure that > the load average is not exceeded. I am not a programmer (not for at least 35 > years, anyway), so I have to leave any such suggestion to the experts. > Well, if we just want to have a fun discussion here are my thoughts. However, the complexity vs usefulness outside of Gentoo is such that I don't see it happening. For the most typical use case - a developer building the same thing over and over (which isn't Gentoo), then make could cache info on resources consumed, and use that to make more educated decisions about how many tasks to launch. That wouldn't help us at all, but it would help the typical make user. However, the typical make user can just tune things in other ways. It isn't going to be possible for make to estimate build complexity in any practical way. Halting problem aside maybe you could build in some smarts looking at the program being executed and its arguments, but it would be a big mess. Something make could do is tune the damping a bit. It could gradually increase the number of jobs it runs and watch the load average, and gradually scale it up appropriately, and gradually scale down if CPU is the issue, or rapidly scale down if swap is the issue. If swapping is detected it could even suspend most of the tasks it has spawned and then gradually continue them as other tasks finish to recover from this condition. However, this isn't going to work as well if portage is itself spawning parallel instances of make - they'd have to talk to each other or portage would somehow need to supervise things. A way of thinking about it is that when you have portage spawning multiple instances of make, that is a bit like adding gain to the --load-average MAKEOPTS. So each instance of make independently looks at load average and takes action. So you have an output (compilers that create load), then you sample that load with a time-weighted average, and then you apply gain to this average, and then use that as feedback. That's basically a recipe for out of control oscillation. You need to add damping and get rid of the gain. Disclaimer: I'm not an engineer and I suspect a real engineer would be able to add a bit more insight. Really though the issue is that this is the sort of thing that only impacts Gentoo and so nobody else is likely to solve this problem for us. -- Rich
Re: [gentoo-user] Jobs and load-average
On Thu, Feb 16, 2023 at 5:32 AM Andreas Fink wrote: > > On Thu, 16 Feb 2023 09:53:30 + > Peter Humphrey wrote: > > > Yes, I was aware of that, but why didn't --load-average=32 take precedence? > This only means that emerge would not schedule additional package job > (where a package job means something like `emerge gcc`) when load > average > 32, howwever if a job is scheduled it's running, independently > of the current load. > While having it in MAKEOPTS, it would be handled by the make system, > which schedules single build jobs, and would stop scheduling additional > jobs, when the load is too high. > > Extreme case: > emerge chromium firefox qtwebengine > --> your load when you do this is pretty much close to 0, i.e. all 3 > packages are being merged simultaneously and each will be built with > -j16. > I.e. for a long time you will have about 3*16=48 single build jobs > running in parallel, i.e. you should see a load going towards 48, when > you do not have anything in your MAKEOPTS. TL;DR - the load-average option results in underdamping, as a result of the delay with the measurement of load average. Keep in mind that load averages are averages and have a time lag, and compilers that are swapping like crazy can run for a fairly long time. So you will probably have fairly severe oscillation in the load if swapping is happening. If your load is under 32, each of your 16 parallel makes, even if running with the limit in MAKEOPTS, will feel free to launch another 256 jobs, because it will take seconds for the 1 minute load average to creep above 32. At that point you have WAY more than 32 tasks running and if they're swapping then half of the processes on your system are probably going to start blocking. So now make (if configured in MAKEOPTS) will hold off on launching anything, but it could take minutes for those swapping compiler jobs to complete the amount of work that would normally take a few seconds. Then as those processes eventually start terminating (assuming you don't get OOM killing or PANICs) your load will start dropping, until eventually it gets back below 32, at which point all those make processes that are just sitting around will wake up and fire off another 50 gcc instances or whatever they get up to before the brakes come back on. The load average setting is definitely useful and I would definitely set it, but when the issue is swapping it doesn't go far enough. Make has no idea how much memory a gcc process will require. Since that is the resource likely causing problems it is hard to efficiently max out your cores without actually accounting for memory use. The best I've been able to do is just set things conservatively so it never gets out of control, and underutilizes CPU in the process. Often it is only parts of a build that even have issues - something big like chromium might have 10,000 tasks that would run fine with -j16 or whatever, but then there is this one part where the jobs all want a ton of RAM and you need to run just that one part at a lower setting. -- Rich
Re: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting
On Thu, Feb 16, 2023 at 6:50 AM John Covici wrote: > > The sending computer has two nics, eno1 for the internal network and > eno2 is on the internet. So, my netconsole stanza said > netconsole=@192.168.0.1/eno1,@192.168.0.2 Is CONFIG_NETCONSOLE enabled for your kernel? I'm not sure if the kernel will assign the names eno1/2 to interfaces - I think those might be assigned by udev, which probably won't have run before the kernel parses this instruction. You might need to use eth0/1 - and your guess is as good as mine which one corresponds to which. If it isn't one of those it might not hurt to put the target mac address in there just to be safe. I haven't needed that but maybe there are situations where ARP won't work (it would be needed if you are crossing subnets, in which case you'd need the gateway MAC). Keep in mind that this is a low-level function that doesn't use any routing/userspace/etc. It was designed to be robust in the event of a PANIC and to be able to be enabled fairly early during boot, so it can't rely on the sorts of things we just take for granted with networking. > > The box which is at 192.168.0.2 has netcat (windows version) and I > tried the following: > netcat -u -v -l 192.168.0.2 and I also tried 192.168.0.1 > which is the ip address of the linux console which I am trying to > debug. > > I also tried 0.0.0.0 which did not work either, but I think the > windows firewall was blocking, and I did fix that, but did not try the > 0.0.0.0 after that. > So I'm pretty sure that netcat requires listing the destination IP, since it has to open a socket to listen on that IP. You can optionally set a source address/port in which case it will ignore anything else, but by default it will accept packets from any source. I was definitely going to suggest making sure that a windows firewall wasn't blocking the inbound connections. That's fairly default behavior on windows. -- Rich
Re: [gentoo-user] Is it OK to get rid of app-alternatives/* ?
On Wed, Feb 15, 2023 at 9:10 AM Michael Orlitzky wrote: > > On 2023-02-15 08:11:46, Neil Bothwick wrote: > > > > If, as you say, it will eventually replace eselect, there is no more > > bloat, just different bloat. It's still just a bunch of symlinks, but > > managed differently. > > > > Should be less, since you already have portage installed but not > necessarily eselect-whatever. > The symlinks are all associated with packages as well, which means that when you uninstall things that will get rid of the symlinks as well. This is really just a best practice all-around. I have a Gentoo system I've been maintaining for a while and I occasionally find orphaned stuff poking around because of special cases of things that weren't managed by the package manager, and so when things were obsoleted they stuck around. The news is needed precisely because the migration involves having the package manager install a bunch of stuff over files not owned by any package. That triggers a warning, but only because the files were in a less than ideal state to start. Things like this and the new user/group packages also reduce the complexity of dependency management because they just turn everything into a package dependency. Less special cases. -- Rich
Re: [gentoo-user] Jobs and load-average
On Wed, Feb 15, 2023 at 4:56 AM Peter Humphrey wrote: > > Not long ago I read that we should allow 2GB RAM for every emerge job - that > is, we should divide our RAM size by 2 to get the maximum number of > simultaneous jobs. I'm trying to get that right, but I'm not there yet. > > I have these entries in make.conf: > EMERGE_DEFAULT_OPTS="--jobs=16 --load-average=32 --autounmask=n --quiet- > unmerge-warn --ke> > MAKEOPTS="-j16" > > Today, though, I saw load averages going up to 72. Can anyone suggest better > values to suit my 24 threads and 64GB RAM? First, keep in mind that --jobs=16 + -j16 can result in up to 256 (16*16) tasks running at once. Of course, that is worst case and most of the time you'll have way less than that. Keep in mind that you need to consider available RAM and not just total RAM. Run free under the conditions where you typically run emerge and see how much available memory it displays. Depending on what you have running it could be much lower than 64GB. Beyond that, unfortunately this is hard to deal with beyond just figuring out what needs more RAM and making exceptions in package.env. Also, RAM pressure could also come from the build directory if it is on tmpfs, which of course many of us use. Some packages that I build with either a greatly reduced -j setting or a non-tmpfs build directory are: sys-cluster/ceph dev-python/scipy dev-python/pandas app-office/calligra net-libs/nodejs dev-qt/qtwebengine dev-qt/qtwebkit dev-lang/spidermonkey www-client/chromium app-office/libreoffice sys-devel/llvm dev-lang/rust (I use the rust binary these days as this has gotten really out of hand) x11-libs/gtk+ These are just packages I've had issues with at some point, and it is possible that some of these packages no longer use as much memory today. -- Rich
Re: [gentoo-user] my 5.15.93 kernel keeps rebooting
On Tue, Feb 14, 2023 at 2:54 PM John Covici wrote: > > On Tue, 14 Feb 2023 14:08:34 -0500, > Rich Freeman wrote: >> > > will be displayed on the console briefly. You can also enable a > > network console, which will send the dmesg output continuously over > > UDP to another device. > > OK, how would I set up logging to a network and what would I have to > do on another computer -- which in my case is Windows? The docs are at: https://www.kernel.org/doc/Documentation/networking/netconsole.txt (you can also google for linux netconsole for some wiki articles on it) I have on my command line: netconsole=@/,@10.1.0.52 That IP is the host I want the log traffic to go to. (Read the docs if you have a more complicated networking setup - I assume that will just run ARP and send stuff out without using a gateway/etc.) Then on a receiving linux host I'd run (I think - it has been a while): nc -u -l -p Now, you mentioned Windows. I've never used it, but nmap has a program available in a windows version called ncat that might do the job: https://nmap.org/ncat/ You just want to make sure you have it listening on port for UDP. Make sure you use UDP or you won't receive anything. If it is working you should get a ton of log spam when your host boots - anything that shows up in dmesg will show up in the network console. It is sent in realtime. -- Rich
Re: [gentoo-user] my 5.15.93 kernel keeps rebooting
On Tue, Feb 14, 2023 at 9:08 AM John Covici wrote: > > Hi. So, foolish me, I decided to go from a working 5.10.155 system to > try latest lts of 5.15 which is 5.15.93. Compile, install went well, > but the system keeps rebooting. It gets all the way and even starts > the local services and then here are the last few lines, which may be > relevant or not: > > Feb 14 06:36:31 ccs.covici.com systemd[1]: Starting local.service... > Feb 14 06:36:31 ccs.covici.com systemd[1]: Starting > systemd-update-utmp-runlevel.service... > Feb 14 06:36:31 ccs.covici.com bash[5753]: rm: cannot remove > '/etc/ppp/provider_is_up': No such file or directory > Feb 14 06:36:31 ccs.covici.com systemd[1]: > systemd-update-utmp-runlevel.service: Deactivated successfully. > Feb 14 06:36:31 ccs.covici.com systemd[1]: Finished > systemd-update-utmp-runlevel.service. > -- Boot 5c394be675854680a9cb616208f374f3 -- > > Any trouble shooting suggestions as to what is making the system > reboot? > Where are you getting this from, the system log/journal? This doesn't seem like a clean shutdown, so if it is a kernel PANIC I wouldn't expect the most critical info to be in the log (since it will stop syncing to protect the filesystem). The details you need probably will be displayed on the console briefly. You can also enable a network console, which will send the dmesg output continuously over UDP to another device. This won't be interrupted by a PANIC unless there is some issue with the hardware or networking stack. If you can get the final messages on dmesg and the panic core dump that would help. The other thing you can do is try to capture a kernel core dump, but that is a bit more complicated to set up. Otherwise your log is just going to say that everything was fine until it wasn't. -- Rich
Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives
On Mon, Dec 19, 2022 at 11:43 AM Mark Knecht wrote: > > On Mon, Dec 19, 2022 at 6:30 AM Rich Freeman wrote: > > > My current solution is: > > 1. Moosefs for storage: amd64 container for the master, and ARM SBCs > > for the chunkservers which host all the USB3 hard drives. > > I'm trying to understand the form factor of what you are mentioning above. > Presumably the chunkservers aren't sitting on a lab bench with USB > drives hanging off of them. Can you point me toward and example of > what you are using? Well, a few basically are just sitting on a bench, but most are in half-decent cases (I've found that Pi4s really benefit from a decent case as they will thermal throttle otherwise). I then have USB3 hard drives attached. I actually still have one RockPro64 with an LSI HBA on a riser card but I'm going to be moving those drives to USB3-SATA adapters because dealing with the kernel patches needed to fix the broken PCIe driver is too much fuss, and the HBA uses a TON of power which I didn't anticipate when I bought it (ugh, server gear). Really at this point for anything new 2GB Pi4s are my preferred go-to with Argon ONE v2 cases. Then I just get USB3 hard drives on sale at Best Buy for ~$15/TB if possible. USB3 will handle a few hard drives depending on how much throughput they're getting, but this setup is more focused on capacity/cost than performance anyway. The low memory requirement for the chunkservers is a big part of why I went with MooseFS instead of Ceph. The OSDs for Ceph recommend something like 4GB per hard drive which adds up very fast. The USB3 hard drives do end up strewn about a fair bit, but they have their own enclosures anyway. I just label them well. > > I've been considering some of these new mini-computers that have > a couple of 2.5Gb/S Ethernet ports and 3 USB 3 ports but haven't > moved forward because I want it packaged in a single case. Yeah, better ethernet would definitely be on my wish list. I'll definitely take a look at the state of those the next time I add a node. > Where does the master reside? In a container on your desktop > machine or is that another element on your network? In an nspawn container on one of my servers. It is pretty easy to set up or migrate so it can go anywhere, but it does benefit from a bit more CPU/RAM. Running it in a container creates obvious dependency challenges if I want to mount moosefs on the same server - that can be solved with systemd dependencies, but it won't figure that out on its own. > > 2. Plex server in a container on amd64 (looking to migrate this to k8s > > over the holiday). > > Why Kubernetes? I run it 24x7. This is half an exercise to finally learn and grok k8s, and half an effort to just develop better container practices in general. Right now all my containers run in nspawn which is actually a very capable engine, but it does nothing for image management, so my containers are more like pets than cattle. I want to get to a point where everything is defined by a few trivially backed-up config files. One thing I do REALLY prefer with nspawn is its flexibility around networking. An nspawn container can use a virtual interface attached to any bridge, which means you can give them their own IPs, routes, gateways, VLANs, and so on. Docker and k8s are pretty decent about giving containers a way to listen on the network for connection (especially k8s ingress or load balancers), but they do nothing really to manage the outbound traffic, which just uses the host network config. On a multi-homed network or when you want to run services for VLANs and so on it seems like a lot of trouble. Sure, you can go crazy with iptables and iproute2 and so on, but I used to do that with non-containerized services and hated it. With nspawn it is pretty trivial to set that stuff up and give any container whatever set of interfaces you want bridged however you want them. I actually fussed a little with running a k8s node inside an nspawn container so that I could just tie pods to nodes to do exotic networking but clusters inside containers (using microk8s which runs in snapd) was just a bridge too far... -- Rich
Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives
On Mon, Dec 19, 2022 at 7:51 AM Wols Lists wrote: > > On 19/12/2022 12:00, Rich Freeman wrote: > > On Mon, Dec 19, 2022 at 12:11 AM Dale wrote: > >> If I like these Raspberry things, may make a media box out of one. I'd > >> like to have a remote tho. > > > So, I've done that. Honestly, these days a Roku is probably the > > better option, or something like a Google Chromecast or the 47 other > > variations on this them. > > Where do you put that 2TB drive on your Roku or Chromecast? > > I'm thinking of building a media server, not to drive the TV, but to > record and store. I thought that was what a media server was! So, he said "media box," which I assumed meant the client that attaches to the TV. There are some canned solutions for media servers - I think the NVidia Shield can run Plex server for example. However, in general server-side I'd go amd64. My current solution is: 1. Moosefs for storage: amd64 container for the master, and ARM SBCs for the chunkservers which host all the USB3 hard drives. With a modest number of them performance is very good, though certainly not as good as Ceph or local storage. (I do have moosefs in my overlay - might try to get that into the main repo when I get a chance.) 2. Plex server in a container on amd64 (looking to migrate this to k8s over the holiday). 3. Rokus or TV apps for the clients. -- Rich
Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives
On Mon, Dec 19, 2022 at 12:11 AM Dale wrote: > > If I like these Raspberry things, may make a media box out of one. I'd > like to have a remote tho. ;-) So, I've done that. Honestly, these days a Roku is probably the better option, or something like a Google Chromecast or the 47 other variations on this them. Keep in mind that low-powered devices like ARM SBCs (or Rokus) are very picky about codecs. If you're running something like Plex the software will offload the transcoding to a server when the codec isn't supported by the player, but if you're doing something more DIY then you need to ensure your library conforms to the specs or your player will just die. Even 1080p can be pushing it for software decoding on an ARM SBC, and forget 4k. Heck, realtime software transcoding 4k is pretty hard on most amd64 servers as well - they will do it but it will use a significant amount of CPU just for one stream. These days I'm using Pis just for moosefs storage, and all the compute-heavy stuff is on amd64. For players I just use TV clients or Rokus plus Plex server on amd64. I don't really get any benefit out of completely DIYing the client side and LIRC is a PITA. -- Rich
Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives
On Sun, Dec 18, 2022 at 4:53 PM Dale wrote: > > It took some digging around but I found out it is a AMD Phenom 9750 quad > core. You might want to hook a Kill-a-watt to that and see how much power it uses. Those older AMD processors were really inefficient. One thing I've come to appreciate with anything that runs 24x7 is to consider the energy bill to run things. That's why the bulk of my storage is running on ARM SBCs. Those things use almost no power when idle, while even a modern amd64 system will often use more at idle than an ARM will at full load. Most of the newer CPUs are MUCH better in performance per watt as well. A server that uses just 50W 24x7 costs about $65/yr (and I have pretty cheap electricity due to a 1yr contract - odds are it costs quite a bit more for most of this list). That's a fairly significant amount of money so it can pay to optimize things, especially if you replace what was a power-hungry performance CPU from 10 years ago with a lower-end CPU from today that probably has double the performance for 10% of the power draw. Obviously if you want bleeding edge the electricity bill won't cover the up-front costs, but lower end CPUs are very cost-effective and efficient compared to really old systems that people tend to use for these kinds of projects. -- Rich
Re: Living in NGL: was: [gentoo-user] NAS and replacing with larger drives
On Sun, Dec 18, 2022 at 2:30 PM Frank Steinmetzger wrote: > > Am Sun, Dec 18, 2022 at 01:07:43PM -0600 schrieb Dale: > > > Mostly, I need a better CPU. If I encrypt anyway. > > Did you ever tell us the exact CPU you have in there? All I can remember is > it has 4 cores. And some AMD processor with a II in its name, but that was > you main rig, right? What encryption algorithm are you using? You should see if this is hardware-accelerated in the kernel for your CPU, or if not if there is another strong algorithm which is. Most newer CPUs will tend to have hardware support for algorithms like AES, and the kernel will use this. This will greatly improve CPU performance. I've run into this issue with zfs on Raspberry Pis. ZFS does the encryption internally, and the openzfs code didn't have support for ARM hardware encryption the last time I checked (this could have changed). I found that dm-crypt works MUCH better on Pis as a result, as the kernel does have ARM encryption hardware support. Again, this all depends on the algorithm. If you're using something exotic odds are the hardware won't handle it natively. -- Rich
Re: [gentoo-user] NAS and replacing with larger drives
On Fri, Dec 9, 2022 at 8:13 AM Michael wrote: > > Actually this had me thinking what is the need to back up the ... Internet? I'm sure the NSA knows the answer to this. Based on discussions I've had with people who are into such things they basically have their own Wayback machine, except it obviously doesn't respect robots.txt or takedown requests. I kind of wish the NSA sold IT services to the general public. I just assume they probably have root on all my devices and their own backups of everything on them. It would be nice if I had a disaster if I could just pay them to buy back a copy of my data, instead of having to have my own completely redundant backups. I'm personally using duplicity for encrypted cloud backups of the stuff that is most critical (documents, recent photos, etc), AWS Glacier for stuff I want long-term backups of (older photos mostly), and then bacula to store local copies of everything I have any interest in because that is easier than trying to restore it all off of Amazon if I lose an array or whatever. AWS Glacier is actually pretty cheap for backup, but be prepared to pay a fair bit for restoration. I'd only need to go to them in a serious disaster like a house fire, so having to pay $100 or whatever to get them to mail me a hard drive with my data isn't really that big of a deal. My backups are generally one-way affairs. -- Rich
Re: [gentoo-user] NAS and replacing with larger drives
On Thu, Dec 8, 2022 at 6:30 PM Dale wrote: > > One thing I like about the Raspberry option, I can upgrade it later. I > can simply take out the old, put in new, upgrade done. If I buy a > prebuilt NAS, they pretty much are what they are if upgrading isn't a > option. Some of the more expensive ones may be upgradable, maybe. The NAS gets you a nice box. The nice box means fixed capacity. I just use USB3 external hard drives. They're cheaper and easy to interface. USB3 also has been less likely to give me ATA interface errors compared to SATA. > I just wonder, could I use that board and just hook it to my USB port > and a external power supply and skip the Raspberry Pi part? I'd bet not > tho. ;-) Not that one, but USB3-SATA interfaces exist and aren't that expensive. You can also get nice little enclosures. You can have as many hard drives as you want on a PC that way, or whatever the USB3 limit is. -- Rich
Re: [gentoo-user] NAS and replacing with larger drives
On Thu, Dec 8, 2022 at 11:56 AM Laurence Perkins wrote: > > Pine64 has an interesting array of SBCs which are both cheaper and (some are) > possibly better suited to becoming a NAS than a Pi. One of them even has a > PCIe socket I think. > I have the RockPro64 and I'll go ahead and warn you that you'll need to patch the kernel if you want to use many PCIe cards. I just tested out a debian image on it and the several year old patch still hasn't made its way into there. The SATA expansion card Pine64 was selling at the time did work out of the box. https://github.com/rockchip-linux/kernel/issues/116 -- Rich -- Rich
Re: [gentoo-user] NAS and replacing with larger drives
On Thu, Dec 8, 2022 at 8:59 AM Frank Steinmetzger wrote: > > You could, but this is either a sink-hole for time, or you need to get up to > speed with cross-compiling and binhosts. I went with the standard Debian and > evaluate Arch from time to time. But I do run Gentoo on my DIY NAS with an > i3-2000. Gentoo has ZFS in portage without overlays, which–for me–is one of > its biggest appeals. ++ Obviously I'm a huge Gentoo fan, but on an ARM SBC unless you're either experimenting or you actually intend to be patching or reconfiguring packages the precompiled option is the way to go. When I'm using less-popular SBCs (ie not Pis) then I will usually look for whatever distros are supporting it in the most first-class way, again, unless I'm experimenting. Then I look for what has the software I need already packaged (again, check the arch because a binary package repo doesn't necessarily include your device, especially if it is 3rd party). I've had to compile things on ARM SBCs and it is SLOW. I have the same philosophy with containers. If I'm just running a service, and not tweaking things, I'll just pick the least-fuss base for my container whatever that is. -- Rich
Re: [gentoo-user] NAS and replacing with larger drives
On Thu, Dec 8, 2022 at 7:37 AM Dale wrote: > > Path two, I've researched building a NAS using a Raspberry Pi 4 8GB as > another option. They come as parts, cases too, but the newer and faster > models of Raspberry Pi 4 with more ram seem to work pretty well. For this sort of application the key improvement of the Pi4 over its predecessors is IO. The Pi4 has USB3 and gigabit ethernet, and they are independent, so you get the full bandwidth of both (in theory). That is a massive step up over USB2 and 100Mbps ethernet that consumes the USB2 bandwidth. I can't really speak to the commercial solutions as I haven't used them. Main concern there is just the limited capacity, lack of expandability, and so on. Some are no doubt better than others in those regards. As far as DIY goes, you can definitely do all of that with a Pi4. Don't expect it to perform as well as sticking it on a decent amd64 motherboard, but for backup and saturating the throughput of 1 hard drive at a time it can probably mostly make do. Encryption can be accomplished either with cryptsetup or a filesystem that has native encryption like ZFS. I've done both on Pi4s for storage. I will warn you that zfs encryption is not hardware-optimized on ARM, so that will not perform very well - it will be completely functional, but you will get CPU-bound. Linux-native encryption (ie cryptsetup/LUKS) will use hardware capabilities on the Pi4, assuming you're using something it supports (I think I'm using AES which performs adequately). For the Pi4 you would need to use USB storage, but for hard drives IMO this is perfectly acceptable, especially on a Pi. The gigabit ethernet and internal IO of the Pi is only going to max out one hard drive no matter how you connect it, so the USB3 interface will not be a bottleneck. On ARM SBCs that have PCIe you don't really get any better performance with an HBA and SATA/SCSI simply because the board IO is already pretty limited. USB3 is actually pretty fast for spinning disks, but depending on the number of hosts/etc it could become a bottleneck on a decent motherboard with a large number of drives. If you're talking about an amd64 with a 10GbE NIC and a decent HBA with sufficient PCIe lanes for both then obviously that is going to saturate more spinning disks. For NVMe you absolutely need to go that route (probably need to consider server-class hardware too). I use USB3 hard drives on Pis for my bulk storage because I care about capacity far more than performance, and with a distributed filesystem the performance is still good enough for what I'm doing. If I needed block storage for containers/VMs/whatever then use a different solution, but that gets expensive fast. Oh, one other thing. One of your issues is that you're using a backup solution that just dumps everything into a single file/directory and requires all the backup storage to be mounted at the same time in a single filesystem. There are solutions that do not have this requirement - particularly ones that are adaptable to tape. Unfortunately the best FOSS option I've found for this on linux is bacula and that is a serious PITA to use. If anybody has a better one I'm all ears (the requirement is to be able to store a backup across multiple hard drives, and this can't involve first storing it all in one place and then splitting it up later, or having more than one storage drive attached at the same time - basically I want to treat hard drives like tapes). If you're storing a LOT of backups then LTO is another option. Every time I do the math on that option it never makes sense unless you're backing up a LOT of data. If you got to a point where your backups consumed 10+ max-capacity hard drives it might start to make sense. Those USB3 hard drives on sale for $15/TB though are just really hard to beat when the tapes aren't all that much cheaper and the drives cost $1k. -- Rich
Re: [gentoo-user] Kernel 6.0 upgrade broke sound to TV over HDMI
On Fri, Nov 25, 2022 at 11:40 PM Dale wrote: > > I found a new version of the nvidia-drivers. I figured it might work > with the new 6.0 kernel so I tested it. Sure enough, it compiled and > installed. And this is why compiling isn't evidence that something works. :) What nvidia driver version are you using? Also, why are you using it at all? The proprietary drivers their pros and cons. The issues you're running into are one of the well-known cons - if you want to run stable kernels, they'll be occasionally breaking, and if you run mainline kernels they'll break all the time. If you're going to run out-of-tree kernels you should almost certainly stick to longterm kernels. Usually people use the proprietary drivers for gaming, high-performance 3D graphics in general, or CUDA (AI, video encoding, etc). If you aren't doing one of those then I suspect the linux-supplied drivers will just work and shouldn't ever break. You can run stable kernels all you want with those. -- Rich
Re: [gentoo-user] Upgrading from 5.14 to 6.0 version
On Mon, Nov 21, 2022 at 1:10 AM Dale wrote: > > I'm back to my old kernel tho since my nvidia-drivers won't work with a > kernel that high. I run into this on rare occasions. They are only rare because you aren't updating regularly. If you want to run external kernel modules like nvidia-drivers or zfs, stick to a longterm kernel. The ABI changes all the time, and so there will frequently be stable kernel version changes that break nvidia-drivers. Then there will be a lag before nvidia-drivers supports the new stable kernel. In the meantime you can't run the latest version, and that can mean security issues. The longterm kernels rarely break nvidia-drivers and get all the security and other fixes. They just don't get new features. -- Rich
Re: [gentoo-user] Re: Upgrading from 5.14 to 6.0 version
On Sat, Nov 12, 2022 at 2:13 PM Wol wrote: > > The idea behind stable kernels is great. The implementation leaves a lot > to be desired and, as always, the reason is not enough manpower. > Two things: first, LTS kernels aren't the same as stable kernels. Dale has been running stable kernels, and gentoo-sources kernels are all stable kernels. Second, I've been running LTS kernels for years without issue. I got into them due to running zfs/btrfs/nvidia. ZFS and nvidia are out of tree modules, and they tend to lag in support for the latest stable branches, so it is a constant battle if you want to run stable. If you run LTS they just work. When I was running btrfs I wanted to stick to LTS mainly because btrfs was constantly breaking things in new releases, which like every other subsystem are introduced in new branches. That was a while ago and maybe btrfs is more stable today. If you run anything out of tree though LTS is a much easier target. Aside from that, new kernel options are almost never added within LTS branch releases, so I just run make oldconfig and I'm done. You do get the rare change, and it is very easy to manage those. The downside is if you want some new kernel feature you won't get it, and you might need to update for support for new chipsets/CPUs if you're upgrading. That isn't a big deal to manage as I don't do it often. I can't remember the last time an LTS kernel blew up on me, but I never rush out to update a kernel the day it is released. Occassionally I do see a regression fixed and it tends to happen fairly quickly. All that said, it would be nice if the kernel had more of a QA process. I think the kernel has basically deferred all of that to distros, which means by running an upstream kernel I get none of it. The upstream kernel config defaults are also less than ideal, which is something distros also manage. -- Rich
Re: [gentoo-user] Upgrading from 5.14 to 6.0 version
On Fri, Nov 11, 2022 at 1:25 AM Dale wrote: > > Is there some major change that causes copying .config file from 5.14 to > 5.18 or higher to break? So, I just upgraded to 5.15 recently and tend to stick to LTS kernels, precisely to minimize this sort of thing. My guess is that you missed something in make oldconfig, but obviously without exact errors that could be completely wrong. I can't speak to 6.0 specifically, but one thing that I've found is a gotcha is when they consolidate config items under higher level ones. So they'll take what used to be 50 questions, and turn it into 1 question with 50 questions underneath it. The 1 question shows up as a new config item, and if you don't appreciate what it does and answer no to it, then you'll disable every item underneath it. Basically, don't assume that any new question is a new feature, and if you say no you'll still have all the pre-existing features. It could be a virtual question and answering no turns off a whole bunch of things that you had previously said yes to. You need to review oldconfig questions pretty carefully. You could try defaulting the answers but even then the defaults aren't always reasonable. They don't just turn on things you don't need. For example, by default linux turns off CGROUP support, and almost everything uses that today. That was just the first thing I checked, and I bet there are a million other gotchas like that. -- Rich
Re: [gentoo-user] Re: Update to /etc/sudoers disables wheel users!!!
On Wed, Oct 26, 2022 at 5:26 PM Grant Edwards wrote: > > On 2022-10-26, Dale wrote: > > Rich Freeman wrote: > >> If you use an x11-based merge tool then it will also refuse to attempt > >> an automatic > >> merge if X11 isn't available. (Obviously you can't actually run the > >> manual merge if the tool uses X11 and that isn't available.) > >> > >> > > > > I'd like to try a GUI based tool. Is that what you talking about? If > > so, name or what package has it? > > At one point, I had one of my systems configured to use "meld" when I > picked "interactive merge" in the etc-update menu, but I've since gone > back to just picking "show differences" in the etc-update menu, then > manually running merge on the two filenames shown. With the > interactive merge option, I was always a bit confused about which file > was the destination and what happened after I exited meld. > I use cfg-update+meld. It can use any 3-way diff/edit tool, but there aren't many of those. I believe the three panels show: Left: the current config file Right: new new packaged config file Center: what the packaged config file was the last time you did an update So Left vs Center shows you what changes you've made vs upstream, and center vs right show you what changes upstream made to their file. So you would look for differences on the right side to see what needs attention in the file, and then work those changes if appropriate into the left file. You just edit the left file to get it the way you want it and save that, and then cfg-update captures the changes in RCS. -- Rich
Re: [gentoo-user] Re: repair of a seg-faulting bin-utils
On Wed, Oct 26, 2022 at 12:24 PM Grant Edwards wrote: > > On 2022-10-26, Corbin wrote: > > Help! > > > > The last update I did built/installed bin-uitls. It is now producing > > seg-faults. I forgot to make a quickpkg of the old bin-utils before > > upgrading. > > The first thing I would do is run a RAM test overnight. IME, > segfaulting binutils or gcc has usually been a hardware problem. Bad disk is obviously another possible issue (saw that on a pi sdcard), but I'd definitely be testing that RAM. Really if you suspect bad RAM it is worth your trouble to just shut down ASAP and test that, because every minute with bad RAM is potentially corrupted files on your hard drive, and rework even if you have a good backup. Another possible issue is bad -march settings. That usually is an issue if you change your CPU and boot off of an existing hard drive. If you're going to upgrade your CPU you should rebuild all of @system (at least) with -march set to something very minimal. Don't assume that a newer CPU does everything an existing one does - they sometimes do drop instructions. You can set -mcpu to whatever you want, as a bad -mcpu will only cause minor performance issues. -- Rich
Re: [gentoo-user] Update to /etc/sudoers disables wheel users!!!
On Wed, Oct 26, 2022 at 1:15 PM Neil Bothwick wrote: > > On Wed, 26 Oct 2022 10:21:06 -0600, Grant Taylor wrote: > > > > dispatch-conf even gives you the opportunity to edit it before > > > applying. > > > > Yep. > > > > I almost always reject the changes suggested on config files that I've > > modified and accept them on files that I've not modified. > > > > I really do wish that there was a better way to manage this, likely > > involving diffs / deltas. E.g. what changed between the N distribution > > file and the N+1 distribution file. Can that same change be safely > > applied to the N' distribution file to create the N'+1 file? > > conf-update allows you to merge the new and old files, prompting you to > pick which to use on each differing section, with a further option to > edit the lines. That way you can keep your changed lines but still add > lines relating to new config options. > It could really use an overhaul but cfg-update does 3-way diffs and auto-merges based on them. Ie, if in a block of text you make a change, and in a new update that particular block of text hasn't changed, then your previous change will get auto-merged. If the upstream file changed in that block of text then you can do a 3-way diff. The tool is really old and barely maintained (I'm caretaking it but don't really want to deal with that - patches welcome). It also uses RCS to store the change history for 3-way merging and that could probably be switched to git or something more modern. If you use an x11-based merge tool then it will also refuse to attempt an automatic merge if X11 isn't available. (Obviously you can't actually run the manual merge if the tool uses X11 and that isn't available.) Using it I find that maybe 95% of my config file changes involve no prompts. Another useful tool is etckeeper which is basically just some integrations for portage around maintaining /etc in git. You can of course just do that manually but it will auto-commit changes if you forget to do so before an update. -- Rich
Re: [gentoo-user] Update to /etc/sudoers disables wheel users!!!
On Wed, Oct 26, 2022 at 3:42 AM Ramon Fischer wrote: > > I do not know, what the developers were thinking to encourage the user > to edit a default file, which gets potentially overwritten after each > package update... > > "etc-update" helps to have an eye on, but muscle memory and fast fingers > are sometimes faster. The Gentoo preference tends to be to follow upstream. So if sudo upstream distributes a file like this that has comments encouraging users to edit it, then that is likely how Gentoo will ship it. If sudo switched to moving everything into an include-based system UPSTREAM then Gentoo would probably start shipping that. If you look at the sudo ebuild you'll see that the config files are 100% upstream. If you look at things like systemd units or udev rules they're much more include-oriented, as this is the upstream preference. Gentoo has emphasized using config file protection early on, and doesn't have any official preference for using included config directories distro-wide. Portage has been moving in this direction for a while though (for the stuff in /etc/portage). -- Rich
Re: Re: [gentoo-user] Cannot shutdown or reboot because of logind disconnection
On Sat, Sep 17, 2022 at 8:21 AM johnstrass wrote: > > > Why is the logind so fragile? Have you checked your logs. I'm guessing that the kernel OOM killer is killing it, and it is kind of hard for a process to not die when the kernel kills it. > Why cannot it be brought up again after the memeory become available again? I suspect it probably could be - probably not a failure mode upstream has paid much attention to. If the OOM killer is loose on your system, general breakage is to be expected. It is actually surprising though that it didn't go after gcc itself. I'd check the logs. You probably could tweak the unit setting so that logind is less likely to get prioritized. Then it might go after something less essential like sshd or postfix. :) Also possible that it isn't logind itself but something else it uses for IPC. Haven't looked into the gory details of how it works. I'm guessing systemd could be coaxed to shut down without it. It might actually do that on its own if you give it time, but I haven't checked the settings. -- Rich
Re: [gentoo-user] Separate /usr partition
On Fri, Sep 16, 2022 at 11:16 AM Peter Humphrey wrote: > > > 1. dracut: 90crypt: Could not find any command of '/lib/systemd/systemd- > cryptsetup cryptsetup'! > > ...and similar for bluetooth. > > What do I have to include in /etc/dracut.conf.d/mine.conf to silence these? I > already omit the relevant modules: > > $ grep -e crypt -e blue /etc/dracut.conf.d/mine.conf > omit_dracutmodules+=" bluetoothd " > omit_dracutmodules+=" systemd-cryptsetup " > omit_dracutmodules+=" cryptsetup " > There are no modules by any of those names, so these config settings are a no-op. systemd-cryptsetup is called by the crypt module There is also a bluetooth module. Modules are located in /usr/lib/dracut/modules.d. I suspect the output of dracut mentions the names of the modules it is loading as well, or probably has a verbosity flag to have it talk more about what it is doing. For the most part modules tend to be automagic. Each one figures out if you're using it, and installs stuff if needed, and if not it no-ops. So if it can't find cryptsetup then it won't go trying to put support for it in the initramfs. I do get though that people prefer to have commands avoid output in a successful state, so omitting those modules should do the trick. Dracut modules are pretty simple in their operation. They all have a module-setup.sh script which is run by dracut and which does any logic, tells dracut what to install in the initramfs, and which registers scripts to run during various phases of boot. I haven't looked at them in ages but I did write up this article on how they work: https://rich0gentoo.wordpress.com/2012/01/21/a-quick-dracut-module/ -- Rich
Re: [gentoo-user] Separate /usr partition
On Wed, Sep 14, 2022 at 12:17 PM Laurence Perkins wrote: > > If something you need for booting with separate /usr is missing that would be > a FSH bug and is probably worth reporting unless you're doing something truly > arcane with your system. > You can always ask upstream but just about everybody is moving away from this. A Gentoo update is in the works that will require anybody using systemd to move everything in / to /usr (it isn't difficult to switch). (/bin would become a symlink to /usr/bin and so on). Probably won't be mandatory for non-systemd users but pretty soon upstreams are going to start assuming that there is no difference between /usr/lib and /lib and so on. Gentoo maintainers could potentially patch that behavior but it currently isn't required to accept bugs that break booting if /usr isn't already mounted. Simplest solution is to use an initramfs, which is basically what 99% of linux users use (just talking about conventional distros here, but I'm guessing Android uses one). Depending on your config things may still work using /usr in the traditional way, but if it doesn't work there is no guarantee that anybody will help you fix that. -- Rich
Re: [gentoo-user] Replace 8TB drive with 10TB drive on encrypted LVM
On Wed, Sep 14, 2022 at 1:40 AM Dale wrote: > > Now to replace my /home drive which is also close to full. It's not > encrypted tho. The biggest difference in this and plain LVM, resizing > with cryptsetup or close and reopen. Keep in mind, while I did all > this, LUKS, cryptsetup, whatever was open. I'm not sure about this > being done while closed. I did find one mention that it had to be open > at certain points. > If you want to do operations on a DM volume, then it needs to exist, which in the cryptsetup world means "open." Most of the details of how to do an operation like moving around encrypted volumes depends on the layers you're using (LUKS, LVM, mdadm, whatever), and how they're stacked. If you're running LVM on top of LUKS then you create new LUKS volumes and add them to LVM and move stuff around. If you're running LUKS on top of LVM then you add unencrypted volumes to LVM, extend the LV underlying LUKS, and then resize the LUKS volume. Last comes resizing whatever filesystems are on top if desired. When you want to get advice around these kinds of operations it is usually necessary to detail exactly how things are set up, because to Linux they're just layers and you can stack them however you want. A script that adds a hard drive to your setup will look different if you're running mdadm on the bottom vs LUKS vs LVM and so on. Plus if you do things in the wrong order there is actually a chance that it will work and leave you with half your filesystem on an encrypted drive and half of it on an unencrypted one, or whatever. DM doesn't care - it just shuffles around blocks and transforms them as told, and as long as you stack things up the same way every time it should still work. It just won't necessarily be doing what you want it to... -- Rich
Re: [gentoo-user] Encrypted hard drives on LVM and urgent power shutdowns.
On Sun, Sep 11, 2022 at 9:56 PM Dale wrote: > > I suspect this would happen on its own but I'd like to make sure. I'd > hate to mess up the file system badly on any of my drives or in a worst > case scenario, brick a hard drive with some 1 in a million chance problem. > I just wanted to comment that LUKS encryption on linux is pretty-much a block-level passthrough. So if your filesystem is journaled and using barriers or syncing to ensure consistency, and you add LUKS to it, then you shouldn't really see any difference in behavior if it is interrupted uncleanly by a power loss. The encryption could add a bit of latency but that shouldn't change much. Of course different filesystems handle interruptions differently, and all those caveats still apply. As far as unmounting goes, you just need to umount the filesystem. umount will block until all writes are synced to disk, and that includes all layers like LVM/LUKS/mdadm/whatever that might be underneath it. If umount returns, then all your data is written to disk and if at that instant you lose power there will be no data loss for that filesystem. I guess if you're using mdadm and you have multiple filesystems not aligned to a stripe boundary, then the raid write hole might still apply, and that is true at anytime whether the filesystem is mounted or not - data on a stripe shared with some other active filesystem could get lost in some situations. Obviously if you lose the key to a LUKS filesystem or if there is some kind of bug in LUKS the use of encryption could hinder data recovery. Beyond that it really shouldn't have any impact on anything. I guess it would also give you more exposure to RAM errors (since that is another code path that stores stuff in RAM). As already discussed, clean shutdowns triggered by NUT/etc are of course best, but the use of LUKS shouldn't change much with the use of a UPS otherwise. -- Rich
Re: [gentoo-user] Limiting amount of memory a program can use.
On Sun, Aug 28, 2022 at 11:32 AM Wols Lists wrote: > > On 28/08/2022 15:21, Rich Freeman wrote: > > Something I wish linux supported was discardable memory, for > > caches/etc. A program should be able to allocate memory while passing > > a hint to the kernel saying that the memory is discardable. > > Linux DOES have that ... > > I'm not sure how to use it, but you can pass a flag to open(), which > says "do not cache this file". That isn't what I was proposing. That has been around for a while. However, it only impacts caching/buffering of open files, which is kernel memory, and not memory held by a process itself. You wouldn't want to limit kernel file caching of a torrent file, since those files are pretty ideal candidates for caching (since the client is likely to send data not long after receiving it). There isn't really any benefit to this either in this case as the kernel already automatically drops file cache memory as needed under memory pressure. I was referring to application caching, which is internal storage used by an application that may or may not have anything to do with open files. A browser cache is a good example of something like this (though that extends to disk caching often and those files would also get cached by the kernel). The kernel can't simply drop this memory when it needs memory since there is no way for it to tell which memory an application owns is used for this purpose, and no way currently to handle the resulting segfaults when the application goes to read that memory. Reading memory is a CPU instruction, not a system call, so it would trigger an exception at the CPU level which must be handled. I'm sure something could be engineered but it is more complex and I'm guessing nobody has seen the need for it yet. Ideally you'd also have ways for the OS to hint how much memory applications ought to consume in this way. After all, you might have two applications running at the same time, which both want to use lots of extra RAM for caching if it is available, but neither has any way of knowing that this situation exists and it is undesirable if all the memory goes to the first application that asks, and also undesirable if after everything is done grabbing memory if there is a bunch of it sitting around doing nothing. TL;DR: cache is not necessarily the same as kernel file cache. -- Rich
Re: [gentoo-user] Limiting amount of memory a program can use.
On Sun, Aug 28, 2022 at 8:24 AM Dale wrote: > > What I would like to do is limit the amount of memory torrent > software can use. While ulimit/cgroups/etc will definitely do the job, they're probably not the solution you want. Those will cause memory allocation to fail, and I'm guessing at that point your torrent software will just die. I'd see if you can do something within the settings of the program to limit its memory use, and then use a resource limit at the OS level as a failsafe, so that a memory leak doesn't eat up all your memory. Otherwise your next email will be asking how to automatically restart a dead service. Systemd has support for that built-in, and there are also options for non-systemd, but you're going to be constantly having restarts and it might not even run for much time at all depending on how bad the problem is. It is always best to tame memory use within an application. Something I wish linux supported was discardable memory, for caches/etc. A program should be able to allocate memory while passing a hint to the kernel saying that the memory is discardable. If the kernel is under memory pressure it can then just deallocate the memory and then have some way to notify the process that the memory no longer is allocated. That might optionally support giving warning first, or it might be some kind of new trappable exception for segfaults to discarded memory. (Since access to memory doesn't involve system calls it might be hard to have more graceful error handling. I guess an option would be to first tell the kernel to lock the memory before accessing it, then release the lock, so that the memory isn't discarded after checking that it is safe.) -- Rich
Re: [gentoo-user] Getting maximum space out of a hard drive
On Fri, Aug 26, 2022 at 10:09 AM Mark Knecht wrote: > > Obviously you can do what you are most comfortable with but to me a NAS > machine with a bunch of external drives does not sound very reliable. > I would have thought the same, but messing around with LizardFS I've found that the USB3 hard drives never disconnect from their Pi4 hosts. I've had more issues with LSI HBAs dying. Of course I have host-level redundancy so if one Pi4 flakes out I can just reboot it with zero downtime - the master server is on an amd64 container. I only have about 2 drives per Pi right now as well - at this point I'd probably add more drives per host but I wanted to get out to 5-6 hosts first so that I get better performance especially during rebuilds. Gigabit networking is definitely a bottleneck, but with all the chunkservers on one switch they each get gigabit full duplex to all the others so rebuilds are still reasonably fast. To go with 10GbE you'd need hardware with better IO than a Pi4 I'd think, but the main bottleneck on the Pi4 I'm having is with encryption which hits the CPU. I am using dm-crypt for this which I think is hardware-optimized. I will say that zfs encryption is definitely not hardware-optimized and really gets CPU-bound, so I'm running zfs on top of dm-crypt. I should probably consider if dm-integrity makes more sense than zfs in this application. -- Rich
Re: [gentoo-user] Getting maximum space out of a hard drive
On Fri, Aug 26, 2022 at 7:26 AM Dale wrote: > > I looked into the Raspberry and the newest version, about $150 now, doesn't > even have SATA ports. The Pi4 is definitely a step up from the previous versions in terms of IO, but it is still pretty limited. It has USB3 and gigabit, and they don't share a USB host or anything like that, so you should get close to full performance out of both. The CPU is of course pretty limited, as is RAM. Biggest benefit is the super-low power consumption, and that is something I take seriously as for a lot of cheap hardware that runs 24x7 the power cost rapidly exceeds the purchase price. I see people buying old servers for $100 or whatever and those things will often go through $100 worth of electricity in a few months. How many hard drives are you talking about? There are two general routes to go for something like this. The simplest and most traditional way is a NAS box of some kind, with RAID. The issue with these approaches is that you're limited by the number of hard drives you can run off of one host, and of course if anything other than a drive fails you're offline. The other approach is a distributed filesystem. That ramps up the learning curve quite a bit, but for something like media where IOPS doesn't matter it eliminates the need to try to cram a dozen hard drives into one host. Ceph can also do IOPS but you're talking 10GbE + NVMe and big bucks, and that is how modern server farms would do it. I'll describe the traditional route since I suspect that is where you're going to end up. If you only had 2-4 drives total you could probably get away with a Pi4 and USB3 drives, but if you want encryption or anything CPU-intensive you're probably going to bottleneck on the CPU. It would be fine if you're more concerned with capacity than storage. For more drives than that, or just to be more robust, then any standard amd64 build will be fine. Obviously a motherboard with lots of SATA ports will help here. However, that almost always is a bottleneck on consumer gear, and the typical solution to that for SATA is a host bus adapter. They're expensive new, but cheap on ebay (I've had them fail though, which is probably why companies tend to sell them while they're still working). They also use a ton of power - I've measured them using upwards of 60W - they're designed for servers where nobody seems to care. A typical HBA can provide 8-32 SATA ports, via mini-SAS breakout cables (one mini-SAS port can provide 4 SATA ports). HBAs tend to use a lot of PCIe lanes - you don't necessarily need all of them if you only have a few drives and they're spinning disks, but it is probably easiest if you get a CPU with integrated graphics and use the 16x slot for the HBA. That or get a motherboard with two large slots (they usually aren't 16x, but getting 4-8x slots on a consumer motherboard isn't super-common). For software I'd use mdadm plus LVM. ZFS or btrfs are your other options, and those can run on bare metal, but btrfs is immature and ZFS cannot be reshaped the way mdadm can, so there are tradeoffs. If you want to use your existing drives and don't have a backup to restore or want to do it live, then the easiest option there is to add one drive to the system to expand capacity. Put mdadm on that drive as a degraded raid1 or whatever, then put LVM on top, and migrate data from an existing disk live over to the new one, freeing up one or more existing drives. Then put mdadm on those and LVM and migrate more data onto them, and so on, until everything is running on top of mdadm. Of course you need to plan how you want the array to look and have enough drives that you get the desired level of redundancy. You can start with degraded arrays (which is no worse than what you have now), then when enough drives are freed up they can be added as pairs to fill it out. If you want to go the distributed storage route then CephFS is the canonical solution at this point but it is RAM-hungry so it tends to be expensive. It is also complex, but there are ansible playbooks and so on to manage that (though playbooks with 100+ plays in them make me nervous). For something simpler MooseFS or LizardFS are probably where I'd start. I'm running LizardFS but they've been on the edge of death for years upstream and MooseFS licensing is apparently better now, so I'd probably look at that first. I did a talk on lizardfs recently: https://www.youtube.com/watch?v=dbMRcVrdsQs -- Rich
Re: [gentoo-user] Getting maximum space out of a hard drive
On Thu, Aug 25, 2022 at 2:59 PM Dale wrote: > > While at it, can I move the drives on LVM to another system without > having to copy anything? Just physically move the drives and LVM see > them correctly on the new system? As long as we aren't talking about boot partitions/sectors, the answer is yes. As long as all the drives are attached and accessible by the kernel (necessary drivers/etc present), then LVM will put them together. It doesn't care where it finds them, as all the necessary metadata is stored on the drives and gets scanned. If you're talking about boot partitions/sectors then it isn't a huge problem, but you do need to ensure the bootloader can find the right drives/etc, as it isn't nearly as flexible and may need updates if the drives switch order/etc, at least for legacy bootloaders. -- Rich
Re: [gentoo-user] Getting maximum space out of a hard drive
On Thu, Aug 25, 2022 at 8:43 AM Dale wrote: > > I've already got data on the drive now with the default settings so it > is to late for the moment however, I expect to need to add drives > later. Keep in mind, I use LVM which means I grow file systems quite > often by adding drives. I don't know if that grows inodes or not. I > suspect it does somehow. It does not. It just means that if you want to reformat it you have to reformat all the drives in the LVM logical volume. :) There are filesystems that don't have fixed limits on inodes on filesystem creation, but ALL filesystems have tradeoffs. This is one of the big limitations of ext4, but ext4 also has a number of advantages over alternatives. I tend to use zfs but it has its own issues. I don't believe inodes are fixed in zfs, but the last time I looked into it there were potential issues with reducing the size of a vdev. (I think that was being worked on but I'm not sure how stable that is, or if it is compatible with grub. Actually, one of my pet peeves has been that finding out exactly what zfs features are compatible with grub is tricky. Oh, you can google it, but I don't think there is any official page that is kept up to date.) -- Rich
Re: [gentoo-user] Getting maximum space out of a hard drive
On Mon, Aug 22, 2022 at 10:50 AM Laurence Perkins wrote: > > Note that 60ish MB/sec is very reasonable for a rotational drive. They *can* > technically go faster, but only if you keep the workload almost entirely > sequential. Most filesystems require a fair amount of seeking to write > metadata, which slows them down quite a bit. > > If you're desperate for performance, you can do things like tell it to ignore > write barriers and turn off various bits of flushing and increase the amount > of allowed dirty write cache. These can be good for a significant > performance boost at the cost of almost certainly corrupting the filesystem > if the system loses power or crashes. > I've also found that on large drives the sequential write speed varies based on position in the drive. If I run something like badblocks on a new hard drive I'll see it start out at something like 200MB/s, and by the end it is around100MB/s. Then at the start of the next pass it will jump back up to 200MB/s. This is just direct block-level sequential writes so it is an ideal use case. As you say, ANY seeking will dramatically reduce the throughput. Time spent seeking is time not spent writing. There is no opportunity to "catch up" as the drive's read/write bandwidth is basically just a function of the recording density and rotational rate and number of platters/etc being read in parallel. If it is seeking it is a lost opportunity to read/write. -- Rich
Re: [gentoo-user] Getting maximum space out of a hard drive
On Sat, Aug 20, 2022 at 3:15 PM Dale wrote: > > Related question. Does encryption slow the read/write speeds of a drive > down a fair amount? This new 10TB drive is maxing out at about > 49.51MB/s or so. Encryption won't impact the write speeds themselves of course, but it could introduce a CPU bottleneck. If you don't have any cores pegged at 100% though I'd say this isn't happening. On x86 encrypting a hard drive shouldn't be a problem. I have seen it become a bottleneck on something like a Pi4 if the encryption isn't directly supported in hardware by the CPU. 50MB/s is reasonable if you have an IOPS-limited workload. It is of course a bit low for something that is bandwidth-limited. If you want to test that I'm not sure rsync is a great way to go. I'd pause that (ctrl-z is fine), then verify that all disk IO goes to zero (might take 30s to clear out the cache). Then I'd use "time dd bs=1M count=2 if=/dev/zero of=/path/to/drive/test" to measure how long it takes to create a 20GB file. Oh, this assumes you're not using a filesystem that can detect all-zeros and compress or make the file sparse. If you get crazy-fast results then I'd do a test like copying a single large file with cp and timing that. Make sure your disk has no IO before testing. If you have two processes accessing at once then you're going to get a huge drop in performance on a spinning disk. That includes one writing process and one reading one, unless the reads all hit the cache. -- Rich
Re: [gentoo-user] Getting maximum space out of a hard drive
On Thu, Aug 18, 2022 at 2:04 PM Dale wrote: > > > Part. # SizePartition TypePartition Name > > 1007.0 KiB free space >19.1 TiB Linux filesystem 10Tb > 1007.5 KiB free space > > > I'm not sure why there seems to be two alignment spots. Is that > normal? Already, there is almost 1TB lost somewhere. 10 TB = 9.09495 TiB. You aren't missing much of anything. And no, I don't want to get into a religious war over base 2 vs base 10, and why it would be confusing if a tape that could store 10MB/m didn't store 10kB/mm but instead stored 10.24 kB/mm. -- Rich
Re: [gentoo-user] Backup program that compresses data but only changes new files.
On Mon, Aug 15, 2022 at 3:41 PM Dale wrote: > > Glad to know what I found was good info. I just wonder how long it will > be before even 10TB drives will be SMR. I also dread having to search > out a 14TB drive later. :/ > I think it will be a long time if ever, and here is why. There are good reasons and bad reasons to use SMR. The reason you would WANT to use SMR is that you have a task that is well-suited to their limitations like backup or applications that can use log-style storage. Ideally you'd want host-managed SMR for this. The benefit is higher density for the cost, so you'd be doing it to get a drive that is cheaper than it otherwise would be. However, these are all things that would appeal to experts who really know what they're doing. The bad reason to use SMR is that you're a manufacturer trying to squeeze out a bit more profit margin, not passing on the savings. In this case you want to sell the drive to somebody who DOESN'T know what they're doing, and make it drive-managed. This is why we've seen SMR in medium-sized drives and not big ones as would be expected if you assumed it would be employed for the good reasons. The only people buying 14TB hard drives are people who tend to know what they're doing, which makes them less of a target for unscrupulous manufacturers. You wouldn't see them as much in small drives as the return in capacity isn't as much. The medium sized drives are big enough to get a return out of using SMR, but small enough that suckers will be willing to buy them. At least, that's my theory... -- Rich
Re: [gentoo-user] Backup program that compresses data but only changes new files.
On Mon, Aug 15, 2022 at 3:28 PM Dale wrote: > > Given my weird way of doing backups, rsync may be the best option. > Plus, easier to restore from as well since it just requires a copy > command, any of them will do. > If you don't make small changes inside of large files, you might actually be fine with tar. It can do compression, though you won't benefit much from that. It can split archives easily. It can do incrementals, but I think they are limited to using metadata so it depends on well-behaved software. What it doesn't do is let you backup just a small part of a file containing larger changes. -- Rich
Re: [gentoo-user] Backup program that compresses data but only changes new files.
On Mon, Aug 15, 2022 at 3:20 PM J. Roeleveld wrote: > > Actually, you can with the "-p / --pause" option. > Also, as per the man-page, if you forget this, the process will simply inform > you the target location is full and you can move slices away to a different > location: > " > If the destination filesystem is too small to contain all the slices of the > backup, the -p option (pausing before starting new slices) might be of > interest. Else, in the case the filesystem is full, dar will suspend the > operation, asking for the user to make free space, then continue its > operation. To make free space, the only thing you cannot do is to touch the > slice being written. > " > > The pause-option will actually stop between slices and you can umount the > target location and mount a different disk there. > > This option has been around for a while. Hmm, sounds kind of non-ideal. It sounds like you can either have it pause when full, or pause between slices. Neither is great. If it pauses when full, and you can't touch the slice being written, then you can't unmount the drive it is being written to. So you end up having to write to a scratch area and keep moving slices off of that onto another drive. At best that is extra IO which slows things down, and of course you need scratch space. If you pause between slices, then you have to have drives of equal size to store to, otherwise you'll have to swap drives that aren't completely full. Ideally you'd want to write until the drive is almost full, then finish a slice and pause. However, it at least seems workable if slow. Just have scratch space for a bunch of slices, let it pause when full, then move slices off as they are done, and accept that your backups will run at maybe 25% of the speed of the scratch drive since it will be constantly seeking between writing new slices and reading old ones. Or if you have enough RAM you could use a tmpfs for that but that seems really cumbersome unless you use very small slices and have the shuffling scripted. -- Rich
Re: [gentoo-user] Backup program that compresses data but only changes new files.
On Mon, Aug 15, 2022 at 2:34 PM J. Roeleveld wrote: > > Actually, there still is a piece of software that does this: > " app-backup/dar " > You can tell it to split the backups into slices of a specific size. dar is a great tool, but unless something changed I don't think you can tell it to pause to remount the destination directory when it fills up. As was pointed out, tar does do this (which I thought was limited to tape devices, but apparently it works for disk as well). For most backup tools, if you want to backup 100TB of data to a bunch of 10TB drives, you basically need to have 100TB of scratch storage lying around to hold the backup before you split it up, or be able to mount all your drives at the same time in some kind of combined filesystem. -- Rich
Re: [gentoo-user] Backup program that compresses data but only changes new files.
On Mon, Aug 15, 2022 at 2:32 PM Wol wrote: > > On 15/08/2022 11:11, Rich Freeman wrote: > > I see lots of talk of NAS and zfs/btrfs and snapshots. IMO these are > > NOT really great solutions for backup. NAS can work of course but it > > is overkill for backup storage. > > Do you want multiple *independent* backups, or do you want *incremental* > backups so you can go back in time. It's nice to have both, but > snapshotting gives you full backups for the price of incremental. Snapshots don't give you backups at all, unless you first make a snapshot and then take a backup (full or incremental) of the snapshot, or serialize them using a send-like mechanism (which can also be full or incremental). If you destroy the drives containing the original copy, then you destroy all the snapshots as well. COW snapshots are great, but they're more about how you handle your LIVE data. They don't address the same failure modes as backups. I use both. -- Rich
Re: [gentoo-user] Backup program that compresses data but only changes new files.
On Mon, Aug 15, 2022 at 3:05 AM Dale wrote: > > Rich Freeman wrote: > > > > Duplicity uses librsync, so it backs up exactly the same data as rsync > > would, except instead of replicating entire files, it creates streams > > of data more like something like tar. So if you back up a million > > small files you might get out 1-3 big files. It can compress and > > encrypt the data as you wish. > > Duplicity sounds interesting except that I already have the drive > encrypted. Then don't encrypt it? Both compression and encryption are optional. > The reason I mentioned being like rsync, I don't want to rebuild a backup > from scratch each time as that would be time consuming. Ah, you just want something that does incremental backups. Duplicity does, along with most decent solutions. I see lots of talk of NAS and zfs/btrfs and snapshots. IMO these are NOT really great solutions for backup. NAS can work of course but it is overkill for backup storage. NAS, zfs, btrfs, and snapshots are all great things to use with your live data. I use several of these myself. Your live data should be protected against bitrot with snapshots/etc. That has nothing to do with why you want backups. We're talking about the storage of backups. While you can store backups on any of these they don't really add much value. Also, you mentioned SMR, and I would definitely not combine SMR with most of those. SMR is perfect for backup. It just isn't perfect for backup using something like rsync that modifies files in place. You want something that only appends to backup files or creates new ones, which is basically how most backup software works except for stuff that works like rsync. The main issue I think you're going to have is having support for multi-volume backups if you need to be able to split a backup across drives. The only thing I've found on Linux that does this is bacula, and it is a royal pain that I'm embarrassed to even mention. If somebody knows of another backup solution that can write the output to disk (a filesystem, not /dev/rmt) and then pause to mount a new disk when one fills up, I'm all ears. For everything else I've tended to see people suggest using lvm/mdadm/whatever combine disks into a single block device so that the backup software doesn't see multiple disks. If you do want to go the route of combining your disks then since you're using SMR I'd probably pick something like lvm that doesn't do any striping/etc and just fills up one disk then moves to the next. Then use a simple filesystem (not btrfs/zfs) that just starts at one end and keeps adding. A log-based filesystem would probably be ideal but I'm not sure if any are decent. You do have the issue of what you do when you start to run out of space, unless you can create multiple sets of disks so that you can complete a new backup before destroying the old one. -- Rich
Re: [gentoo-user] Backup program that compresses data but only changes new files.
On Sun, Aug 14, 2022 at 6:44 PM Dale wrote: > > I plan to buy another hard drive pretty soon. Next month is possible. > If there is nothing available that does what I want, is there a way to > use rsync and have it set to backup files starting with "a" through "k" > to one spot and then backup "l" through "z" to another? I could then > split the files into two parts. Oh, I didn't comment on this part, so sorry for the double reply. If you need backups that span multiple disks your options are very limited unfortunately. Most linux backup software might output multiple files but it dumps them all in one place and much of it assumes that all the files are in one place for restoration. Here are the options I've found: 1. You can use lvm/zfs/btrfs/whatever to combine multiple disks to make them look like one disk. This is a workaround, and obviously you're limited to however many disks you can physically mount at one time. 2. You can use bacula, which does support changing media, since it was designed for tape, but unlike tar it can output to a directory. However, this is not very well-supported and it can be a pain. This is what I'm doing for large-scale backups. I basically treat a hard drive like a giant tape. It is fussy to set up and use, and bacula itself is VERY fussy to use. Oh, and make sure you REALLY understand it and do some restoration tests because otherwise you could paint yourself into a corner. I always backup my database, and I have the bacula software itself running in a container and after every backup I just create a tarball of the whole container and stick that on the backup disk (it isn't big, and that solves the bootstrapping problem). Don't ever use bacula to back up itself - it is terrible for that. 3. Obviously if you have a scratch disk big enough to hold everything temporarily that also works. You can do your backup, then copy it off to other drives however you want. -- Rich
Re: [gentoo-user] Backup program that compresses data but only changes new files.
On Sun, Aug 14, 2022 at 6:44 PM Dale wrote: > > Right now, I'm using rsync which doesn't compress files but does just > update things that have changed. I'd like to find some way, software > but maybe there is already a tool I'm unaware of, to compress data and > work a lot like rsync otherwise. So, how important is it that it work exactly like rsync? I use duplicity, in part because I've been using it forever. Restic seems to be a similar program most are using these days which I haven't looked at super-closely but I'd look at that first if starting out. Duplicity uses librsync, so it backs up exactly the same data as rsync would, except instead of replicating entire files, it creates streams of data more like something like tar. So if you back up a million small files you might get out 1-3 big files. It can compress and encrypt the data as you wish. The downside is that you don't end up with something that looks like your original files - you have to run the restore process to extract them all back out. It is extremely space-efficient though - if 1 byte changes in the middle of a 10GB file you'll end up just backing up maybe a kilobyte or so (whatever the block size is), which is just like rsync. Typically you rely on metadata to find files that change which is fast, but I'm guessing you can tell these programs to do a deep scan which of course requires reading the entire contents, and that will discover anything that was modified without changing ctime/mtime. The output files can be split to any size, and the index info (the metadata) is separate from the raw data. If you're storing to offline/remote/cloud/whatever storage typically you keep the metadata cached locally to speed retrieval and to figure out what files have changed for incrementals. However, if the local cache isn't there then it will fetch just the indexes from wherever it is stored (they're small). It has support for many cloud services - I store mine to AWS S3. There are also some options that are a little closer to rsync like rsnapshot and burp. Those don't store compressed (unless there is an option for that or something), but they do let you rotate through multiple backups and they'll set up hard links/etc so that they are de-duplicated. Of course hard links are at the file level so if 1 byte inside a file changes you'll end up with two full copies. It will still only transfer a single block so the bandwidth requirements are similar to rsync. -- Rich
Re: [gentoo-user] About to have fiber internet and need VPN info
On Sun, Aug 7, 2022 at 11:36 AM Michael wrote: > > The best a well configured VPN tunnel can offer is a secure connection between > client and VPN server, which is handy if you are out and about using untrusted > and insecure WiFi hotspots. > > The only other reason for using a VPN service is to present a different > geolocation for the purpose of overcoming country-specific website > restrictions. I think ONLY is a bit strong here. A VPN effectively makes it impossible for your ISP to know who you're talking to, and it obscures your IP from hosts you are connecting to. Sure, there are ways to defeat this, but most of them are only applicable for state-level actors, and the methods available to ordinary companies can only identify at best a unique browser profile, which only lets them correlate traffic with those they share info with to the degree that you use a single browser profile across those platforms. For non-web traffic there are generally fewer attacks available. Many of the attacks that are often cited like DNS-based attacks are not that difficult to prevent (eg by ensuring your DNS traffic goes out over the VPN). If there are sites you browse using a different browser profile (ideally on a VM/etc), and you never use that browser profile for ecommerce or activity associated with your normal social media accounts, then it is unlikely that those sites will actually be able to identify you. Really the biggest pain with the VPNs is the number of websites that actively try to block connections from them or flood you with CAPTCHAs. Many more mainstream social media sites/etc also effectively require association with a mobile phone number, or trigger this behavior if they don't like your IP address. Obviously VPNs can be abused to attack hosts or evade bans and generally cause trouble, which is a frustration for those who simply don't want companies to know who you are. Bottom line is that just because the NSA can track your connections doesn't mean that every random webserver on the planet can do so. The few government agencies that are likely to be that well-connected are also very interested in keeping the extent of their capabilities hidden from each other, and so when they intercept your data they're going to guard it even more carefully than you would. A solution doesn't need to be able to defeat the NSA to be useful. -- Rich
Re: [gentoo-user] About to have fiber internet and need VPN info
On Sat, Jul 16, 2022 at 6:57 AM Dale wrote: > > I also want to use a VPN but only for some programs. Example, I want > Ktorrent and a couple Firefox profiles to use VPNs but at least one > Firefox profile I want to remain outside of VPN. I can't keep up with which VPNs are more or less scummy at any moment in time, but I will comment on this bit and on the concept in general. Controlling which HOSTS use a VPN is pretty straightforward via the routing tables. If you have a decent DHCP server and can issue routers to individual hosts you can do it that way (most consumer routers probably won't support this with their built-in DHCP). Controlling it at the software level is a real PITA. On an OS like Windows I don't think it is even possible unless via SOCKS or whatever. On Linux you can do it with iproute2 and often netfilter is needed as well. Look up policy-based routing, and be prepared to do some studying. I'll tell you right now you probably don't want to do it this way. I think for outbound-only connections it isn't as hard to do it at a uid level, so if you run software under different uids that would make it easier. If you want to handle inbound connections on servers and have the replies not go out over the normal destination-based route then you need to mark the connections using netfilter and then set a policy routing for the replies, otherwise your reply traffic will go out over the wrong router and have the wrong IP and the other end won't associate it with the connection. I imagine you run into the same problems with any kind of use of NAT for inbound forwarded traffic in a multi-homed situation. Controlling routes by container is also a potential issue. If you're using a container technology that uses virtual interfaces that get their own IPs/routing/etc then it is easy - same as host-level routing. If you're using something like Docker/k8s where it wants all the outbound traffic to just go out from the host then it can be a pain. I think they can do macvlan but I think that has its own issues. That is actually something I'm trying to figure out for myself. Ok, topic change: the threat model. As others have pointed out, the VPN changes WHO can see your traffic, and that's mainly it. I think this is still a useful consideration, because in many places your ISP is chosen by where you live, but with a VPN provider you can choose anyone you want. The ISP has no reason to earn your trust because you're a captive audience, while a VPN provider who gets outed for leaking private info basically is out of business. So I think there is a benefit. However, you're going to be reducing your risk of being traced by private companies here, like advertisers, intellectual property enforcement companies, and so on. If you're worried about the NSA or some other state-level actor then you need to do a LOT more to evade them. I just assume the NSA has root on all my hosts already, and I wish that they'd at least offer to sell backups of my systems back to me so that I didn't need to keep my own... :) -- Rich
Re: [gentoo-user] Selective news?
On Sat, Jul 30, 2022 at 10:27 AM Peter Humphrey wrote: > > Then I went to my ~amd64 machine and looked for the same news item - and > there was no sign of anything about either pulse or pipe. This machine has no > working sound hardware, but even so, how did the news function know not to > fetch that item? From: https://gitweb.gentoo.org/data/gentoo-news.git/tree/2022-07-29-pipewire-sound-server/2022-07-29-pipewire-sound-server.en.txt Display-If-Installed: media-video/pipewire Display-If-Installed: media-sound/pulseaudio Display-If-Installed: media-sound/pulseaudio-daemon Display-If-Installed: media-libs/libpulse If you have any of those packages installed, the news will display. We try to filter news so that users aren't bombarded by things that aren't relevant to them. For example, a few weeks ago there was apparently a corruption in the mu MUA (which I'd never heard of), which would be really important to know about if you were one of the 0.001% of Gentoo users who rely on it. -- Rich
Re: [gentoo-user] Re: python mess - random winge!
On Tue, Jul 5, 2022 at 1:10 PM Wols Lists wrote: > > On 05/07/2022 17:44, Rich Freeman wrote: > > hat and its dependencies. Obviously you need to be caught up before > > things get removed from the repo, but the offending package itself > > will get removed when that happens anyway. > > > > You can always just globally keep the older version around longer if > > you don't want to deal with a bunch of cruft in > > /etc/portage/package.use. The news item explains how to do this. > > I'd be inclined to lock the older version of the package you want so it > won't upgrade automatically (don't know how to do this, probably put > that particular version in @world), and also give that a python 9 > dependency in package.use. The news item explains how to get the upgrade earlier, delay the upgrade entirely, or have support for both versions. I'd suggest doing it the way the news item suggests, because when you start fighting the devs who maintain the python update logic, you're probably going to find yourself swearing a lot. -- Rich
Re: [gentoo-user] Re: python mess - random winge!
On Tue, Jul 5, 2022 at 12:36 PM Jack wrote: > > On 2022.07.05 12:24, Grant Edwards wrote: > > On 2022-07-05, William Kenworthy wrote: > > > > > I synced portage a couple of days now and now my systems are > > rebuilding > > > python modules for 3.10 without any input from me [...] > > > > Every time there's a Python upgrade like this, it turns into a bit of > > an ordeal because I always have a small handful of packages that don't > > support the newer version. > > > > The news item offers no advice on what to do in this situation other > > than completely postponing the upgrade of everything (which doesn't > > seem like the best choice.) > > > > It would be nice if the news item explained how to let the upgrade > > procede while holding back a few packages. > > > > Can you set 3_9 and 3_10 globally, and then disable 3_10 for a few > > individual packages that can't be built with 3_10? > As far as I can tell, you just need to add python_targets_python3_9 for > the package in the appropriate package.use file. > That and its dependencies. Obviously you need to be caught up before things get removed from the repo, but the offending package itself will get removed when that happens anyway. You can always just globally keep the older version around longer if you don't want to deal with a bunch of cruft in /etc/portage/package.use. The news item explains how to do this. -- Rich
Re: [gentoo-user] sync-type: rsync vs git
On Wed, Apr 27, 2022 at 10:22 AM Grant Edwards wrote: > > Is there any advantage (either to me or the Gentoo community) to > continue to use rsync and the rsync pool instead of switching the > rest of my machines to git? > > I've been very impressed with the reliability and speed of sync > operations using git they never take more than a few seconds. With git you might need to occasionally wipe your repository to delete history if you don't want it to accumulate (I don't think there is a way to do that automatically but if you can tell git to drop history let me know). Of course that history can come in handy if you need to revert something/etc. If you sync infrequently - say once a month or less frequently, then I'd expect rsync to be faster. This is because git has to fetch every single set of changes since the last sync, while rsync just compares everything at a file level. Over a long period of time that means that if a package was revised 4 times and old versions were pruned 4 times, then you end up fetching and ignoring 2-3 versions of the package that would just never be fetched at all with rsync. That can add up if it has been a long time. On the other hand, if you sync frequently (especially daily or more often), then git is FAR less expensive in both IO and CPU on both your side and on the server side. Your git client and the server just communicate what revision they're at, the server can see all the versions you're missing, and send the history in-between. Then your client can see what objects it is missing that it wants and fetch them. Since it is all de-duped by its design anything that hasn't changed or which the repo has already seen will not need to be transferred. With rsync you need to scan the entire filesystem metadata at least on both ends to figure out what has changed, and if your metadata isn't trustworthy you need to hash all the file contents (which isn't done by default). Since git is content-hashed you basically get more data integrity than the default level for rsync and the only thing that needs to be read is the git metadata, which is packed efficiently. Bottom line is that I think git just makes more sense these days for the typical gentoo user, who is far more likely to be interested in things like changelogs and commit histories than users of other distros. I'm not saying it is always the best choice for everybody, but you should consider it and improve your git-fu if you need to. Oh, and if you want the equivalent of an old changelog, just go into a directory and run "git whatchanged ." -- Rich
Re: [gentoo-user] Re: How to make a boot menu?
On Sun, Apr 17, 2022 at 3:00 PM Martin Vaeth wrote: > > Yes, without a manually written grub.cfg you get none of these features - > the default grub.cfg is just horrible. > Well, the most powerful feature is probably still available: > The possibility to edit the kernel's command line, partition and path which > theoretically can cover everything else, though it is rather inconvenient. The GRUB bootloader just parses its config file, which can be manually edited as you point out. You also have grub-mkconfig which outputs a config file for the bootloader to use, and it is typically used to scan /boot and find all the kernels/initramfs present and create menu items. grub-mkconfig just runs a bunch of shell scripts to generate everything, so you can have it autogenerate anything you want. It seems like a rough way to do it would be to just copy the regular linux once for each runlevel so that you end up with each kernel show up more than once, and then the individual runlevels can be tweaked accordingly. Obviously it would be more elegant to add a loop over a configuration variable. I'm not aware of anybody having actually done this, however, so you'd have to DIY. -- Rich
Re: [gentoo-user] Re: systemd-boot on openrc
On Sun, Apr 17, 2022 at 1:05 PM Martin Vaeth wrote: > > Peter Humphrey wrote: > > On Sunday, 17 April 2022 16:42:35 -00 Martin Vaeth wrote: > >> Peter Humphrey wrote: > >> > On Sunday, 17 April 2022 14:54:50 -00 Rich Freeman wrote: > >> >> Can't you just fix your USE flags with systemd-utils? Why revert? > >> > > >> > No, because the flag I'd need is 'boot', and that triggers switching from > >> > elogind to systemd. > >> > >> No, USE=boot for systemd-util does not trigger anything like that. > > > > I meant, if I set that flag, portage wants me to remove elogind andinstall > > systemd. > > Maybe, but the fault is certainly not this flag but something else. > For instance, that you do not have keyworded something which you should have. It would probably be helpful to post more relevant output, like portage output including --verbose and so on so that it is clear what it is actually doing. systemd-utils blocks systemd, so I can't see how it could force you to install systemd (after all, it just supplies things that are otherwise bundled with systemd already). Maybe in addition to setting the boot USE flag you also changed something else? -- Rich -- Rich
Re: [gentoo-user] systemd-boot on openrc
On Sun, Apr 17, 2022 at 9:03 AM Peter Humphrey wrote: > > On Sunday, 17 April 2022 12:13:06 -00 Neil Bothwick wrote: > > --->8 > > It looks like this is cause my using mixed keywords, amd64 for udev and > > ~amd64 for systemd-boot/utils. Does keywording udev-250 resolve the > > blocks? > > Yes, after keywording several others, thus: > > ~sys-apps/systemd-tmpfiles-249.9 > ~sys-apps/systemd-utils-250.4 > ~sys-fs/udev-250 > ~virtual/tmpfiles-0-r2 > > But then, after rebooting because of the udev update, systemd-boot-250-r1 has > come in. I can't revert those keywords though, because then I'd have to ditch > elogind in favour of systemd. I really do not want to do that. Can't you just fix your USE flags with systemd-utils? Why revert? If I need to bump a package up to ~arch temporarily usually I just do it with an atom like "
