Re: recommended partitions to backup with dump
On Thu, 25 Aug 2022 07:59:00 - (UTC) Stuart Henderson wrote: > /var, maybe exclude /var/cache. (maybe also /var/log, but it can be > useful to have). > Is there a way to exclude directories within a selected volume for a full backup? It looks as though nodump only works for levels above 0. -- Edward Ahlsen-Girard Ft Walton Beach, FL
Re: recommended partitions to backup with dump
On 2022-08-24, Shadrock Uhuru wrote: > hi everyone > after losing a considerable amount of data that i had accumulated over the > last year or so > by trying to remove a directory called '~' that i had created by mistake > in a sub directory of my home directory with rm -rf ~ > which of course started to eat through my home directory with a vengence, > i managed to stop it before it went to far, > i didn't have any recent backups, As you probably now figured out, use quotes: '~' "rm -rf -i $whatever" if you have any suspicion that the name might be "difficult". (-i must come *after* -f, so there's little point aliasing rm to rm -i). > needless to say i've learning my lesson about having a good policy of regular > backups. > what are the recommended partition to backup if > > 1 i want to do a fresh reinstall e.g. to move to a larger hard drive. > 2 for a disaster recovery like what i experienced above. > > i will be using ville walveranta's autodump 1.5a script > which does a full dump on sundays and incremental dumps during the week, That's a far more complicated script tgan I woukd consider using for backups. Do you understand what it does and how it works? A backup done to the same machine is not really a backup. > i already have /home /etc and /root set for backup, > are there any other partitions i should bear in mind ? /var, maybe exclude /var/cache. (maybe also /var/log, but it can be useful to have). -- Please keep replies on the mailing list.
Re: recommended partitions to backup with dump
Remember that some that some things need to be explicitly dumped - eg databases and repositories because when you do restore, you might want to restore to an upgraded OS version. Rather than use dump, I use gtar - I have restored stuff after 30 years with gtar, to completely different OSes - eg OS/2 to OBSD! * If you do GFS (four tapes: grandfather, father, son, and NEXT), each tape is approximately used once per month, and can be expected to have quite a long life. The "Son" tape is kept near the drive, the "father" tape goes off-site. You tape from the disk backup (done with rsync) machine - databases and repositories can have hot standbys on the same machine. In case of disaster recovery, step 1 is to duplicate the "father" tape off site. Then you still have a chance of knowing how bad things are before you accidentally overwrite the "son". Once a year, you can put one father tape aside for archive, and replace with a fresh tape. Possibly twice a year - to maintain a second, completely separate, archive. If you don't use tapes, expect to lose data one day. I am probably the only person to have restored data from VMS to BBC Micro using 1600BPI reel to reel tapes. (Probably the only person to have a BBC micro with both reel-to-reel tape and ST506 interfaces). Andrew * ICL George 2 to VMS only worked with no compression, and George 2 to Cray :-) required an assembler program to convert between the two different the 6-bit ASCII character sets on 7-track 556 bpi tapes.
Re: recommended partitions to backup with dump
On 8/24/22 13:28, Shadrock Uhuru wrote: hi everyone after losing a considerable amount of data that i had accumulated over the last year or so by trying to remove a directory called '~' that i had created by mistake in a sub directory of my home directory with rm -rf ~ which of course started to eat through my home directory with a vengence, i managed to stop it before it went to far, i didn't have any recent backups, needless to say i've learning my lesson about having a good policy of regular backups. what are the recommended partition to backup if 1 i want to do a fresh reinstall e.g. to move to a larger hard drive. 2 for a disaster recovery like what i experienced above. i will be using ville walveranta's autodump 1.5a script which does a full dump on sundays and incremental dumps during the week, i already have /home /etc and /root set for backup, are there any other partitions i should bear in mind ? shadrock I agree with "know exactly what you need" or "save everything" If backups are cheap in time and money just save everything as often as you can - daily if you do a lot online. Rebuilding from incremental dumps can be painful. Even if dumping everything I'd consider leaving out any directory with "cache" in its name That can potentially save many gigs of storage and hours of backup time. Thunderbird and firefox usually have multiple gig databases. If backups are expensive and you're being very, very selective, I'd add to your list: the output of pkg_info /usr/local, /var/mail potentially /usr/src, /var/www, /var/unbound, /var/nsd any directories I added or I'm not sure about any system directories where I've modified files Any volumes not part of the base system have their own dump schedule. hth Geoff Steckel
Re: recommended partitions to backup with dump
On 8/24/22 13:28, Shadrock Uhuru wrote: hi everyone after losing a considerable amount of data that i had accumulated over the last year or so by trying to remove a directory called '~' that i had created by mistake in a sub directory of my home directory with rm -rf ~ which of course started to eat through my home directory with a vengence, i managed to stop it before it went to far, i didn't have any recent backups, needless to say i've learning my lesson about having a good policy of regular backups. what are the recommended partition to backup if 1 i want to do a fresh reinstall e.g. to move to a larger hard drive. 2 for a disaster recovery like what i experienced above. i will be using ville walveranta's autodump 1.5a script which does a full dump on sundays and incremental dumps during the week, i already have /home /etc and /root set for backup, are there any other partitions i should bear in mind ? shadrock /root and /etc should be on the root partition ( / , sd0a, typically). There is *generally* not much data of substance in the directory /root, but that depends on your environment. Also depending on your environment, there's often a lot of really important stuff in /var. Or not. You may have local scripts hiding out in /usr/local/*bin. If you want a "Bare Metal" restoration, you really need everything. I kinda think of 'dump' as a bare-metal restoration tool, though it can definitely restore individual files. The real answer, though, is "you backup everything you need". OpenBSD installs are so small, the vast majority of your system is often so much bigger, might as well just back up everything, or exclude things that are more trouble than they are worth (/mnt, /tmp leap into mind). After you establish your backup system, build and validate a new system based on that backup, both a "fresh install" and a "unhappy event" case. I'm rather a fan of "know where your important files are" and restore by building a new system, installing the required applications, then copying over the config files and the data directories. Thus I tend to be partial to rsync backups using the --link-dest option rather than dump(8)s of file systems. Both have their place, and they really aren't competitors. I have a sample starting point rsync --link-dest script here: https://holland-consulting.net/scripts/ibs/ Nick.
Re: recommended partitions to backup with dump
Hello Shadrock
24.08.2022 20:28 tarihinde Shadrock Uhuru yazdı:
1 i want to do a fresh reinstall e.g. to move to a larger hard drive.
2 for a disaster recovery like what i experienced above.
These arguments will require a "full backup". It means backup all
partitions, so you can migrate all data (which means both os and personal)
So you can restore it all ("as they were"). Incrementel backups require
level 0 back up and restore process will require all incremental
backsups in sequential so restoring process will recover everything
properly.
My humble advice is running a full backup of all partitions regularly
(aka bare metal backup). Time intervals depends on activities and policy.
Sincerely yours,
OpenPGP_0x648AAD2AAA3BAD5F_and_old_rev.asc
Description: OpenPGP public key
OpenPGP_signature
Description: OpenPGP digital signature
Re: recommended partitions to backup with dump
On 2022-08-24 12:51:16-0500, Allan Streib wrote: > On Wed, Aug 24, 2022, at 12:28, Shadrock Uhuru wrote: > > i already have /home /etc and /root set for backup, > > are there any other partitions i should bear in mind ? > > I always backup /var The above make sense to me also. Exploring man 7 hier might also be interesting, and possibly seeing what is on a newly installed system, and what is not.
Re: recommended partitions to backup with dump
On Wed, Aug 24, 2022, at 12:28, Shadrock Uhuru wrote: > i already have /home /etc and /root set for backup, > are there any other partitions i should bear in mind ? I always backup /var Allan
recommended partitions to backup with dump
hi everyone after losing a considerable amount of data that i had accumulated over the last year or so by trying to remove a directory called '~' that i had created by mistake in a sub directory of my home directory with rm -rf ~ which of course started to eat through my home directory with a vengence, i managed to stop it before it went to far, i didn't have any recent backups, needless to say i've learning my lesson about having a good policy of regular backups. what are the recommended partition to backup if 1 i want to do a fresh reinstall e.g. to move to a larger hard drive. 2 for a disaster recovery like what i experienced above. i will be using ville walveranta's autodump 1.5a script which does a full dump on sundays and incremental dumps during the week, i already have /home /etc and /root set for backup, are there any other partitions i should bear in mind ? shadrock
Re: How to compact partitions (disklabel)?
> Am 13.06.2022 um 10:21 schrieb Stuart Henderson : > > On 2022-06-13, Mike Fischer wrote: >> After solving a recent problem on a VM where the /usr/local was full I was >> left with a disklabel that had a hole of unused space in it (see below for >> details). I was wondering if there is a way to compact the partitions, i.e. >> move the partitions following the deleted one up to fill the hole, >> potentially leaving corresponding free space at the end. >> >> I’d prefer to not have to use dd(1) on the raw device to move the data? I’d >> hope for something that is smart enough to adjust the disklabel after moving >> the bytes. Wishful thinking? > > There's no good way to do this. My preference would be to attach a new > virtual disk, partition either manually or according to current auto > defaults for the larger disk, dump|restore and run installboot, then > remove the old virtual disk. Ok, thanks! I thought I missed something ;-) > >> 16 partitions: >> #size offset fstype [fsize bsize cpg] >> f: 5056800 8025952 4.2BSD 2048 16384 12960 # /usr > > You might find this a little tight too after some updates. # df -h Filesystem SizeUsed Avail Capacity Mounted on /dev/sd0a 615M108M476M18%/ /dev/sd0k 3.7G798M2.7G22%/home /dev/sd0d 863M8.0K820M 0%/tmp /dev/sd0f 2.3G1.7G555M76%/usr /dev/sd0g 648M299M317M48%/usr/X11R6 /dev/sd0l 4.8G2.2G2.4G48%/usr/local /dev/sd0j 5.2G2.0K4.9G 0%/usr/obj /dev/sd0i 1.4G968M402M71%/usr/src /dev/sd0e 1.3G425M806M35%/var # 24% (555M) free seems ok for now, but thanks for the heads-up. Mike
Re: How to compact partitions (disklabel)?
On 2022-06-13, Mike Fischer wrote: > After solving a recent problem on a VM where the /usr/local was full I was > left with a disklabel that had a hole of unused space in it (see below for > details). I was wondering if there is a way to compact the partitions, i.e. > move the partitions following the deleted one up to fill the hole, > potentially leaving corresponding free space at the end. > > I’d prefer to not have to use dd(1) on the raw device to move the data? I’d > hope for something that is smart enough to adjust the disklabel after moving > the bytes. Wishful thinking? There's no good way to do this. My preference would be to attach a new virtual disk, partition either manually or according to current auto defaults for the larger disk, dump|restore and run installboot, then remove the old virtual disk. > 16 partitions: > #size offset fstype [fsize bsize cpg] > f: 5056800 8025952 4.2BSD 2048 16384 12960 # /usr You might find this a little tight too after some updates. -- Please keep replies on the mailing list.
How to compact partitions (disklabel)?
Hi! After solving a recent problem on a VM where the /usr/local was full I was left with a disklabel that had a hole of unused space in it (see below for details). I was wondering if there is a way to compact the partitions, i.e. move the partitions following the deleted one up to fill the hole, potentially leaving corresponding free space at the end. I’d prefer to not have to use dd(1) on the raw device to move the data? I’d hope for something that is smart enough to adjust the disklabel after moving the bytes. Wishful thinking? Details: Partition sd0h, ≈2.42 GB in size, containing /usr/local was full on a 20 GB virtual disk in VMWare Fusion, used for OpenBSD 7.1 stable, amd64. The partitions where originally created using the defaults in OpenBSD 6.8 IIRC. I enlarged the virtual disk in VMWare by 5 GB to 25 GB and then in single user mode I added a new sd0l partition using disklabel(8), created a file system on it, mounted the new file system and used dump(8)/restore(8) to copy the data. Then I modified /etc/fstab to use sd0l instead of sd0h and rebooted. Lastly I used disklabel(8) to delete sd0h. This left the aforementioned hole of unused data on disk. (For completeness sake I also adjusted the MBR using fdisk(8) to make the OpenBSD partition reflect the new size. But I’m not sure if that was even required. Seemed to work fine without that change.) The current disklabel looks like this: # disklabel sd0 # /dev/rsd0c: type: SCSI disk: SCSI disk label: VMware Virtual S duid: e592eaa53f566380 flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 2610 total sectors: 52428800 boundstart: 64 boundend: 52428800 drivedata: 0 16 partitions: #size offset fstype [fsize bsize cpg] a: 1299584 64 4.2BSD 2048 16384 10153 # / b: 2148640 1299648swap# none c: 524288000 unused d: 1833600 3448288 4.2BSD 2048 16384 12960 # /tmp e: 2744064 5281888 4.2BSD 2048 16384 12960 # /var f: 5056800 8025952 4.2BSD 2048 16384 12960 # /usr g: 1381856 13082752 4.2BSD 2048 16384 10710 # /usr/X11R6 i: 3059360 19538944 4.2BSD 2048 16384 12960 # /usr/src j: 11279680 22598304 4.2BSD 2048 16384 12960 # /usr/obj k: 8051648 33877984 4.2BSD 2048 16384 12960 # /home l: 10499168 41929632 4.2BSD 2048 16384 12960 # /usr/local # So partitions i through l would need to move. Thanks! Mike
Re: Auto layout for disk partitions - a new user's perspective
For people who really want to tinker with the ports system, perhaps non intuitively, bulk(8) is the manpage that contains the most exhaustive set of information about how you might want to configure your system and the choices involved.
Re: Auto layout for disk partitions - a new user's perspective
"James Mintram" writes: > For context, I need erlang 24 + elixir 13 and the current packages > are older than that. Which is why I have found myself working > with ports almost immediately (pro level yak shaving..) There were a post in ports mailing list with a patch for erlang port update: https://marc.info/?l=openbsd-ports=162162511924040=2 You may want to use it as starting point. -- Renato
Re: Auto layout for disk partitions - a new user's perspective
As a long time OpenBSD user I install from packages but also build from ports. There is a usage case for both, but realize building packages is not a "standard" system. Twenty years ago building packages from ports was the norm, but not today. 73 diana On April 18, 2022 9:35:27 AM MDT, Thomas Frohwein wrote: > > >I think it might be worth repeating that packages are the recommended >way to use third-party software. And that's also a great justification >why there is no /usr/ports partition on a default install. > >Unless you are doing ports development work, you shouldn't need the >ports tree.
Re: Auto layout for disk partitions - a new user's perspective
On Mon, Apr 18, 2022 at 04:15:46PM +, James Mintram wrote: > Thanks for all of the very useful replies, I have managed to get > everything working. > > For context, I need erlang 24 + elixir 13 and the current packages > are older than that. Which is why I have found myself working > with ports almost immediately (pro level yak shaving..) > > I ended up carving out some space from the /home partition > for ports and setting WRKOBJDIR as recommended. > > As for the /usr/src folder, I personally like to work with git > due to familiarity with the tools. I have seen got, and like the > idea of separating the repository from the worktree, so I will > look into using that. It also possible to change the sizes of the auto layout: choose " (E)dit auto layout" during install and type: R You can use suffixes and use increments, e.g. +4G to add 4G to a partition. -Otto > > On Mon, 18 Apr 2022, at 3:35 PM, Thomas Frohwein wrote: > > On Mon, Apr 18, 2022 at 01:36:18PM -, Stuart Henderson wrote: > > > > [...] > > > > > > 2) Should there be a /usr/local/pobj partition created with correct > > > > mount > > > > options? (I appreciate building ports is an "advanced" thing to do - > > > > but it > > > > feels weird having to mess with partition layout after a fresh install > > > > just to > > > > build them) > > > > > > Ports doesn't use /usr/local/pobj by default (you can set it via WRKOBJDIR > > > in mk.conf, but /usr/local isn't a great place for a filesystem with rapid > > > changes during a port build). Also, /usr/local/pobj *is* normally > > > wxallowed. > > > > > > If you are using ports I would strongly recommend a separate filesystem > > > for /usr/ports, either with default ports-related directories (i.e. don't > > > change dirs in mk.conf) and set that wxallowed, or with a separate > > > WRKOBJDIR > > > on a wxallowed filesystem. > > > > I think it might be worth repeating that packages are the recommended > > way to use third-party software. And that's also a great justification > > why there is no /usr/ports partition on a default install. > > > > Unless you are doing ports development work, you shouldn't need the > > ports tree. There are rare ports which don't have a package (for > > license reasons). If you need one of them, CVS has the advantage over > > git that you can checkout a subdirectory. If you do this for an > > individual port, the space requirements should be minimal. Still, for > > regular use you shouldn't need to deal with any of this. > > > > >
Re: Auto layout for disk partitions - a new user's perspective
On Mon, Apr 18, 2022 at 01:36:18PM -, Stuart Henderson wrote: [...] > > 2) Should there be a /usr/local/pobj partition created with correct mount > > options? (I appreciate building ports is an "advanced" thing to do - but it > > feels weird having to mess with partition layout after a fresh install just > > to > > build them) > > Ports doesn't use /usr/local/pobj by default (you can set it via WRKOBJDIR > in mk.conf, but /usr/local isn't a great place for a filesystem with rapid > changes during a port build). Also, /usr/local/pobj *is* normally wxallowed. > > If you are using ports I would strongly recommend a separate filesystem > for /usr/ports, either with default ports-related directories (i.e. don't > change dirs in mk.conf) and set that wxallowed, or with a separate WRKOBJDIR > on a wxallowed filesystem. I think it might be worth repeating that packages are the recommended way to use third-party software. And that's also a great justification why there is no /usr/ports partition on a default install. Unless you are doing ports development work, you shouldn't need the ports tree. There are rare ports which don't have a package (for license reasons). If you need one of them, CVS has the advantage over git that you can checkout a subdirectory. If you do this for an individual port, the space requirements should be minimal. Still, for regular use you shouldn't need to deal with any of this.
Re: Auto layout for disk partitions - a new user's perspective
the installer creates partition layouts for a variety of _regular_ usage patterns. Both of these situations you describe are not the normal pattern. We don't want to over-allocate space to specific purposes like that. Other systems do one giant root partition and then avoid these space issues. There are downsides with the policy of creating seperate partitions like we do, so that we can vary mountpoint flat options. We kind of shrug and cope with it. James Mintram wrote: > Hi. I am new to OpenBSD, so these questions come from my first > experience with the system. > > I selected the auto layout option when partitioning my 256GB drive. I have > then found issues while doing the following: > > 1) Cloning src from the github mirror and checking it out, completely fills > the /usr/src parition. > > 2) Building some ports fail because /usr/local/pobj is not on an wxallowed fs. > > > My questions are: > > 1) Should the default /usr/src partition be bigger on large disks? > > 2) Should there be a /usr/local/pobj partition created with correct mount > options? (I appreciate building ports is an "advanced" thing to do - but it > feels weird having to mess with partition layout after a fresh install just > to > build them)
Re: Auto layout for disk partitions - a new user's perspective
On 2022-04-18, James Mintram wrote: > Hi. I am new to OpenBSD, so these questions come from my first > experience with the system. > > I selected the auto layout option when partitioning my 256GB drive. I have > then found issues while doing the following: > > 1) Cloning src from the github mirror and checking it out, completely fills > the /usr/src parition. > > 2) Building some ports fail because /usr/local/pobj is not on an wxallowed fs. > > My questions are: > > 1) Should the default /usr/src partition be bigger on large disks? /usr/src is sized for just a checkout rather than a full repository mirror with history. This is normal for cvs (and if you do have a full repo mirror with cvs, that would be in a different place than /usr/src). If you're using the git conversion you could do a shallow checkout, or use a larger fs, or place it elsewhere. On a typical system I don't think it's helpful to have this much larger (though it is now starting to get a little tight for a checkout so maybe it could go up a few hundred MB). /usr/src isn't needed on a typical machine and raising the size will impact on sizes of other partitions, which might make it more likely people run into harder problems later.. > 2) Should there be a /usr/local/pobj partition created with correct mount > options? (I appreciate building ports is an "advanced" thing to do - but it > feels weird having to mess with partition layout after a fresh install just > to > build them) Ports doesn't use /usr/local/pobj by default (you can set it via WRKOBJDIR in mk.conf, but /usr/local isn't a great place for a filesystem with rapid changes during a port build). Also, /usr/local/pobj *is* normally wxallowed. If you are using ports I would strongly recommend a separate filesystem for /usr/ports, either with default ports-related directories (i.e. don't change dirs in mk.conf) and set that wxallowed, or with a separate WRKOBJDIR on a wxallowed filesystem. -- Please keep replies on the mailing list.
Re: Generating Disk Labels for New Fdisk Partitions
On Wed, Jan 19, 2022 at 10:13:02PM +, thatfatblack...@disroot.org wrote: > After installing OpenBSD I realized I was going to need another > partition, so I went ahead and created it with fdisk. > > Problem is that partition is not showing up like sd0i or sd0j. This is expected behaviour. Since your disk has a 'real' disklabel on it, any other fdisk partitions that you create will not be parsed and added automatically. > I need a way to generate it without having to reinstall OpenBSD. You need to add the details of the new, non-OpenBSD partition to the disklabel manually. This is not difficult. If you invoke disklabel with the -E parameter, you will be able to edit the disklabel partitions in the same way that you did in the installer, so you can use option 'a' to add a partition, and fill in the correct parameters, such as offset, size, etc. If you don't know the parameters, invoke disklabel with the -d parameter. This will show you how the kernel would parse the partitions if there was no disklabel already present. From this information, you should be able to pick out which is your new partition, probably i or j, note the details down, then add them as described above.
Re: Non-default partitions and upgrades
On Mon, Apr 12, 2021 at 08:08:12PM -0700, Paul Pace wrote: > Hello! > > I generally try and run things as a project recommends, but I am wondering > about running different additional partitions (e.g., add /var/www) or > changing partition letter (e.g., move /var to the end for convenient VPS > expansion). > > I know it isn't the biggest thing in the world, but would this ever have an > impact on running version upgrades? > > Thank you, > > Paul > That would work , unless you do crazy things. The upgrade script mounts the filesystems using the fstab on the system to be upgraded. -Otto
Non-default partitions and upgrades
Hello! I generally try and run things as a project recommends, but I am wondering about running different additional partitions (e.g., add /var/www) or changing partition letter (e.g., move /var to the end for convenient VPS expansion). I know it isn't the biggest thing in the world, but would this ever have an impact on running version upgrades? Thank you, Paul
Re: Default partitions allocate only 1GB to /
On Sun, Feb 28, 2021 at 08:30:14PM +0100, Janne Johansson wrote: Is /var a filesystem of its own? Otherwise it could be /var/tmp or some other place under /var which is used for unpacking packages. Yes, /var is on its own filesystem, with 10.4G available.
Re: Default partitions allocate only 1GB to /
On Sun, Feb 28, 2021 at 05:17:15PM +, James Cook wrote: > This makes little sense to me. Why should deleting a 20MB file on a > filesystem with >700MB free space be sufficient for the install to go > through? Especially when the install obviously doesn't need that much space > on the filesystem in question? That doesn't make sense to me either. Something strange is going on. Maybe someone else will have a guess. It did occur to me that between the first (unsuccessful) and 2nd (successful) attempts I also rebooted the machine and ran `pkg_add -u`.
Re: Default partitions allocate only 1GB to /
Den sön 28 feb. 2021 kl 14:51 skrev : > I deleted the file and `pkg_add libreoffice` worked as expected. > Post-install I still have 746MB free in /, according to `df -h`. > > This makes little sense to me. Why should deleting a 20MB file on a > filesystem with >700MB free space be sufficient for the install to go > through? Especially when the install obviously doesn't need that much > space on the filesystem in question? > > (space available in /usr/local went from 11.4G, pre-install, to 10.8G, > post-install... was `pkg_add` trying to stage files in /, even though > /tmp is a separate filesystem?) Is /var a filesystem of its own? Otherwise it could be /var/tmp or some other place under /var which is used for unpacking packages. -- May the most significant bit of your life be positive.
Re: Default partitions allocate only 1GB to /
On Sat, Feb 27, 2021 at 11:52:39PM +, James Cook wrote: Sorry, you're right, pkg_add can add files to /. But generally those will be quite small (/etc/make2fs.conf sounds like a configuration file). How big is your root partition, and how much space is used? For example mine is like this after several months of use and many packages installed, indicating the installer's default behaviour has worked well for me: falsifian angel ~ $ df -h / Filesystem SizeUsed Avail Capacity Mounted on /dev/sd2a 989M199M741M21%/ My root partition is about the same -- circa 1GB in size, about 700MB free. According to `df -h` my /user/local has 11.4GB available and /usr has 3.5GB, so there *should* be plenty of space for Libreoffice.
Re: Default partitions allocate only 1GB to /
On Sat, Feb 27, 2021 at 11:52:39PM +, James Cook wrote: If you have a lot more space used, you could try to figure out what's using it. My go-to command is "du -xah /|sort -h|less" That's a neat command, and amazingly enough it did the trick: there was a 20MB file, INS@yjf(...) located in the root directory. It looked like a copy of the kernel binary which had been saved while I was messing about with kernel configuration options. I deleted the file and `pkg_add libreoffice` worked as expected. Post-install I still have 746MB free in /, according to `df -h`. This makes little sense to me. Why should deleting a 20MB file on a filesystem with >700MB free space be sufficient for the install to go through? Especially when the install obviously doesn't need that much space on the filesystem in question? (space available in /usr/local went from 11.4G, pre-install, to 10.8G, post-install... was `pkg_add` trying to stage files in /, even though /tmp is a separate filesystem?)
Re: Default partitions allocate only 1GB to /
On Sat, Feb 27, 2021 at 03:27:41PM -0600, Edgar Pettijohn wrote: Its more likely that you accidentaly used dd to write to a usb stick and instead wrote to a file in /dev. Thats the only way I've ever had this problem. You're right -- I had written a file to /dev. After deleting it, the problem still comes up, unfortunately.
Re: Default partitions allocate only 1GB to /
On Sat, Feb 27, 2021 at 08:27:07PM +, James Cook wrote: Something's strange about your setup. The installer normally creates a separate partition for /usr and maybe /usr/local. If you're using pkg_add, then packages go in /usr/local, so they shouldn't end up on your root partition. If your disk is really tiny the installer won't create a separate /usr partition, but in that case it won't make a separate /home either. As far as I know everything was installed using defaults. Doing `pkg_add libreoffice` the installer is definitely looking at both / and /usr/local/ ... and it gives an odd bytecount for /: ``` Error: /dev/sda1 on / is not large enough (/etc/mke2fs.conf) /dev/sda1 on /: 956 bytes (missing 86470 blocks) /dev/sd1h on /usr/local: 4513435 bytes ``` Later it gives different byte counts for both values.
Re: Default partitions allocate only 1GB to /
On Sat, Feb 27, 2021 at 11:21:45PM +, tetrahe...@danwin1210.me wrote: > On Sat, Feb 27, 2021 at 08:27:07PM +, James Cook wrote: > > Something's strange about your setup. The installer normally creates a > > separate partition for /usr and maybe /usr/local. If you're using > > pkg_add, then packages go in /usr/local, so they shouldn't end up on > > your root partition. > > > > If your disk is really tiny the installer won't create a separate /usr > > partition, but in that case it won't make a separate /home either. > > As far as I know everything was installed using defaults. > > Doing `pkg_add libreoffice` the installer is definitely looking at both / > and /usr/local/ ... and it gives an odd bytecount for /: > > ``` > Error: /dev/sda1 on / is not large enough (/etc/mke2fs.conf) > /dev/sda1 on /: 956 bytes (missing 86470 blocks) > /dev/sd1h on /usr/local: 4513435 bytes > ``` > > Later it gives different byte counts for both values. > doas du -xh / should help you locate whats going on. Edgar
Default partitions allocate only 1GB to /
When installing OpenBSD, the default partition layout only allocates 1GB to / ... most of the disk space is allocated to /home. Once you start installing packages, / quickly grows beyond 1GB, and it looks like even some large packages exceed the available space on their own: Error: /dev/sda1 on / is not large enough Is there an easy fix for this that I'm missing somewhere, or is this a poorly chosen default?
Re: Default partitions allocate only 1GB to /
On Sat, Feb 27, 2021 at 11:21:45PM +, tetrahe...@danwin1210.me wrote: > On Sat, Feb 27, 2021 at 08:27:07PM +, James Cook wrote: > > Something's strange about your setup. The installer normally creates a > > separate partition for /usr and maybe /usr/local. If you're using > > pkg_add, then packages go in /usr/local, so they shouldn't end up on > > your root partition. > > > > If your disk is really tiny the installer won't create a separate /usr > > partition, but in that case it won't make a separate /home either. > > As far as I know everything was installed using defaults. > > Doing `pkg_add libreoffice` the installer is definitely looking at both / > and /usr/local/ ... and it gives an odd bytecount for /: > > ``` > Error: /dev/sda1 on / is not large enough (/etc/mke2fs.conf) Sorry, you're right, pkg_add can add files to /. But generally those will be quite small (/etc/make2fs.conf sounds like a configuration file). How big is your root partition, and how much space is used? For example mine is like this after several months of use and many packages installed, indicating the installer's default behaviour has worked well for me: falsifian angel ~ $ df -h / Filesystem SizeUsed Avail Capacity Mounted on /dev/sd2a 989M199M741M21%/ If you have a lot more space used, you could try to figure out what's using it. My go-to command is "du -xah /|sort -h|less" > /dev/sda1 on /: 956 bytes (missing 86470 blocks) > /dev/sd1h on /usr/local: 4513435 bytes > ``` > > Later it gives different byte counts for both values. -- James
Re: Default partitions allocate only 1GB to /
On Sat, Feb 27, 2021 at 03:32:44PM +, tetrahe...@danwin1210.me wrote: > When installing OpenBSD, the default partition layout only allocates 1GB to > / ... most of the disk space is allocated to /home. > > Once you start installing packages, / quickly grows beyond 1GB, and it looks > like even some large packages exceed the available space on their own: > Error: /dev/sda1 on / is not large enough > > Is there an easy fix for this that I'm missing somewhere, or is this a > poorly chosen default? > Its more likely that you accidentaly used dd to write to a usb stick and instead wrote to a file in /dev. Thats the only way I've ever had this problem. Edgar
Re: Default partitions allocate only 1GB to /
On Sat, Feb 27, 2021 at 03:32:44PM +, tetrahe...@danwin1210.me wrote: > When installing OpenBSD, the default partition layout only allocates 1GB to > / ... most of the disk space is allocated to /home. > > Once you start installing packages, / quickly grows beyond 1GB, and it looks > like even some large packages exceed the available space on their own: > Error: /dev/sda1 on / is not large enough > > Is there an easy fix for this that I'm missing somewhere, or is this a > poorly chosen default? Something's strange about your setup. The installer normally creates a separate partition for /usr and maybe /usr/local. If you're using pkg_add, then packages go in /usr/local, so they shouldn't end up on your root partition. If your disk is really tiny the installer won't create a separate /usr partition, but in that case it won't make a separate /home either. -- James
Re: The 16 partitions thread
I'd guess it's like dealing with children: "How come I can't lick the window when it's frozen? Gerri's mom lets her!"^1. Dhu On Thu, 30 Apr 2020 07:22:35 -0500 Ed Ahlsen-Girard wrote: > Some people read replies in misc and say, "wow, Theo and the OBSD devs > are obnoxiously harsh.' > > I read the 16 partitions thread and think, "I marvel at their patience > with interlocutors who have not read the relevant source code and give > no indication that they would understand it if they did." > > -- > > Edward Ahlsen-Girard > Ft Walton Beach, FL > > > -- Je suis Canadien. Ce n'est pas Francais ou Anglaise. C'est une esp`ece de sauvage: ne obliviscaris, vix ea nostra voco;-)
Re: The 16 partitions thread
On Thu, Apr 30, 2020 at 11:13 AM Consus wrote: > On Thu, Apr 30, 2020 at 07:22:35AM -0500, Ed Ahlsen-Girard wrote: > > I read the 16 partitions thread and think, "I marvel at their patience > > with interlocutors who have not read the relevant source code and give > > no indication that they would understand it if they did." > > Yeah, stupid users, right? Who needs them anyways. > You say that sarcastically, but that only shows that you don't know anything about OpenBSD. It has been stated many many times that OpenBSD is written by the developers, for those said developers. And they're sharing it with us. Think about that a little and then look at what you said again.
Re: The 16 partitions thread
On Thu, Apr 30, 2020 at 07:22:35AM -0500, Ed Ahlsen-Girard wrote: > Some people read replies in misc and say, "wow, Theo and the OBSD devs > are obnoxiously harsh.' > > I read the 16 partitions thread and think, "I marvel at their patience > with interlocutors who have not read the relevant source code and give > no indication that they would understand it if they did." Yeah, stupid users, right? Who needs them anyways.
The 16 partitions thread
Some people read replies in misc and say, "wow, Theo and the OBSD devs are obnoxiously harsh.' I read the 16 partitions thread and think, "I marvel at their patience with interlocutors who have not read the relevant source code and give no indication that they would understand it if they did." -- Edward Ahlsen-Girard Ft Walton Beach, FL
Re: More than 16 partitions
On Sat, Apr 25, 2020 at 2:41 PM Theo de Raadt wrote: > > Amelia A Lewis wrote: > > > So, and I recognize that the answer might reasonably be "go read more > > code and figure it out yourself," a question for Theo and others if you > > have a moment: why couldn't an arch expand past sixteen? It seems, both > > from the math calculating struct size (which may be mistaken, in which > > case I apologize) and in the comment for MAXMAXPARTITIONS that more > > *are* possible. > > Because there is another reason. Here are the device nodes for > two sequentially-numbered disks. > > brw-r- 1 root operator4, 0 Apr 17 11:50 sd0a > brw-r- 1 root operator4, 1 Apr 17 11:50 sd0b > brw-r- 1 root operator4, 2 Apr 17 11:50 sd0c > brw-r- 1 root operator4, 3 Apr 17 11:50 sd0d > brw-r- 1 root operator4, 4 Apr 17 11:50 sd0e > brw-r- 1 root operator4, 5 Apr 17 11:50 sd0f > brw-r- 1 root operator4, 6 Apr 17 11:50 sd0g > brw-r- 1 root operator4, 7 Apr 17 11:50 sd0h > brw-r- 1 root operator4, 8 Apr 17 11:50 sd0i > brw-r- 1 root operator4, 9 Apr 17 11:50 sd0j > brw-r- 1 root operator4, 10 Apr 17 11:50 sd0k > brw-r- 1 root operator4, 11 Apr 17 11:50 sd0l > brw-r- 1 root operator4, 12 Apr 17 11:50 sd0m > brw-r- 1 root operator4, 13 Apr 17 11:50 sd0n > brw-r- 1 root operator4, 14 Apr 17 11:50 sd0o > brw-r- 1 root operator4, 15 Apr 17 11:50 sd0p > brw-r- 1 root operator4, 16 Apr 17 11:50 sd1a > brw-r- 1 root operator4, 17 Apr 17 11:50 sd1b > brw-r- 1 root operator4, 18 Apr 17 11:50 sd1c > brw-r- 1 root operator4, 19 Apr 17 11:50 sd1d > brw-r- 1 root operator4, 20 Apr 17 11:50 sd1e > brw-r- 1 root operator4, 21 Apr 17 11:50 sd1f > brw-r- 1 root operator4, 22 Apr 17 11:50 sd1g > brw-r- 1 root operator4, 23 Apr 17 11:50 sd1h > brw-r- 1 root operator4, 24 Apr 17 11:50 sd1i > brw-r- 1 root operator4, 25 Apr 17 11:50 sd1j > brw-r- 1 root operator4, 26 Apr 17 11:50 sd1k > brw-r- 1 root operator4, 27 Apr 17 11:50 sd1l > brw-r- 1 root operator4, 28 Apr 17 11:50 sd1m > brw-r- 1 root operator4, 29 Apr 17 11:50 sd1n > brw-r- 1 root operator4, 30 Apr 17 11:50 sd1o > brw-r- 1 root operator4, 31 Apr 17 11:50 sd1p > > Look very carefully at this column ^^ > Are they allocated in the kernel in a linear fashion? If not, you could allocate additional nodes under a spare major for the extra partitions. If so, well I'm just talking out of my arse. I'd see for myself if I could find where they're allocated. I'll have more of a deep dive later. -- Aaron Mason - Programmer, open source addict I've taken my software vows - for beta or for worse
Re: More than 16 partitions
> Medoesn't a care a flying fsck about what is "trendy". Is this the most ironic sentence ever posted on here? Dubiously censoring an expletive with a common 'Unix' utility isn't motivated by some sort of desire to feel like a part of the righteous ones? Come on.
Re: More than 16 partitions
If you didn't make any of this up, you dumbed it down to the point where there's no useful info left. You seem to operate on the assumption that merely dissing the work of companies and from ecosystems you don't like, as though it's the 'trendy' thing to do, is enough for you to get by on this forum without scrutiny. -- Patrick Harper paia...@fastmail.com On Thu, 23 Apr 2020, at 18:06, zeurk...@volny.cz wrote: > "Groot" wrote: > > I've tried and failed to create more than 16 > > partitions on OpenBSD. First of all I don't > > understand the difference between the operations > > performed by fdisk and disklabel. Is it that > > OpenBSD sees partitions differently? First we > > create an OpenBSD partition with fdisk and then > > with disklabel we can create at the most 16 more > > filesystem partitions within it. > > Traditionally, BSD has used only its own disklabel(5). Unfortunately, > mess-dos on the IBM pee-cee set a competing standard, the "Master Boot > Record", with a separate partition table (and a lot of kludging to > support more than 4 partitions). While it was (and AFAIK remains) > possible to use the whole disk the traditional way (only a BSD > disklabel, as on e.g. sparc64), it has become common practice to wrap > the BSD stuff in a mess-dos partition, with the caveat that some of the > mess-dos partition entries are duplicated in the BSD label. > > Thus, the BSD label is essentially OpenBSD's version of the structure of > things on the disk. But is an imperfect version: 16 partitions *is* the > limit for an OpenBSD label, and, of course, mess-dos partition > identifiers (which are more *ahem* fine-grained) are not used. To top it > off, partitions which rest within the mess-dos OpenBSD partition are not > necessarily represented on the mess-dos level (this would count, from > the mess-dos perspective, as overlap between partitions and thus confuse > a great many tools). > > Then GPT entered the story to make the mess complete. But me'll remain > blissfully unaware of the inner workings of that particular clusterfsck, > if you don't mind ;) > > It's no shame to be confused by this garbage. Almost all of us'd like > better, but for the above hysterical raisins, it's not so easy to make > it so. > > --zeurkous. > > -- > Friggin' Machines! > >
Re: More than 16 partitions
Amelia A Lewis wrote: > So, and I recognize that the answer might reasonably be "go read more > code and figure it out yourself," a question for Theo and others if you > have a moment: why couldn't an arch expand past sixteen? It seems, both > from the math calculating struct size (which may be mistaken, in which > case I apologize) and in the comment for MAXMAXPARTITIONS that more > *are* possible. Because there is another reason. Here are the device nodes for two sequentially-numbered disks. brw-r- 1 root operator4, 0 Apr 17 11:50 sd0a brw-r- 1 root operator4, 1 Apr 17 11:50 sd0b brw-r- 1 root operator4, 2 Apr 17 11:50 sd0c brw-r- 1 root operator4, 3 Apr 17 11:50 sd0d brw-r- 1 root operator4, 4 Apr 17 11:50 sd0e brw-r- 1 root operator4, 5 Apr 17 11:50 sd0f brw-r- 1 root operator4, 6 Apr 17 11:50 sd0g brw-r- 1 root operator4, 7 Apr 17 11:50 sd0h brw-r- 1 root operator4, 8 Apr 17 11:50 sd0i brw-r- 1 root operator4, 9 Apr 17 11:50 sd0j brw-r- 1 root operator4, 10 Apr 17 11:50 sd0k brw-r- 1 root operator4, 11 Apr 17 11:50 sd0l brw-r- 1 root operator4, 12 Apr 17 11:50 sd0m brw-r- 1 root operator4, 13 Apr 17 11:50 sd0n brw-r- 1 root operator4, 14 Apr 17 11:50 sd0o brw-r- 1 root operator4, 15 Apr 17 11:50 sd0p brw-r- 1 root operator4, 16 Apr 17 11:50 sd1a brw-r- 1 root operator4, 17 Apr 17 11:50 sd1b brw-r- 1 root operator4, 18 Apr 17 11:50 sd1c brw-r- 1 root operator4, 19 Apr 17 11:50 sd1d brw-r- 1 root operator4, 20 Apr 17 11:50 sd1e brw-r- 1 root operator4, 21 Apr 17 11:50 sd1f brw-r- 1 root operator4, 22 Apr 17 11:50 sd1g brw-r- 1 root operator4, 23 Apr 17 11:50 sd1h brw-r- 1 root operator4, 24 Apr 17 11:50 sd1i brw-r- 1 root operator4, 25 Apr 17 11:50 sd1j brw-r- 1 root operator4, 26 Apr 17 11:50 sd1k brw-r- 1 root operator4, 27 Apr 17 11:50 sd1l brw-r- 1 root operator4, 28 Apr 17 11:50 sd1m brw-r- 1 root operator4, 29 Apr 17 11:50 sd1n brw-r- 1 root operator4, 30 Apr 17 11:50 sd1o brw-r- 1 root operator4, 31 Apr 17 11:50 sd1p Look very carefully at this column ^^
Re: More than 16 partitions
I have a dread sense that I'm going to regret asking questions, but I'm
going to do it anyway, because hey, what the hell, I can always drink
bleach.
On Fri, 24 Apr 2020 19:09:53 -0600, Theo de Raadt wrote:
[snip]
>
> Reality hasn't changed. A sector is still 512 bytes, and
> disklabel has to fit in it.
I doubt that it will improve anything, but some of us (well, at least
me) got intrigued by your answers and the history (I thought that
sixteen was chosen because four bits and it was significant for that
reason, but that was clearly leaping to the same sorts of conclusions
from a different direction than disk size, which was embarrassing). So
I went and had a look at the struct definition.
Some summary for misc-readers who are not Theo or other devs; y'all
skip over me doing base-level analysis, 'kay? Otherwise you'll be rilly
annoyed with me for seeming to talk down to you, but this is for the
other folks who responded on thread without looking at disklabel.h.
/usr/src/sys/disklabel.h, the struct disklabel is the bit in question:
struct disklabel {
u_int32_t d_magic; /* the magic number */
u_int16_t d_type; /* drive type */
u_int16_t d_subtype;/* controller/d_type specific */
char d_typename[16]; /* type name, e.g. "eagle" */
char d_packname[16]; /* pack identifier */
/* disk geometry: */
u_int32_t d_secsize;/* # of bytes per sector */
u_int32_t d_nsectors; /* # of data sectors per track
*/
u_int32_t d_ntracks;/* # of tracks per cylinder */
u_int32_t d_ncylinders; /* # of data cylinders per unit
*/
u_int32_t d_secpercyl; /* # of data sectors per
cylinder */
u_int32_t d_secperunit; /* # of data sectors (low part)
*/
u_char d_uid[8]; /* Unique label identifier. */
/*
* Alternate cylinders include maintenance, replacement,
configuration
* description areas, etc.
*/
u_int32_t d_acylinders; /* # of alt. cylinders per unit
*/
/* hardware characteristics: */
u_int16_t d_bstarth;/* start of useable region
(high part) */
u_int16_t d_bendh; /* size of useable region (high
part) */
u_int32_t d_bstart; /* start of useable region */
u_int32_t d_bend; /* end of useable region */
u_int32_t d_flags; /* generic flags */
#define NDDATA 5
u_int32_t d_drivedata[NDDATA]; /* drive-type specific
information */
u_int16_t d_secperunith;/* # of data sectors (high
part) */
u_int16_t d_version;/* version # (1=48 bit
addressing) */
#define NSPARE 4
u_int32_t d_spare[NSPARE]; /* reserved for future use */
u_int32_t d_magic2; /* the magic number (again) */
u_int16_t d_checksum; /* xor of data incl. partitions
*/
/* filesystem and partition information: */
u_int16_t d_npartitions; /* number of partitions in
following */
u_int32_t d_bbsize; /* size of boot area at sn0,
bytes */
u_int32_t d_sbsize; /* max size of fs superblock,
bytes */
/* at this point, the struct as a whole occupies 160 bytes, in general
metadata
about the disklabel-as-a-whole, not any specific partition. next is
partitions: */
struct partition { /* the partition table */
u_int32_t p_size; /* number of sectors (low part)
*/
u_int32_t p_offset; /* starting sector (low part) */
u_int16_t p_offseth;/* starting sector (high part)
*/
u_int16_t p_sizeh; /* number of sectors (high
part) */
u_int8_t p_fstype; /* filesystem type, see below */
u_int8_t p_fragblock; /* encoded filesystem
frag/block */
u_int16_t p_cpg;/* UFS: FS cylinders per group
*/
} d_partitions[MAXPARTITIONS]; /* actually may be more */
};
struct partition is 16 bytes, times MAXPARTITIONS.
Note that MAXPARTITIONS isn't defined in this header; it's in the
per-arch disklabel.h (/usr/src/arch/*/include/disklabel.h). On the
other hand, at the top of the file is this:
/*
* The absolute maximum number of disk partitions allowed.
* This is the maximum value of MAXPARTITIONS for which 'struct
disklabel'
* is <= DEV_BSIZE bytes long. If MAXPARTITIONS is greater than this,
beware.
*/
#define MAXMAXPARTITIONS22
#if MAXPARTITIONS > MAXMAXPARTITIONS
#warn beware: MAXPARTITIONS bigger than MAXMAXPARTITIONS
#endif
Now, to cut to the chase, /usr/src/arch/*/include/disklabel.h all have
"#define MAXPARTITIONS 16" (whitespace varies, but the number does
Re: More than 16 partitions
Theo de Raadt writes: > Reality hasn't changed. A sector is still 512 bytes, and > disklabel has to fit in it. OK. Allan
Re: More than 16 partitions
Strahil Nikolov wrote: > On April 25, 2020 4:09:53 AM GMT+03:00, Theo de Raadt > wrote: > >Allan Streib wrote: > > > >> Theo de Raadt writes: > >> > >> > OpenBSD has apparently become popular amongst people who can't > >think > >> > and connect "real world constraints" and "reality" with "no > >alternative > >> > decision was possible". This is very common amongst people who > >won't > >> > lift their finger. > >> > >> I'm not the one complaining about the 16 partition limit, and I'm not > >> asking for anything to change. I've only said I think it's something > >> that is the way it is because of the design decisions made on the > >basis > >> of "reality" at the time, and which probably didn't contemplate the > >day > >> when everyone would have multi-terabyte hard drives and that people > >> might want more than 16 partitions. I stand corrected on that > >> speculation if I'm wrong. > > > >Reality hasn't changed. A sector is still 512 bytes, and > >disklabel has to fit in it. > > > >You are not LISTENING. > > Does this mean that with a sector of 4096 (modern HDDs/SSDs) and a patch - > we can have larger disklabel ? We access all disks as multiples of 512, no matter what their underlying storage layout is. We align some stuff later, but that does not change the fact "struct disklabel" is precisely 512 bytes longer. Feel free to make your own version of code that works on some machines, but not other machines. Feel free to figure out for yourself how it solves nothing.
Re: More than 16 partitions
On April 25, 2020 4:09:53 AM GMT+03:00, Theo de Raadt wrote: >Allan Streib wrote: > >> Theo de Raadt writes: >> >> > OpenBSD has apparently become popular amongst people who can't >think >> > and connect "real world constraints" and "reality" with "no >alternative >> > decision was possible". This is very common amongst people who >won't >> > lift their finger. >> >> I'm not the one complaining about the 16 partition limit, and I'm not >> asking for anything to change. I've only said I think it's something >> that is the way it is because of the design decisions made on the >basis >> of "reality" at the time, and which probably didn't contemplate the >day >> when everyone would have multi-terabyte hard drives and that people >> might want more than 16 partitions. I stand corrected on that >> speculation if I'm wrong. > >Reality hasn't changed. A sector is still 512 bytes, and >disklabel has to fit in it. > >You are not LISTENING. Does this mean that with a sector of 4096 (modern HDDs/SSDs) and a patch - we can have larger disklabel ? Best Regards, Strahil Nikolov
Re: More than 16 partitions
Allan Streib wrote: > Theo de Raadt writes: > > > OpenBSD has apparently become popular amongst people who can't think > > and connect "real world constraints" and "reality" with "no alternative > > decision was possible". This is very common amongst people who won't > > lift their finger. > > I'm not the one complaining about the 16 partition limit, and I'm not > asking for anything to change. I've only said I think it's something > that is the way it is because of the design decisions made on the basis > of "reality" at the time, and which probably didn't contemplate the day > when everyone would have multi-terabyte hard drives and that people > might want more than 16 partitions. I stand corrected on that > speculation if I'm wrong. Reality hasn't changed. A sector is still 512 bytes, and disklabel has to fit in it. You are not LISTENING.
Re: More than 16 partitions
Theo de Raadt writes: > OpenBSD has apparently become popular amongst people who can't think > and connect "real world constraints" and "reality" with "no alternative > decision was possible". This is very common amongst people who won't > lift their finger. I'm not the one complaining about the 16 partition limit, and I'm not asking for anything to change. I've only said I think it's something that is the way it is because of the design decisions made on the basis of "reality" at the time, and which probably didn't contemplate the day when everyone would have multi-terabyte hard drives and that people might want more than 16 partitions. I stand corrected on that speculation if I'm wrong. Allan
Re: More than 16 partitions
Ingo Schwarze wrote: The limitation to 16 partitions definitely feels painful to me. There is softraid(4). The only discipline that supports a single chunk is crypto. Make a couple of OpenBSD RAID partitions, set them up as crypto, partition those new crypto pseudo-devices, up to 16 partitions each, mkfs, mount, profit. From what I understand, crypto is not that big a performance hit if you have AES on your AMD64 intruction set. I just tried it on a small 16GB usb flash drive. The only issue was getting a clean reboot (rc.local to configure the pseudo-devices succeeds but mount happens too early). John
Re: More than 16 partitions
A little bit of fun, slightly related to some of the discussion: [1] is something that comes into my mind each time i read some of the emails [2] is coming next [1] https://www.youtube.com/watch?v=nlt5Wa13fFU [2] https://www.youtube.com/watch?v=zIV4poUZAQo
Re: More than 16 partitions
Allan Streib wrote: > Theo de Raadt writes: > > > Allan Streib wrote: > > > >> Seems like one of those numbers that was chosen long ago, when disks > >> had orders of magnitude less storage capacity they have now, and 16 > >> partitions really would have been more than enough. > > > > the word "chosen" makes it seem like such an arbitrary decision. > > No, didn't mean to imply arbitrary or ill-considered, more that > someone(s) decided that to be an adequate number, considering various > requirements and constraints. At the time, that would probably not have > included the common availability of multi-terabyte drives. Obviously I > wasn't there. Incorrect. And wow, you demonstrate an amazing inability to read. All the offsets and sizes have to fit in a 512 byte sector. It wasn't chosen. THERE WAS NO OTHER WAY. OpenBSD has apparently become popular amongst people who can't think and connect "real world constraints" and "reality" with "no alternative decision was possible". This is very common amongst people who won't lift their finger.
Re: More than 16 partitions
Theo de Raadt writes: > Allan Streib wrote: > >> Seems like one of those numbers that was chosen long ago, when disks >> had orders of magnitude less storage capacity they have now, and 16 >> partitions really would have been more than enough. > > the word "chosen" makes it seem like such an arbitrary decision. No, didn't mean to imply arbitrary or ill-considered, more that someone(s) decided that to be an adequate number, considering various requirements and constraints. At the time, that would probably not have included the common availability of multi-terabyte drives. Obviously I wasn't there. Allan
Re: More than 16 partitions
Allan Streib wrote:
> Seems like one of those numbers that was chosen long ago, when disks
> had orders of magnitude less storage capacity they have now, and 16
> partitions really would have been more than enough.
the word "chosen" makes it seem like such an arbitrary decision.
As currently written, the partition table must fit in a single sector.
Read disklabel.h and figure out how this part works:
struct partition { /* the partition table */
u_int32_t p_size; /* number of sectors (low part) */
u_int32_t p_offset; /* starting sector (low part) */
u_int16_t p_offseth;/* starting sector (high part) */
u_int16_t p_sizeh; /* number of sectors (high part) */
u_int8_t p_fstype; /* filesystem type, see below */
u_int8_t p_fragblock; /* encoded filesystem frag/block */
u_int16_t p_cpg;/* UFS: FS cylinders per group */
} d_partitions[MAXPARTITIONS]; /* actually may be more */
...
#define DL_GETDSIZE(d) (((u_int64_t)(d)->d_secperunith << 32) + \
(d)->d_secperunit)
#define DL_SETDSIZE(d, n) do { \
u_int64_t x = (n); \
(d)->d_secperunith = x >> 32; \
(d)->d_secperunit = x; \
} while (0)
#define DL_GETBSTART(d) (((u_int64_t)(d)->d_bstarth << 32) + \
(d)->d_bstart)
#define DL_SETBSTART(d, n) do { \
u_int64_t x = (n); \
(d)->d_bstarth = x >> 32; \
(d)->d_bstart = x; \
} while (0)
etc etc etc
Long time ago, it was chosen that cars should have 4 wheels.
Re: More than 16 partitions
Ingo Schwarze writes: > The limitation to 16 partitions definitely feels painful to me. Well, one pragmatic solution is to add another disk -- 16 more partitions. Not always possible, granted. Seems like one of those numbers that was chosen long ago, when disks had orders of magnitude less storage capacity they have now, and 16 partitions really would have been more than enough. Allan
Re: More than 16 partitions
Lars, Your email didn't contain a diff. Is there a reason for that? I'm wondering whether it is because it is too difficult for you, or maybe it is too difficult for everyone, or maybe you are simply talking out of your ass by trying to assign work to other people because that is your nature? > HAMMER2 could be ported. There is much collaboration between OpenBSD > and DragonflyBSD already (drivers for example). > https://www.dragonflybsd.org/docs/handbook/environmentquickstart/#index3h2 > > On Thu, 2020-04-23 at 16:48 -0400, Eric Furman wrote: > > ZFS cannot be ported to OBSD. It has an unacceptable license. > > If something like it were to be used on OBSD it would have to be > > written from scratch with a BSD license and it has already been > > discussed at length on this list how hard that is. > > Besides it is not really necessary. ZFS is overly complex and not > > needed in most cases. > -- > Lars Schotte > Mudroňova 13 > 92101 Piešťany >
Re: More than 16 partitions
HAMMER2 could be ported. There is much collaboration between OpenBSD and DragonflyBSD already (drivers for example). https://www.dragonflybsd.org/docs/handbook/environmentquickstart/#index3h2 On Thu, 2020-04-23 at 16:48 -0400, Eric Furman wrote: > ZFS cannot be ported to OBSD. It has an unacceptable license. > If something like it were to be used on OBSD it would have to be > written from scratch with a BSD license and it has already been > discussed at length on this list how hard that is. > Besides it is not really necessary. ZFS is overly complex and not > needed in most cases. -- Lars Schotte Mudroňova 13 92101 Piešťany
Re: More than 16 partitions
Hi Strahil, Strahil Nikolov wrote on Thu, Apr 23, 2020 at 11:16:41PM +0300: > And who the hell needs more than 16 partitions ? Me, and i'm quite sure many do. It's certainly not a good idea to combine any partitions that are separate in a default install because there are good reasons for all the separations. Then, on a development machine, i want separate partitions for src, xenocara, ports and the related obj partitions. Then i need a noperm partition for building releases. And because i sometimes work on free software projects outside OpenBSD, i want a partition for source code control checkouts of other projects. I certainly don't want to to mix /co/ that into /home/ or /usr/src/. At this point, here is the bare minimum i typically have: 01 a / 02 b swap 03 c whole disk 04 d /tmp nodev nosuid 05 e /var nodev nosuid 06 f /usr nodev 07 g /usr/local nodev wxallowed 08 h /usr/src nodev nosuid 09 i /usr/obj nodev nosuid 10 j /usr/xenocaranodev nosuid 11 k /usr/xobjnodev nosuid 12 l /usr/ports nodev nosuid 13 m /usr/ports/pobj nodev nosuid wxallowed 14 n /co nodev nosuid 15 o /co/destdir nodev noexec noperm 16 p /homenodev nosuid So, i'm out of partitions before i even start doing anything even mildly special. I don't even have a single spare partition, even though having that would be quite useful for better keeping track of unused space on the disk that i might keep around for maybe using it later. The limitation to 16 partitions definitely feels painful to me. > Why not we just port ZFS from FreeBSD, or LVM from Linux For starters, neither are free software. Both are encumbered with very restrictive licenses that prevent using them last time i looked (CDDL and GPL). Besides, OpenBSD values code that is small, simple, and easy to audit, so i'm not sure whether these would qualify even if they were free. So trying to port them would be a waste of time. Trying to port HAMMER from DragonFly might be worthwhile, but a huge and difficult project, and i'm not sure it is related to the limitation of the number of partitions. Yours, Ingo
Re: More than 16 partitions
On 2020-04-23, Ian Darwin wrote: > So: I was able to newfs, mount, and use an OpenBSD partition which > disklabel called 'a' and which had no trace of an fdisk partition around it. > > As Allan pointed out, this is not for booting from - none of those > fdisk partitions looks very healthy. biosboot(8) has an MBR boot signature. If the BIOS doesn't check for a valid MBR partition table--some do, some don't--then it should be able to directly run biosboot(8) from sector 0. installboot(8) tries to prevent such a configuration, but it could be tweaked, or you could try to tweak the disklabel and set the type to floppy, because floppies don't have MBR partitions. -- Christian "naddy" Weisgerber na...@mips.inka.de
Re: More than 16 partitions
On 2020-04-24 04:45, zeurk...@volny.cz wrote: > Your point is well-taken (though this is just the way mespeaks); yet, > Theo is a native speaker No-one is a native speaker of this made up crap, mecraps
Re: More than 16 partitions
On Thu, Apr 23, 2020 at 10:29:01PM +0200, Francois Pussault wrote: > I agree ; Using more than 10 partitions is rare but in case of NFS or other > network shares of course. > 16 is really enough in my point of view. > I've got to disgree with this one. I'm doing porting work. I yank out all of the directories except /usr/ports itself, using mk.conf. I then also make another partition /usr/ports/mystuff umount /usr/ports/mystuff umount /usr/ports newfs /usr/ports, etc. remount /usr/ports, mkdir /usr/ports/mystuff, remount /usr/ports/mystuff tar xzvf ports.tar.gz into /usr/ports and I can continue on working, without having lost any work I'm still examining. Working with retail equipment at home for a normal desktop. 16 OK Power often fails or hardware fails. Working on a server. Power almost never fails, nor the hardware. At home I run built-in HD, USB flash and USB HD. 16 is no problem with three HD's. I can ro lots of stuff and I need to. I'm not doing any porting at home, only on server hardware. Too tired of reliability issues at home. That's just what I(me)thinks. |-} There be-is-are some very good, cheap, rugged and waterproof USB HD's out there. Very portable(s). Bye, Chris
Re: More than 16 partitions
On 2020-04-23 11:45, zeurk...@volny.cz wrote: "Jan Betlach" wrote: For a non-native English speaker like myself, it is very difficult to read your mestuff... Your point is well-taken (though this is just the way mespeaks); yet, Theo is a native speaker, and he seems to have completely missed the content of merecent responses. Weird, isn't it? Anyway, as this would appear to be quite OT, me'd suggest we continue this (if at all) in private mail... Jan Take care, --zeurkous. Say Hi to Boss Nass for me next time you're in Otoh Gunga. Jordan
Re: More than 16 partitions
On Thu, Apr 23, 2020 at 04:42:53PM -0400, Allan Streib wrote: > > So, can I setup openBSD labels on x86_64 without legacy/GPT partition > > first ? > > IIRC yes you can, as long as you don't need to boot from that disk. Easily confirmed (a few false starts deleted from this transcript): $ uname -a OpenBSD foo.darwinsys.com 6.7 GENERIC.MP#145 amd64 # Here I plugged in a cheap USB device $ dmesg | tail -4 umass0: using SCSI over Bulk-Only scsibus4 at umass0: 2 targets, initiator 0 sd2 at scsibus4 targ 1 lun 0: removable serial.18a5023805270130 sd2: 3750MB, 512 bytes/sector, 768 sectors # Trash any existing fdisk and disklabel info # dd if=/dev/random of=/dev/rsd2c bs=512 count=100 100+0 records in 100+0 records out 51200 bytes transferred in 0.068 secs (742845 bytes/sec) # disklabel sd2 # /dev/rsd2c: type: SCSI disk: SCSI disk label: Store n Go Drive duid: flags: bytes/sector: 512 sectors/track: 63 tracks/cylinder: 255 sectors/cylinder: 16065 cylinders: 478 total sectors: 768 boundstart: 0 boundend: 768 drivedata: 0 16 partitions: #size offset fstype [fsize bsize cpg] c: 7680 unused i: 7679944 56 MSDOS # fdisk sd2 # confirm there is no fdisk table, just random rubbish Disk: sd2 geometry: 478/255/63 [768 Sectors] Offset: 0 Signature: 0x111 Starting Ending LBA Info: #: id C H S - C H S [ start:size ] --- 0: 82 77157 27 55 - 172421 98 24 [ 1239528960: 1530420603 ] Linux swap 1: 64 10096 3 23 - 176047 141 26 [ 162192451: 2666011513 ] NetWare 2.xx 2: 6E 252409 74 42 - 209458 117 56 [ 4054955288: 3604962205 ] 3: A9 19978 12 42 - 22375 228 62 [ 320947367:38521434 ] NetBSD # disklabel -E sd2 Label editor (enter '?' for help at any prompt) sd2> p OpenBSD area: 0-768; size: 768; free: 768 #size offset fstype [fsize bsize cpg] c: 7680 unused sd2> a partition: [a] offset: [0] 64 size: [7679936] 100M FS type: [4.2BSD] sd2*> w sd2> q No label changes. # newfs /dev/rsd2a /dev/rsd2a: 101.9MB in 208768 sectors of 512 bytes 4 cylinder groups of 25.48MB, 1631 blocks, 3328 inodes each super-block backups (for fsck -b #) at: 32, 52224, 104416, 156608, # mount /dev/sd2a /mnt $ ls /mnt $ date | doas dd of=/mnt/date.txt 0+1 records in 0+1 records out 29 bytes transferred in 0.000 secs (322584 bytes/sec) $ ls /mnt date.txt $ cat /mnt/date.txt Thu Apr 23 18:55:35 EDT 2020 # fdisk sd2 # still no fdisk table Disk: sd2 geometry: 478/255/63 [768 Sectors] Offset: 0 Signature: 0x111 Starting Ending LBA Info: #: id C H S - C H S [ start:size ] --- 0: 82 77157 27 55 - 172421 98 24 [ 1239528960: 1530420603 ] Linux swap 1: 64 10096 3 23 - 176047 141 26 [ 162192451: 2666011513 ] NetWare 2.xx 2: 6E 252409 74 42 - 209458 117 56 [ 4054955288: 3604962205 ] 3: A9 19978 12 42 - 22375 228 62 [ 320947367:38521434 ] NetBSD # So: I was able to newfs, mount, and use an OpenBSD partition which disklabel called 'a' and which had no trace of an fdisk partition around it. As Allan pointed out, this is not for booting from - none of those fdisk partitions looks very healthy.
Re: More than 16 partitions
> > From: Strahil Nikolov > Sent: Thu Apr 23 22:16:41 CEST 2020 > To: , Theo de Raadt , > > Cc: Martin Schröder > Subject: Re: More than 16 partitions > > > On April 23, 2020 10:46:44 PM GMT+03:00, Theo de Raadt > wrote: > >You need to stop making this mailing list just about you. > > > >STFU. > > > > > > wrote: > > > >> "Martin Schröder" wrote: > >> > Am Do., 23. Apr. 2020 um 21:31 Uhr schrieb : > >> >> No problem. Would it be too crude a suggestion that we go back to > >the > >> >> content now...? > >> > > >> > You didn't provide any patch. > >> > >> That is entirely correct. > >> > >> --zeurkous. > >> > >> -- > >> Friggin' Machines! > >> > > Some of these e-mails were useful others not... > > So, can I setup openBSD labels on x86_64 without legacy/GPT partition first ? > And who the hell needs more than 16 partitions ? Why not we just port ZFS > from FreeBSD, or LVM from Linux and get over it ? hello, I agree ; Using more than 10 partitions is rare but in case of NFS or other network shares of course. 16 is really enough in my point of view. > > P.S.: The last one was not a real question, but I would like to hear if > anyone has attempted to port any of these 2 . > > Best Regards, > Strahil Nikolov > Cordialement Francois Pussault 10 chemin de négo saoumos apt 202 - bat 2 31300 Toulouse +33 6 17 230 820 fpussa...@contactoffice.fr
Re: More than 16 partitions
On Thu, Apr 23, 2020 at 08:14:25PM +0200, Jan Betlach wrote: > For a non-native English speaker like myself, it is very difficult to read > your mestuff… One may practice by reading Gollum/Smeagol-passages..
Re: More than 16 partitions
For a non-native English speaker like myself, it is very difficult to read your mestuff… Jan On 23 Apr 2020, at 19:47, zeurk...@volny.cz wrote: theo wrote: That is a rewriting of history. It's history the way meknows it. Mecertainly predates some of it. The disklabel format predates the PC. Indeed. Mewasn't sure where and when exactly it appeared, so meleft that bit out. But medid know it was older, and metried to communicate that fact (obviously mefailed -- meapologizes). It came from the the ancient attempt to handle things in CSRG's 4.3reno/4.4 work on the hp300. It was probably a rewrite of the native HPUX disk format. Hmm, hp300, eh? This was then put on all the other architectures, as a unified view of the disk. It was modified and extended on as as-needed basis. Rewriting the history like this is pathetic inaccurate and narrowminded. Your history is absolutely false and you've made up a bunch of balony. So, what did memake up? Did mepresent a timeline? An exact order of events? Did mepresent a scientific study? Or didmejust try to give an overview of things in terms that Groot (and many others, mesuspects) may just understand? It is not true, and even a elementary review of the history of disklabel.h back into the early NetBSD tree will make it clear what's going on. Like mesaid, it's the history the way meknows it. Me's not a bloody authority on the history of either BSD or the IBM pee-cee, *at all*. Perhaps meshould've made that clearer. OH, and I did most of the early work post-CSRG, because we needed to "emulate" this on SunOS, and I ported Torek's sparc code into NetBSD. Mehas _no doubt at all_ that you know BSD (including its history) better than me (that is, of course, an understatement). I urge you to stop posting such balony. Then it's me turn to urge you to not read me overview as an historical account of any exactness. After all, the goal, for me, was trying to help Groot understand the relationships he sought clarification for. Perhaps meindeed should've included a disclaimer. Then again, mehas no offical role here (nor does mewant one), and in no way are me words to be taken for the one and universal truth. Can we please just assume that Groot is mature enough to be able to form his own view based on our individual contributions? Me'd like that. --zeurkous. -- Friggin' Machines!
Re: More than 16 partitions
On Thu, Apr 23, 2020, at 4:16 PM, Strahil Nikolov wrote: > So, can I setup openBSD labels on x86_64 without legacy/GPT partition > first ? > And who the hell needs more than 16 partitions ? Why not we just port > ZFS from FreeBSD, or LVM from Linux and get over it ? > > P.S.: The last one was not a real question, but I would like to hear > if anyone has attempted to port any of these 2 . ZFS cannot be ported to OBSD. It has an unacceptable license. If something like it were to be used on OBSD it would have to be written from scratch with a BSD license and it has already been discussed at length on this list how hard that is. Besides it is not really necessary. ZFS is overly complex and not needed in most cases.
Re: More than 16 partitions
> So, can I setup openBSD labels on x86_64 without legacy/GPT partition first ? IIRC yes you can, as long as you don't need to boot from that disk. Allan
Re: More than 16 partitions
On April 23, 2020 10:46:44 PM GMT+03:00, Theo de Raadt wrote: >You need to stop making this mailing list just about you. > >STFU. > > > wrote: > >> "Martin Schröder" wrote: >> > Am Do., 23. Apr. 2020 um 21:31 Uhr schrieb : >> >> No problem. Would it be too crude a suggestion that we go back to >the >> >> content now...? >> > >> > You didn't provide any patch. >> >> That is entirely correct. >> >> --zeurkous. >> >> -- >> Friggin' Machines! >> Some of these e-mails were useful others not... So, can I setup openBSD labels on x86_64 without legacy/GPT partition first ? And who the hell needs more than 16 partitions ? Why not we just port ZFS from FreeBSD, or LVM from Linux and get over it ? P.S.: The last one was not a real question, but I would like to hear if anyone has attempted to port any of these 2 . Best Regards, Strahil Nikolov
Re: More than 16 partitions
You need to stop making this mailing list just about you. STFU. wrote: > "Martin Schröder" wrote: > > Am Do., 23. Apr. 2020 um 21:31 Uhr schrieb : > >> No problem. Would it be too crude a suggestion that we go back to the > >> content now...? > > > > You didn't provide any patch. > > That is entirely correct. > > --zeurkous. > > -- > Friggin' Machines! >
RE: More than 16 partitions
"Martin Schröder" wrote: > Am Do., 23. Apr. 2020 um 21:31 Uhr schrieb : >> No problem. Would it be too crude a suggestion that we go back to the >> content now...? > > You didn't provide any patch. That is entirely correct. --zeurkous. -- Friggin' Machines!
Re: More than 16 partitions
Am Do., 23. Apr. 2020 um 21:31 Uhr schrieb : > No problem. Would it be too crude a suggestion that we go back to the > content now...? You didn't provide any patch.
RE: More than 16 partitions
"Christian Groessler" wrote: > On 4/23/20 7:57 PM, zeurk...@volny.cz wrote: > >> theo wrote: >>> You made it all up. >> That's an easy accusation, with an easy response: No, medid not make any >> of it up > > > Could you refrain from using your idiotic "me.."-words? Fine, me'll try and keep the idiotic ones out. > Thanks No problem. Would it be too crude a suggestion that we go back to the content now...? --zeurkous. -- Friggin' Machines!
Re: More than 16 partitions
On 4/23/20 7:57 PM, zeurk...@volny.cz wrote: theo wrote: You made it all up. That's an easy accusation, with an easy response: No, medid not make any of it up Could you refrain from using your idiotic "me.."-words? Thanks
RE: More than 16 partitions
"Jan Betlach" wrote: > For a non-native English speaker like myself, it is very difficult to > read your mestuff... Your point is well-taken (though this is just the way mespeaks); yet, Theo is a native speaker, and he seems to have completely missed the content of merecent responses. Weird, isn't it? Anyway, as this would appear to be quite OT, me'd suggest we continue this (if at all) in private mail... > Jan Take care, --zeurkous. -- Friggin' Machines!
RE: More than 16 partitions
theo wrote: > You made it all up. That's an easy accusation, with an easy response: No, medid not make any of it up. --zeurkous. -- Friggin' Machines!
Re: More than 16 partitions
You made it all up. wrote: > theo wrote: > > That is a rewriting of history. > > It's history the way meknows it. Mecertainly predates some of it. > > > The disklabel format predates the PC. > > Indeed. Mewasn't sure where and when exactly it appeared, so meleft that > bit out. But medid know it was older, and metried to communicate that > fact (obviously mefailed -- meapologizes). > > > It came from the the ancient attempt to handle things in CSRG's > > 4.3reno/4.4 work on the hp300. It was probably a rewrite of the > > native HPUX disk format. > > Hmm, hp300, eh? > > > This was then put on all the other architectures, as a unified > > view of the disk. It was modified and extended on as as-needed > > basis. > > > > Rewriting the history like this is pathetic inaccurate and > > narrowminded. Your history is absolutely false and you've made > > up a bunch of balony. > > So, what did memake up? Did mepresent a timeline? An exact order of > events? Did mepresent a scientific study? Or didmejust try to give an > overview of things in terms that Groot (and many others, mesuspects) may > just understand? > > > It is not true, and even a elementary > > review of the history of disklabel.h back into the early NetBSD > > tree will make it clear what's going on. > > Like mesaid, it's the history the way meknows it. Me's not a bloody > authority on the history of either BSD or the IBM pee-cee, *at all*. > > Perhaps meshould've made that clearer. > > > OH, and I did most of the early work post-CSRG, because we needed > > to "emulate" this on SunOS, and I ported Torek's sparc code into > > NetBSD. > > Mehas _no doubt at all_ that you know BSD (including its history) better > than me (that is, of course, an understatement). > > > I urge you to stop posting such balony. > > Then it's me turn to urge you to not read me overview as an historical > account of any exactness. > > After all, the goal, for me, was trying to help Groot understand the > relationships he sought clarification for. > > Perhaps meindeed should've included a disclaimer. Then again, mehas no > offical role here (nor does mewant one), and in no way are me words to > be taken for the one and universal truth. > > Can we please just assume that Groot is mature enough to be able to form > his own view based on our individual contributions? > > Me'd like that. > > --zeurkous. > > -- > Friggin' Machines!
RE: More than 16 partitions
theo wrote: > That is a rewriting of history. It's history the way meknows it. Mecertainly predates some of it. > The disklabel format predates the PC. Indeed. Mewasn't sure where and when exactly it appeared, so meleft that bit out. But medid know it was older, and metried to communicate that fact (obviously mefailed -- meapologizes). > It came from the the ancient attempt to handle things in CSRG's > 4.3reno/4.4 work on the hp300. It was probably a rewrite of the > native HPUX disk format. Hmm, hp300, eh? > This was then put on all the other architectures, as a unified > view of the disk. It was modified and extended on as as-needed > basis. > > Rewriting the history like this is pathetic inaccurate and > narrowminded. Your history is absolutely false and you've made > up a bunch of balony. So, what did memake up? Did mepresent a timeline? An exact order of events? Did mepresent a scientific study? Or didmejust try to give an overview of things in terms that Groot (and many others, mesuspects) may just understand? > It is not true, and even a elementary > review of the history of disklabel.h back into the early NetBSD > tree will make it clear what's going on. Like mesaid, it's the history the way meknows it. Me's not a bloody authority on the history of either BSD or the IBM pee-cee, *at all*. Perhaps meshould've made that clearer. > OH, and I did most of the early work post-CSRG, because we needed > to "emulate" this on SunOS, and I ported Torek's sparc code into > NetBSD. Mehas _no doubt at all_ that you know BSD (including its history) better than me (that is, of course, an understatement). > I urge you to stop posting such balony. Then it's me turn to urge you to not read me overview as an historical account of any exactness. After all, the goal, for me, was trying to help Groot understand the relationships he sought clarification for. Perhaps meindeed should've included a disclaimer. Then again, mehas no offical role here (nor does mewant one), and in no way are me words to be taken for the one and universal truth. Can we please just assume that Groot is mature enough to be able to form his own view based on our individual contributions? Me'd like that. --zeurkous. -- Friggin' Machines!
Re: More than 16 partitions
That is a rewriting of history. The disklabel format predates the PC. It came from the the ancient attempt to handle things in CSRG's 4.3reno/4.4 work on the hp300. It was probably a rewrite of the native HPUX disk format. This was then put on all the other architectures, as a unified view of the disk. It was modified and extended on as as-needed basis. Rewriting the history like this is pathetic inaccurate and narrowminded. Your history is absolutely false and you've made up a bunch of balony. It is not true, and even a elementary review of the history of disklabel.h back into the early NetBSD tree will make it clear what's going on. OH, and I did most of the early work post-CSRG, because we needed to "emulate" this on SunOS, and I ported Torek's sparc code into NetBSD. I urge you to stop posting such balony. wrote: > "Groot" wrote: > > I've tried and failed to create more than 16 > > partitions on OpenBSD. First of all I don't > > understand the difference between the operations > > performed by fdisk and disklabel. Is it that > > OpenBSD sees partitions differently? First we > > create an OpenBSD partition with fdisk and then > > with disklabel we can create at the most 16 more > > filesystem partitions within it. > > Traditionally, BSD has used only its own disklabel(5). Unfortunately, > mess-dos on the IBM pee-cee set a competing standard, the "Master Boot > Record", with a separate partition table (and a lot of kludging to > support more than 4 partitions). While it was (and AFAIK remains) > possible to use the whole disk the traditional way (only a BSD > disklabel, as on e.g. sparc64), it has become common practice to wrap > the BSD stuff in a mess-dos partition, with the caveat that some of the > mess-dos partition entries are duplicated in the BSD label. > > Thus, the BSD label is essentially OpenBSD's version of the structure of > things on the disk. But is an imperfect version: 16 partitions *is* the > limit for an OpenBSD label, and, of course, mess-dos partition > identifiers (which are more *ahem* fine-grained) are not used. To top it > off, partitions which rest within the mess-dos OpenBSD partition are not > necessarily represented on the mess-dos level (this would count, from > the mess-dos perspective, as overlap between partitions and thus confuse > a great many tools). > > Then GPT entered the story to make the mess complete. But me'll remain > blissfully unaware of the inner workings of that particular clusterfsck, > if you don't mind ;) > > It's no shame to be confused by this garbage. Almost all of us'd like > better, but for the above hysterical raisins, it's not so easy to make > it so. > > --zeurkous. > > -- > Friggin' Machines! >
RE: More than 16 partitions
"Groot" wrote: > I've tried and failed to create more than 16 > partitions on OpenBSD. First of all I don't > understand the difference between the operations > performed by fdisk and disklabel. Is it that > OpenBSD sees partitions differently? First we > create an OpenBSD partition with fdisk and then > with disklabel we can create at the most 16 more > filesystem partitions within it. Traditionally, BSD has used only its own disklabel(5). Unfortunately, mess-dos on the IBM pee-cee set a competing standard, the "Master Boot Record", with a separate partition table (and a lot of kludging to support more than 4 partitions). While it was (and AFAIK remains) possible to use the whole disk the traditional way (only a BSD disklabel, as on e.g. sparc64), it has become common practice to wrap the BSD stuff in a mess-dos partition, with the caveat that some of the mess-dos partition entries are duplicated in the BSD label. Thus, the BSD label is essentially OpenBSD's version of the structure of things on the disk. But is an imperfect version: 16 partitions *is* the limit for an OpenBSD label, and, of course, mess-dos partition identifiers (which are more *ahem* fine-grained) are not used. To top it off, partitions which rest within the mess-dos OpenBSD partition are not necessarily represented on the mess-dos level (this would count, from the mess-dos perspective, as overlap between partitions and thus confuse a great many tools). Then GPT entered the story to make the mess complete. But me'll remain blissfully unaware of the inner workings of that particular clusterfsck, if you don't mind ;) It's no shame to be confused by this garbage. Almost all of us'd like better, but for the above hysterical raisins, it's not so easy to make it so. --zeurkous. -- Friggin' Machines!
RE: More than 16 partitions
Haai, theo wrote: > Groot wrote: > >> I've tried and failed to create more than 16 >> partitions on OpenBSD. First of all I don't >> understand the difference between the operations >> performed by fdisk and disklabel. Is it that >> OpenBSD sees partitions differently? First we >> create an OpenBSD partition with fdisk and then >> with disklabel we can create at the most 16 more >> filesystem partitions within it. >> >> But if I want more than 16 filesystem partitions >> on a single disk naturally I try to create more >> fdisk partitions and then within it lay more fs- >> partitions. When I try that OpenBSD doesn't >> recognize those partitions and I can't even run >> 'newfs' on those partitions. Is there a way for >> OpenBSD to deal with more than 16 partitions? >> > > Sorry, there is no way. > > You can create a vnd, on a file, and then that vnd can contain > partitions. But it might suck. Me's configured a pair of consecutive softraid(4) partitions in concat mode. Sees relatively light use, but performance has been adequate. Previously, meused crypto w/ a passphrase of '0' (automagically read from a file at boot), but that was just needlessly complicated. Fixing vnd(4) to work on devices, or just adding a 'concat-of-1-volume' (perhaps call it 'L'ayer) "level" to softraid, is what me'd recommend. But that doesn't seem to help Groot (right now). Hope the above helps, --zeurkous. -- Friggin' Machines!
Re: More than 16 partitions
Groot wrote: > I've tried and failed to create more than 16 > partitions on OpenBSD. First of all I don't > understand the difference between the operations > performed by fdisk and disklabel. Is it that > OpenBSD sees partitions differently? First we > create an OpenBSD partition with fdisk and then > with disklabel we can create at the most 16 more > filesystem partitions within it. > > But if I want more than 16 filesystem partitions > on a single disk naturally I try to create more > fdisk partitions and then within it lay more fs- > partitions. When I try that OpenBSD doesn't > recognize those partitions and I can't even run > 'newfs' on those partitions. Is there a way for > OpenBSD to deal with more than 16 partitions? > Sorry, there is no way. You can create a vnd, on a file, and then that vnd can contain partitions. But it might suck.
Re: Re-organising partitions without re-installation
On 25/12/19 10:30 pm, Sriram Narayanan wrote: > On Wed, 25 Dec 2019 at 7:41 PM, Stuart Longland > wrote: > >> Both VMs should probably be re-built from scratch as a matter of sanity, >> but I can do that at leisure now, what I have, works. > > > What hypervisor are you using? Does it support an API to create VM from ISO > images and to launch VMs from templates? I'm using OpenNebula atop Linux KVM with a Ceph storage back-end. https://hackaday.io/project/10529-solar-powered-cloud-computing It does have templates, but only one of the VMs concerned are actually being managed by OpenNebula (my own; sjl-router). The other (corerouter) is actually a bare KVM virtual machine managed outside OpenNebula as the plan was to set up corosync to auto-migrate it and the VM that runs the OpenNebula front-end between the two compute nodes. Since it is the route by which the OpenNebula VM reaches the host nodes, it can't be managed by OpenNebula. So templates are not a solution. … > This would help you to cut over when needed, cut back in case of issues, > and have the ability to recover thanks to your automation. It would, and if I were managing dozens of them all with a largely identical pattern, I'd definitely look into it. My VMs tend to be pets, not cattle¹. They're all highly specialised to the task they're performing and none of them are really alike, so templating really doesn't work in that context. Regards, -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere. 1. http://cloudscaling.com/blog/cloud-computing/the-history-of-pets-vs-cattle/ (In truth, my hosts are also pet-like, because I only have a small number of them.)
Re: Re-organising partitions without re-installation
On Wed, 25 Dec 2019 at 7:41 PM, Stuart Longland wrote: > On 24/12/19 9:16 pm, Dumitru Moldovan wrote: > > Maybe it would be worth mentioning in the FAQ? I could only find it > > here: https://www.openbsd.org/faq/upgrade63.html, but then it was not > > mentioned for newer releases. > > > > Another remedy is to follow the `Files to remove` section in the FAQ, > > e.g. for 6.6: https://www.openbsd.org/faq/upgrade66.html#RmFiles. The > > FAQ article for the 6.3 upgrade suggests sysclean does that too. This > > seems to be a byproduct of the design, meaning it doesn't specifically > > remove those files, but it should remove them, as long as all installed > > packages are updated and no longer need them. But this is just my > > reading of the sysclean man page. > > Yeah I had done that… actually for another router VM I had to do a very > brutal equivalent of it when it ran out of disk space mid-update in the > installer… I basically blew away /usr/* (minus directories that are on > different partitions like 'local') figuring it'd re-instate the files > when it unpacked the newer file sets. > > This lead to some missing files in /usr/share/relink but I was able to > re-instate those from another 6.6 VM that did update cleanly > (ironically, the very one that prompted this discussion). > > So far, both have now run `syspatch`, and I've got kernel re-linking > working on both now. We shall see. > > Both VMs should probably be re-built from scratch as a matter of sanity, > but I can do that at leisure now, what I have, works. What hypervisor are you using? Does it support an API to create VM from ISO images and to launch VMs from templates? Do you have an inventory of the configuration files and the settings in these files? You may want to set up automation to produce a new openbsd vm and deliver configurations into it via configuration management and then switch to this vm after testing. This would help you to cut over when needed, cut back in case of issues, and have the ability to recover thanks to your automation. > -- > Stuart Longland (aka Redhatter, VK4MSL) > > I haven't lost my mind... > ...it's backed up on a tape somewhere. > >
Re: Re-organising partitions without re-installation
On 24/12/19 9:16 pm, Dumitru Moldovan wrote: > Maybe it would be worth mentioning in the FAQ? I could only find it > here: https://www.openbsd.org/faq/upgrade63.html, but then it was not > mentioned for newer releases. > > Another remedy is to follow the `Files to remove` section in the FAQ, > e.g. for 6.6: https://www.openbsd.org/faq/upgrade66.html#RmFiles. The > FAQ article for the 6.3 upgrade suggests sysclean does that too. This > seems to be a byproduct of the design, meaning it doesn't specifically > remove those files, but it should remove them, as long as all installed > packages are updated and no longer need them. But this is just my > reading of the sysclean man page. Yeah I had done that… actually for another router VM I had to do a very brutal equivalent of it when it ran out of disk space mid-update in the installer… I basically blew away /usr/* (minus directories that are on different partitions like 'local') figuring it'd re-instate the files when it unpacked the newer file sets. This lead to some missing files in /usr/share/relink but I was able to re-instate those from another 6.6 VM that did update cleanly (ironically, the very one that prompted this discussion). So far, both have now run `syspatch`, and I've got kernel re-linking working on both now. We shall see. Both VMs should probably be re-built from scratch as a matter of sanity, but I can do that at leisure now, what I have, works. -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
Re: Re-organising partitions without re-installation
On Mon, Dec 23, 2019 at 10:42:47PM +, rgci...@disroot.org wrote: December 24, 2019 4:42 AM, "Dumitru Moldovan" wrote: On Sun, Dec 22, 2019 at 10:56:20AM +1000, Stuart Longland wrote: So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64. Later that got updated to 6.2, then 6.3, 6.4… Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch: I have a similar issue with my desktop. I tried to outsmart the automatic installer to squeeze as much space as possible for /home on a desktop with an 80GB SSD. Which worked out OK for a few upgrade cycles, always from stable version to next stable version. However, after a couple of years, I had to unbreak an update that didn't fit any more in /usr. To my surprise, I had lots of old libs from previous releases left on disk. Had to manually remove a few of the older unused libs from /usr to be able to redo the update successfully. My understanding is that this is by design. In an update, some libs are overwritten (if they keep the same file name), but others are left on disk (theoretically unused) when lib versions are incremented. I can see a few ways in which this eases updates for people following -current, such as the OpenBSD devs, so it's a small price to pay. one thing that is useful is sysclean(8) my process now after a doas sysupgrade is 1) doas sysclean; and review the output 2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created 3) doas sysclean | xargs doas rm -rf Thanks for pointing out I missed sysclean. I used it myself, at least after the last upgrade, as I see I have it installed and `sysclean -a` only finds my custom x.org config in /etc. Maybe it would be worth mentioning in the FAQ? I could only find it here: https://www.openbsd.org/faq/upgrade63.html, but then it was not mentioned for newer releases. Another remedy is to follow the `Files to remove` section in the FAQ, e.g. for 6.6: https://www.openbsd.org/faq/upgrade66.html#RmFiles. The FAQ article for the 6.3 upgrade suggests sysclean does that too. This seems to be a byproduct of the design, meaning it doesn't specifically remove those files, but it should remove them, as long as all installed packages are updated and no longer need them. But this is just my reading of the sysclean man page.
Re: Re-organising partitions without re-installation
On 24/12/19 12:51 pm, Edgar Pettijohn wrote: > > On Dec 23, 2019 4:42 PM, rgci...@disroot.org wrote: >> >> December 24, 2019 4:42 AM, "Dumitru Moldovan" wrote: >> one thing that is useful is sysclean(8) >> >> my process now after a doas sysupgrade is >> 1) doas sysclean; and review the output >> 2) vise /etc/sysclean.ignore; so that sysclean ignores special files i >> created > > Just wanted to emphasize step 2 above or else you will delete stuff you > shouldn't. Yes, I just installed it, and ran it through `tee` so I could review the delete list before I passed it to `rm` (and manually edit it). There were a few configuration directories in `/etc` for non-base stuff (e.g. `collectd`'s password file, `vpnc`, etc) that I had to prune out and add to /etc/sysclean.ignore. That put a dint in the used space: > sjl-router# df -h > Filesystem SizeUsed Avail Capacity Mounted on > /dev/sd0a 129M 77.8M 44.6M64%/ > /dev/sd0k 472M 28.0K448M 0%/home > /dev/sd0d 198M 50.0K188M 0%/tmp > /dev/sd0f 2.3G1.2G 1022M55%/usr > /dev/sd0h 2.1G324M1.6G16%/usr/local > /dev/sd0j 1.3G2.0K1.2G 0%/usr/obj > /dev/sd0i 1.0G2.0K974M 0%/usr/src > /dev/sd0e 209M 73.2M125M37%/var I can understand the update tool being conservative about what it deletes, who knows what is linked to those .so files without scanning each and every ELF binary? (hello Gentoo revdep-rebuild!) Keeping them there is definitely the KISS approach. I'm just re-running `syspatch` to see if I can get the remainder of the patches in. If this fails, I might see if I can dig up some docs on how this disklabel and ffs stuff works and see if I can teach `gparted` about it. Something tells me it's not the complicated mess that LVM2 is, and it handles that just fine. Many thanks all. -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
Re: Re-organising partitions without re-installation
On Dec 23, 2019 4:42 PM, rgci...@disroot.org wrote: > > December 24, 2019 4:42 AM, "Dumitru Moldovan" wrote: > > > On Sun, Dec 22, 2019 at 10:56:20AM +1000, Stuart Longland wrote: > > > >> So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64. > >> Later that got updated to 6.2, then 6.3, 6.4… > >> > >> Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch: > > > > I have a similar issue with my desktop. I tried to outsmart the > > automatic installer to squeeze as much space as possible for /home on a > > desktop with an 80GB SSD. Which worked out OK for a few upgrade cycles, > > always from stable version to next stable version. > > > > However, after a couple of years, I had to unbreak an update that didn't > > fit any more in /usr. To my surprise, I had lots of old libs from > > previous releases left on disk. Had to manually remove a few of the > > older unused libs from /usr to be able to redo the update successfully. > > > > My understanding is that this is by design. In an update, some libs are > > overwritten (if they keep the same file name), but others are left on > > disk (theoretically unused) when lib versions are incremented. I can > > see a few ways in which this eases updates for people following > > -current, such as the OpenBSD devs, so it's a small price to pay. > > one thing that is useful is sysclean(8) > > my process now after a doas sysupgrade is > 1) doas sysclean; and review the output > 2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created Just wanted to emphasize step 2 above or else you will delete stuff you shouldn't. > 3) doas sysclean | xargs doas rm -rf > > yorosiku ~ > +1 for sysclean
Re: Re-organising partitions without re-installation
On Mon, Dec 23, 2019 at 3:10 PM Stuart Longland wrote: ... > Where do you get `sysclean` from? I don't seem to have it: > > sjl-router# man sysclean > > > man: No entry for sysclean in the manual. > > sjl-router# which sysclean > > which: sysclean: Command not found. > $ pkg_info sysclean Information for http://mirrors.sonic.net/pub/OpenBSD/snapshots/packages/amd64/sysclean-2.8.tgz Comment: list obsolete files between OpenBSD upgrades Description: sysclean is a script designed to help remove obsolete files between OpenBSD upgrades. sysclean compares a reference root directory against the currently installed files, taking files from both the base system and packages into account. sysclean does not remove any files on the system. It only reports obsolete filenames or packages using out-of-date libraries. Maintainer: Sebastien Marie WWW: https://github.com/semarie/sysclean/ $
Re: Re-organising partitions without re-installation
On 24/12/19 8:42 am, rgci...@disroot.org wrote: >> My understanding is that this is by design. In an update, some libs are >> overwritten (if they keep the same file name), but others are left on >> disk (theoretically unused) when lib versions are incremented. I can >> see a few ways in which this eases updates for people following >> -current, such as the OpenBSD devs, so it's a small price to pay. > one thing that is useful is sysclean(8) > > my process now after a doas sysupgrade is > 1) doas sysclean; and review the output > 2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created > 3) doas sysclean | xargs doas rm -rf > > yorosiku ~ Where do you get `sysclean` from? I don't seem to have it: > sjl-router# man sysclean > > man: No entry for sysclean in the manual. > sjl-router# which sysclean > which: sysclean: Command not found. Regards, -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
Re: Re-organising partitions without re-installation
December 24, 2019 4:42 AM, "Dumitru Moldovan" wrote: > On Sun, Dec 22, 2019 at 10:56:20AM +1000, Stuart Longland wrote: > >> So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64. >> Later that got updated to 6.2, then 6.3, 6.4… >> >> Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch: > > I have a similar issue with my desktop. I tried to outsmart the > automatic installer to squeeze as much space as possible for /home on a > desktop with an 80GB SSD. Which worked out OK for a few upgrade cycles, > always from stable version to next stable version. > > However, after a couple of years, I had to unbreak an update that didn't > fit any more in /usr. To my surprise, I had lots of old libs from > previous releases left on disk. Had to manually remove a few of the > older unused libs from /usr to be able to redo the update successfully. > > My understanding is that this is by design. In an update, some libs are > overwritten (if they keep the same file name), but others are left on > disk (theoretically unused) when lib versions are incremented. I can > see a few ways in which this eases updates for people following > -current, such as the OpenBSD devs, so it's a small price to pay. one thing that is useful is sysclean(8) my process now after a doas sysupgrade is 1) doas sysclean; and review the output 2) vise /etc/sysclean.ignore; so that sysclean ignores special files i created 3) doas sysclean | xargs doas rm -rf yorosiku ~
Re: Re-organising partitions without re-installation
On Sun, Dec 22, 2019 at 10:56:20AM +1000, Stuart Longland wrote: So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64. Later that got updated to 6.2, then 6.3, 6.4… Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch: I have a similar issue with my desktop. I tried to outsmart the automatic installer to squeeze as much space as possible for /home on a desktop with an 80GB SSD. Which worked out OK for a few upgrade cycles, always from stable version to next stable version. However, after a couple of years, I had to unbreak an update that didn't fit any more in /usr. To my surprise, I had lots of old libs from previous releases left on disk. Had to manually remove a few of the older unused libs from /usr to be able to redo the update successfully. My understanding is that this is by design. In an update, some libs are overwritten (if they keep the same file name), but others are left on disk (theoretically unused) when lib versions are incremented. I can see a few ways in which this eases updates for people following -current, such as the OpenBSD devs, so it's a small price to pay. But yes, it would be awesome if installations that use -stable would not have to pay this price. After all, only libs from current version of the base system should be needed. If you built something linked to an older lib from a previous OS version, it should be recompiled after an updated. This could also be a security issue if you're sloppy and use binaries linked to old base libs that are no longer updated for security issues. If I missed anything, would love to be corrected! And sorry, I don't have a solution for this. Other than the obvious one of packaging the files in base. Is this a heresy?
Re: Re-organising partitions without re-installation
Stuart Longland writes: > > 16 partitions: > > #size offset fstype [fsize bsize cpg] > > a: 268416 64 4.2BSD 2048 16384 2097 # / > > b: 373010 268480swap# none > > c: 167772160 unused > > d: 413056 641504 4.2BSD 2048 16384 3227 # /tmp > > e: 435744 1054560 4.2BSD 2048 16384 3390 # /var > > f: 5006848 1490304 4.2BSD 2048 16384 12958 # /usr > > h: 4403456 6497152 4.2BSD 2048 16384 12958 # > > /usr/local > > i: 2138976 10900608 4.2BSD 2048 16384 12958 # /usr/src > > j: 2746048 13039584 4.2BSD 2048 16384 12958 # /usr/obj > > k: 986208 15785632 4.2BSD 2048 16384 7674 # /home > > Question is, how do I re-organise this space? There is sufficient space > there. /usr/obj and /usr/src are pretty much unused. /usr/local could > be made smaller too as could /home. Copy contents of /home, if any, to /var; Remove partitions i, j and k, replacing with a single large i to mount at /home; format new, larger partition i and restore the contents of /home from the backup; update /etc/fstab. Alternatively back up the contents of /usr/local as well and replace partition h with a larger one if you don't need a /home (except that now sysupgrade does, so...). Reboot not required although you will need to stop/start anything holding files open. Matthew
Re-organising partitions without re-installation
Hi all, So, a few years ago now, I deployed a router VM with OpenBSD 6.1 AMD64. Later that got updated to 6.2, then 6.3, 6.4… Yesterday I updated it to 6.5, then 6.6… now I'm trying to run syspatch: > sjl-router# syspatch > > Get/Verify syspatch66-010_libcaut... 100% > || 20185 > KB00:16 > Installing patch 010_libcauth > No space left on sd0a, aborting > sjl-router# df -h > Filesystem SizeUsed Avail Capacity Mounted on > /dev/sd0a 129M 98.0M 24.4M80%/ > /dev/sd0k 472M 28.0K448M 0%/home > /dev/sd0d 198M 76.0K188M 0%/tmp > /dev/sd0f 2.3G1.3G911M60%/usr > /dev/sd0h 2.1G551M1.4G27%/usr/local > /dev/sd0j 1.3G2.0K1.2G 0%/usr/obj > /dev/sd0i 1.0G2.0K974M 0%/usr/src > /dev/sd0e 209M119M 79.1M60%/var > sjl-router# uname -a > OpenBSD sjl-router.redhatters.home 6.6 GENERIC#353 amd64 8GB seemed like a reasonable amount for something that would just be routing. And looking at that `df` output, it would appear that there's about 2.5GB locked away, in partitions that the original automatic layout dictated I should have, but then didn't utilise. I'm thankful I had the foresight of overruling its decision to allocate space to /usr/X11R6… as this machine does not have X installed. (Why would a router need that anyway?) > sjl-router# disklabel sd0 > # /dev/rsd0c: > type: SCSI > disk: SCSI disk > label: Block Device > duid: d7b965d8cdeaeef2 > flags: > bytes/sector: 512 > sectors/track: 63 > tracks/cylinder: 255 > sectors/cylinder: 16065 > cylinders: 1044 > total sectors: 16777216 > boundstart: 64 > boundend: 16771860 > drivedata: 0 > > 16 partitions: > #size offset fstype [fsize bsize cpg] > a: 268416 64 4.2BSD 2048 16384 2097 # / > b: 373010 268480swap# none > c: 167772160 unused > d: 413056 641504 4.2BSD 2048 16384 3227 # /tmp > e: 435744 1054560 4.2BSD 2048 16384 3390 # /var > f: 5006848 1490304 4.2BSD 2048 16384 12958 # /usr > h: 4403456 6497152 4.2BSD 2048 16384 12958 # /usr/local > i: 2138976 10900608 4.2BSD 2048 16384 12958 # /usr/src > j: 2746048 13039584 4.2BSD 2048 16384 12958 # /usr/obj > k: 986208 15785632 4.2BSD 2048 16384 7674 # /home Question is, how do I re-organise this space? There is sufficient space there. /usr/obj and /usr/src are pretty much unused. /usr/local could be made smaller too as could /home. OpenBSD has growfs(8). I note it is called growfs and not resizefs nor shrinkfs. The steps I believe I'd need to perform are: - shrink /home to 200MB - re-locate /home to the end - blow away /usr/src and /usr/obj - shrink /usr/local to 1GB - re-locate /usr/local to just before /home - re-locate /usr to just before /usr/local - re-locate /var to just before /usr - re-locate /tmp to to just before /var - now grow / to fill the available space In days gone by, there was PartitionMagic for doing this. Under Linux today, there's gparted. OpenBSD complicates things because it ignores the native disklabel format of the host platform (i.e. MS-DOS disklabel / GUID partition table) in favour of its own BSD slice system. So such a tool has to not only understand ffs, but it also must understand the BSD disklabel embedded in the partition allocated to OpenBSD. Re-installing is something I did under the following conditions: - Before the existence of the aforementioned partition management tools - When I *really* screwed up I'm not after a GUI tool to do this (although some sort of visualisation is helpful in my experience, I can also use a spreadsheet to work out the numbers), but I really don't think "reinstall" should be the default answer to all this as that is really a measure of last resort. Is there such a tool for manipulating partitions in this manner? -- Stuart Longland (aka Redhatter, VK4MSL) I haven't lost my mind... ...it's backed up on a tape somewhere.
Re: Best Practices for growing disk partitions on a server
On 2019-11-19, Steve Litt wrote: > In OpenBSD is there such a thing as a bind mount like they have in > Linux? No. The closest is probably "mount from 127.0.0.1 over NFS".
Re: Best Practices for growing disk partitions on a server
On Tue, 19 Nov 2019 12:52:52 +0200 Dumitru Moldovan wrote: > > 1. Install the same OS on new hardware, in your case with a better > partitioned drive. > > 2. rsync everything relevant to the new machine (but make sure it > still boots afterwards and functions as expected, so amend boot > manager files, hardware-dependant configs etc.). > > 3. Update relevant DNS records for affected services AND set packet > redirection for services on the old machines until DNS propagation > does its thing (which is usually much longer than expected, esp. > for public services, even if you lower TTLs well in advance). > > If you don't have spare hardware for the migration, I guess you could > use a spare VM until you repartition drives in your old hardware as > needed. Also rsync to a USB hard drive, so even if everything including your VM goes to hell you can still restore. I just bought a 5TB USB drive for $99 at Costco. SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr
Re: Best Practices for growing disk partitions on a server
On Mon, 18 Nov 2019 18:28:55 - (UTC) Stuart Henderson wrote: > On 2019-11-17, Lev Lazinskiy wrote: > > Hi folks, > > > > I am new to openBSD, so forgive me if I am missing something > > obvious. > > > > I recently installed openBSD on a server using the auto-partition > > layout during installation and am quickly starting to run out of > > disk space. > > > > I have read the section in the FAQ [1] regarding how to grow a disk > > partition, but I am confused on the best way to actually do this. > > > > Specifically, I am trying to grow /usr and /home but they are "busy" > > when I try to follow these steps on a running server. > > > > Is the assumption that you are supposed to reboot the server with > > the ISO attached and pop into a shell to complete these steps? > > > > [1] https://www.openbsd.org/faq/faq14.html#GrowPartition > > > > This faq entry tells you to use growfs, which won't work for /usr > from a standard auto-defaults install because it requires empty space > immediately following the partition to grow into. > > On a larger disk it often will work for /home because auto-defaults > place it at the end of the disk, size it at max 300G, and leave the > space following it empty. > > Sometimes you can shunt things around - if you have free space at the > end of the disk you can move files from the filesystem immediately > following /usr there, and growfs into the now-vacated space - > sometimes you have another partition that is A) larger than /usr and > B) that is larger than you need, in which case you copy files so you > can swap them around. Or again if you have free space at the end of > the disk, after what you'd want to grow /home into, you can make a > new partition there, copy the files, and leave the former /usr > partition empty. But it's quite delicate work and is often easier to > reinstall. In OpenBSD is there such a thing as a bind mount like they have in Linux? SteveT Steve Litt November 2019 featured book: Manager's Guide to Technical Troubleshooting Second edition http://www.troubleshooters.com/mgr
Re: Best Practices for growing disk partitions on a server
On Tue, Nov 19, 2019 at 12:31:25PM +0200, Dumitru Moldovan wrote: On Sun, Nov 17, 2019 at 03:12:06PM -0800, Lev Lazinskiy wrote: This makes sense, but I was curious what the recommended approach is for a server that you cannot simply reinstall. A humble piece of advice from a fellow system admin... Never ever build a system that "you cannot simply reinstall." There should be at least two ways to redo everything: 1. From scratch using documentation or preferably a modern deployment system such as Ansible, Salt, etc. 2. From backup. This should be drilled from time to time, even without a practical need, just to make sure things work as expected. Yes, practice is not as simple as theory and I admit being guilty at times of every possible mistake in not following the above. :-] But, realistically speaking, as long as a system still functions, it should be reproducible. Document the setup, backup its configs and saved data, etc. May be bad advice for your situation, but this is what I used to do when migrating setups that were not documented or easy to redeploy: 1. Install the same OS on new hardware, in your case with a better partitioned drive. 2. rsync everything relevant to the new machine (but make sure it still boots afterwards and functions as expected, so amend boot manager files, hardware-dependant configs etc.). 3. Update relevant DNS records for affected services AND set packet redirection for services on the old machines until DNS propagation does its thing (which is usually much longer than expected, esp. for public services, even if you lower TTLs well in advance). If you don't have spare hardware for the migration, I guess you could use a spare VM until you repartition drives in your old hardware as needed. If you don't care about service disruption, I guess you could skip redirection. Hope that helps!
Re: Best Practices for growing disk partitions on a server
On Sun, Nov 17, 2019 at 03:12:06PM -0800, Lev Lazinskiy wrote: This makes sense, but I was curious what the recommended approach is for a server that you cannot simply reinstall. A humble piece of advice from a fellow system admin... Never ever build a system that "you cannot simply reinstall." There should be at least two ways to redo everything: 1. From scratch using documentation or preferably a modern deployment system such as Ansible, Salt, etc. 2. From backup. This should be drilled from time to time, even without a practical need, just to make sure things work as expected. Yes, practice is not as simple as theory and I admit being guilty at times of every possible mistake in not following the above. :-] But, realistically speaking, as long as a system still functions, it should be reproducible. Document the setup, backup its configs and saved data, etc.
Re: Best Practices for growing disk partitions on a server
Stuart Henderson wrote: > On 2019-11-17, Lev Lazinskiy wrote: > > Hi folks, > > > > I am new to openBSD, so forgive me if I am missing something obvious. > > > > I recently installed openBSD on a server using the auto-partition layout > > during installation and am quickly starting to run out of disk space. > > > > I have read the section in the FAQ [1] regarding how to grow a disk > > partition, but I am confused on the best way to actually do this. > > > > Specifically, I am trying to grow /usr and /home but they are "busy" > > when I try to follow these steps on a running server. > > > > Is the assumption that you are supposed to reboot the server with the > > ISO attached and pop into a shell to complete these steps? > > > > [1] https://www.openbsd.org/faq/faq14.html#GrowPartition > > > > This faq entry tells you to use growfs, which won't work for /usr > from a standard auto-defaults install because it requires empty space > immediately following the partition to grow into. > > On a larger disk it often will work for /home because auto-defaults > place it at the end of the disk, size it at max 300G, and leave the space > following it empty. > > Sometimes you can shunt things around - if you have free space at the end > of the disk you can move files from the filesystem immediately following > /usr there, and growfs into the now-vacated space - sometimes you have > another partition that is A) larger than /usr and B) that is larger than > you need, in which case you copy files so you can swap them around. > Or again if you have free space at the end of the disk, after what you'd > want to grow /home into, you can make a new partition there, copy the > files, and leave the former /usr partition empty. But it's quite delicate > work and is often easier to reinstall. > > Interested what you are bumping into on /usr, on a recently installed > system the default size is usually ok for most uses unless it's a small > disk or unless you are building ports and don't have a separate /usr/ports > partition. (If you *are* using ports and have free space at the end - > as above - you could create a new partition to hold /usr/ports in there > and move *those* files across - that's a much easier proposition than > moving the whole of /usr. Which is to say, there are all sorts of complexities, and you'll have to work through them. On the other hand learning how to save all your files, and then do a fresh install, is like gaining a superpower.
Re: Best Practices for growing disk partitions on a server
On 2019-11-17, Lev Lazinskiy wrote: > Hi folks, > > I am new to openBSD, so forgive me if I am missing something obvious. > > I recently installed openBSD on a server using the auto-partition layout > during installation and am quickly starting to run out of disk space. > > I have read the section in the FAQ [1] regarding how to grow a disk > partition, but I am confused on the best way to actually do this. > > Specifically, I am trying to grow /usr and /home but they are "busy" > when I try to follow these steps on a running server. > > Is the assumption that you are supposed to reboot the server with the > ISO attached and pop into a shell to complete these steps? > > [1] https://www.openbsd.org/faq/faq14.html#GrowPartition > This faq entry tells you to use growfs, which won't work for /usr from a standard auto-defaults install because it requires empty space immediately following the partition to grow into. On a larger disk it often will work for /home because auto-defaults place it at the end of the disk, size it at max 300G, and leave the space following it empty. Sometimes you can shunt things around - if you have free space at the end of the disk you can move files from the filesystem immediately following /usr there, and growfs into the now-vacated space - sometimes you have another partition that is A) larger than /usr and B) that is larger than you need, in which case you copy files so you can swap them around. Or again if you have free space at the end of the disk, after what you'd want to grow /home into, you can make a new partition there, copy the files, and leave the former /usr partition empty. But it's quite delicate work and is often easier to reinstall. Interested what you are bumping into on /usr, on a recently installed system the default size is usually ok for most uses unless it's a small disk or unless you are building ports and don't have a separate /usr/ports partition. (If you *are* using ports and have free space at the end - as above - you could create a new partition to hold /usr/ports in there and move *those* files across - that's a much easier proposition than moving the whole of /usr.
