Re: filesystem code integer and many inodes
Hi Otto, * Otto Moerbeek wrote: > > Did anyone look closer at this? > > Did anyone test? I compiled kernel + world with your diff in a VM and performed the following tests. * fdisk, disklablel, newfs of an external disk * disklabel, growfs, fsck on a partition on an external disk * fsirand on a partition (not sure how to test this but fsirand -p showed different numbers afterwards...) * Extract src.tgz, powered off the VM, reboot and fsck upon boot * Same while extracting src.tgz on a second mounted disk To the best of my knowledge the tools did the jobs they were supposed to. Let me know if there is something else I should test. Cheers Matthias
Re: filesystem code integer and many inodes
On Fri, May 29, 2020 at 09:30:04AM +0200, Otto Moerbeek wrote:
> On Thu, May 28, 2020 at 12:54:41PM -0600, Todd C. Miller wrote:
>
> > On Thu, 28 May 2020 20:53:07 +0200, Otto Moerbeek wrote:
> >
> > > Here's the separate diff for the prefcg loops. From FreeBSD.
> >
> > OK millert@
> >
> > - todd
> >
>
> And here's the updated diff against -current. I removed a redundant
> cast in a fs_ipg * fs_ncg multiplication in fsck_ffs. Since both are
> u_int32 and we know the product is <= UINT_MAX, so we do not need to
> cast.
>
> I would like to make some progress here, I have a followup diff to
> speed up Phase 5 of fsck_ffs...
Did anyone look closer at this?
Did anyone test?
-Otto
>
> Index: sbin/clri/clri.c
> ===
> RCS file: /cvs/src/sbin/clri/clri.c,v
> retrieving revision 1.20
> diff -u -p -r1.20 clri.c
> --- sbin/clri/clri.c 28 Jun 2019 13:32:43 - 1.20
> +++ sbin/clri/clri.c 29 May 2020 07:23:27 -
> @@ -68,7 +68,8 @@ main(int argc, char *argv[])
> char *fs, sblock[SBLOCKSIZE];
> size_t bsize;
> off_t offset;
> - int i, fd, imax, inonum;
> + int i, fd;
> + ino_t imax, inonum;
>
> if (argc < 3)
> usage();
> Index: sbin/dumpfs/dumpfs.c
> ===
> RCS file: /cvs/src/sbin/dumpfs/dumpfs.c,v
> retrieving revision 1.35
> diff -u -p -r1.35 dumpfs.c
> --- sbin/dumpfs/dumpfs.c 17 Feb 2020 16:11:25 - 1.35
> +++ sbin/dumpfs/dumpfs.c 29 May 2020 07:23:27 -
> @@ -69,7 +69,7 @@ union {
> #define acg cgun.cg
>
> int dumpfs(int, const char *);
> -int dumpcg(const char *, int, int);
> +int dumpcg(const char *, int, u_int);
> int marshal(const char *);
> int open_disk(const char *);
> void pbits(void *, int);
> @@ -163,6 +163,7 @@ dumpfs(int fd, const char *name)
> size_t size;
> off_t off;
> int i, j;
> + u_int cg;
>
> switch (afs.fs_magic) {
> case FS_UFS2_MAGIC:
> @@ -172,7 +173,7 @@ dumpfs(int fd, const char *name)
> afs.fs_magic, ctime());
> printf("superblock location\t%jd\tid\t[ %x %x ]\n",
> (intmax_t)afs.fs_sblockloc, afs.fs_id[0], afs.fs_id[1]);
> - printf("ncg\t%d\tsize\t%jd\tblocks\t%jd\n",
> + printf("ncg\t%u\tsize\t%jd\tblocks\t%jd\n",
> afs.fs_ncg, (intmax_t)fssize, (intmax_t)afs.fs_dsize);
> break;
> case FS_UFS1_MAGIC:
> @@ -198,7 +199,7 @@ dumpfs(int fd, const char *name)
> printf("cylgrp\t%s\tinodes\t%s\tfslevel %d\n",
> i < 1 ? "static" : "dynamic",
> i < 2 ? "4.2/4.3BSD" : "4.4BSD", i);
> - printf("ncg\t%d\tncyl\t%d\tsize\t%d\tblocks\t%d\n",
> + printf("ncg\t%u\tncyl\t%d\tsize\t%d\tblocks\t%d\n",
> afs.fs_ncg, afs.fs_ncyl, afs.fs_ffs1_size,
> afs.fs_ffs1_dsize);
> break;
> default:
> @@ -223,9 +224,9 @@ dumpfs(int fd, const char *name)
> (intmax_t)afs.fs_cstotal.cs_ndir,
> (intmax_t)afs.fs_cstotal.cs_nifree,
> (intmax_t)afs.fs_cstotal.cs_nffree);
> - printf("bpg\t%d\tfpg\t%d\tipg\t%d\n",
> + printf("bpg\t%d\tfpg\t%d\tipg\t%u\n",
> afs.fs_fpg / afs.fs_frag, afs.fs_fpg, afs.fs_ipg);
> - printf("nindir\t%d\tinopb\t%d\tmaxfilesize\t%ju\n",
> + printf("nindir\t%d\tinopb\t%u\tmaxfilesize\t%ju\n",
> afs.fs_nindir, afs.fs_inopb,
> (uintmax_t)afs.fs_maxfilesize);
> printf("sbsize\t%d\tcgsize\t%d\tcsaddr\t%jd\tcssize\t%d\n",
> @@ -238,10 +239,10 @@ dumpfs(int fd, const char *name)
> printf("nbfree\t%d\tndir\t%d\tnifree\t%d\tnffree\t%d\n",
> afs.fs_ffs1_cstotal.cs_nbfree, afs.fs_ffs1_cstotal.cs_ndir,
> afs.fs_ffs1_cstotal.cs_nifree,
> afs.fs_ffs1_cstotal.cs_nffree);
> - printf("cpg\t%d\tbpg\t%d\tfpg\t%d\tipg\t%d\n",
> + printf("cpg\t%d\tbpg\t%d\tfpg\t%d\tipg\t%u\n",
> afs.fs_cpg, afs.fs_fpg / afs.fs_frag, afs.fs_fpg,
> afs.fs_ipg);
> - printf("nindir\t%d\tinopb\t%d\tnspf\t%d\tmaxfilesize\t%ju\n",
> + printf("nindir\t%d\tinopb\t%u\tnspf\t%d\tmaxfilesize\t%ju\n",
> afs.fs_nindir, afs.fs_inopb, afs.fs_nspf,
> (uintmax_t)afs.fs_maxfilesize);
> printf("sbsize\t%d\tcgsize\t%d\tcgoffset %d\tcgmask\t0x%08x\n",
> @@ -261,7 +262,7 @@ dumpfs(int fd, const char *name)
> afs.fs_sblkno, afs.fs_cblkno, afs.fs_iblkno, afs.fs_dblkno);
> printf("cgrotor\t%d\tfmod\t%d\tronly\t%d\tclean\t%d\n",
> afs.fs_cgrotor, afs.fs_fmod, afs.fs_ronly, afs.fs_clean);
> - printf("avgfpdir %d\tavgfilesize %d\n",
> + printf("avgfpdir %u\tavgfilesize %u\n",
>
Re: filesystem code integer and many inodes
On Fri, May 29, 2020 at 09:30:04AM +0200, Otto Moerbeek wrote: > On Thu, May 28, 2020 at 12:54:41PM -0600, Todd C. Miller wrote: > > > On Thu, 28 May 2020 20:53:07 +0200, Otto Moerbeek wrote: > > > > > Here's the separate diff for the prefcg loops. From FreeBSD. > > > > OK millert@ > > > > - todd > > > > And here's the updated diff against -current. I removed a redundant > cast in a fs_ipg * fs_ncg multiplication in fsck_ffs. Since both are > u_int32 and we know the product is <= UINT_MAX, so we do not need to > cast. > > I would like to make some progress here, I have a followup diff to > speed up Phase 5 of fsck_ffs... This last line was directed at other tech@ subscribers and not so much at millert@. Please review and/or test. Thanks! -Otto
Re: filesystem code integer and many inodes
On Thu, May 28, 2020 at 12:54:41PM -0600, Todd C. Miller wrote:
> On Thu, 28 May 2020 20:53:07 +0200, Otto Moerbeek wrote:
>
> > Here's the separate diff for the prefcg loops. From FreeBSD.
>
> OK millert@
>
> - todd
>
And here's the updated diff against -current. I removed a redundant
cast in a fs_ipg * fs_ncg multiplication in fsck_ffs. Since both are
u_int32 and we know the product is <= UINT_MAX, so we do not need to
cast.
I would like to make some progress here, I have a followup diff to
speed up Phase 5 of fsck_ffs...
-Otto
Index: sbin/clri/clri.c
===
RCS file: /cvs/src/sbin/clri/clri.c,v
retrieving revision 1.20
diff -u -p -r1.20 clri.c
--- sbin/clri/clri.c28 Jun 2019 13:32:43 - 1.20
+++ sbin/clri/clri.c29 May 2020 07:23:27 -
@@ -68,7 +68,8 @@ main(int argc, char *argv[])
char *fs, sblock[SBLOCKSIZE];
size_t bsize;
off_t offset;
- int i, fd, imax, inonum;
+ int i, fd;
+ ino_t imax, inonum;
if (argc < 3)
usage();
Index: sbin/dumpfs/dumpfs.c
===
RCS file: /cvs/src/sbin/dumpfs/dumpfs.c,v
retrieving revision 1.35
diff -u -p -r1.35 dumpfs.c
--- sbin/dumpfs/dumpfs.c17 Feb 2020 16:11:25 - 1.35
+++ sbin/dumpfs/dumpfs.c29 May 2020 07:23:27 -
@@ -69,7 +69,7 @@ union {
#define acgcgun.cg
intdumpfs(int, const char *);
-intdumpcg(const char *, int, int);
+intdumpcg(const char *, int, u_int);
intmarshal(const char *);
intopen_disk(const char *);
void pbits(void *, int);
@@ -163,6 +163,7 @@ dumpfs(int fd, const char *name)
size_t size;
off_t off;
int i, j;
+ u_int cg;
switch (afs.fs_magic) {
case FS_UFS2_MAGIC:
@@ -172,7 +173,7 @@ dumpfs(int fd, const char *name)
afs.fs_magic, ctime());
printf("superblock location\t%jd\tid\t[ %x %x ]\n",
(intmax_t)afs.fs_sblockloc, afs.fs_id[0], afs.fs_id[1]);
- printf("ncg\t%d\tsize\t%jd\tblocks\t%jd\n",
+ printf("ncg\t%u\tsize\t%jd\tblocks\t%jd\n",
afs.fs_ncg, (intmax_t)fssize, (intmax_t)afs.fs_dsize);
break;
case FS_UFS1_MAGIC:
@@ -198,7 +199,7 @@ dumpfs(int fd, const char *name)
printf("cylgrp\t%s\tinodes\t%s\tfslevel %d\n",
i < 1 ? "static" : "dynamic",
i < 2 ? "4.2/4.3BSD" : "4.4BSD", i);
- printf("ncg\t%d\tncyl\t%d\tsize\t%d\tblocks\t%d\n",
+ printf("ncg\t%u\tncyl\t%d\tsize\t%d\tblocks\t%d\n",
afs.fs_ncg, afs.fs_ncyl, afs.fs_ffs1_size,
afs.fs_ffs1_dsize);
break;
default:
@@ -223,9 +224,9 @@ dumpfs(int fd, const char *name)
(intmax_t)afs.fs_cstotal.cs_ndir,
(intmax_t)afs.fs_cstotal.cs_nifree,
(intmax_t)afs.fs_cstotal.cs_nffree);
- printf("bpg\t%d\tfpg\t%d\tipg\t%d\n",
+ printf("bpg\t%d\tfpg\t%d\tipg\t%u\n",
afs.fs_fpg / afs.fs_frag, afs.fs_fpg, afs.fs_ipg);
- printf("nindir\t%d\tinopb\t%d\tmaxfilesize\t%ju\n",
+ printf("nindir\t%d\tinopb\t%u\tmaxfilesize\t%ju\n",
afs.fs_nindir, afs.fs_inopb,
(uintmax_t)afs.fs_maxfilesize);
printf("sbsize\t%d\tcgsize\t%d\tcsaddr\t%jd\tcssize\t%d\n",
@@ -238,10 +239,10 @@ dumpfs(int fd, const char *name)
printf("nbfree\t%d\tndir\t%d\tnifree\t%d\tnffree\t%d\n",
afs.fs_ffs1_cstotal.cs_nbfree, afs.fs_ffs1_cstotal.cs_ndir,
afs.fs_ffs1_cstotal.cs_nifree,
afs.fs_ffs1_cstotal.cs_nffree);
- printf("cpg\t%d\tbpg\t%d\tfpg\t%d\tipg\t%d\n",
+ printf("cpg\t%d\tbpg\t%d\tfpg\t%d\tipg\t%u\n",
afs.fs_cpg, afs.fs_fpg / afs.fs_frag, afs.fs_fpg,
afs.fs_ipg);
- printf("nindir\t%d\tinopb\t%d\tnspf\t%d\tmaxfilesize\t%ju\n",
+ printf("nindir\t%d\tinopb\t%u\tnspf\t%d\tmaxfilesize\t%ju\n",
afs.fs_nindir, afs.fs_inopb, afs.fs_nspf,
(uintmax_t)afs.fs_maxfilesize);
printf("sbsize\t%d\tcgsize\t%d\tcgoffset %d\tcgmask\t0x%08x\n",
@@ -261,7 +262,7 @@ dumpfs(int fd, const char *name)
afs.fs_sblkno, afs.fs_cblkno, afs.fs_iblkno, afs.fs_dblkno);
printf("cgrotor\t%d\tfmod\t%d\tronly\t%d\tclean\t%d\n",
afs.fs_cgrotor, afs.fs_fmod, afs.fs_ronly, afs.fs_clean);
- printf("avgfpdir %d\tavgfilesize %d\n",
+ printf("avgfpdir %u\tavgfilesize %u\n",
afs.fs_avgfpdir, afs.fs_avgfilesize);
printf("flags\t");
if (afs.fs_magic == FS_UFS2_MAGIC ||
@@ -296,8 +297,8 @@ dumpfs(int fd, const char *name)
if (pread(fd, (char
Re: filesystem code integer and many inodes
On Thu, 28 May 2020 20:53:07 +0200, Otto Moerbeek wrote: > Here's the separate diff for the prefcg loops. From FreeBSD. OK millert@ - todd
Re: filesystem code integer and many inodes
On Tue, May 26, 2020 at 04:11:50PM +0200, Otto Moerbeek wrote:
> On Tue, May 26, 2020 at 03:54:15PM +0200, Otto Moerbeek wrote:
>
> > On Tue, May 26, 2020 at 07:51:28AM -0600, Todd C. Miller wrote:
> >
> > > On Tue, 26 May 2020 12:07:21 +0200, Otto Moerbeek wrote:
> > >
> > > > Apart from the noting the strange Subject: I also like to mention one
> > > > change in the way cylinder groups are scanned. The current code scans
> > > > forward and backward, which causes an uneven distribution of full cgs
> > > > (the upper end of the cgs will get full first). Fix that by always
> > > > scanning forward, wrapping to cg 0 if needed.
> > >
> > > Should that be a separate commit? I can't find any problems
> > > with the diff but I haven't tried running with it yet.
> > >
> > > - todd
> >
> > Yeah, I can do that. Note that it must be comitted first, since the
> > loop condition is always true if I change the loop var to unsigned.
> >
> > -Otto
> >
>
> And a new diff. I accidentally capitalized a letter just before sending.
> Thanks to naddy for spotting that.
Here's the separate diff for the prefcg loops. From FreeBSD.
OK?
-Otto
Index: ffs_alloc.c
===
RCS file: /cvs/src/sys/ufs/ffs/ffs_alloc.c,v
retrieving revision 1.111
diff -u -p -r1.111 ffs_alloc.c
--- ffs_alloc.c 28 May 2020 15:48:29 - 1.111
+++ ffs_alloc.c 28 May 2020 18:47:53 -
@@ -515,7 +518,7 @@ ffs_dirpref(struct inode *pip)
maxcontigdirs = 1;
/*
-* Limit number of dirs in one cg and reserve space for
+* Limit number of dirs in one cg and reserve space for
* regular files, but only if we have no deficit in
* inodes or space.
*
@@ -524,16 +527,17 @@ ffs_dirpref(struct inode *pip)
* We scan from our preferred cylinder group forward looking
* for a cylinder group that meets our criterion. If we get
* to the final cylinder group and do not find anything,
-* we start scanning backwards from our preferred cylinder
-* group. The ideal would be to alternate looking forward
-* and backward, but tha tis just too complex to code for
-* the gain it would get. The most likely place where the
-* backward scan would take effect is when we start near
-* the end of the filesystem and do not find anything from
-* where we are to the end. In that case, scanning backward
-* will likely find us a suitable cylinder group much closer
-* to our desired location than if we were to start scanning
-* forward from the beginning for the filesystem.
+* we start scanning forwards from the beginning of the
+* filesystem. While it might seem sensible to start scanning
+* backwards or even to alternate looking forward and backward,
+* this approach fails badly when the filesystem is nearly full.
+* Specifically, we first search all the areas that have no space
+* and finally try the one preceding that. We repeat this on
+* every request and in the case of the final block end up
+* searching the entire filesystem. By jumping to the front
+* of the filesystem, our future forward searches always look
+* in new cylinder groups so finds every possible block after
+* one pass over the filesystem.
*/
for (cg = prefcg; cg < fs->fs_ncg; cg++)
if (fs->fs_cs(fs, cg).cs_ndir < maxndir &&
@@ -542,7 +546,7 @@ ffs_dirpref(struct inode *pip)
if (fs->fs_contigdirs[cg] < maxcontigdirs)
goto end;
}
- for (cg = prefcg - 1; cg >= 0; cg--)
+ for (cg = 0; cg < prefcg; cg++)
if (fs->fs_cs(fs, cg).cs_ndir < maxndir &&
fs->fs_cs(fs, cg).cs_nifree >= minifree &&
fs->fs_cs(fs, cg).cs_nbfree >= minbfree) {
@@ -555,7 +559,7 @@ ffs_dirpref(struct inode *pip)
for (cg = prefcg; cg < fs->fs_ncg; cg++)
if (fs->fs_cs(fs, cg).cs_nifree >= avgifree)
goto end;
- for (cg = prefcg - 1; cg >= 0; cg--)
+ for (cg = 0; cg < prefcg; cg++)
if (fs->fs_cs(fs, cg).cs_nifree >= avgifree)
goto end;
end:
Re: filesystem code integer and many inodes
On Tue, May 26, 2020 at 03:54:15PM +0200, Otto Moerbeek wrote:
> On Tue, May 26, 2020 at 07:51:28AM -0600, Todd C. Miller wrote:
>
> > On Tue, 26 May 2020 12:07:21 +0200, Otto Moerbeek wrote:
> >
> > > Apart from the noting the strange Subject: I also like to mention one
> > > change in the way cylinder groups are scanned. The current code scans
> > > forward and backward, which causes an uneven distribution of full cgs
> > > (the upper end of the cgs will get full first). Fix that by always
> > > scanning forward, wrapping to cg 0 if needed.
> >
> > Should that be a separate commit? I can't find any problems
> > with the diff but I haven't tried running with it yet.
> >
> > - todd
>
> Yeah, I can do that. Note that it must be comitted first, since the
> loop condition is always true if I change the loop var to unsigned.
>
> -Otto
>
And a new diff. I accidentally capitalized a letter just before sending.
Thanks to naddy for spotting that.
-Otto
Index: sbin/clri/clri.c
===
RCS file: /cvs/src/sbin/clri/clri.c,v
retrieving revision 1.20
diff -u -p -r1.20 clri.c
--- sbin/clri/clri.c28 Jun 2019 13:32:43 - 1.20
+++ sbin/clri/clri.c26 May 2020 09:41:18 -
@@ -68,7 +68,8 @@ main(int argc, char *argv[])
char *fs, sblock[SBLOCKSIZE];
size_t bsize;
off_t offset;
- int i, fd, imax, inonum;
+ int i, fd;
+ ino_t imax, inonum;
if (argc < 3)
usage();
Index: sbin/dumpfs/dumpfs.c
===
RCS file: /cvs/src/sbin/dumpfs/dumpfs.c,v
retrieving revision 1.35
diff -u -p -r1.35 dumpfs.c
--- sbin/dumpfs/dumpfs.c17 Feb 2020 16:11:25 - 1.35
+++ sbin/dumpfs/dumpfs.c26 May 2020 09:41:18 -
@@ -69,7 +69,7 @@ union {
#define acgcgun.cg
intdumpfs(int, const char *);
-intdumpcg(const char *, int, int);
+intdumpcg(const char *, int, u_int);
intmarshal(const char *);
intopen_disk(const char *);
void pbits(void *, int);
@@ -163,6 +163,7 @@ dumpfs(int fd, const char *name)
size_t size;
off_t off;
int i, j;
+ u_int cg;
switch (afs.fs_magic) {
case FS_UFS2_MAGIC:
@@ -172,7 +173,7 @@ dumpfs(int fd, const char *name)
afs.fs_magic, ctime());
printf("superblock location\t%jd\tid\t[ %x %x ]\n",
(intmax_t)afs.fs_sblockloc, afs.fs_id[0], afs.fs_id[1]);
- printf("ncg\t%d\tsize\t%jd\tblocks\t%jd\n",
+ printf("ncg\t%u\tsize\t%jd\tblocks\t%jd\n",
afs.fs_ncg, (intmax_t)fssize, (intmax_t)afs.fs_dsize);
break;
case FS_UFS1_MAGIC:
@@ -198,7 +199,7 @@ dumpfs(int fd, const char *name)
printf("cylgrp\t%s\tinodes\t%s\tfslevel %d\n",
i < 1 ? "static" : "dynamic",
i < 2 ? "4.2/4.3BSD" : "4.4BSD", i);
- printf("ncg\t%d\tncyl\t%d\tsize\t%d\tblocks\t%d\n",
+ printf("ncg\t%u\tncyl\t%d\tsize\t%d\tblocks\t%d\n",
afs.fs_ncg, afs.fs_ncyl, afs.fs_ffs1_size,
afs.fs_ffs1_dsize);
break;
default:
@@ -223,9 +224,9 @@ dumpfs(int fd, const char *name)
(intmax_t)afs.fs_cstotal.cs_ndir,
(intmax_t)afs.fs_cstotal.cs_nifree,
(intmax_t)afs.fs_cstotal.cs_nffree);
- printf("bpg\t%d\tfpg\t%d\tipg\t%d\n",
+ printf("bpg\t%d\tfpg\t%d\tipg\t%u\n",
afs.fs_fpg / afs.fs_frag, afs.fs_fpg, afs.fs_ipg);
- printf("nindir\t%d\tinopb\t%d\tmaxfilesize\t%ju\n",
+ printf("nindir\t%d\tinopb\t%u\tmaxfilesize\t%ju\n",
afs.fs_nindir, afs.fs_inopb,
(uintmax_t)afs.fs_maxfilesize);
printf("sbsize\t%d\tcgsize\t%d\tcsaddr\t%jd\tcssize\t%d\n",
@@ -238,10 +239,10 @@ dumpfs(int fd, const char *name)
printf("nbfree\t%d\tndir\t%d\tnifree\t%d\tnffree\t%d\n",
afs.fs_ffs1_cstotal.cs_nbfree, afs.fs_ffs1_cstotal.cs_ndir,
afs.fs_ffs1_cstotal.cs_nifree,
afs.fs_ffs1_cstotal.cs_nffree);
- printf("cpg\t%d\tbpg\t%d\tfpg\t%d\tipg\t%d\n",
+ printf("cpg\t%d\tbpg\t%d\tfpg\t%d\tipg\t%u\n",
afs.fs_cpg, afs.fs_fpg / afs.fs_frag, afs.fs_fpg,
afs.fs_ipg);
- printf("nindir\t%d\tinopb\t%d\tnspf\t%d\tmaxfilesize\t%ju\n",
+ printf("nindir\t%d\tinopb\t%u\tnspf\t%d\tmaxfilesize\t%ju\n",
afs.fs_nindir, afs.fs_inopb, afs.fs_nspf,
(uintmax_t)afs.fs_maxfilesize);
printf("sbsize\t%d\tcgsize\t%d\tcgoffset %d\tcgmask\t0x%08x\n",
@@ -261,7 +262,7 @@ dumpfs(int fd, const char *name)
afs.fs_sblkno, afs.fs_cblkno, afs.fs_iblkno, afs.fs_dblkno);
Re: filesystem code integer and many inodes
On Tue, May 26, 2020 at 07:51:28AM -0600, Todd C. Miller wrote: > On Tue, 26 May 2020 12:07:21 +0200, Otto Moerbeek wrote: > > > Apart from the noting the strange Subject: I also like to mention one > > change in the way cylinder groups are scanned. The current code scans > > forward and backward, which causes an uneven distribution of full cgs > > (the upper end of the cgs will get full first). Fix that by always > > scanning forward, wrapping to cg 0 if needed. > > Should that be a separate commit? I can't find any problems > with the diff but I haven't tried running with it yet. > > - todd Yeah, I can do that. Note that it must be comitted first, since the loop condition is always true if I change the loop var to unsigned. -Otto
Re: filesystem code integer and many inodes
On Tue, 26 May 2020 12:07:21 +0200, Otto Moerbeek wrote: > Apart from the noting the strange Subject: I also like to mention one > change in the way cylinder groups are scanned. The current code scans > forward and backward, which causes an uneven distribution of full cgs > (the upper end of the cgs will get full first). Fix that by always > scanning forward, wrapping to cg 0 if needed. Should that be a separate commit? I can't find any problems with the diff but I haven't tried running with it yet. - todd
Re: filesystem code integer and many inodes
On Tue, May 26, 2020 at 11:58:39AM +0200, Otto Moerbeek wrote: > Hi, > > In theory ffs code support a maximum of UINT_MAX inodes, but in > practice, due to integer overflows in the current code, the limit is > INT_MAX inodes. > > This fixes that, and allows me to create and use filesystems with more > than INT_MAX inodes. This is partly from FreeBSD code. > > Main change is in fs.h, modifying a few fields to unsigned, most > notably fs_ipg (inodes per cylinder group) and fs_ncg (number of > cylinder groups). In various places fs_ipg * fs_ncg is computed, so > both should be unsigned. I also made sure cg indexes are unsigned and > ino_t is used for inode numbers. > > Tested on a 30TB partition with various parameters (notably -f x -b y > and -i z combinations). > > Please test and/or review, Apart from the noting the strange Subject: I also like to mention one change in the way cylinder groups are scanned. The current code scans forward and backward, which causes an uneven distribution of full cgs (the upper end of the cgs will get full first). Fix that by always scanning forward, wrapping to cg 0 if needed. -Otto
