Re: dump LOB status
> On 26. Sep 2020, at 9.31, Otto Moerbeek wrote:
> Indeed, that commit was reverted in FreeBSD. This should do better. I
> do not like the assert FreeBSD has, so I turned into an quit().
Works for me. Thanks!
> Index: tape.c
> ===
> RCS file: /cvs/src/sbin/dump/tape.c,v
> retrieving revision 1.45
> diff -u -p -r1.45 tape.c
> --- tape.c28 Jun 2019 13:32:43 - 1.45
> +++ tape.c26 Sep 2020 06:30:37 -
> @@ -330,7 +330,10 @@ flushtape(void)
> }
>
> blks = 0;
> - if (spcl.c_type != TS_END) {
> + if (spcl.c_type != TS_END && spcl.c_type != TS_CLRI &&
> + spcl.c_type != TS_BITS) {
> + if (spcl.c_count > TP_NINDIR)
> + quit("c_count too large\n");
> for (i = 0; i < spcl.c_count; i++)
> if (spcl.c_addr[i] != 0)
> blks++;
Re: dump LOB status
> On 24. Sep 2020, at 15.36, Otto Moerbeek wrote: > > On Tue, Sep 22, 2020 at 08:37:22PM +0300, Juha Erkkilä wrote: >> Actually, I tested this again and now it appears >> dump and restore both work correctly. Previously, >> I first tested dump/restore with an empty filesystem, >> then with some files, and it may be that the second >> time I was accidentally testing restore with the first >> dump file. >> >> My tests were only with a small amount of files, >> I will do a better test with proper data (about >> 0.5 terabytes and over 10 files) and will >> report again here in a next few days. > > Lookin through FreeBSD commits I think you want the main.c one as > well, otherwise silent corruption of the dump is still possible. > > -Otto With that patch I get a message: fatal: morestack on g0 DUMP: fs is too large for dump! DUMP: The ENTIRE dump is aborted. This is on a 2 terabyte filesystem with 0.5 terabytes of data “successfully” backed up (or at least I considered the backup and restore as successful).
Re: dump LOB status
> On 22. Sep 2020, at 9.00, Otto Moerbeek wrote: > > On Mon, Sep 21, 2020 at 10:23:55PM +0300, Juha Erkkilä wrote: >> >> It looks like the same issue has been fixed in >> FreeBSD: https://svnweb.freebsd.org/base?view=revision=334979 >> <https://svnweb.freebsd.org/base?view=revision=334979> >> >> The diff applies cleanly to the current OpenBSD source tree. > > Maybe by hand, but not by using patch(1), the context differs a bit. > > Next obvious question: did you test if it fixes your problem? That > means, do you get a dump that can be restored again? > > -Otto Two of my previous mails did not make it to the list for some reason, but that does not matter. I tested this with 0.5 terabytes and approximately 70 thousand files, with level 0 and 1 dumps, doing some additions/deletions/moves between dumps (no inplace modifications to files, though). It appears both dump and restore worked correctly. I did not check all file contents though, but compared path listings and did contents check to some randomly sampled files. I will also test the patch Otto sent to this list a while ago. This may take some time.
Re: dump LOB status
> On 16. Sep 2020, at 20.27, Juha Erkkilä wrote: > > >> On 16. Sep 2020, at 0.18, Kenneth Gober wrote: >> I took a very quick look at the source and it appears that 213 is shown in >> octal. I believe that the 200 bit indicates that a core file was produced, >> and 13 is probably a signal number (13 octal equals 11 decimal which would >> be SIGSEGV). I am not sure whether the size of the file system is itself >> the cause, I have been using dump(8) to back up a large (currently 6.7TB) >> volume to tape for years (several tapes, actually) and it works fine, >> although that system is still on 6.1/amd64. I looked in CVS and didn't see >> any obvious diffs between 6.1 and 6.6 that jumped out at me as potential >> causes, so perhaps the issue has been latent for a long time and I haven't >> seen it because it's triggered by the particulars of one or more files >> rather than the overall file system size. Maybe if an individual file gets >> too big, or is too 'sparse' or something? > > I can reproduce this on -current from Fri Sep 11 11:30:09 > with a freshly created and an empty filesystem of 2 terabytes. It looks like the same issue has been fixed in FreeBSD: https://svnweb.freebsd.org/base?view=revision=334979 <https://svnweb.freebsd.org/base?view=revision=334979> The diff applies cleanly to the current OpenBSD source tree.
Re: dump LOB status
> On 16. Sep 2020, at 0.18, Kenneth Gober wrote: > I took a very quick look at the source and it appears that 213 is shown in > octal. I believe that the 200 bit indicates that a core file was produced, > and 13 is probably a signal number (13 octal equals 11 decimal which would > be SIGSEGV). I am not sure whether the size of the file system is itself > the cause, I have been using dump(8) to back up a large (currently 6.7TB) > volume to tape for years (several tapes, actually) and it works fine, > although that system is still on 6.1/amd64. I looked in CVS and didn't see > any obvious diffs between 6.1 and 6.6 that jumped out at me as potential > causes, so perhaps the issue has been latent for a long time and I haven't > seen it because it's triggered by the particulars of one or more files > rather than the overall file system size. Maybe if an individual file gets > too big, or is too 'sparse' or something? I can reproduce this on -current from Fri Sep 11 11:30:09 with a freshly created and an empty filesystem of 2 terabytes.
Re: dump LOB status
> On 15. Sep 2020, at 18.54, Jose Soares wrote: > > Thank you, Stuart. > > I am facing this when issuing the dump command of a "large" file system > (2.7TB). > dump command has finished successfully for the other smaller file systems. > > # df -h > Filesystem SizeUsed Avail Capacity Mounted on > /dev/wd0a 2.0G237M1.6G12%/ > /dev/sd0d 10.8T2.7T7.6T26%/home > /dev/wd0d 3.9G146K3.7G 0%/tmp > /dev/wd0f 3.9G956M2.8G25%/usr > /dev/wd0g 2.0G253M1.6G13%/usr/X11R6 > /dev/wd0h 5.9G 15.6M5.6G 0%/usr/local > /dev/wd0j 3.1G2.0K3.0G 0%/usr/obj > /dev/wd0i 2.0G2.0K1.9G 0%/usr/src > /dev/wd0e 5.9G106M5.5G 2%/var > > The only contribution I was able to find via Google was > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244470 where a similar > problem was being reported also regarding a dump of a large file system, > but for FreeBSD. > > Any suggestion to get the dump working or to better understand what is > happening? Segfault on dump, tape.c line 335 spcl.c_addr[I], it overflows. A workaround is to raise TP_BSIZE from 1024 to something bigger (maybe 8192?) in /usr/include/protocols/dumprestore.h and recompile dump. Not a proper fix! (Also happened to me maybe a week ago, recent -current, indeed the filesystem was big (2 terabytes)).
pledge(2) and pselect(2)
Hi, Is there any reason pselect() is omitted from /usr/src/sys/kern/kern_pledge.c? There is poll(), ppoll(), kevent(), kqueue() and select() for PLEDGE_STDIO. Juha Index: sys/kern/kern_pledge.c === RCS file: /home/cvs/OpenBSD/src/sys/kern/kern_pledge.c,v retrieving revision 1.93 diff -u -p -r1.93 kern_pledge.c --- sys/kern/kern_pledge.c 31 Oct 2015 02:46:23 - 1.93 +++ sys/kern/kern_pledge.c 31 Oct 2015 06:58:37 - @@ -158,6 +158,7 @@ const u_int pledge_syscalls[SYS_MAXSYSCA [SYS_kevent] = PLEDGE_STDIO, [SYS_kqueue] = PLEDGE_STDIO, [SYS_select] = PLEDGE_STDIO, + [SYS_pselect] = PLEDGE_STDIO, [SYS_fstat] = PLEDGE_STDIO, [SYS_fsync] = PLEDGE_STDIO,
