Re: FYI: what it takes for RAM+swap to build devel/llvm40 with 4 processors or cores and WITH__DEBUG= (powerpc64 example)

[Mark Millard]

[I add some what-it-take-for-an-upgrade information.]

On 2017-Mar-12, at 6:53 PM, Mark Millard  wrote:

> Summary: RAM+(peak swap) was about 26 GiBytes.
> Also: about 118 GiByte /usr/obj/. . ./llvm40/ area.
> (2 processors, 2 cores each, all in use;
>  WITH_DEBUG= used)
> 
> The peak usage times were when the 4 cores were
> each busy running ld at the same time.
> 
> [So far as I know FreeBSD does not report peak swap usage
> "since boot". So I do not have a cross check on if I missed
> seeing a higher peak then I report in the details below.]
> 
> What all this note spans as part of the build:
> 
> # more /var/db/ports/devel_llvm40/options
> # This file is auto-generated by 'make config'.
> # Options for llvm40-4.0.0.r4
> _OPTIONS_READ=llvm40-4.0.0.r4
> _FILE_COMPLETE_OPTIONS_LIST=CLANG DOCS EXTRAS LIT LLD LLDB
> OPTIONS_FILE_SET+=CLANG
> OPTIONS_FILE_SET+=DOCS
> OPTIONS_FILE_SET+=EXTRAS
> OPTIONS_FILE_SET+=LIT
> OPTIONS_FILE_SET+=LLD
> OPTIONS_FILE_SET+=LLDB
> 
> The system clang 4.0 was used to do the build. A port
> binutils was used (-B${LOCALBASE}/bin/ in CFLAGS,
> CXXFLAGS, an CPPFLAGS). The kernel was non-debug generally
> but buildworld buildkernel did not have MALLOC_PRODUCTION= .
> The llvm40 build did have MALLOC_PRODUCTION= .
> 
> # uname -paKU
> FreeBSD FBSDG5L 12.0-CURRENT FreeBSD 12.0-CURRENT  r314687M  powerpc 
> powerpc64 1200023 1200023
> 
> Most of what I have access to for FreeBSD does not have a
> big enough configuration to do a WITH_DEBUG= build of llvm40
> on a machine with 4 cores, all in use.
> 
> One type of environment that does is an old PowerMac G5
> so-called "Quad Core" that has 16 GiBytes of RAM, 17 GiBytes
> of swap, and a 480 GiByte SSD (but extra over provisioned so
> it appears even smaller for the file system+swap).
> 
> Watching with top the peak swap usage that I saw was
> 56% of the 17 GiByte --so call it 10 GiBytes or so.
> 
> So something like 16 GiBytes RAM + 10 GiBytes swap
> and so something like 26 GiByte total.
> 
> I used portmaster with -DK. Afterwards the /usr/obj/
> sub-area for llvm40 used totaled to a size of:
> 
> # du -sg /usr/obj/portswork/usr/ports/devel/llvm40
> 118   /usr/obj/portswork/usr/ports/devel/llvm40
> 
> So around 118 GiBytes of disk space.
> 
> Showing the major space usage contributions:
> 
> # du -sg /usr/obj/portswork/usr/ports/devel/llvm40/work/.build/* 
> /usr/obj/portswork/usr/ports/devel/llvm40/work/stage/usr/local/llvm40/*
> . . .
> 29/usr/obj/portswork/usr/ports/devel/llvm40/work/.build/bin
> . . .
> 29/usr/obj/portswork/usr/ports/devel/llvm40/work/.build/lib
> . . .
> 12/usr/obj/portswork/usr/ports/devel/llvm40/work/.build/tools
> . . .
> 26
> /usr/obj/portswork/usr/ports/devel/llvm40/work/stage/usr/local/llvm40/bin
> . . .
> 24
> /usr/obj/portswork/usr/ports/devel/llvm40/work/stage/usr/local/llvm40/lib
> . . .
> 
> 
> 
> Side notes that are more system specific:
> 
> The timestamps on the script output file indicate that
> the build took about 8 hours 24 minutes.
> 
> The powerpc64 system used was built with the system clang
> 4.0 compiler and a port-based binutils. This is despite that
> clang 4.0 produces code that has any thrown C++ exceptions
> completely non-functional for powerpc64 (program crashes
> via signals reporting problems).



I upgraded from llvm40 r4 to final. An interesting result was
its creation of a backup package for llvm40-4.0.0.r4:

about 13 cpu-core-hours running pkg create

(Remember: I've been building with WITH_DEBUG= ) Its
single-threaded status stands out via elapsed time
approximately matching.

I'll note that it was somewhat under 6 elapsed hours for
staging to have been populated (-j4 with 4 cores present
helps for this part).

(Of course these elapsed-time figures are rather system
dependent, although the ratio might be more stable.)



Also interesting was:

Installed packages to be REMOVED:
llvm40-4.0.0.r4

Number of packages to be removed: 1

The operation will free 49 GiB.


===
Mark Millard
markmi at dsl-only.net

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: NFSv2 boot & OLD_NFSV2

[Toomas Soome]

> On 26. märts 2017, at 23:00, Rick Macklem  wrote:
> 
> Just in case it wasn't clear, I think this is a good idea and I think
> you have a handle on any potential problems.
> 
> Good luck with it, rick

aye, thanks, just wanted to give people some time to react. And got some stupid 
cold meanwhile:D

rgds,
toomas

> 
> From: Toomas Soome >
> Sent: Tuesday, March 21, 2017 5:04:59 AM
> To: Daniel Braniss
> Cc: Baptiste Daroussin; Rick Macklem; FreeBSD Current
> Subject: Re: NFSv2 boot & OLD_NFSV2
> 
> On 21. märts 2017, at 10:50, Daniel Braniss   >> wrote:
> 
> 
> On 21 Mar 2017, at 10:13, Baptiste Daroussin  >> 
> wrote:
> 
> On Tue, Mar 21, 2017 at 09:58:21AM +0200, Daniel Braniss wrote:
> 
> On 20 Mar 2017, at 23:55, Toomas Soome  >> wrote:
> 
> 
> On 20. märts 2017, at 23:53, Rick Macklem   >> wrote:
> 
> Baptiste Daroussin wrote:
> On Mon, Mar 20, 2017 at 08:22:12PM +0200, Toomas Soome wrote:
> Hi!
> 
> The current boot code is building NFSv3, with preprocessor conditional 
> OLD_NFSV2. Should NFSv2 code still be kept around or can we burn it?
> 
> rgds,
> toomas
> 
> I vote burn
> 
> Bapt
> I would be happy to see NFSv2 go away. However, depending on how people 
> configure
> their diskless root fs, they do end up using NFSv2 for their root fs.
> 
> Does booting over NFSv3 affect this?
> 
> I think the answer is no for a FreeBSD server (since the NFSv2 File Handle is 
> the same as
> the NFSv3 one, except padded with 0 bytes to 32bytes long).
> However, there might be non-FreeBSD NFS servers where the NFSv2 file handle 
> is different
> than the NFSv3 one and for that case, the user would need NFSv2 boot code (or
> reconfigure their root fs to use NFSv3).
> 
> To be honest, I suspect few realize that they are using NFSv2 for their root 
> fs.
> (They'd see it in a packet trace or via "nfsstat -m", but otherwise they 
> probably
> think they are using NFSv3 for their root fs.)
> 
> rick
> 
> if they do not suspect, they most likely use v3 - due to simple fact that you 
> have to rebuild loader to use NFSv2 - it is compile time option.
> 
> 
> old systems, 8.x, still use/boot v2, and so do old linux.
> NetApp has discontinued support for v2, so we had to move this machines to 
> use FreeBSD server and the day was
> saved. So, till these machines get upgraded/discontinued we have a problem. 
> There are several solutions
> to this issue, but as long as it's a matter of getting rid for the sake of 
> it, I would vote to keep it a while longer.
> 
> danny
> 
> 
> Given you are speaking of 8.x I suppose you are using the loader that comes 
> with
> it, meaning you are safe if we remove it from the loader in 12.0 (note as said
> by Toomas that is it is already off by default in the 12.0 loader) am I 
> missing
> something?
> 
> 
> as usual, did not read the whole thread, I assumed - wrongly - that support 
> for v2 would be discontinued.
> removing v2 support from the boot process is fine! great, go for it. It will 
> only involve newer
> hosts, and simplifying the boot process is always a good idea.
> 
> sorry for the noise.
> danny
> 
> 
> 
> yes, just to clarify,  the current loader code (in current), is having NFS 
> code implemented as:
> 
> #ifdef OLD_NFSV2
> 
> v2 implementation is here
> 
> #else
> 
> v3 implementation is here
> 
> #endif
> 
> Which does mean that pxeboot/loader.efi is built by default to use v3 only, 
> but we do have 2 parallel implementations of the NFS readers. And yes, the 
> question is just about boot loader reader code (we do not implement NFS 
> writes) - and we are *not* talking about server side there.
> 
> Indeed it also is possible to merge those 2 version implementations, but to 
> be honest, I see very little point of doing that either, even if there is 
> some setup still with v2 only server, there is still an option just to use 
> TFTP based boot - especially given that current boot loader does provide 
> parallel option to use either NFS or TFTP (via dhcp option 150), with 
> existing binaries - that is, without having to re-compile.
> 
> rgds,
> toomas
> 
> ___
> freebsd-current@freebsd.org  mailing list
> https://lists.freebsd.org/mailman/listinfo/freebsd-current 
> 
> To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org 
> "

Re: NFSv2 boot & OLD_NFSV2

[Rick Macklem]

Just in case it wasn't clear, I think this is a good idea and I think
you have a handle on any potential problems.

Good luck with it, rick

From: Toomas Soome 
Sent: Tuesday, March 21, 2017 5:04:59 AM
To: Daniel Braniss
Cc: Baptiste Daroussin; Rick Macklem; FreeBSD Current
Subject: Re: NFSv2 boot & OLD_NFSV2

On 21. märts 2017, at 10:50, Daniel Braniss 
> wrote:


On 21 Mar 2017, at 10:13, Baptiste Daroussin 
> wrote:

On Tue, Mar 21, 2017 at 09:58:21AM +0200, Daniel Braniss wrote:

On 20 Mar 2017, at 23:55, Toomas Soome > 
wrote:


On 20. märts 2017, at 23:53, Rick Macklem 
> wrote:

Baptiste Daroussin wrote:
On Mon, Mar 20, 2017 at 08:22:12PM +0200, Toomas Soome wrote:
Hi!

The current boot code is building NFSv3, with preprocessor conditional 
OLD_NFSV2. Should NFSv2 code still be kept around or can we burn it?

rgds,
toomas

I vote burn

Bapt
I would be happy to see NFSv2 go away. However, depending on how people 
configure
their diskless root fs, they do end up using NFSv2 for their root fs.

Does booting over NFSv3 affect this?

I think the answer is no for a FreeBSD server (since the NFSv2 File Handle is 
the same as
the NFSv3 one, except padded with 0 bytes to 32bytes long).
However, there might be non-FreeBSD NFS servers where the NFSv2 file handle is 
different
than the NFSv3 one and for that case, the user would need NFSv2 boot code (or
reconfigure their root fs to use NFSv3).

To be honest, I suspect few realize that they are using NFSv2 for their root fs.
(They'd see it in a packet trace or via "nfsstat -m", but otherwise they 
probably
think they are using NFSv3 for their root fs.)

rick

if they do not suspect, they most likely use v3 - due to simple fact that you 
have to rebuild loader to use NFSv2 - it is compile time option.


old systems, 8.x, still use/boot v2, and so do old linux.
NetApp has discontinued support for v2, so we had to move this machines to use 
FreeBSD server and the day was
saved. So, till these machines get upgraded/discontinued we have a problem. 
There are several solutions
to this issue, but as long as it's a matter of getting rid for the sake of it, 
I would vote to keep it a while longer.

danny


Given you are speaking of 8.x I suppose you are using the loader that comes with
it, meaning you are safe if we remove it from the loader in 12.0 (note as said
by Toomas that is it is already off by default in the 12.0 loader) am I missing
something?


as usual, did not read the whole thread, I assumed - wrongly - that support for 
v2 would be discontinued.
removing v2 support from the boot process is fine! great, go for it. It will 
only involve newer
hosts, and simplifying the boot process is always a good idea.

sorry for the noise.
danny



yes, just to clarify,  the current loader code (in current), is having NFS code 
implemented as:

#ifdef OLD_NFSV2

v2 implementation is here

#else

v3 implementation is here

#endif

Which does mean that pxeboot/loader.efi is built by default to use v3 only, but 
we do have 2 parallel implementations of the NFS readers. And yes, the question 
is just about boot loader reader code (we do not implement NFS writes) - and we 
are *not* talking about server side there.

Indeed it also is possible to merge those 2 version implementations, but to be 
honest, I see very little point of doing that either, even if there is some 
setup still with v2 only server, there is still an option just to use TFTP 
based boot - especially given that current boot loader does provide parallel 
option to use either NFS or TFTP (via dhcp option 150), with existing binaries 
- that is, without having to re-compile.

rgds,
toomas

___
freebsd-current@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-current
To unsubscribe, send any mail to "freebsd-current-unsubscr...@freebsd.org"

Re: r315684 panic: sleepq_add: td 0xfffff80003c01a40 to sleep on wchan 0xfffff80006f0873c with sleeping prohibited

[Gleb Smirnoff]

On Sat, Mar 25, 2017 at 11:45:29AM +0200, Konstantin Belousov wrote:
K> On Fri, Mar 24, 2017 at 08:31:42PM -0700, Gleb Smirnoff wrote:
K> >   Darren,
K> > 
K> > On Sat, Mar 25, 2017 at 03:03:14AM +0200, Konstantin Belousov wrote:
K> > K> On Fri, Mar 24, 2017 at 05:40:15PM +, Darren wrote:
K> > K> > I am getting this panic every hour to every couple of hours.
K> > K> > 
K> > K> > FreeBSD asrock 12.0-CURRENT FreeBSD 12.0-CURRENT #0 r315684: Thu Mar 
23 14:56:45 EDT 2017 darren@asrock:/usr/obj/usr/src/sys/GENERIC  amd64
K> > K> > I manually typed out the following, apologize for any typos. 
K> > K> > 
K> > K> > 
K> > K> > panic: sleepq_add: td 0xf80003c01a40 to sleep on wchan 
0xf80006f0873c with sleeping prohibited
K> > K> > cpuid = 0
K> > K> > time = 1490372797
K> > K> > KDB: stack backtrace:
K> > K> > db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 
0xfe0072e33690
K> > K> > vpanic() at vpanic+0x19c/frame 0xfe0072e33710
K> > K> > kassert_panic() at kassert_panic+0x126/frame 0xfe0072e33780
K> > K> > sleepq_add() at sleepq_add+0x34f/frame 0xfe0072337d0
K> > K> > _sleep() at _sleep+0x28d/frame 0xfe0072e33870
K> > K> > soclose() at soclose+0xda/frame 0xfe0072e338b0
K> > K> > _fdrop() at _fdrop+0x1a/frame 0xfe0072e338d0
K> > K> > sendfile_iodone() at sendfile_iodone+0x19d/frame 0xfe0072e33910
K> > K> > vnode_pager_generic_getpages_done_async() at 
vnode_pager_generic_getpages_done_async+037/frame 0xfe0072e33930
K> > K> > bufdone() at bufdone+0x64/frame 0xfe0072e33960
K> > K> > g_io_deliver() at g_io_deliver+0x276/frame 0xfe0072e339b0
K> > K> > g_io_deliver() at g_io_deliver+0x276/frame 0xfe0072e33a00
K> > K> > g_disk_done() at g_disk_done+0x104/frame 0xfe0072e33a40
K> > K> > xpt_done_process() at xpt_done_process+0x35f/frame 0xfe0072e33a80
K> > K> > xpt_done_direct() at ahci_ch_intr_direct+0xd5/frame 0xfe0072e33af0
K> > K> > ahci_itr() at ahci_intr+0x102/frame 0xfe0072e33b20
K> > K> > intr_event_execute_handlers() at 
intr_event_execute_handlers+0x99/frame 0xfe0072e33b60
K> > K> > ithread_loop() at ithread_loop+0xb6/frame 0xfe0072e33bb0
K> > K> > fork_exit() at fork_exit+0x84/frame 0xfe0072e33bf0
K> > K> > fork_trampoline() at fork_trampoline+0xe/frame 0xfe0072e33bf0
K> > K> > --- trap 0, rip = 0, rsp = 0, rbp = 0 ---
K> > K> > KDB: enter: panic
K> > K> > [ thread pid 12 tid 100038 ]
K> > K> > Stopped at  kdb_enter+0x3b: movq    $0,kdb_why
K> > K> > db>
K> > K> 
K> > K> Indeed, the context where sendfile_iodone() is executed, cannot call 
fdrop().
K> > 
K> > Can you please test the attached patch?
K> > 
K> > -- 
K> > Totus tuus, Glebius.
K> 
K> > Index: sys/kern/kern_sendfile.c
K> > ===
K> > --- sys/kern/kern_sendfile.c   (revision 315926)
K> > +++ sys/kern/kern_sendfile.c   (working copy)
K> > @@ -296,8 +296,9 @@ sendfile_iodone(void *arg, vm_page_t *pg, int coun
K> >CURVNET_RESTORE();
K> >}
K> >  
K> > -  /* XXXGL: curthread */
K> > -  fdrop(sfio->sock_fp, curthread);
K> > +  ACCEPT_LOCK();
K> > +  SOCK_LOCK(so);
K> > +  sorele(so);
K> >free(sfio, M_TEMP);
K> >  }
K> >  
K> > @@ -860,7 +861,9 @@ prepend_header:
K> >} else {
K> >sfio->sock_fp = sock_fp;
K> >sfio->npages = npages;
K> > -  fhold(sock_fp);
K> > +  SOCK_LOCK(so);
K> > +  soref(so);
K> > +  SOCK_UNLOCK(so);
K> >error = (*so->so_proto->pr_usrreqs->pru_send)
K> >(so, PRUS_NOTREADY, m, NULL, NULL, td);
K> >sendfile_iodone(sfio, NULL, 0, 0);
K> 
K> With this patch, what prevents a close of the sfio->sock_fp file, which is
K> needed to get the pointer to socket ?

You are right, patch is unfinished. Here is better one.

-- 
Totus tuus, Glebius.
Index: sys/kern/kern_sendfile.c
===
--- sys/kern/kern_sendfile.c	(revision 315926)
+++ sys/kern/kern_sendfile.c	(working copy)
@@ -80,7 +80,7 @@ struct sf_io {
 	volatile u_int	nios;
 	u_int		error;
 	int		npages;
-	struct file	*sock_fp;
+	struct socket	*so;
 	struct mbuf	*m;
 	vm_page_t	pa[];
 };
@@ -255,7 +255,7 @@ static void
 sendfile_iodone(void *arg, vm_page_t *pg, int count, int error)
 {
 	struct sf_io *sfio = arg;
-	struct socket *so;
+	struct socket *so = sfio->so;
 
 	for (int i = 0; i < count; i++)
 		if (pg[i] != bogus_page)
@@ -267,8 +267,6 @@ sendfile_iodone(void *arg, vm_page_t *pg, int coun
 	if (!refcount_release(>nios))
 		return;
 
-	so = sfio->sock_fp->f_data;
-
 	if (sfio->error) {
 		struct mbuf *m;
 
@@ -296,8 +294,9 @@ sendfile_iodone(void *arg, vm_page_t *pg, int coun
 		CURVNET_RESTORE();
 	}
 
-	/* XXXGL: curthread */
-	fdrop(sfio->sock_fp, curthread);
+	ACCEPT_LOCK();
+	SOCK_LOCK(so);
+	sorele(so);
 	free(sfio, M_TEMP);
 }
 
@@ -858,9 +857,11 @@