On Wed, 06 Dec, Richard Hilditch wrote: > I am having problems with failing to get memory on a system using LiS (2.18.0 > plus our patches which are close to 2.19.0) under stress. It looks like each > call to allocb maps down to a kmalloc call with memory type GFP_ATOMIC. > > Obviously this is useful if the message is required in interrupt context but > a > large part of our Streams code is not running in interrupt context. > > What is the background to this? > > Could we consider changing so that for instance if we called allocb with > BPRI_HI it used GFP_ATOMIC and otherwise used GFP_KERNEL? >
Traditionally SVR4 performs atomic memory allocations for the entire triplet (message block, data block and data buffer). While it is possible to use non-atomic memory allocators for the data buffer and then use esballoc(9) to obtain a message and data block, allocation for the message and data block will still be atomic, not solving your problem. OSF/HP-UX (Mentat) defined another priority flag (BPRI_WAITOK) as did other STREAMS implementations such as Solaris (BPRI_FT). Linux Fast-STREAMS implements both these flags (BPRI_WAITOK and BPRI_FT). They become particularly useful when allocating message blocks from a blocking process context, such as in the Stream Head, or STREAMS open procedure (e.g. allocating an stroptions structure for an M_SETOPTS message). So, for example, from the STREAMS open procedure (blocking context), one simply does something like: while (!(mp = allocb(sizeof(struct stroptions), BPRI_WAITOK))) ; and one is guaranteed of obtaining a message block or the calling process will block. Allocation of the entire triplet (message block, data block and data buffer) is performed at GFP_KERNEL instead of GFP_ATOMIC. BPRI_WAITOK cannot be used from within a put or service procedure (which by definition is a non-blocking context), but can be used from open or close procedures as well as any blocking context outside of STREAMS (separate kernel thread, user-kernel interface). For more description of this feature, see: http://www.openss7.org/man/man2html?9+allocb That said, if you are failing allocations of message blocks it is likely because you are exceeding the maximum number of message blocks allocated in the system. SVR4 had a tunable and many other STREAMS implementations have tunables that determine at what percentage of STREAMS dedicated memory message block allocations will fail (it is normally before the system completely runs out of memory). For Linux Fast-STREAMS, the sysctl tunable streams.nstrmsgs in (/proc/sys/streams/nstrmsgs) determines the maximum number of allocated message blocks. (Currently set to 262144 message blocks.) LiS defaults to allowing message block allocations up to the limit of memory, but has to option to set a limit base on memory usage. That said, if you are failing memory allocations it is likely because there is no memory (or no memory for STREAMS) left available. The most common cause of this situation is not GFP_ATOMIC vs. GFP_KERNEL, but a module or driver with a buffer leak. Another possibility is a system with just too little memory to be useful. Hope that helps. --brian -- Brian F. G. Bidulock ¦ The reasonable man adapts himself to the ¦ [email protected] ¦ world; the unreasonable one persists in ¦ http://www.openss7.org/ ¦ trying to adapt the world to himself. ¦ ¦ Therefore all progress depends on the ¦ ¦ unreasonable man. -- George Bernard Shaw ¦ _______________________________________________ Linux-streams mailing list [email protected] http://www.openss7.org/mailman/listinfo/linux-streams
