On Tue, Dec 9, 2025 at 10:39 AM Harry Yoo <[email protected]> wrote: > > On Mon, Dec 08, 2025 at 07:51:40PM +0100, Vlastimil Babka wrote: > > On 12/7/25 14:59, Harry Yoo wrote: > > > On Wed, Dec 03, 2025 at 07:15:12PM +0800, Hao Li wrote: > > >> On Wed, Dec 03, 2025 at 02:46:22PM +0900, Harry Yoo wrote: > > >> > On Tue, Dec 02, 2025 at 05:00:08PM +0800, Hao Li wrote: > > >> > > Introduce barn_get_full_sheaf(), a helper that detaches a full sheaf > > >> > > from > > >> > > the per-node barn without requiring an empty sheaf in exchange. > > >> > > > > >> > > Use this helper in __pcs_replace_empty_main() to change how an empty > > >> > > main > > >> > > per-CPU sheaf is handled: > > >> > > > > >> > > - If pcs->spare is NULL and pcs->main is empty, first try to > > >> > > obtain a > > >> > > full sheaf from the barn via barn_get_full_sheaf(). On success, > > >> > > park > > >> > > the empty main sheaf in pcs->spare and install the full sheaf as > > >> > > the > > >> > > new pcs->main. > > >> > > > > >> > > - If pcs->spare already exists and has objects, keep the existing > > >> > > behavior of simply swapping pcs->main and pcs->spare. > > >> > > > > >> > > - Only when both pcs->main and pcs->spare are empty do we fall > > >> > > back to > > >> > > barn_replace_empty_sheaf() and trade the empty main sheaf into > > >> > > the > > >> > > barn in exchange for a full one. > > >> > > > >> > Hi Hao, > > >> > > > >> > Yeah this is a very subtle difference between __pcs_replace_full_main() > > >> > and __pcs_replace_empty_main(), that the former installs the full main > > >> > sheaf in pcs->spare, while the latter replaces the empty main sheaf > > >> > with > > >> > a full sheaf from the barn without populating pcs->spare. > > >> > > >> Exactly. > > >> > > >> > Is it intentional, Vlastimil? > > > > > > Let's first see if Vlastimil had an intention, and... > > > > Hm I don't think I aimed to make this difference on purpose, but I didn't > > also aim to make the alloc/free paths completely symmetric. Rather the goal > > was just to do what seemed the best option in each situation. And probably > > getting a full sheaf and populating spare never seemed to be an important > > case to warrant the extra code for a situation that's only transient after > > boot (see below). > > > > >> > > This makes the empty-main path more symmetric with > > >> > > __pcs_replace_full_main(), > > >> > > which for a full main sheaf parks the full sheaf in pcs->spare and > > >> > > pulls an > > >> > > empty sheaf from the barn. It also matches the documented design > > >> > > more closely: > > >> > > > > >> > > "When both percpu sheaves are found empty during an allocation, an > > >> > > empty > > >> > > sheaf may be replaced with a full one from the per-node barn." > > >> > > > >> > I'm not convinced that this change is worthwhile by adding more code; > > >> > you probably need to make a stronger argument for why it should be > > >> > done. > > >> > > >> Hi Harry, > > >> > > >> Let me explain my intuition in more detail. > > >> > > >> Previously, when pcs->main was empty and pcs->spare was NULL, we used > > >> barn_replace_empty_sheaf() to trade the empty main sheaf into the barn > > >> in exchange for a full one. As a result, pcs->main became full, but > > >> pcs->spare remained NULL. Later, when frees filled pcs->main again, > > >> __pcs_replace_full_main() had to call into the barn to obtain an empty > > >> sheaf, because there was still no local spare to use. > > > > As Harry suggests, that assumes a specific pattern where we exhaust main > > sheaf first and then we fill it fully back. > > Right. > > > But even then this can only > > happen once per cpu and then we have populated the spare and are very > > unlikely to run into this situation again. > > Good point! > > > Also it's unlikely that full sheaves even exist in the barn during this > > early stage when we would request them. That assumes cpus behave differently > > and some have returned full sheaves to the barn before other cpus have > > consumed their first full sheaf and request another. > > Right. > > > More likely both barn_replace_empty_sheaf() and barn_get_empty_sheaf() will > > fail and we do alloc_full_sheaf(). > > > > And then... I think I can see an issue in > > __pcs_replace_empty_main() that's more likely to be suboptimal than the lack > > of symmetry you point out. > > > When we reach the last part below "we can reach > > here only when gfpflags_allow_blocking..." and we have empty pcs->main, a > > full sheaf from alloc_full_sheaf() and no spare, we should be doing > > "pcs->spare = pcs->main" and not barn_put_empty_sheaf(). Right? This is what > > can delay populating the spare more likely I think. > > That makes sense to me. > > > >> With this patch, when pcs->main is empty and pcs->spare is NULL, > > >> __pcs_replace_empty_main() instead uses barn_get_full_sheaf() to pull a > > >> full sheaf from the barn while keeping the now‑empty main sheaf locally > > >> as pcs->spare. The next time pcs->main becomes full, > > >> __pcs_replace_full_main() can simply swap main and spare, with no barn > > >> operations and no need to allocate a new empty sheaf. > > > > > > I'm not still sure that either way is superior, as it really depends on > > > the alloc/free pattern. If the CPU keeps allocating more objects, keeping > > > the empty sheaf is unnecessary, but we don't know what the alloc/free > > > pattern will be. > > > > Yeah. > > > > > So strong opinion from me, but I think it'd be better make > > > __pcs_replace_{full,empty}_main() handle it consistently, > > > if there is no special intention. > > > > I'd rather see some numbers. But the suboptimality pointed out above is more > > obvious to me. Do you agree and want to send a patch? :) > > I agree and would like Hao Li to try this path as he raised this topic, > if he's interested ;)
Thanks Harry for reviewing and letting me work on this as a newcomer to SLUB. > > > >> In other words, although we still need one barn operation when main > > >> first becomes empty in __pcs_replace_empty_main(), we avoid a future > > >> barn operation on the subsequent “main full” path in > > >> __pcs_replace_full_main. > > >> > > >> Thanks. > > >> > > >> > > > >> > > Signed-off-by: Hao Li <[email protected]> > > -- > Cheers, > Harry / Hyeonggon
