Dave Marquardt wrote: > "Roland" == Roland Mainz <roland.mainz at nrubsig.org> writes: > Roland> Dave Marquardt wrote: [snip] > >> More recently, more work was done for LPOOB, but I'm not as familiar > >> with that code. > > Roland> Ok.. but even for B61 64k pages are not used by default (or $ ksh93 -c > Roland> 'pmap -s -x $$ ; true' # shows wrong values (note the "true", > otherwise > Roland> ksh93 will |exec()| the last command and pmap shows the page map for > Roland> itself)). > > So it appears there hasn't been tuning done to help this particular > case.
There are many other cases where the "Large Page Out of The Box" (=LPOOB) will AFAIK never work. For example the hit&&run usage of a shell (e.g. $ ksh93 -c 'one_cmd' #) will AFAIK use multiple 8k pages for the stack but never use 64k pages until the stack usage exceeds 64k, right ? If that's "true" then the LPOOB behaviour is sub-optimal since once the 64k pages get used most of the execution time may be over (and AFAIK LPOOB currently won't use 64k pages on UltraSPARC 3/4 anyway). A similar problem exists for gzip/bzip2/libz.so.1 where the use of 4M or 512k pages gives almost no benefit while the explicit use of 64k pages for the in/out buffers delivers a significant performance boost even on a Blade1000 and a SF68k with the "crippled MMU" (=compared to the Spitfire MMU design). The important part seems to be to restrict the amount of 64k pages which are actively being used to avoid that the small set of MMU entries which can hold any page size gets exhausted (or better: "overflows") - and LPOOB is AFAIK little help here because it cannot distinguish between normal pages and the hotspots (maybe it would be possible by doing some statistics but even that would only be done with a huge delay and then most of the execution time is over). Or short: IMO it makes still sense to do explict usage of the -xpagesize* option _and_ 64k pages if their amount is restriced and the usage predictable (but I agree with you - the usage of 64k pages for the heap+anon memory may be an overkill for MMUs like those in the UltraSPARC-3/4 with the current implementation of the MMU code in Solaris which hardcodes the large tables to 8k and 4M pagesizes (which is IMO a bug and should be changed to 8k+<preferred_largepage_size_of_process>)). > Roland> [snip] > >> Another issue is that you may still have some number of stray 8K pages > >> due to alignment issues, e.g. the heap doesn't start on a 64K > >> boundary. It's possible you take care of that with a mapfile, but I > >> haven't reviewed all the Makefiles to see if that's the case. > > Roland> Right now we don't use mapfiles for that since we only map stack&&heap > Roland> with 64k pages (AFAIK you're thinking about things like text/data > Roland> segments, right ?). We still have "stray" 8k pages coming from > Roland> allocations before the "-xpagesize_*=64k"-option has an effect - but > Roland> AFAIK we can ignore this since this memory is not used in the "hot > Roland> codepaths". > > I'm specifically thinking of the BSS & heap boundary. Heap starts at > the end of BSS, and last I knew we weren't mapping BSS on large pages. How would such a mapfile look like ? [snip] > Roland> What about applying the Niagara1 defaults for UltraSPARC-1/2 > Roland> CPUs, too ? > > That was exactly my thought for the first round of tuning for > UltraSPARC 1&2. > > I've opened an RFE, CR 6569725: > > 6569725 Add better large page out of box support for UltraSPARC I and II Thanks! :-) > As I said, I doubt Sun management will want to invest in this area due > to low return on investment for Sun, but certainly someone else in the > OpenSolaris community could take it on, or perhaps some Sun employee > in his spare time. Well, since UltraSPARc-1/2 and Niagara1 AFAIK share almost the same MMU implementation... wouldn't it be possible to just "copy over" the Niagara1 settings (yes, I know... Niagara1 trades the 512k pages for 256M pages which will require some adjustments... but I guess the main LPOOB settings can be carried over...) ? ---- Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.mainz at nrubsig.org \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 7950090 (;O/ \/ \O;)
