On Sat, Sep 22, 2012 at 11:41 AM, Cedric Blancher <cedric.blanc...@googlemail.com> wrote: > On 22 September 2012 10:14, Cedric Blancher > <cedric.blanc...@googlemail.com> wrote: >> On 17 September 2012 00:15, Roland Mainz <roland.ma...@nrubsig.org> wrote: >>> * Issues: >>> - We have to make sure that memory allocation doesn't get excessive >>> - Currently it seems that the stream returned by |sftmp()| somehow >>> causes the shell not to use |mmap()| to read the temporary file's >>> (used if we spill over or call an external process) content even for >>> "insanely" (in this context :-) ) large file sizes (e.g. 2^20 bytes) >>> ... erm... what exactlly causes this ? >> >> BTW: mmap() is not used for ANY files on Fedora Linux with a 64bit >> ksh93. I've just observed that shcomp munches a 100MB script via >> read() and not mmap(), which is IMO unfortunate. > > Thanks to Olga I narrowed the issue down to MAP_TYPE. The sfio code > assumes mmap() is available when MAP_TYPE is set but newer versions of > Linux no longer set MAP_TYPE. No MAP_TYPE, no mmap(). What a shit. > Who's actually monitoring performance on Linux? > Calm down, Ced.
I tried to have a stake in this since neither Glenn or Roland seem to reply and the problem is IMO severe. of course, first I'd like to thank Al Viro and Linus Torvalds for giving me some insight into the Linux VM system and the Linux mmap() implementation. First cause of the problem is that src/lib/libast/features/mmap sets does not set #define _mmap_worthy to 2. The kernel.org people shook their heads in disbelieve and said the iffe test is just 'bollocks' - per definition of mmap() it should always be preferred if you intend to read more than one or two pages worth of data. The test doesn't quality as proper test anyway since it operates on a tiny amount of data (Al said around 10000 times more data must be processed to make this test valid) and does not test different filesystems. The second cause is that all these calls pass a valid buf to sfsetbuf() - which disabled mmap() usage. ksh -c 'builtin cat ; cat ../tst/bigscript.sh >trash ; true' ksh -c 'x=$( < ../tst/bigscript.sh ) ; true' ksh -c 'builtin tail ; tail -101000000 ../tst/bigscript.sh >trash ; true' I think that fact that these three do not use mmap() is a severe regression and MUST be fixed before the next alpha. Irek _______________________________________________ ast-developers mailing list ast-developers@research.att.com https://mailman.research.att.com/mailman/listinfo/ast-developers
