On Mon, Sep 17, 2012 at 12:15 AM, Roland Mainz <roland.ma...@nrubsig.org> wrote:
> ksh93 currently uses a memory buffer for x=$(...) and x=${...;}
> command substitutions to buffer the data from stdout in memory as long
> as no external process is called or the buffer doesn't overflow.
> AFAIK this code is used (src/cmd/ksh93/sh/subshell.c, line 576):
> -- snip --
> 575 /* use sftmp() file for standard output */
> 576 if(!(iop = sftmp(PIPE_BUF)))
> 577 {
> 578 sfswap(sp->saveout,sfstdout);
> 579
> errormsg(SH_DICT,ERROR_system(1),e_tmpcreate);
> 580 }
> -- snip --
>
> ... question: Why is this buffer so small (it uses |PIPE_BUF| in
> ast-ksh.2012-09-11 ... which 512 on most systems... except AFAIK Linux
> which uses 4096) ?
>
> IMO it may be better to have a size (e.g. 8 times the system's default
> page size is typically optimal on SPARC machines... on other
> architectures this may vary) which is greater than the threshhold used
> by sfio to use |mmap()| to read from files. The basic idea is to make
> sure that we only spill to a file if the results are guranteed to be
> large enough so we can always |mmap()| the results.
>
> * Issues:
> - We have to make sure that memory allocation doesn't get excessive
> - Currently it seems that the stream returned by |sftmp()| somehow
> causes the shell not to use |mmap()| to read the temporary file's
> (used if we spill over or call an external process) content even for
> "insanely" (in this context :-) ) large file sizes (e.g. 2^20 bytes)
> ... erm... what exactlly causes this ?
BTW: It may be nice to have temporary file size statistics in
.sh.statss which measure...
- ... minimum file size used by x=$(...) and x=${...;}
- ... maximum file size used by x=$(...) and x=${...;}
- ... how often the memory buffer used by x=$(...) and x=${...;} was
converted to a file ("spill-over" or external process)
----
Bye,
Roland
--
__ . . __
(o.\ \/ /.o) roland.ma...@nrubsig.org
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 3992797
(;O/ \/ \O;)
_______________________________________________
ast-developers mailing list
ast-developers@research.att.com
https://mailman.research.att.com/mailman/listinfo/ast-developers
