I am missing something. How is the kernel wasting space with
read/write string literals? You say 8-9 MB. How is changing this to
read only going to make any difference (except to cause BAD TRAP
panics when something tries to modify the literal)?
As for fork, you make it sound like all read/write pages are copied
everytime a fork is done. This should only be true for pages that are
modified and copied due to copy-on-write. This should not be true of
all read/write pages.
At one time, fork had enhancements to "anticipate" certain pages
needing copying (some of the stack, for instance), and copying them up
front. I don't know if it still does this (too lazy to dig in the
code). Certainly not every page is copied, especially not for large
data segments.
As for making the compiler by default put string literals in read only
segments, I would prefer the compiler to be less restrictive. I know
what I'm doing, and if I do it incorrectly, it should be my problem,
not the compiler's.
Also, the kernel stack space is not used for dynamically allocated
space, except temporarily for stack frames for function calls. Oh, I
see, you want to use the stack instead of kmem_alloc...??? If you
make the kernel stacks very large, you run into virtual address size
constraints on x86. If the dynamic memory allocators are slow, fix
them. If things are not freeing space after allocation, fix it.
Putting this in the stack where it is fast and automatically cleaned
up on return is not an answer.
Just my 2 cents.
max
Quoting Roland Mainz <[EMAIL PROTECTED]>:
Garrett D'Amore wrote:
Bart Smaalders wrote:
> Matty wrote:
>> On 3/2/07, Michael Shapiro <[EMAIL PROTECTED]> wrote:
>>
>>
>> Just out of curiosity, is there a reason intrd was written in Perl?
>
> The fellow who wrote it liked perl? poold is written in perl and
> merging was/is anticipated?
>
> Perl wouldn't be my first choice for intrd's implementation language,
> but it wouldn't be my last, either.
I am going to put my $0.02 here as well.
I have written (and continue to write) a lot of perl over the years
(probably several hundred KLOC), and for some tasks it can't be beat.
HOWEVER, I would really, really have preferred to see PSARC discourage
or even disallow the use of perl for "core platform technology". (This
goes for Python, Tcl, and some of the other scripting languages that
abound these days.)
There are several reasons for this belief, listed here:
1) Perl is a massive beast. The perl5.8.4 delivery in S10 (in
/usr/perl5) is 36 Megabytes. This creates a substantial hurdle for
folks trying to build a minimal/minimized Solaris (think of use in
appliances).
[snip]
Uhm... I guess there are MUCH bigger problems in Solaris right now. The
36MB are disk space but Solaris currently suffers from problems like
excessive memory usage which hurts the usage of Solaris in embedded
environments much more than "perl".
Just one example:
ISO C/C++ string literals are currently read-write when C/C++ code gets
compiled with Sun Workshop/Forte/Studio while the same application
compiled with gcc treats string literals as read-only data (the gcc
people even claim that Sun Workshop/Forte/Studio's behaviour violates
the ANSI C standard... ;-( ).
This issue sounds minor but the problem summs up to an incredible loss:
1. The kernel alone wastes around 8-9MB with read-write string literals
2. Userspace (assuming a simple user logged into CDE) wastes around
15-20MB (and some large standalone applications exceed this value (or
did exceed... for example I fixed Mozilla/FireFox myself and ksh93 in
Solaris uses "-xstrconst" by default (otherwise the large strings
containing the manual pages for builtin commands would hurt us badly for
each single |fork()|))). Note that this is _per_ user. Multiple copies
of the same application/tool/utility/shell will copy those data even if
they are only "read" (and copy-on-write will not help since string
literals are shattered over the whole read-write data section since the
compiler/linker does not sort data by usage or datatype (the 4k(8k page
granulation is worse, using largepages makes this problem even much more
worse)).
Now think about how this affects performance on a large multiuser
machine with 500 SunRay users or shells/utilities/etc. which call
|fork()| very often. Solaris wastes an incredible amount of memory and
CPU time just because the compiler's default setting is to treat string
literals as read-write instead of read-only.
IMO the problem needs to be handled from two sides:
1. Make the default setting for Sun Studio 12 "-xstrconst" and add an
option "-xstrnoconst" for backwards-compatibilty (CC:'ing
[EMAIL PROTECTED] for that). THIS IS VERY IMPORTANT to get the
"switch" done ASAP (see below)
2. Crawl over the OS/Net sources and add "-xstrconst" to the matching
Makefiles (the "compile OS/Net with gcc"-project already did lots of
work&&testing in this area which means the risk should be quite low)
[1] should make sure that future projects do not have to set explicitly
"-xstrconst" for each bit they compile (most projects from the "open
source" world treat string lierals as read-only anyway since this is the
default since gcc2.x and gcc4.x no longer has an option to make string
literals writeable). IMO this is the most important part since it should
elimitate the issue from userspace applications once Sun Studio 12 gets
used. The current default setting to treat string lierals is IMO not
good since it affects both memory usage and performance of applications
very badly
[2] should elimitate the problem from the OS/Net codebase and prepares
the tree for the upcoming "switch" in Sun Studio 12
Another issue is the way how applications in Solaris allocate temporary
memory (this applies to the kernel but a "solution" is much more tricky
because the kernel's stack can't grow which limits it to 4k on i86 and
8k on UltraSPARC (UltraSPARC may be in a better position since it could
map the stack with 64k pages (which runs into other problems since lots
of code doesn't expect a "custom" page size for kernel stacks... ;-((
))) - IMO in many places the usage of |malloc()| should be replaced with
|alloca()| or C99's variable length arrays assuming the size does not
exceed a given size (some sample code can be found in
http://mail.opensolaris.org/pipermail/perf-discuss/2007-February/001589.html).
This solution would have lots of advantages, for example:
- Multithreaded applications would not go through the heap each time,
e.g. avoid locking/syscall overhead in the worst case
- Better locality of data - the thread's stack is assumed to be closer
to the running CPU than a "random" page in the heap
- It is "easier" (the explanation of this is little bit beyond the scope
of this email, basically some UltraSPARC CPUs seems to have trouble
handling more than two different pagsizes at the same time except the
number of pages for a 3rd/4th size is very small) to map the stack with
largepages than the whole heap of an application
- Avoid taxing the heap with lots of small temporary allocations
----
Bye,
Roland
--
__ . . __
(o.\ \/ /.o) [EMAIL PROTECTED]
\__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
/O /==\ O\ TEL +49 641 7950090
(;O/ \/ \O;)
_______________________________________________
opensolaris-code mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/opensolaris-code
_______________________________________________
opensolaris-code mailing list
[email protected]
http://mail.opensolaris.org/mailman/listinfo/opensolaris-code
