hey nicolay, the sn1, s8, s9, and s10 brands use all add 1024 to a syscall number to indicate a native system call. the catch with all these brands is that only the brand emulation library makes native system calls, so we don't have to hack libc, ld, and any other library that directly makes system calls to add 1024 to the syscall number.
wrt modifying libc, ld, and any other libraries that make system calls directly, for prototyping purposes i'd just recommend updating the syscall wrappers so that they always add a offset to the syscall number. that offset would be stored in a global variable and would default to 0. then when the brand library is initialized it could call a function in libc which sets this value. it's very likely that this would have an unnaccaptebale performance impact on non-branded processes, and eventually we could probably come up with an alternate mechanism (say adding some nops to the syscall wrappers that can be hot patched). but i wouldn't recommend spending a lot of time coming up with the perfect fix for this now. there are plenty of much more complicated problems that you might want to concentrate on. ;) ed On Tue, Aug 04, 2009 at 01:51:07AM -0700, Nicolay wrote: > Hi, > > let's say we have a following hypothetical situation. There is a brand which > makes use of sysenter / syscall instruction for entering the kernel. Surely > branded system call numbers and native Solaris system call numbers overlap to > some extent. Thus the following trick is used: if system call number (which > came from the branded process) is higher than, say, 1024, then brand sysenter > / syscall callback would consider it a request for a native call. The call > number would be reduced by 1024 and passed to the standard system call > dispatcher. However, if system call number is less than 1024 it is considered > a branded system call number and is handled accordingly, normally by bouncing > to the userspace brand support library and emulating system call there. > > Here goes the funny part. Brand support library surely uses libc. But libc is > unaware of our tricks with system call numbers. As far as I know libc is > using at least to methods for entering the kernel depending on the desired > number of return values: via sysenter / syscall (for one return argument) and > via interrupt gate (int 0x91; for two return arguments). So calling a system > call via interrupt gate will result in a normal call completion as we did not > mess with the b_int91 callback. But calling a system call via sysenter / > syscall will go wrong - the kernel (more precisely, our brand callback) will > think that someone wants to call branded system call and trampoline back to > the support library. This is definitely not the way we wanted things to go. > Most probably this would result in an infinite recursion or in branded > process getting itself killed. > > So we need a way to tell libc that other system call numbers should be used > where appropriate. How to archive it? My idea is to somehow patch either libc > image in memory or on disk, adjusting system call numbers as necessary. > However I lack experience and knowledge to do this and predict possible > negative implications. It would be very nice if someone could point me to the > right direction, maybe explain how libc image may be patched only for branded > processes. > > Also I'm open to discussion. If my solution seems hacky or otherwise > unacceptable, please suggest yours! Particularly I'm not insisting that 1024+ > system call number sharing scheme should be used. > -- > This message posted from opensolaris.org > _______________________________________________ > brandz-discuss mailing list > brandz-discuss at opensolaris.org
