The patch causes a notable performance improvement on busy systems
which provide a native openat() system call.
The advantage is that David's new /proc emulation works everywhere in
AST and the native /proc file system is by passed.
I noticed this with a test script, which sorts files into directories,
and keeps a directory fd open per destination directory. Suddenly,
with Roland's patch, the script became MUCH faster, because it now
uses openat() for ~{fd}/destdir instead of going through the native
/proc file system.
Olga
On Thu, Jul 4, 2013 at 7:47 AM, Roland Mainz <[email protected]> wrote:
> On Wed, Jul 3, 2013 at 2:06 AM, Roland Mainz <[email protected]> wrote:
>> On Tue, Jul 2, 2013 at 8:04 PM, Roland Mainz <[email protected]>
>> wrote:
>>> On Tue, Jul 2, 2013 at 4:11 PM, Irek Szczesniak <[email protected]>
>>> wrote:
>>>> On Sun, Jun 30, 2013 at 11:53 PM, Roland Mainz <[email protected]>
>>>> wrote:
>>>>> 2013/6/28 Glenn Fowler <[email protected]>:
>>>>>>
>>>>>> the AT&T Software Technology ast alpha 2013-06-28 source release
>>>>>> has been posted to the download site
>>>>>> http://www.research.att.com/sw/download/alpha/
>>>>>> the package names and md5 checksums are
>>>>>> INIT eddbf89d061348519d86f2618b708a94
>>>>>> ast-base a745a7d4ce6f53c2e4134af4cc835ff7
>>>>>> ast-open fdb74839ff041e34c800c333188a050e
>>>>>> ast-ksh 8f22428cf30af7146bd210664c2fd166
>>>>>> the md5 sums should match the ones listed on the download page
>>>>>>
>>>>>> NOTE NOTE NOTE NOTE NOTE NOTE
>>>>>>
>>>>>> (*) *at() emulations for systems that need it
>>>>>> (*) O_CLOEXEC F_DUPFD_CLOEXEC SOCK_CLOEXEC
>>>>>> (*) accept4 pipe2
>>>>>> (*) syscall restart on EINTR controlled by
>>>>>> astserial(AST_SERIAL_RESTART,op)
>>>>>> (*) REALLY IMPORTANT: works on modern linux, some problems on solaris
>>>>>> REASON: one of the ast team will not be logging on for 16 days and
>>>>>> didn't have time left to figure out the solaris problems
>>>>>> most likely in one of these
>>>>>> src/lib/libast/comp/at.c
>>>>>> src/lib/libast/port/intercept.c
>>>>>> figuring that out left as a summer homework assignment
>>>>>
>>>>> Homework more or less done for Solaris (including |O_XATTR|/|O_SEARCH|
>>>>> support) and fixing Linux's |O_PATH| support... the prototype patch is
>>>>> attached as "astksh20130628_solaris_fixes001.diff.txt".
>>> [snip]
>>>>> - IMO it would be nice to add |ioctl()| calls per ioctl type with
>>>>> proper prototypes, e.g. replace something like
>>>>> |ioctl(JOBTTY,TIOCSETD,&linedisc)| with
>>>>> |ioctl_TIOCSETD(JOBTTY,&linedisc)|. The point is to get stronger C
>>>>> type checks and properly pass the types to the real |ioctl()|. Right
>>>>> now the code assumes a specific layout and padding of the varargs data
>>>>> which is *NOT* true and *NOT* portable (maybe we should just wait
>>>>> until something like IA64 breaks and comes back with more teeth&&slime
>>>>> to bite you... :-) ).
>>>>> Another nice side-effect would be that we can do per-|ioctl()|-type
>>>>> breakpoints... :-)
>>>>
>>>> FYI Solaris 7 still doesn't build.
>>>
>>> Erm... general issue: It would be nice to have a bit more details than
>>> just "... it doesn't work... " ... ;-)
>>>
>>> ... but I'm aware of the problem... right now I'm tracking the issues
>>> on SuSE 9.3 (thanks go to SuSE for helping out :-) ) and Solaris 8.
>>> _Maybe_ I have a patch in a few hours...
>>
>> Attached (as "astksh20130628_solaris_fixes002.diff.txt") is the new
>> patch. It fixes ast-ksh.2013-06-28 on Solaris 8 and SuSE 9.3 (old
>> Linux which even predates any of the |*at()|/|openat()|-APIs).
>>
>> Notes (Glenn/David: Please *read* this and comment if you don't like it):
>> - A lot of calls in libshell have been changed from using |open(...)|
>> to |openat(shp->pwdfd, ...| ... mainly to make sure the code goes
>> through the intercept and |*at()|-API emulation code so that all
>> emulated |O_*| flags are processed correctly and are not passed to the
>> native OS'es |open()| syscall and cause trouble there.
>> The other reason is to make libshell independent from the global cwd
>> so multiple libshell objects can work in a single process and don't
>> stomp on each other (the only design issue is that |spawnvex()| needs
>> to be enhanced to take a cwd fd as argument in the future to set the
>> cwd for a child process (AFAIK this is compatible to |vfork()| since
>> we only call |fchdir()| after |vfork()| in the child but do not write
>> into global data)).
>>
>> - src/cmd/ksh93/bltins/cd_pwd.c ... I restored the original comments
>> about |O_SEARCH| ... please leave the algorithm as-is to make sure
>> cd(1) is standard-conformant
>>
>> - src/lib/libast/comp/at.c ... added |renameat()| emulation and added
>> a missing flag for |unlinkat()| ... I still have to review the code
>> completely and add safety checks for unsupported flags... they should
>> return an error instead of doing something which may not be intended
>>
>> - src/lib/libast/features/fcntl.c ... added the __USE_GNU&&co. hacks
>> from my original patch... please don't complain... AFAIK there is no
>> way to get |O_PATH| working properly otherwise (and |O_DIRECTORY| on
>> older Linux versions (like SuSE 9.3))
>>
>> - src/lib/libast/path/pathopen.c ... off-by-one error which caused $(
>> < {fd}/filetoread.txt ) to fail on SuSE 9.3
>>
>> - src/lib/libast/port/intercept.c ... added some "filters" to remove
>> emulation flags so they do not bleed-through to the native syscalls
>
> Attached (as "astksh20130628_solaris_fixes003.diff.txt") is an updated
> version of the patch which fixes the issues which came up during code
> review:
> - Fixed error handling in cd(1) for NFSv4/CIFS/SMBFS XATTR directories
> - Opening the history files now goes through the
> |*at()|-emulation&&intercept code to make sure extra flags+signal
> restart is handled properly (tested)
>
> BTW: No, going through the |*at()| emulation is not slower unless the
> |fd| argument in |openat(fd,...)| differs between individual calls
> (well... at least the code in
> http://svn.nrubsig.org/svn/people/gisburn/code/openat_emu/openat_emu.c
> did maintain a cache (which is valid until |fchdir()|/|chdir()| is
> called) and AFAIK the |*at()|-emulation in libast should do the same).
>
> Comments/rants/etc. welcome...
>
> ----
>
> Bye,
> Roland
>
> --
> __ . . __
> (o.\ \/ /.o) [email protected]
> \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer
> /O /==\ O\ TEL +49 641 3992797
> (;O/ \/ \O;)
>
> _______________________________________________
> ast-developers mailing list
> [email protected]
> http://lists.research.att.com/mailman/listinfo/ast-developers
>
--
, _ _ ,
{ \/`o;====- Olga Kryzhanovska -====;o`\/ }
.----'-/`-/ [email protected] \-`\-'----.
`'-..-| / http://twitter.com/fleyta \ |-..-'`
/\/\ Solaris/BSD//C/C++ programmer /\/\
`--` `--`
_______________________________________________
ast-developers mailing list
[email protected]
http://lists.research.att.com/mailman/listinfo/ast-developers
