Re: [ast-users] [ast-developers] new home for AST and UWIN software
Thanks Lefty for all the work getting this up and running. I know dgk has some patches in the works. He may be offline for the next few weeks. On Tue, Jan 19, 2016, 4:06 PM Eleftherios Koutsofioswrote: > hi AST and UWIN users. > > as many of you noticed, the download site on www.research.att.com went > off the air shortly before the end of the year due to some security issue. > > the timing was unfortunate because several people including me were on > vacation so it's been down for a long time. > > but we've finally managed to move most of that software on GitHub. > you can find the AST and UWIN software packages under > https://github.com/att/: > /att/ast and /att/uwin. > (btw. the /att tree on GitHub hosts a lot of open source software developed > by the AT Research group. feel free to browse. I'll be putting up some > of my code there soon). > > /att/ast corresponds to the ast-open package. it includes the software that > was also available under individual packages, like ast-ksh, ast-dss, etc., > so I decided to only create this one. > it has 3 branches, matching the old structure: master (i.e. official), > alpha, > and beta. beta is the most recent one. it includes the last package I had > gotten from Glenn and Dave with some minor fixes to get it to compile on > some > new OS versions, like Centos 7 and Ubuntu 14. > > /att/uwin is the source code for the UWIN system. it has a master and a > beta > branch. I don't have an environment to build and test this on, so I > don't know > how well it builds. > > cloning either of these git repos is equivalent to downloading the INIT and > ast-open (or INIT and uwin) packages from the old site and then running: > ./bin/package read. > so the next step after the clone step is to run: ./bin/package make. > vanilla build, where no previous version of NMAKE is available should still > work and on some systems that was actually the way to go for me. > > as an example, to get and compile the beta branch of AST: > > git clone --branch beta https://github.com/att/ast.git > cd ast > ./bin/package make > > very little of the documentation from the old site has moved to the GitHub > site, I'll try to migrate the rest later, I just wanted to get the software > up again. > > thanks > > lefteris > > -- > E. Koutsofios > AT Labs - Research > > ___ > ast-developers mailing list > ast-develop...@lists.research.att.com > http://lists.research.att.com/mailman/listinfo/ast-developers > ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] ast-ksh failing to build on RHEL7
sorry for the delay still recovering from vacation run this: --- cd /home/qbarnes/ksh-20140625/arch/linux.i386-64/src/lib/libast iffe -d2 -v -X ast -X std -c 'cc -D_BLD_DLL -fPIC -D_BLD_ast -O2' - run /home/qbarnes/ksh-20140625/src/lib/libast/features/socket iffe.out 21 --- and send me iffe.out just in case its a shell problem you can try different shells: for sh in /bin/sh ksh etc. do SHELL=$sh $sh iffe -v -X ast -X std -c 'cc -D_BLD_DLL -fPIC -D_BLD_ast -O2' - run /home/qbarnes/ksh-20140625/src/lib/libast/features/socket iffe-${sh##*/}.out done and compare iffe-*.out On Sun, Jul 6, 2014 at 7:46 PM, Quentin Barnes qbar...@gmail.com wrote: I downloaded the new beta INIT.2014-06-25 and ast-ksh.2014-06-25. The build went fine on the RHEL6 Linux Distro (gcc 4.4.7) but went off the rails using RHEL7 (gcc 4.8.2) failing to build the ast library due to compilation failures. If you compare the build logs, the first substantial difference shows up around line 1323 with the iffe messing up sys/socket and later with dirent. Both header problems lead directly to compile errors later. You can check out the logs posted here: RHEL6 build (good): https://pastee.org/p679h RHEL7 build (bad): https://pastee.org/5y5nk If I compare the arch/linux.i386-64/srclib/libast/ast_socket.h files, the RHEL7 one is missing everything including and after the /* the socket part of ast_intercept.h */ comment line. Any further information I can provide or ideas to try? Quentin ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] ast beta software download update
try again On Wed, Apr 16, 2014 at 11:51 AM, Glenn Fowler glenn.s.fow...@gmail.comwrote: sorry for the problems -- this is what happens when 15 year old procedures change it will be fixed in about 12 hours when I get back to a terminal On Wed, Apr 16, 2014 at 3:34 AM, Dr. Werner Fink wer...@suse.de wrote: On Wed, Apr 16, 2014 at 08:07:11AM +0200, Michal Hlavinka wrote: Hi Glenn, when I try to download it, the files are empty. Indeed Werner -- Having a smoking section in a restaurant is like having a peeing section in a swimming pool. -- Edward Burr ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] ast beta software download update
sorry for the problems -- this is what happens when 15 year old procedures change it will be fixed in about 12 hours when I get back to a terminal On Wed, Apr 16, 2014 at 3:34 AM, Dr. Werner Fink wer...@suse.de wrote: On Wed, Apr 16, 2014 at 08:07:11AM +0200, Michal Hlavinka wrote: Hi Glenn, when I try to download it, the files are empty. Indeed Werner -- Having a smoking section in a restaurant is like having a peeing section in a swimming pool. -- Edward Burr ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
[ast-users] ast beta software download update
the ast beta 2014-04-15 source release has been posted to the download site gsf.cococlyde.org = download = beta the package names and md5 checksums are INIT 09fd885fdfa18cce16787b991dec55e1 ast-base 313056c51d69c49f0436aba80045bb4a ast-open e65c37b618c9efad76a2af5a4ffc65c9 ast-ksh 254f08e7741c7174e88c9cb187401fd5 only a few more betas before the official release ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] vmalloc memory allocations via shared memory ? / was: Re: [patch] vmalloc |mmap(MAP_ANON)| fixes for fragmentation issues (on Solaris) ... / was: Re: Severe performance regression betw
On Mon, Dec 9, 2013 at 5:17 PM, Roland Mainz roland.ma...@nrubsig.orgwrote: On Mon, Dec 9, 2013 at 10:53 PM, Roland Mainz roland.ma...@nrubsig.org wrote: On Fri, Dec 6, 2013 at 5:40 AM, Glenn Fowler glenn.s.fow...@gmail.com wrote: On Thu, Dec 5, 2013 at 4:50 PM, Irek Szczesniak iszczesn...@gmail.com wrote: [snip] Erm... Solaris (|__SunOS|) was once (pre-vmalloc-rewrite) excempt from this functionality since it cannot overcommit memory (except if someone uses |MAP_NORESERVE| or uses kernel debugging options in /etc/system) ... ... attached (as astksh20131010_vmalloc_sunos_fragmentation_fix001.diff.txt) is a patch which... 1. ... restores this exception for Solaris 2. ... bumps the |mmap()| size to 4MB for 32bit processes and 16MB for 64bit processes since both values are more or less the points where the fragmentation stops. Note that this does *not* mean it will use so much memory... it only means that it reserves this amount of memory and the real allocation happens on the first read, write or execute access of the matching MMU page. This also means there is no performance difference between a 1MB |mmap(MAP_ANON)| and a 128MB |mmap(MAP_ANON)| since it only reserves memory but does not initalise/allocate it yet... this happens on the first time it's accessed. The other reasons for the 4MB/16MB size were: x86 has 2MB largepages, allowing a ksh process to benefit from such pages, additionaly most AST (including ksh93) applications consume a few MB of memory... so there is a good chance that the typical application/shell memory consumtion completly fits into that 4MB chunk. 64bit processes get four times as much memory since it's expected that they may operate on much larger datasets (and see the comment about fragmentation above) Just to demonstrate reservation vs. real usage via Solaris pmap: -- snip -- $ ksh -c 'print hello ; pmap -x $$ ; true' | egrep '16384.*anon' FD7FFDA0 16384148 20 - rw---[ anon ] -- snip -- The test shows that of 16384k only 148k have really been touched... the difference (16384-148) is reserved by the shell process but not used. 3. Linux has /proc/sys/vm/overcommit_memory which is either 0 or 1 to describe whether the kernel permits overcommitment of memory or not. AFAIK a simple function could be written which returns |-1| (not not permit overcommitment), |0| (don't know) or |1| (does permit overcommitment) ... and if the function returns |-1| vmalloc should do the same as on Solaris 4. The patch removes one unneccesary |memset(p, 0, size)| which was touching pages and therefore allocating them Note that if I use VMALLOC_OPTIONS=getmem=safe with the patch above vmalloc seems to resort to try shared memory: -- snip -- shmget(IPC_PRIVATE, 67108864, 0600|IPC_CREAT) = 8 brk(0x00603480) = 0 shmat(8, 0, 0600) = 0xFD7FFAA0 shmdt(0xFD7FFAA0) = 0 shmat(8, 0xDFAFFFA83000, 0600) Err#22 EINVAL shmat(8, 0xEE97FD241000, 0600) Err#22 EINVAL shmat(8, 0xF7FFFE0BFBE2, 0600) Err#22 EINVAL shmat(8, 0xFBFFFDC5FB41, 0600) Err#22 EINVAL shmat(8, 0xFDFFFDA2FAF08000, 0600) Err#22 EINVAL shmat(8, 0xFEFFFD917AC84000, 0600) Err#22 EINVAL shmat(8, 0xFF7FFD88BAB42000, 0600) Err#22 EINVAL shmat(8, 0xFFBFFD845AAA1000, 0600) Err#22 EINVAL shmat(8, 0xFFDFFD822AA5, 0600) Err#22 EINVAL shmat(8, 0xFFEFFD8112A28000, 0600) Err#22 EINVAL shmat(8, 0xFFF7FD8086A14000, 0600) Err#22 EINVAL shmat(8, 0xFFFBFD8040A0A000, 0600) Err#22 EINVAL shmat(8, 0xFFFDFD801DA05000, 0600) Err#22 EINVAL shmat(8, 0xFFFEFD800C202000, 0600) Err#22 EINVAL shmat(8, 0x7D8003601000, 0600) Err#22 EINVAL shmat(8, 0xBD7FFF00, 0600) = 0xBD7FFF00 shmdt(0xBD7FFF00) = 0 shmat(8, 0xBD7FFB00, 0600) = 0xBD7FFB00 shmdt(0xBD7FFB00) = 0 shmat(8, 0xBD7FF700, 0600) = 0xBD7FF700 shmdt(0xBD7FF700) = 0 -- snip -- ... note that such an allocation is... erm... not wise... because shared memory is usually a resource which system-wide... which means if many shell processes use shared memory it won't be available for other proceses (like databases) anymore.' if you look at that particular code its probing process address boundaries and then releasing after the probe (shmdt()) ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] [patch] vmalloc |mmap(MAP_ANON)| fixes for fragmentation issues (on Solaris) ... / was: Re: Severe performance regression between ksh 2010-03-05 and 2013-10-08
if that memset(0) is in vmopen() then im not sure its unnecessary run these tests to check your patch with different sizes and with/without the memset(0) bin/package use cd builtin nmake test On Mon, Dec 9, 2013 at 4:53 PM, Roland Mainz roland.ma...@nrubsig.orgwrote: On Fri, Dec 6, 2013 at 5:40 AM, Glenn Fowler glenn.s.fow...@gmail.com wrote: On Thu, Dec 5, 2013 at 4:50 PM, Irek Szczesniak iszczesn...@gmail.com wrote: On Wed, Dec 4, 2013 at 3:02 PM, Glenn Fowler glenn.s.fow...@gmail.com wrote: On Sun, Dec 1, 2013 at 4:58 PM, Lionel Cons lionelcons1...@gmail.com wrote: On 1 December 2013 17:26, Glenn Fowler glenn.s.fow...@gmail.com wrote: I believe this is related to vmalloc changes between 2013-05-31 and 2013-06-09 re-run the tests with export VMALLOC_OPTIONS=getmem=safe if that's the problem then it gives a clue on a general solution details after confirmation timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0; while read i ; do key=$i$((k++)) ; a[$key]=$i ; done ; printf %s\n ${a[@]} ; } ; print ${.sh.version} ; nanosort xxx yyy' Version AIJMP 93v- 2013-10-08 real 34.60 user 33.27 sys1.19 VMALLOC_OPTIONS=getmem=safe timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0; while read i ; do key=$i$((k++)) ; a[$key]=$i ; done ; printf %s\n ${a[@]} ; } ; print ${.sh.version} ; nanosort xxx yyy' Version AIJMP 93v- 2013-10-08 real 15.34 user 14.67 sys0.52 So your hunch that VMALLOC_OPTIONS=getmem=safe fixes the problem is correct. What does VMALLOC_OPTIONS=getmem=safe do? vmalloc has an internal discipline/method for getting memory from the system several methods are available with varying degrees of thread safety etc. see src/lib/libast/vmalloc/vmdcsystem.c for the code and src/lib/libast/vmalloc/malloc.c for the latest VMALLOC_OPTIONS description (vmalloc.3 update shortly) ** getmemory=f enable f[,g] getmemory() functions if supported, all by default ** anon: mmap(MAP_ANON) ** break|sbrk: sbrk() ** native: native malloc() ** safe: safe sbrk() emulation via mmap(MAP_ANON) ** zero: mmap(/dev/zero) i believe the performance regression with anon is that on linux mmap(0MAP_ANON|MAP_PRIVATE...), which lets the system decide the address, returns adjacent (when possible) region addresses from highest to lowest order and the reverse order at minimum tends to fragment more memory zero has the same hi=lo characteristic i suspect it adversely affects the vmalloc coalescing algorithm but have not dug deeper for now the probe order in vmalloc/vmdcsystem.c was simply changed to favor safe MAP_FIXED should be avoided because its only there for special purposes like the runtime linker ld.so.1 or debuggers. Using this for a general-purpose memory allocator causes serious problems: 1. On some systems this is a privileged operation and only available for users with root privileges 2. SPARC T4 with 256GB and Solaris 11.1 the use of 'safe' degraded the performance from 9 seconds to almost 15 minutes because it utterly destroys the systems concept of large pages. If two MAP_FIXED mappings follow directly each other the system downgrades the page size to the smallest possible size, even trying to break up larger pages, which in turn must be done by a special deamon (vmtasks) 3. MAP_PRIVATE|MAP_FIXED|MAP_ANON may no longer be available in future versions of Solaris 4. Using the 'safe' allocator on SmartOS (solaris 11 clone) triggers a SEGV: map(0xCD800B482000, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANON, 4294967295, 0) = 0xCD800B482000 sigaction(SIGSEGV, 0xFD7FFFDFDE50, 0xFD7FFFDFDED0) = 0 Incurred fault #6, FLTBOUNDS %pc = 0x0052FE06 siginfo: SIGSEGV SEGV_MAPERR addr=0xCD800B582000 Received signal #11, SIGSEGV [caught] siginfo: SIGSEGV SEGV_MAPERR addr=0xCD800B582000 lwp_sigmask(SIG_SETMASK, 0x0400, 0x, 0x, 0x) = 0xFFBFFEFF [0x] edit src/lib/libast/vmalloc/vmmaddress.c and change #define VMCHKMEM0 this affects vmalloc detecting overbooked memory but will disable the MAP_FIXED codepath Erm... Solaris (|__SunOS|) was once (pre-vmalloc-rewrite) excempt from this functionality since it cannot overcommit memory (except if someone uses |MAP_NORESERVE| or uses kernel debugging options in /etc/system) ... ... attached (as astksh20131010_vmalloc_sunos_fragmentation_fix001.diff.txt) is a patch which... 1. ... restores this exception for Solaris 2. ... bumps the |mmap()| size to 4MB for 32bit processes
Re: [ast-users] Severe performance regression between ksh 2010-03-05 and 2013-10-08
edit src/lib/libast/vmalloc/vmmaddress.c and change #define VMCHKMEM0 this affects vmalloc detecting overbooked memory but will disable the MAP_FIXED codepath On Thu, Dec 5, 2013 at 4:50 PM, Irek Szczesniak iszczesn...@gmail.comwrote: On Wed, Dec 4, 2013 at 3:02 PM, Glenn Fowler glenn.s.fow...@gmail.com wrote: On Sun, Dec 1, 2013 at 4:58 PM, Lionel Cons lionelcons1...@gmail.com wrote: On 1 December 2013 17:26, Glenn Fowler glenn.s.fow...@gmail.com wrote: I believe this is related to vmalloc changes between 2013-05-31 and 2013-06-09 re-run the tests with export VMALLOC_OPTIONS=getmem=safe if that's the problem then it gives a clue on a general solution details after confirmation timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0; while read i ; do key=$i$((k++)) ; a[$key]=$i ; done ; printf %s\n ${a[@]} ; } ; print ${.sh.version} ; nanosort xxx yyy' Version AIJMP 93v- 2013-10-08 real 34.60 user 33.27 sys1.19 VMALLOC_OPTIONS=getmem=safe timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0; while read i ; do key=$i$((k++)) ; a[$key]=$i ; done ; printf %s\n ${a[@]} ; } ; print ${.sh.version} ; nanosort xxx yyy' Version AIJMP 93v- 2013-10-08 real 15.34 user 14.67 sys0.52 So your hunch that VMALLOC_OPTIONS=getmem=safe fixes the problem is correct. What does VMALLOC_OPTIONS=getmem=safe do? vmalloc has an internal discipline/method for getting memory from the system several methods are available with varying degrees of thread safety etc. see src/lib/libast/vmalloc/vmdcsystem.c for the code and src/lib/libast/vmalloc/malloc.c for the latest VMALLOC_OPTIONS description (vmalloc.3 update shortly) ** getmemory=f enable f[,g] getmemory() functions if supported, all by default ** anon: mmap(MAP_ANON) ** break|sbrk: sbrk() ** native: native malloc() ** safe: safe sbrk() emulation via mmap(MAP_ANON) ** zero: mmap(/dev/zero) i believe the performance regression with anon is that on linux mmap(0MAP_ANON|MAP_PRIVATE...), which lets the system decide the address, returns adjacent (when possible) region addresses from highest to lowest order and the reverse order at minimum tends to fragment more memory zero has the same hi=lo characteristic i suspect it adversely affects the vmalloc coalescing algorithm but have not dug deeper for now the probe order in vmalloc/vmdcsystem.c was simply changed to favor safe MAP_FIXED should be avoided because its only there for special purposes like the runtime linker ld.so.1 or debuggers. Using this for a general-purpose memory allocator causes serious problems: 1. On some systems this is a privileged operation and only available for users with root privileges 2. SPARC T4 with 256GB and Solaris 11.1 the use of 'safe' degraded the performance from 9 seconds to almost 15 minutes because it utterly destroys the systems concept of large pages. If two MAP_FIXED mappings follow directly each other the system downgrades the page size to the smallest possible size, even trying to break up larger pages, which in turn must be done by a special deamon (vmtasks) 3. MAP_PRIVATE|MAP_FIXED|MAP_ANON may no longer be available in future versions of Solaris 4. Using the 'safe' allocator on SmartOS (solaris 11 clone) triggers a SEGV: map(0xCD800B482000, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANON, 4294967295, 0) = 0xCD800B482000 sigaction(SIGSEGV, 0xFD7FFFDFDE50, 0xFD7FFFDFDED0) = 0 Incurred fault #6, FLTBOUNDS %pc = 0x0052FE06 siginfo: SIGSEGV SEGV_MAPERR addr=0xCD800B582000 Received signal #11, SIGSEGV [caught] siginfo: SIGSEGV SEGV_MAPERR addr=0xCD800B582000 lwp_sigmask(SIG_SETMASK, 0x0400, 0x, 0x, 0x) = 0xFFBFFEFF [0x] Irek ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Severe performance regression between ksh 2010-03-05 and 2013-10-08
I believe this is related to vmalloc changes between 2013-05-31 and 2013-06-09 re-run the tests with export VMALLOC_OPTIONS=getmem=safe if that's the problem then it gives a clue on a general solution details after confirmation On Thu, Nov 28, 2013 at 1:28 PM, Lionel Cons lionelcons1...@gmail.comwrote: On 28 November 2013 08:58, Glenn Fowler glenn.s.fow...@gmail.com wrote: here are some points of reference showing real user sys times these were manually sampled to pinpoint jumps in performance from 206 ksh binaries from 2006-11-22 through 2013-09-10 ksh-2009-11-17 0m12.06s 0m11.82s 0m0.14s ksh-2009-12-04 0m13.84s 0m13.58s 0m0.16s ksh-2010-06-16 0m15.15s 0m14.92s 0m0.17s ksh-2010-11-16 0m14.06s 0m13.82s 0m0.16s ksh-2011-04-11 0m12.72s 0m12.44s 0m0.17s ksh-2011-06-21 0m12.58s 0m12.35s 0m0.15s ksh-2011-09-21 0m20.58s 0m20.27s 0m0.19s ksh-2013-04-11 0m23.40s 0m23.15s 0m0.17s ksh-2013-05-13 0m13.83s 0m13.61s 0m0.12s ksh-2013-05-31 0m14.15s 0m13.93s 0m0.11s ksh-2013-06-06 0m27.15s 0m26.87s 0m0.14s On Wed, Nov 27, 2013 at 5:38 PM, Lionel Cons lionelcons1...@gmail.com wrote: We've observing a *severe* performance regression between ksh 2010-03-05 and 2013-10-08 on Solaris 11, AMD64, LANG is en_US.UTF-8: # prepare $ timex seq 140 xxx # run new ksh $ timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0; while read i ; do key=$i$((k++)) ; a[$key]=$i ; done ; printf %s\n ${a[@]} ; } ; print ${.sh.version} ; nanosort xxx yyy' Version AIJMP 93v- 2013-10-08 real 32.59 user 32.19 sys0.30 # run old ksh - much faster $ timex /bin/ksh -c 'function nanosort { typeset -A a ; integer k=0; while read i ; do key=$i$((k++)) ; a[$key]=$i ; done ; printf %s\n ${a[@]} ; } ; print ${.sh.version} ; nanosort xxx yyy' Version JM 93t+ 2010-03-05 real 14.59 user 13.92 sys0.56 Can anyone explain this? IO-wise the new ksh is better but consumes much more CPU time, while the old ksh issues more IO requests but consumes only half as much CPU time. Lionel I looks that the problem is related to the function scope, without a function scope the loop takes 24 seconds, and with function scope it takes 32 seconds: timex ~/bin/ksh -c 'nanosort() { typeset -A a ; integer k=0; while read i ; do key=$i$((k++)) ; a[$key]=$i ; done ; printf %s\n ${a[@]} ; } ; print ${.sh.version} ; nanosort xxx yyy' Version AIJMP 93v- 2013-10-08 real 24.98 user 24.57 sys0.32 timex ~/bin/ksh -c 'function nanosort { typeset -A a ; integer k=0; while read i ; do key=$i$((k++)) ; a[$key]=$i ; done ; printf %s\n ${a[@]} ; } ; print ${.sh.version} ; nanosort xxx yyy' Version AIJMP 93v- 2013-10-08 real 32.79 user 32.39 sys0.31 Lionel ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] If this is a bug, what ksh version must I upgrade to?
in a private note Dan asked if ksh had ~(...) magic for doing the rightmost match and I correctly said NO it turns out ast regex and ksh extended patterns can do rightmost match with negation rather than magic flags/options it just took a night's sleep to figure out a neat use for negation this pattern will return the token of the rightmost match in ${.sh.match[5]} ~(-glr)!(__+(+([A-Z0-9])?(_))__)@(__+(+([A-Z0-9])?(_))__) -g non-greedy -l no left anchor -r no right anchor On Wed, Oct 30, 2013 at 10:01 AM, dan.rickh...@comcast.net wrote: Glenn, Thank you very much. Your suggestion that I use ~(-g) did the trick. Not only is the code is blazing fast, but, since I needed the left-most match, it (serendipitously) fixed the problem that my previous code was finding the right-most match. Here are the before and after versions. x=YY:__AB_CD_EF12_34_56PQ_RS_TU__:ZZ [[ $x == @(*__+(+([A-Z0-9])?(_))__*) ]] print “${.sh.match[2]} PQ_RS_TU [[ $x == ~(-g:@(*__+(+([A-Z0-9])?(_))__*)) ]] print “${.sh.match[2]} AB_CD_EF ** BTW — This is used in some code that marches along such lines, from left to right, finding and replacing tokens bounded by double underscores. Each token could be one or more numbers and/or uppercase letters and/or underscores. Where the token's embedded underscores must not be adjacent to another underscore, or appear at the token's ends. I also wanted to always be able to recover the match from the same index-numbered element of {.sh.match}, in this case index 2. Thanks, Dan -- *From: *Glenn Fowler glenn.s.fow...@gmail.com *To: *Dan Rickhoff dan.rickh...@comcast.net *Cc: *ast-users@lists.research.att.com *Sent: *Tuesday, October 29, 2013 7:24:26 AM *Subject: *Re: [ast-users] If this is a bug, what ksh version must I upgrade to? its a performance problem with the underlying regex whenever (...) groups are involved it has to work harder if you only care about *any* match vs the longest of the leftmost matches then prefix the pattern with ~(-g) which means not greedy or minimal this loop shows the time deterioration x= for ((i = 1; i = 20; i++)) do x=x$x time -f %E $SHELL -c [[ x__${x}__x == *@(__+(+(x)?(_))__)* ]]; printf '%d %2d ' $? $i done On Tue, Oct 29, 2013 at 8:40 AM, Dan Rickhoff dan.rickh...@comcast.netwrote: If this is a ksh bug, what ksh version should I upgrade to? On: OS: Red Hat Enterprise Linux Server release 6.1 (Santiago) ksh: version sh (ATT Research) 93t+ 2010-06-21 Elapsed time less than 2 tenths of a second: $ time -f ‘%E\n' ksh -e '[[ A___C_Z___F == *@(__+(+([A-Z0-9])?(_))__)* ]]' 0:00.14 However, if that string is extended by adding, say, seven more Zs, then the elapsed mushrooms to almost 10 seconds. $ time -f '%E\n' ksh -e '[[ A___C____F == *@(__+(+([A-Z0-9])?(_))__)* ]]' 0:09.96 This appears to be a ksh bug (a memory leak?), what ksh version must I upgrade to to get past it? Please let me know if I should provide further information. Thanks, Dan ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] If this is a bug, what ksh version must I upgrade to?
its a performance problem with the underlying regex whenever (...) groups are involved it has to work harder if you only care about *any* match vs the longest of the leftmost matches then prefix the pattern with ~(-g) which means not greedy or minimal this loop shows the time deterioration x= for ((i = 1; i = 20; i++)) do x=x$x time -f %E $SHELL -c [[ x__${x}__x == *@(__+(+(x)?(_))__)* ]]; printf '%d %2d ' $? $i done On Tue, Oct 29, 2013 at 8:40 AM, Dan Rickhoff dan.rickh...@comcast.netwrote: If this is a ksh bug, what ksh version should I upgrade to? On: OS: Red Hat Enterprise Linux Server release 6.1 (Santiago) ksh: version sh (ATT Research) 93t+ 2010-06-21 Elapsed time less than 2 tenths of a second: $ time -f ‘%E\n' ksh -e '[[ A___C_Z___F == *@(__+(+([A-Z0-9])?(_))__)* ]]' 0:00.14 However, if that string is extended by adding, say, seven more Zs, then the elapsed mushrooms to almost 10 seconds. $ time -f '%E\n' ksh -e '[[ A___C____F == *@(__+(+([A-Z0-9])?(_))__)* ]]' 0:09.96 This appears to be a ksh bug (a memory leak?), what ksh version must I upgrade to to get past it? Please let me know if I should provide further information. Thanks, Dan ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] [ast-developers] Temporary project work at Google Code?
thanks for the suggestion, its added to the list of possibilities today dgk and I may finally be in a place to start building again its amazing how systems (in this case gsfdgk home systems), left to their own designs, become harder to replicate as time passes in my particular case I have forgotten all of the little personalization tweaks that make a stock system my system although forgetting may not matter that much because those tweaks are the least stable part of linux distros On Tue, Oct 29, 2013 at 6:07 AM, Lionel Cons lionelcons1...@gmail.comwrote: On 28 October 2013 03:07, Irek Szczesniak iszczesn...@gmail.com wrote: Glenn, is it possible to temporary do some work on ast-ksh and ast-open using Google Code and push another alpha in the first week of November 2013? I think Roland Mainz provided some good patches and IMO I could provide further patches if we can get things moving in the first week of November. +1 It would be good if development work could continue this month. Lionel ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] dgk gsf status
thanks for the kind remarks based on a few comment a clarification is in order in this statement AST means AST+UWIN -- UWIN will not fall by the wayside, at least not as long as dgk and I have friends and family with windows machines that need periodic fumigation: Both dgk and gsf will continue to work on AST software, and might actually have more time (at least in the short run) to focus on it. On Tue, Oct 1, 2013 at 2:55 PM, Glenn Fowler glenn.s.fow...@gmail.comwrote: As has been pointed out several times on the AST and UWIN lists, ATT gives very little support to OpenSouce software, which is why we have so few people involved with our rather large collection of AST software. In spite of this, ksh, nmake, vczip, UWIN and other AST tools continue to be used in several ATT projects. It turns out that software isn't the only thing lacking support: both dgk (ATT fellow, 36 years of service) and gsf (ATT fellow, 29 years of service) have been terminated, effective October 10. Our third major partner, Phong Vo (ATT fellow, 32 years of service), left a few months ago for Google. The UWIN maintainer, Jeff Fellin, is still with ATT and provides UWIN support for some critical operations. Both dgk and gsf will continue to work on AST software, and might actually have more time (at least in the short run) to focus on it. The download site and mail groups will remain within ATT for at least the next several months. Our ATT colleague, dr.ek, AST user and bug detector, will maintain the site. We have secured the astopen.org domain and are investigating non-ATT hosting options, including a repository with bug tracking. The process of change will take time; the patience of the user community will be greatly appreciated. Its quite a shock to have 3 weeks to plan personal, career, and hacking futures after working in an environment that has essentially been stable for almost 30 years. The user groups will be informed as plans solidify. New email: dgk dgk...@gmail.com gsf glenn.s.fow...@gmail.com Switch over to these soon: ATT does not forward email of former employees. Thanks -dgk -gsf ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Number of allocated bytes through pointer?
On Fri, 27 Sep 2013 10:09:11 +0200 Simon Toedt wrote: Does libast's malloc() implementation have an api to return the number of bytes which were allocated for a pointer returned by malloc() or realloc()? it used to but not since the thread-safe-efficient retooling ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] [ast-developers] 'printf foo bar; head -1 bar' - ksh93 builtin vs OSX portability?
this was an interesting diversion a text file with size0 and newline!=last-char has an incomplete line the head and tail descriptions do not mention incomplete lines but head has some text that is equivalent to incomplete line counts as a line: When a file contains less than number lines, it shall be copied to standard output in its entirety. This shall not be an error. for consistency tail should treat incomplete lines the same way head does given that, both the current ast head and ast tail are inconsistent w.r.t. incomplete lines head was easy to fix by implementing the entirety text above tail was a bit more interesting because it if revealed a bug in sfmove(from,to,size,record_sep) when record_sep is '\n' -- it loses the data for an incomplete line fixing tail involved checking if sfmove() hit an incomplete line and then calling sfmove(...,-1) to move all remaining *and* fixing sfmove(...,-1) to recognize that an incomplete line was hit regression tests were added to src/cmd/builtin and now head and tail align with gnu from what I can tell the standard conformance tests completely miss this issue this will be in the next alpha and that will be early next week On Thu, 19 Sep 2013 22:36:44 +0200 Roland Mainz wrote: On Thu, Sep 19, 2013 at 8:53 PM, FELLIN, JEFFREY K (JEFF) j...@research.att.com wrote: When I test printf foo bar there is no newline in the file: $ printf foo bar $ od -bc bar 000 146 157 157 f o o 003 So if head -1 is suppose to print the first line, there is no end of line, hence to no line. Given the definition of head prints lines, wouldn't a file containing no newline chars, not have any output from head? One counterargument may be that we're talking about the last line, which may/should be always printed, regardless whether it has a newline or not. The other issue is that I can't find a head(1) implementation which behaves like AST head(1) with Irek's example: On Solaris 10 (/usr/bin/head is the natiev SystemV implementation, not the head(1) from AST) I get this: -- snip -- $ bash -c 'printf foo bar ; /usr/bin/head -1 bar ; printf '\n'' foo $ bash -c 'printf foo bar ; /usr/gnu/bin/head -1 bar ; printf '\n'' foo $ bash -c 'printf foo bar ; /usr/bsd/bin/head -1 bar ; printf '\n'' foo -- snip -- On SuSE/Linux I get this for GNU coreutils and busybox head(1): -- snip -- $ bash -c 'printf foo bar ; /usr/bin/head -1 bar ; printf '\n'' foo $ printf foo bar ; busybox head -1 bar ; printf '\n' foo -- snip -- Erm... this looks pretty much uniform... ;-/ Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.ma...@nrubsig.org \__\/\/__/ MPEG specialist, CJAVASunUnix programmer /O /==\ O\ TEL +49 641 3992797 (;O/ \/ \O;) ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Thank you for the grep builtin!
On Fri, 13 Sep 2013 17:14:03 +0200 Lionel Cons wrote: On 18 June 2013 10:20, Glenn Fowler g...@research.att.com wrote: you showed ast grep strace output but not gnu grep gnu /usr/bin/grep does read of varying chunk sizes on my redhat linux for NFS files the chunk sizes were around 32Ki I added some test options to the SFIO_OPTIONS env var SFIO_OPTIONS=nomaxmap # disable mmap(), force read() SFIO_OPTIONS=maxmap=-1 # map entire file SFIO_OPTIONS=maxmap=1Gi # map 1Gi chunks etc. as long ast the buffer isn't trivially small I don't think lines spanning buffer boundaries will be a drag on timing you might be able to set up a few experiments to see if there is a knee in the performance curves where the space-time tradeoffs meet The test results are in the email below: - bigger mmap() windows for sfio file IO. The break even we calculated is somewhere between 108-140MB window size on an Intel Sandy Bridge four way server and 144-157 for an AMD Jaguar prototype machine (X1150). This is likely to 'fix' most of the grep performance issue. Also forwarding a clarifying comment: explain them that this is NOT akin to buffer bloat. It only the defines the maximum size of the address space window a process can have for this file. The kernel itself is in charge to select an appropriate number of pages it can donate to pass data through this window thanks for the report based on this for the next alpha the default will be 128Mi on machines where sizeof(void*)=8 it can be tweaked with more feedback from other users / systems ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Thank you for the grep builtin!
On Fri, 13 Sep 2013 23:14:22 +0200 =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= wrote: Glenn, shared mmap() mapping do not have any impact on fork() performance, at least on VM architectures who can share pages (this is common practice since at least SystemV, and no modern Unix or Linux exists which does not do copy-on-write, but more on that below) The pages are not even touched, or looked at at fork() time, so even millions of mmap() pages have no impact. Only if the pages are touched the VM system will realize a fork() has happened, and *may* create a copy-on-write copy if you write to it. If you only read the pages nothing will happen. thanks we weren't concerned about the pages themselves but the TLB or whatever the vm system uses to keep track of pages that has to be duped on fork(), no? or are you saying even that is copy on write? ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Thank you for the grep builtin!
we're getting close again we're not interested in the pages but the metadata for the pages this may be based on incorrect assumptions ... 1Gib mapped and 8Kib page size = 131072 entries for address-to-page lookup at fork() time the parent process has that 131072 entry table in hand what does the child get? a copy of that 131072 entry table or a reference? On Fri, 13 Sep 2013 23:26:34 +0200 =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= wrote: No, this is not copy on write, this is check-what-to-do-on-access-when-not-mapped. The short explanation is, that the fork() is not the time when an action in the VM system will happen, its the time of the first access to a page, which is not mapped yet, in the current process, when an action will happen. What is copied at fork() time, is the range information, i.e. mapping from/to/flags, but not the individual pages. So the number of mapped areas is a concern at fork() time, but not their size. Olga On Fri, Sep 13, 2013 at 11:20 PM, Glenn Fowler g...@research.att.com wrote: On Fri, 13 Sep 2013 23:14:22 +0200 =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= wrote: Glenn, shared mmap() mapping do not have any impact on fork() performance, at least on VM architectures who can share pages (this is common practice since at least SystemV, and no modern Unix or Linux exists which does not do copy-on-write, but more on that below) The pages are not even touched, or looked at at fork() time, so even millions of mmap() pages have no impact. Only if the pages are touched the VM system will realize a fork() has happened, and *may* create a copy-on-write copy if you write to it. If you only read the pages nothing will happen. thanks we weren't concerned about the pages themselves but the TLB or whatever the vm system uses to keep track of pages that has to be duped on fork(), no? or are you saying even that is copy on write? -- , __ , { \/`o;-Olga Kryzhanovska -;o`\/ } .'-/`-/ olga.kryzhanov...@gmail.com \-`\-'. `'-..-| / http://twitter.com/fleyta \ |-..-'` /\/\ Solaris/BSD//C/C++ programmer /\/\ `--` `--` ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] NaN and their payloads
I'm aware of NaN payload but have not seen C code that * has MIN/MAX macros for the min/max payload value for a give FP type * sets a NaN value with a specific payload * extracts a payload from a NaN are there standard api's for this? for the casual reader, comparisons on NaN are unordered: f = NaN; f == f = false f != f = false so almost all post-facto ops on NaNs are NaN-specific functions On Tue, 3 Sep 2013 02:50:31 +0200 Tina Harriott wrote: I'm currently looking into a complex mathematical simulation (fluid dynamics) and found references to NaN (Not A Number) with payloads. The Nan values are used to represent 'no data' in the datasets and the payloads are used for context information. The code has almost 2 million lines of code and removing the feature would be a major effort. As we are in the process of using ksh93 as interface to the simulation using libshell we need access to Nan values and their payloads. Can anyone give an estimate how hard it would be to add support for Nan payloads to ksh93? Tina -- Tina Harriott - Women in Mathematics Contact: tina.harriott.m...@gmail.com ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] [ast-developers] arithmetric function int() does not work
On Tue, 3 Sep 2013 04:35:56 +0200 Tina Harriott wrote: The arithmetic function int() does not work in sh (ATT Research) 93v- 2013-08-27 I get this: ksh -c 'print -- $(( log2( int(pow(2,69) )) ))' 69 But it should print 'nan', as it does if I use explicitly an intermediate integer variable: ksh -c 'integer i; print -- $(( i=pow(2,69) , log2(i) ))' nan in ksh93 the int() arith function is an alias for the shell floor() which calls the C library floorl() (or floor() if long double not available) C floor() has no too big to fit in C integral value error if int(f) is supposed to act like the C cast (int)f then the ksh int() needs to be its own function that implements the cast (ksh_int_type)f ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Anyone got /dev/file@ working?
On Fri, 30 Aug 2013 04:03:57 +0200 Cedric Blancher wrote: On 28 August 2013 16:32, Lionel Cons lionelcons1...@gmail.com wrote: Did anyone got /dev/file@ working in ast-ksh.20130814 or is that feature broken there? This is again broken in ast-ksh.20130829: ksh -c 'redirect {n}/dev/file@sync/etc ; true' /home/ced/bin/ksh: /dev/file@sync/etc: cannot open [No such file or directory] the feature is there the spelling is different the original spelling was in a non-release patch proposal current spelling is as follows for reasons already posted /dev/file/sync:etc # relative /dev/file/sync:/etc # absolute ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Anyone got /dev/file@ working?
On Fri, 30 Aug 2013 04:33:09 +0200 Cedric Blancher wrote: On 30 August 2013 04:29, Glenn Fowler g...@research.att.com wrote: On Fri, 30 Aug 2013 04:03:57 +0200 Cedric Blancher wrote: On 28 August 2013 16:32, Lionel Cons lionelcons1...@gmail.com wrote: Did anyone got /dev/file@ working in ast-ksh.20130814 or is that feature broken there? This is again broken in ast-ksh.20130829: ksh -c 'redirect {n}/dev/file@sync/etc ; true' /home/ced/bin/ksh: /dev/file@sync/etc: cannot open [No such file or directory] the feature is there the spelling is different the original spelling was in a non-release patch proposal current spelling is as follows for reasons already posted /dev/file/sync:etc # relative /dev/file/sync:/etc # absolute Well, I did like /dev/file@options/ because it was clear where the options are. It looked familiar and user-friendly. Sounds like I have to get used to the new Windows-like style does it work? also, I don't know what you mean windows like I get hives or BSOD doing anything windows-like ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Errors while trying to build ksh.2013-08-07 on debian testing
thanks for the report this will be fixed in the next alpha On Sun, 11 Aug 2013 22:35:02 +0200 Giovanni Rapagnani wrote: This is a multi-part message in MIME format. --01090103030704000604 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Hello, I am trying to build ast-ksh 2013-08-07 on debian testing 'jessie'. I downloaded the packages INIT.2013-08-07.tgz and ast-ksh.2013-08-07.tgz, verified checksum, unpacked INIT* and copied ast-ksh* into lib/package/tgz. Then I executed: $ bin/package read $ bin/package make debug=1 Unfortunately I am getting lots of error related to missing header files (see output of 'bin/package results' in attachment). bison and yacc version are: bison (GNU Bison) 2.7.12-4996 gcc version: gcc (Debian 4.7.3-4) 4.7.3 Could someone help me to build ast-ksh 2013-08-07 ? Best regards. -- Giovanni Rapagnani --01090103030704000604 Content-Type: text/plain; charset=UTF-8; name=make_errors.txt Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=make_errors.txt == /home/gio/test/sources/build/arch/linux.i386-64/lib/package/gen/make.out == package: make start at Sun Aug 11 22:23:04 CEST 2013 in /home/gio/test/sources/build/arch/linux.i386-64 conf: read /home/gio/test/sources/build/src/lib/libast/comp/conf.tab conf: probe for ABI_AIO_XFER_MAX limits.h value conf: probe for ABI_ASYNCHRONOUS_IO limits.h value conf: probe for ABI_ASYNC_IO limits.h value conf: probe for AUDIT limits.h value conf: probe for AVAIL_PROCESSORS limits.h value conf: probe for CHAR_BIT limits.h value conf: probe for CHAR_MAX limits.h value conf: probe for CHAR_MIN limits.h value conf: probe for CHILD_MAX limits.h value conf: probe for CLOCKS_PER_SEC limits.h value conf: probe for CPU_KEYBITS1 limits.h value conf: probe for CPU_VERSION limits.h value conf: probe for EXEC_INTERPRETER_LENGTH limits.h value conf: probe for HOSTID limits.h value conf: probe for HW_SERIAL limits.h value conf: probe for INT_MIN limits.h value conf: probe for IO_TYPE limits.h value conf: probe for IP_SECOPTS limits.h value conf: probe for _POSIX_JOB_CONTROL minmax value conf: probe for KERN_POINTERS limits.h value conf: probe for KERN_SIM limits.h value conf: probe for _POSIX_LFS_CFLAGS minmax value conf: probe for LLONG_MAX limits.h value conf: probe for LLONG_MIN limits.h value conf: probe for LONG_MAX limits.h value conf: probe for LONG_MIN limits.h value conf: probe for MB_LEN_MAX limits.h value conf: probe for MCAS_OFFSET limits.h value conf: probe for MIN_HOLE_SIZE limits.h value conf: probe for MMAP_FIXED_ALIGNMENT limits.h value conf: probe for MSEM_LOCKID limits.h value conf: probe for NL_MAX limits.h value conf: probe for NL_SPECMAX limits.h value conf: probe for NPROC_CONF limits.h value conf: probe for NPROC_ONLN limits.h value conf: probe for NSS_BUFLEN_GROUP limits.h value conf: probe for NSS_BUFLEN_PASSWD limits.h value conf: probe for NUM_PROCESSORS limits.h value conf: probe for OPEN_MAX limits.h value conf: probe for OPEN_MAX_CEIL limits.h value conf: probe for OSREL_MAJ limits.h value conf: probe for OSREL_MIN limits.h value conf: probe for OSREL_PATCH limits.h value conf: probe for _POSIX_PAGESIZE minmax value conf: probe for PID_MAX limits.h value conf: probe for _SVID_PID_MAX minmax value conf: probe for PROC_RSRC_MGR limits.h value conf: probe for PTHREAD_THREADS_MAX limits.h value conf: probe for PTRDIFF_MAX limits.h value conf: probe for PTRDIFF_MIN limits.h value conf: probe for RELEASE limits.h value conf: probe for _POSIX_SAVED_IDS minmax value conf: probe for SCHAR_MAX limits.h value conf: probe for SCHAR_MIN limits.h value conf: probe for SECURITY_CLASS limits.h value conf: probe for _AST_SF_BUFSIZE minmax value conf: probe for _AST_SH minmax value conf: probe for SHRT_MIN limits.h value conf: probe for SIG_ATOMIC_MAX limits.h value conf: probe for SIG_ATOMIC_MIN limits.h value conf: probe for SLVM_MAXNODES limits.h value conf: probe for SOFTPOWER limits.h value conf: probe for STD_BLK limits.h value conf: probe for SYSPID_MAX limits.h value conf: probe for TMP_MAX limits.h value conf: probe for UCHAR_MAX limits.h value conf: probe for UCHAR_MIN limits.h value conf: probe for UID_MAX limits.h value conf: probe for ULLONG_MAX limits.h value conf: probe for ULONG_MAX limits.h value conf: probe for USHRT_MAX limits.h value conf: probe for VERSION_88 limits.h value conf: probe for VERSION_88 limits.h value conf: probe for VERSION_90 limits.h value conf: probe for VERSION_90 limits.h value conf: probe for VERSION_93 limits.h value conf: probe for VERSION_93 limits.h value conf: probe for WCHAR_MAX limits.h value conf: probe for WCHAR_MIN limits.h value conf: probe for WINT_MAX limits.h value conf: probe for WINT_MIN limits.h value In file included from
Re: [ast-users] Unit of least precision (ULP)/spacing between floating point number problems
On Sun, 11 Aug 2013 22:57:40 +0200 Roland Mainz wrote: On Sun, Aug 11, 2013 at 6:15 PM, Cedric Blancher cedric.blanc...@gmail.com wrote: On 11 August 2013 10:43, Tina Harriott tina.harriott.m...@gmail.com wrote: On Wed, Jul 24, 2013 at 7:28 PM, Glenn Fowler g...@research.att.com wrote: On Wed, 24 Jul 2013 19:02:39 +0200 Tina Harriott wrote: Here's one of my little tough problems which I am unable to solve myself, even after looking at the source code of ast-ksh. The problem below requires a good understanding how floating point numbers are implemented in computers. I'm trying to prototype code and like to iterate over a small, linear area by using the C library function nextafter() to step forward the smallest possible distance between each step, and print the number of iterations needed to cover the distance between 4 and 4.0001: ksh -c 'float v ; integer iter ; for ((iter=0,v=4 ; v 4.0001 iter 1000; v=nextafter(v,4.0001))) ; do ((iter++));done;print $iter ' 2305843 The result 2305843 is correct. However, if I use typeset -E (or just typeset) to declare the variable v the loop runs forever, or in this case until it hits iter 1000 which I added as safeguard later: ksh -c 'typeset -E v ; integer iter ; for ((iter=0,v=4 ; v 4.0001 iter 1000; v=nextafter(v,4.0001))) ; do ((iter++));done;print $iter ' 1000 Can anyone explain this? float is an alias this shows the alias definition type float which is typeset -lE this documents -l typeset --?l so for your example typeset -E gave you a double v and typeset -lE would give you a long double v But why does nextafter() misbehave if I want to use a datatype smaller than long double? Accuracy is a good thing, but in this case we iterate too fine-grained, meaning the code should iterate over the smallest possible steps of a double, but not over the smallest possible steps of a long double. Does anyone have a good idea how to fix this in ksh? Grumpf... yes. Technically I feared that day may come when |nextafter()| and |nexttoward()| were added in ksh93... ;-/ The issue is more or less like this: Both |nextafter(f|l|)\(\)| and |nexttoward(f|l|)\(\)| step over the smallest possible quantity for the specific { |float|, |double|, |long double| }-datatype and therefore (for example) using |nextafterl()| (intended for |long double|) for a |float| doesn't work because it does so small steps that they cannot be represented in a |float| ... that causes the endless loop in Tina's example. The fix would be to remember the datatype (e.g. { |float|, |double|, |long double| }) for a given variable and pass that down to |arith_exec()| and call the specific version of |nextafter()| and |nexttoward()| for that datatype, for example: - variables declared via typeset -s -E/-X should use |nextafterf()|/|nexttowardf()| - variables declared via typeset-E/-X should use |nextafter()|/|nexttoward()| - variables declared via typeset -l -E/-X should use |nextafterl()|/|nexttowardl()| ... if the platforms libc/libm do not have a matching |nextafter(f|l|)\(\)|/|nexttoward(f|l|)\(\)| variant for the input datatype then the function not found-error should be thrown. this looks like the problem Note that we do _not_ have to change the logic for all math functions... AFAIK |nextafter()| and |nexttoward()| are the only exceptions which require special handling... Glenn: What do you think ? I'll meet with dgk shortly ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Errors while trying to build ksh.2013-08-07 on debian testing
On Mon, 12 Aug 2013 15:44:17 -0400 Glenn Fowler wrote: On Mon, 12 Aug 2013 21:35:53 +0200 Joshuah Hurst wrote: On Mon, Aug 12, 2013 at 9:32 PM, Glenn Fowler g...@research.att.com wrote: I'm working on this offline with Giovanni its a fault in the packaging Are you sure? /usr/include/sys/socket.h should be just there on a development system. If it isn't there then he lacks essential packages to compile anything more than helloworld.c look closer at the error messages its complaining about ../include/sys/socket.h and that's my lazy bad + a bug in iffe had I chosen to use it thanks btw its always good to have extra eyes on a problem ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Regression in ksh93v-: echo does no longer accept escape sequences
sorry for the delay on responding this will show the universe setting getconf UNIVERSE the universe is similar to gnu's POSIXLY_CORRECT and handle parts of std utilities that are undefined or implementation-dependent att and ucb are the expected values ast source for the most part checks for att for system-5 style ksh also check for ucb for berkley style this can change the setting for the current shell and children getconf UNIVERSE - ucb the setting is maintained in an _*_ env var so its not tamper proof the initial setting is done in src/lib/libast/port/astconf.c by looking at the PATH env var checking for system-5-ish and berkley-ish dirs from left to right the leftmost one wins if its att then echo groks neither \ nor -options, otherwise it does On Mon, 29 Jul 2013 11:23:17 +0200 =?KOI8-R?B?z8zYx8Egy9LZ1sHOz9fTy8HR?= wrote: I can confirm the problem on Suse Linux 12.3, but not on Solaris 11/i386. I think the magic switch in AST echo which selects the proper emulation mode (BSD, SYSV) is broken. Glenn, how again can I affect the UNIVERSE setting in AST? There was a magic, hidden, environment variable to do it. Olga On Mon, Jul 29, 2013 at 11:10 AM, PHILIPP, Axel, Dr. axel.phil...@mtu.de wrote: Version AJM 93u+ 2012-08-01 : cp003421 ksh -c 'echo \nTest\n' Test Version AIJM 93v- 2013-07-24 : cp003421 ksh93v -c 'echo \nTest\n' \nTest\n cp003421 cat /etc/SuSE-release SUSE Linux Enterprise Desktop 11 (x86_64) VERSION = 11 PATCHLEVEL = 2 Best Regards Axel Philipp -- MTU Aero Engines AG Geschaeftsfuehrung/Board of Management: Egon W. Behle, Vorsitzender/CEO; Dr. Rainer Martens, Michael Schreyögg, Dr. Stefan Weingartner, Reiner Winkler Vorsitzender des Aufsichtsrats/Chairman of the Supervisory Board: Klaus Eberhardt Sitz der Gesellschaft/Registered Office: Muenchen Handelsregister/Commercial Register: Muenchen HRB 157206 Diese E-Mail sowie ihre Anhaenge enthalten MTU-eigene vertrauliche oder rechtlich geschuetzte Informationen. Wenn Sie nicht der beabsichtigte Empfaenger sind, informieren Sie bitte den Absender und loeschen Sie diese E-Mail sowie die Anhaenge. Das unbefugte Speichern, Kopieren oder Weiterleiten ist nicht gestattet. This e-mail and any attached documents are proprietary to MTU, confidential or protected by law. If you are not the intended recipient, please advise the sender and delete this message and its attachments. Any unauthorised storing, copying or distribution is prohibited. ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users -- , __ , { \/`o;-Olga Kryzhanovska -;o`\/ } .'-/`-/ olga.kryzhanov...@gmail.com \-`\-'. `'-..-| / http://twitter.com/fleyta \ |-..-'` /\/\ Solaris/BSD//C/C++ programmer /\/\ `--` `--` ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Unit of least precision (ULP)/spacing between floating point number problems
On Wed, 24 Jul 2013 19:02:39 +0200 Tina Harriott wrote: Here's one of my little tough problems which I am unable to solve myself, even after looking at the source code of ast-ksh. The problem below requires a good understanding how floating point numbers are implemented in computers. I'm trying to prototype code and like to iterate over a small, linear area by using the C library function nextafter() to step forward the smallest possible distance between each step, and print the number of iterations needed to cover the distance between 4 and 4.0001: ksh -c 'float v ; integer iter ; for ((iter=0,v=4 ; v 4.0001 iter 1000; v=nextafter(v,4.0001))) ; do ((iter++));done;print $iter ' 2305843 The result 2305843 is correct. However, if I use typeset -E (or just typeset) to declare the variable v the loop runs forever, or in this case until it hits iter 1000 which I added as safeguard later: ksh -c 'typeset -E v ; integer iter ; for ((iter=0,v=4 ; v 4.0001 iter 1000; v=nextafter(v,4.0001))) ; do ((iter++));done;print $iter ' 1000 Can anyone explain this? float is an alias this shows the alias definition type float which is typeset -lE this documents -l typeset --?l so for your example typeset -E gave you a double v and typeset -lE would give you a long double v ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Demo code and contrib packages?
On Wed, 24 Jul 2013 18:52:14 +0200 Tina Harriott wrote: Does ast-ksh have extension packages like demo and contrib like bash and zsh do? in ksh user extension are called builtins and have an main(argc,argv,context) api its described in the nav bar at http://www.research.att.com/sw/download/ AST = ksh = builtins ksh has some builtins implemented directly in its source there and many other ast builtins implemented in src/lib/libcmd ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] ACL lists not preserved when copying files with ksh cp
On Wed, 24 Jul 2013 18:52:57 +0200 Tina Harriott wrote: On 23 July 2013 20:43, Glenn Fowler g...@research.att.com wrote: On Tue, 23 Jul 2013 19:16:43 +0200 Tina Harriott wrote: I hope this is the right place to report to. On Suse Linux nfs4 ACL lists are not preserved if I copy files with ksh's builtin cp command. To demonstrate: 1. touch aaa 2. nfs4_setfacl -a A::testuser@localdomain:RX aaa 3. nfs4_getfacl aaa D::OWNER@:x A::OWNER@:rwatTcCy A::1000:rxtcy - new ACL entry A::GROUP@:rtcy A::EVERYONE@:rtcy 4. ksh -c 'builtin cp; cp aaa aaa_copy' 5. nfs4_getfacl aaa_copy D::OWNER@:x A::OWNER@:rwatTcCy A::GROUP@:rxtcy A::EVERYONE@:rtcy The new ACL entry is missing in the copy. cp options -a and -p have no effect. Is this functionality missing or just broken. ACL support is IMO a mandatory enterprise system feature and needs to be supported. missing on the todo list How long will it take to implement it? acls have always been a portability sore point we avoided doing anything because no-one has presented an api that handles all our needs across varying architectures/implementations in particular we need an api that converts a string rep to binary converts a binary rep to string applies a binary acl to a file/fd retrives a binary acl from a file/fd I don't use acls because whenever they have been forced on me I manage to get painted into all sorts of corners that prevent work at inopportune times a thing I really don't like is they bleed into non-acl apis/commands in strange ways should ls/chown/chmod/mv/ln grok acls? what about other commands/apis that copy files and don't use cp(1) or pax(1)? how much stuff needs to be added around each open(O_CREAT) to make acls seamless? is there an acl equivalent to umask(1)/umask(2)? ast encompasses a lot of apis/commands the main reason behind doing it in the first place is uniform semantics across all of ast I don't see uniformity in acls at the moment but I can be convinced ... ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Demo code and contrib packages?
On Wed, 24 Jul 2013 22:15:01 +0200 Tina Harriott wrote: On 24 July 2013 19:32, Glenn Fowler g...@research.att.com wrote: On Wed, 24 Jul 2013 18:52:14 +0200 Tina Harriott wrote: Does ast-ksh have extension packages like demo and contrib like bash and zsh do? in ksh user extension are called builtins and have an main(argc,argv,context) api its described in the nav bar at http://www.research.att.com/sw/download/ AST = ksh = builtins ksh has some builtins implemented directly in its source there and many other ast builtins implemented in src/lib/libcmd I think you misunderstood the question. For example, bash provides demo and contrib packages. Demo contains demo scripts to show the various features of a shell, and contrib contains extra functionality written by people who are not in the core development team but are otherwise thought to be useful. yes I did misunderstand currently no demo or contrib packages ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] ACL lists not preserved when copying files with ksh cp
On Tue, 23 Jul 2013 19:16:43 +0200 Tina Harriott wrote: I hope this is the right place to report to. On Suse Linux nfs4 ACL lists are not preserved if I copy files with ksh's builtin cp command. To demonstrate: 1. touch aaa 2. nfs4_setfacl -a A::testuser@localdomain:RX aaa 3. nfs4_getfacl aaa D::OWNER@:x A::OWNER@:rwatTcCy A::1000:rxtcy - new ACL entry A::GROUP@:rtcy A::EVERYONE@:rtcy 4. ksh -c 'builtin cp; cp aaa aaa_copy' 5. nfs4_getfacl aaa_copy D::OWNER@:x A::OWNER@:rwatTcCy A::GROUP@:rxtcy A::EVERYONE@:rtcy The new ACL entry is missing in the copy. cp options -a and -p have no effect. Is this functionality missing or just broken. ACL support is IMO a mandatory enterprise system feature and needs to be supported. missing on the todo list ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] No libast in INIT package?
packages come in 2 flavors: source and binary in general it is best to build from source to rule out os/compiler diffs between the ast build farm and other systems all ast source packages require the INIT package the usual procedure is to download INIT and the ast-* package you want mkdir -p bin lib/package/tgz # download INIT*.tgz ast-*.tgz into lib/package/tgz # download package into bin/package chmod +x bin/package bin/package read bin/package make if you take these *exact* steps and get failure then you can email me or the list with the contents of the file named by the path printed on the standard output by bin/package results path attached or included in the message background details -- ignore if all you want to do is build and use INIT contains scripts and commands to bootstrap build the other packages from source ast uses ast nmake as its build engine any packages you build with out nmake in scope are automatically built by the bootstrap commands/scripts the bootstrap build is primariliy for build stuff required by nmake if you want to develop (edit/patch/recompile/test) ast source it would be best to download an ast package that contains nmake (ast-base ast-open) because the bootstrap build may not behave nicely in a development cycle On Mon, 22 Jul 2013 15:40:36 + Reimer, George wrote: INIT contains only the build system and a way to download the individual source packages. ast-open is AFAIK the recommended way to obtain all stuff, including libast. I cannot test and verify the following as extensively as I'd like, because the server on which I can run the INIT package sits behind corporate firewalls and cannot download packages. But I believe that your statement above is false. INIT does *not* contain a functioning build system because INIT, by itself, cannot even execute components like ${INSTALLROOT}/bin/mkdir which is called by ${PACKAGEROOT}/bin/package setup: root@lipossrp01ga: # echo $PACKAGEROOT /data/archive/asttest /data/archive/asttest root@lipossrp01ga: # echo $INSTALLROOT /data/archive/asttest/arch/ibm.risc /data/archive/asttest root@lipossrp01ga: # bin/package setup Could not load program mkdir: Dependent module libast.so could not be loaded. Could not load module libast.so. System error: No such file or directory /data/archive/asttest root@lipossrp01ga: # Even though AST versions of standard Unix commands like 'mkdir' are also part of the ast-open package, these commands are included as part of the INIT package with the intent (I imagine) of being able to execute them in the course of using the INIT package to retrieve and build other AST packages. Perhaps it is not necessary to run bin/package setup in order to do that, and perhaps no other use is made of $INSTALLROOT/bin/mkdir? (Nor of 'ratz', 'mamake', 'proto' and 'release', all in $INSTALLROOT/bin and which also fail with the same error?) Again, firewalls make it impractical for me to try all this out. But it seems to me that if you are going to include such binaries as part of the INIT package they should be able to execute, either by including libast.so ( possibly other dependencies?) or supplying statically-linked versions. Yours, George Reimer -Original Message- From: Irek Szczesniak [mailto:iszczesn...@gmail.com] Sent: Friday, July 19, 2013 3:20 PM To: Reimer, George Cc: ast-users@lists.research.att.com Subject: Re: [ast-users] No libast in INIT package? On Fri, Jul 12, 2013 at 5:03 PM, Reimer, George george.rei...@fisglobal.com wrote: Hello, I'm new to the AST software archive and perhaps I am using things improperly, but as best as I understand the instructions at https://urldefense.proofpoint.com/v1/url?u=http://www2.research.att.co m/sw/download/k=%2FbkpAUdJWZuiTILCq%2FFnQg%3D%3D%0Ar=W%2F5gbnNveYGG1 XCX8vQsiY6orNnjdBtrtn%2BMb0MZjfw%3D%0Am=m7vyR2p8akeCzEfg2jZunNTQkLwg3 mbCAIr7NRhnMUE%3D%0As=0ee56e80e9ec3a07f15d8040ddfcad1f6ecd925567b3cfa941e39cbd4109a9bd , I should be able to use the package script in the INIT package to retrieve and install other AST packages, though it's also possible to retrieve and install at least some of them manually. INIT contains only the build system and a way to download the individual source packages. ast-open is AFAIK the recommended way to obtain all stuff, including libast. Irek _ The information contained in this message is proprietary and/or confidential. If you are not the intended recipient, please: (i) delete the message and all copies; (ii) do not disclose, distribute or use the message in any manner; and (iii) notify the sender immediately. In addition, please be aware that any message addressed to our domain is subject to archiving and review by persons other than the intended recipient. Thank
Re: [ast-users] No libast in INIT package?
On Mon, 22 Jul 2013 11:59:10 -0400 Glenn Fowler wrote: packages come in 2 flavors: source and binary in general it is best to build from source to rule out os/compiler diffs between the ast build farm and other systems all ast source packages require the INIT package the usual procedure is to download INIT and the ast-* package you want one important clarification here (its documented on the download site too) source packages are binary independent this means that the same source package should build on all systems source package names match *.-MM-DD.tgz binary package names match *.-MM-DD.*.tgz if you are building from source make sure to download source packages finally, to build from source you need working cc/ar/ld ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] [ast-developers] Matching accented é with [=e=] using AST tr
On Mon, 22 Jul 2013 12:02:15 +0200 Cedric Blancher wrote: On 18 July 2013 16:14, Glenn Fowler g...@research.att.com wrote: this is fixed for the next alpha later today Thanks... ast-ksh.20130719 fixes this. finally -- right? I'm not pleased with the speed of the fix but absent an api for equivalence class testing (besides regex) that's the only way I could see to do it at the moment Uh, and thanks for bumping the tr --version number so we can add checks :) sorry that 2012-07-19 wasn't announced it had a bunch of roland-related patches and I wanted him to have first crack there are a few problems with it that will be fixed in todays alpha ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] No libast in INIT package?
packages marked standalone have their own package-specific instructions this is described in the paragraph immediately before the package listing On Mon, 22 Jul 2013 16:51:14 + Reimer, George wrote: I will try this. In fact I have started to try this, I manually downloaded the sfio package. But I do also want to emphasize the significant differences between the instructions you've provided me below with those available to new/prospective users of the AST software at the AST download page. As it stands now, right at: http://www2.research.att.com/~gsf/download/ I read: + The INIT package is required by all but the standalone and self extracting archive packages. It contains the package(1) script that controls package installation and building. The packages and instructions are probably very different than what you may have used before; the initial setup can be tedious. However, once that's done, downloading, updating, reading, building, testing and installing can all be scripted from the command line: package setup # instead of CLICK to download -- you must still use the license agreement name and password package make # if you have source packages package test package install DIRECTORY # if you want a separate installation See the download menu on the left for more information. Follow the package name links below to view the package descriptions, then use package setup (installing source or installing binaries) to download. package setup downloads the package closure + And besides that, when I decided to use the sfio package for testing I clicked on the download link for it at the above URL and read: sfio Software Download Package List The table below lists the releases and versions for the sfio package. Use the package setup command to download the latest source or binary package by name, or click the RELEASE entries below to download specific packages. The latest releases are marked with *. The concept that I am trying to get across here is that I and other aspiring AST software users who try and follow these instructions will retrieve the INIT package, install it and then follow the instructions above to build other interesting-looking packages and we will start by running bin/package setup ... and we will get an error when mkdir fails to load because it cannot find libast.so. Indeed, your instructions below to manually run the system mkdir are necessary *because* it does not work when you try and run bin/package setup. It fails. It fails because it tries to run the included AST version of 'mkdir' which cannot run because it cannot find 'libast.so'. Which is not included in the INIT package. Yours, George Reimer -Original Message- From: Glenn Fowler [mailto:g...@research.att.com] Sent: Monday, July 22, 2013 11:59 AM To: ast-users@lists.research.att.com; Reimer, George Subject: Re: [ast-users] No libast in INIT package? packages come in 2 flavors: source and binary in general it is best to build from source to rule out os/compiler diffs between the ast build farm and other systems all ast source packages require the INIT package the usual procedure is to download INIT and the ast-* package you want mkdir -p bin lib/package/tgz # download INIT*.tgz ast-*.tgz into lib/package/tgz # download package into bin/package chmod +x bin/package bin/package read bin/package make if you take these *exact* steps and get failure then you can email me or the list with the contents of the file named by the path printed on the standard output by bin/package results path attached or included in the message background details -- ignore if all you want to do is build and use INIT contains scripts and commands to bootstrap build the other packages from source ast uses ast nmake as its build engine any packages you build with out nmake in scope are automatically built by the bootstrap commands/scripts the bootstrap build is primariliy for build stuff required by nmake if you want to develop (edit/patch/recompile/test) ast source it would be best to download an ast package that contains nmake (ast-base ast-open) because the bootstrap build may not behave nicely in a development cycle On Mon, 22 Jul 2013 15:40:36 + Reimer, George wrote: INIT contains only the build system and a way to download the individual source packages. ast-open is AFAIK the recommended way to obtain all stuff, including libast. I cannot test and verify the following as extensively as I'd like, because the server on which I can run the INIT package sits behind corporate firewalls and cannot download packages. But I believe that your statement above is false. INIT does *not* contain a functioning build system
Re: [ast-users] ksh93 openat error
this patch + fixes for features/fcntl.c rolled in for the next alpha On Fri, 19 Jul 2013 20:49:54 +0200 Irek Szczesniak wrote: --001a11c28aa2dd24c204e1e1caeb Content-Type: text/plain; charset=ISO-8859-1 On Fri, Jul 19, 2013 at 1:54 PM, Philippe Bergheaud philippe.berghe...@fr.ibm.com wrote: On the following platform: Linux version 2.6.32-5-686 (Debian 2.6.32-46) gcc version 4.3.5 (Debian 4.3.5-4) ksh93 compiles, but fails to initialize: $ strace ~/ast-base.2013-06-28/arch/linux.i386/bin/ksh [...] openat(AT_FDCWD, ., O_RDONLY|O_NONBLOCK|O_DIRECT) = -1 EINVAL (Invalid argument) write(2, ksh: Can't obtain directory fd. ..., 51ksh: Can't obtain directory fd. [Invalid argument]) = 51 --- SIGSEGV (Segmentation fault) @ 0 (0) --- +++ killed by SIGSEGV +++ Memory fault the cause for EINVAL is the flag displayed by strace as O_DIRECT it happens that O_DIRECT and O_SEARCH collide: both of them are 04 nullifying O_SEARCH with #define O_SEARCH 0 is a possible workaround (but probably not a fix) Roland Mainz send a large patch cluster to ast-develop...@research.att.com to address these problems. I've attached the patches for you. Irek --001a11c28aa2dd24c204e1e1caeb Content-Type: text/plain; charset=US-ASCII; name=astksh20130628_solaris_fixes005.diff.txt Content-Disposition: attachment; filename=astksh20130628_solaris_fixes005.diff.txt Content-Transfer-Encoding: base64 X-Attachment-Id: f_hjbqrri40 ZGlmZiAtciAtdSBidWlsZF9pMzg2XzY0Yml0X2RlYnVnL3NyYy9jbWQva3NoOTMvYmx0aW5zL2Nk X3B3ZC5jIGJ1aWxkX2kzODZfNjRiaXRfZGVidWdfcGF0Y2hlZC9zcmMvY21kL2tzaDkzL2JsdGlu cy9jZF9wd2QuYwotLS0gc3JjL2NtZC9rc2g5My9ibHRpbnMvY2RfcHdkLmMJMjAxMy0wNi0yNiAy MjoxMjoyOC4wMDAwMDAwMDAgKzAyMDAKKysrIHNyYy9jbWQva3NoOTMvYmx0aW5zL2NkX3B3ZC5j CTIwMTMtMDctMDggMTQ6MzM6MjYuMDM2ODA3NTc5ICswMjAwCkBAIC01NSw5ICs1NSwxNyBAQAog ICovCiBpbnQgc2hfZGlyb3BlbmF0KFNoZWxsX3QgKnNocCwgaW50IGRpciwgY29uc3QgY2hhciAq cGF0aCwgYm9vbCB4YXR0cikKIHsKKyNpZmRlZiBPX0RJUkVDVE9SWQorI2RlZmluZSBPX2RpcmVj dG9yeSAoT19ESVJFQ1RPUlkpCisjZWxzZQorI2RlZmluZSBPX2RpcmVjdG9yeSAoMCkKKyNlbmRp ZgorCiAJaW50IGZkLHNoZmQ7Ci0JaW50IHNhdmVkZXJybm89ZXJybm87CisJaW50IHNhdmVkZXJy bm87CisjaWZuZGVmIE9fRElSRUNUT1JZCiAJc3RydWN0IHN0YXQgZnM7CisjZW5kaWYKICNpZm5k ZWYgT19YQVRUUgogCU5PVF9VU0VEKHhhdHRyKTsKICNlbmRpZgpAQCAtNzAsMjQgKzc4LDU5IEBA CiAJCWlmKChhcGZkID0gb3BlbmF0KGRpciwgcGF0aCwgT19SRE9OTFl8T19OT05CTE9DS3xPX2Ns b2V4ZWMpKT49MCkKIAkJewogCQkJZmQgPSBvcGVuYXQoYXBmZCwgZV9kb3QsIE9fWEFUVFJ8T19j bG9leGVjKTsKKwkJCXNhdmVkZXJybm89ZXJybm87CiAJCQljbG9zZShhcGZkKTsKKwkJCWVycm5v PXNhdmVkZXJybm87CisJCX0KKwkJZWxzZQorCQl7CisJCQlyZXR1cm4gLTE7CiAJCX0KIAl9CiAJ ZWxzZQogI2VuZGlmCi0JCWZkID0gb3BlbmF0KGRpciwgcGF0aCwgT19TRUFSQ0h8T19OT05CTE9D S3xPX2Nsb2V4ZWMpOworCXsKKwkJLyoKKwkJICogT3BlbiBkaXJlY3RvcnkuIEZpcnN0IHdlIHRy eSB3aXRob3V0IHxPX1NFQVJDSHwgYW5kCisJCSAqIGlmIHRoaXMgZmFpbHMgd2l0aCBFQUNDRVNT IHdlIHRyeSB3aXRoIHxPX1NFQVJDSHwKKwkJICogYWdhaW4uCisJCSAqIFRoaXMgaXMgcmVxdWly ZWQgLi4uCisJCSAqIC0gLi4uIGJlY2F1c2Ugc29tZSBwbGF0Zm9ybXMgbWF5IHJlcXVpcmUgdGhh dCBpdCBjYW4KKwkJICogb25seSBiZSB1c2VkIGZvciBkaXJlY3RvcmllcyB3aGlsZSBzb21lIGZp bGVzeXN0ZW1zCisJCSAqIChlLmcuIFJlaXNlcjQgb3IgSFNNIHN5c3RlbXMpIGFsbG93IGEgfGZj aGRpcigpfCBpbnRvCisJCSAqIGZpbGVzLCB0b28pCisJCSAqIC0gLi4uIHRvIHByZXNlcnZlIHRo ZSBzZW1hbnRpY3Mgb2YgImNkIiwgZS5nLgorCQkgKiBvdGhlcndpc2UgImNkIiB3b3VsZCByZXR1 cm4gW05vIGFjY2Vzc10gaW5zdGVhZCBvZgorCQkgKiBbTm90IGEgZGlyZWN0b3J5XSBmb3IgZmls ZXMgb24gZmlsZXN5c3RlbXMgd2hpY2ggZG8KKwkJICogbm90IGFsbG93IGEgImNkIiBpbnRvIGZp bGVzLgorCQkgKiAtIC4uLiB0byBhbGxvdyB0aGF0IGEKKwkJICogJCByZWRpcmVjdCB7bn08L2V0 YyA7IGNkIC9kZXYvZmQvJG4gIyB3b3JrcyBvbiBtb3N0CisJCSAqIHBsYXRmb3Jtcy4KKwkJICov CisKKwkJZmQgPSBvcGVuYXQoZGlyLCBwYXRoLCBPX2RpcmVjdG9yeXxPX05PTkJMT0NLfE9fY2xv ZXhlYyk7CisJCWlmICgoZmQgPCAwKSAmJiAoZXJybm8gPT0gRUFDQ0VTKSkKKwkJeworCQkJZmQg PSBvcGVuYXQoZGlyLCBwYXRoLCBPX1NFQVJDSHxPX2RpcmVjdG9yeXxPX05PTkJMT0NLfE9fY2xv ZXhlYyk7CisJCX0KKwl9CiAKIAlpZihmZCA8IDApCisJewogCQlyZXR1cm4gZmQ7CisJfQorCisj aWZuZGVmIE9fRElSRUNUT1JZCiAJaWYgKCFmc3RhdChmZCwgJmZzKSAmJiAhU19JU0RJUihmcy5z dF9tb2RlKSkKIAl7CiAJCWNsb3NlKGZkKTsKIAkJZXJybm8gPSBFTk9URElSOwogCQlyZXR1cm4g LTE7CiAJfQorI2VuZGlmCiAKIAkvKiBNb3ZlIGZkIHRvIGEgbnVtYmVyID4gMTAgYW5kIHJlZ2lz dGVyIHRoZSBmZCBudW1iZXIgd2l0aCB0aGUgc2hlbGwgKi8KLQlzaGZkID0gZmNudGwoZmQsIEZf ZHVwZmRfY2xvZXhlYywgMTApOworCXNoZmQgPSBzaF9mY250bChmZCwgRl9kdXBmZF9jbG9leGVj LCAxMCk7CiAJc2F2ZWRlcnJubz1lcnJubzsKIAljbG9zZShmZCk7CiAJZXJybm89c2F2ZWRlcnJu bzsKQEAgLTE0OSw3ICsxOTIsNyBAQAogCXJlZ2lzdGVyIFNoZWxsX3QgKnNocCA9IGNvbnRleHQt PnNocDsKIAlpbnQgc2F2ZXJybm89MDsKIAlpbnQgcnZhbDsKLQlib29sIGZsYWc9ZmFsc2UseGF0 dHI9ZmFsc2UsIHVzZV9kZXZmZD1mYWxzZTsKKwlib29sIHBmbGFnPWZhbHNlLHhhdHRyPWZhbHNl LCB1c2VfZGV2ZmQ9ZmFsc2U7CiAJY2hhciAqb2xkcHdkLCpjcDsKIAlzdHJ1Y3Qgc3RhdCBzdGF0 YjsKIAlpbnQgZGlyZmQgPSBzaHAtPnB3ZGZkOwpAQCAtMTY0LDEwICsyMDcsMTAgQEAKIAkJCWRp cmZkID0gb3B0X2luZm8ubnVtOwogCQkJYnJlYWs7CiAJCWNhc2UgJ0wnOgotCQkJZmxhZyA9IGZh bHNlOworCQkJcGZsYWcgPSBmYWxzZTsKIAkJCWJyZWFrOwogCQljYXNlICdQJzoKLQkJCWZsYWcg
Re: [ast-users] [ast-developers] Matching accented é with [=e=] using AST tr
this is fixed for the next alpha later today On Thu, 14 Mar 2013 14:19:58 +0100 Cedric Blancher wrote: How do I match accented e (i.e. é) using an equivalence class in AST tr? Doing that in sed is easy: ~/bin/sed -r s/[[=e=]]/X/g 8é8 ; printf \n 8X8 But in tr I am not able to get it working: ksh -c 'builtin tr ; tr -Cd [=e=] 1e2é3 ; print' e AFAIK this should print eé. I used: version tr (ATT Research) 2012-11-12 version sed (ATT Research) 2012-03-28 Ced -- Cedric Blancher cedric.blanc...@googlemail.com Institute Pasteur ___ ast-developers mailing list ast-develop...@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-developers ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Looking for MVS ssh access to test latest ast-ksh.2013-06-28 ...
ibm used to offer time-limited access to z/os accounts On Wed, 10 Jul 2013 03:47:19 +0200 Roland Mainz wrote: Hi! Does anyone here have an MVS system where I could log-in via ssh to buildtest the latest ksh93 version (ast-ksh.2013-06-28) and check for any bugs ? Bye, Roland -- __ . . __ (o.\ \/ /.o) roland.ma...@nrubsig.org \__\/\/__/ MPEG specialist, CJAVASunUnix programmer /O /==\ O\ TEL +49 641 3992797 (;O/ \/ \O;) ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] [ast-developers] SFIO sfgetd() support for C99 printf(%a) format?
On Sat, 13 Jul 2013 10:48:55 +0200 Lionel Cons wrote: Does SFIO's sfgetd() have support for C99 printf(%a) format? both sfputu()/sfgetu() for integers and sfputd()/sfgetd() for floating point use sfio-specific encodings sfscanf() handles %a format via ast { strtof strtod strtold strntof strntod strntold } all implemented by the libast private header src/lib/libast/sfio/sfstrtof.h ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] [ast-developers] SFIO sfgetd() support for C99 printf(%a) format?
On Tue, 16 Jul 2013 16:58:34 +0200 Lionel Cons wrote: On 16 July 2013 16:53, Glenn Fowler g...@research.att.com wrote: On Sat, 13 Jul 2013 10:48:55 +0200 Lionel Cons wrote: Does SFIO's sfgetd() have support for C99 printf(%a) format? both sfputu()/sfgetu() for integers and sfputd()/sfgetd() for floating point use sfio-specific encodings Unfortunately that encoding predates IEEE754, i.e. there is no nan/-nan, working infinite, subnormal numbers or negative zero. That allows only for very very limited use. good point the floating point sign is encoded as a byte 0: positive 1: negative we could retain these for backwards compatibility and use a leading 2 to mark IEEE754 binary interchange format, at the start just for numbers not encodable by the old sfio format do you have a www ref for the exact description of the binary interchange format? ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] bug: kill(1) in ksh93 doesn't check for kill(3p) and killpg(3p) errors
On Mon, 17 Jun 2013 16:00:21 +0200 Cedric Blancher wrote: On 17 June 2013 15:50, Glenn Fowler g...@research.att.com wrote: On Mon, 17 Jun 2013 14:59:38 +0200 Cedric Blancher wrote: I found a bug in ksh93's kill(1) implementation: It doesn't check for errors when kill(3p) and killpg(3p) are used. However posix says for kill(3p) and killpg(3p): RETURN VALUE Upon successful completion, 0 shall be returned. Otherwise, -1 shall be returned and errno set to indicate the error. ERRORS The kill() function shall fail if: EINVAL The value of the sig argument is an invalid or unsupported signal number. EPERM The process does not have permission to send the signal to any receiving process. ESRCH No process or process group can be found corresponding to that specified by pid. So the two syscalls can fail, and ksh93 kill(1) should check this. can you point to the specific lines in the code that led you to this conclusion function job_kill(): 1214 } 1215 else 1216 r = kill(pid,sig); 1217 if(r=0 !stopsig) 1218 { 1219 if(pw-p_flagP_STOPPED) 1220 pw-p_flag = ~(P_STOPPED|P_SIGNALLED); 1221 if(sig) 1222 kill(pid,SIGCONT); 1223 } 1224 } 1225 else 1226 { 1227 if(qflag) 1228 goto no_sigqueue; 1229 if((r = killpg(-pid,sig))=0 !stopsig) 1230 { 1231 job_unstop(job_bypid(pw-p_pid)); 1232 if(sig) None of them checks for errors from kill(pid,sig) or killpg() to print an error. none in the section of code you mention (job_kill() which is called by b_kill() via job_walk()) translates to 3 job control specific instances: kill(pid,SIGCONT); killpg(-pid,SIGCONT); kill(pw-p_pid,SIGCONT); and if you look close they are encased in logic that already has done a kill/killpg that does check the other 4 instances are checked: r = kill(pid,sig); if((r = killpg(-pid,sig))=0 !stopsig) r = kill(pid,sig); r = killpg(-pid,sig); r = killpg(pid,sig); I noticed this when David reported that on Linux 3% of the USR1 and USR2 signals get lost, which indicates a kernel bug. This is the only place I could find which may be a culprit for 'silently' loosing signals. If this isn't the culprit then Linux has a bug. Ced -- Cedric Blancher cedric.blanc...@googlemail.com Institute Pasteur ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Directory walker API which takes directory fd as input?
no -- that's a good point we would probably add fts_openat(path, fd, flags, comparf) and extend the *at() semantics to allow path==0 to use fd already in hand On Mon, 17 Jun 2013 16:22:36 +0200 Lionel Cons wrote: Does libast have a directory walker API which can take a directory fd (for openat()) as input to define the starting point? Lionel ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] [ast-developers] VMALLOC_OPTIONS='abort' no longer calling abort() on heap corruption?
On Thu, 6 Jun 2013 12:52:14 +0200 Lionel Cons wrote: A colleague is asking: Is VMALLOC_OPTIONS='abort' no longer calling abort() on heap corruption when libast::free() is passed a pointer which does not originate from libast::malloc()/calloc()? libast version is ast-ksh.2013-05-24 thanks -- this will be fixed in the next alpha ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] Thank you for the grep builtin!
great - any soft numbers on improvement? On Thu, 6 Jun 2013 11:23:27 +0200 Lionel Cons wrote: I'd like say Thank you for adding the grep builtin to ksh93. Today we figured out that this gave a major performance boost and helped a lot working around shoddy or broken grep implementations. Thank you! Lionel ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users
Re: [ast-users] segfault in child shell--debugging steps?
On Thu, 3 Jan 2013 00:25:34 -0500 Aaron Davies wrote: i've been seeing a fairly consistent segfault in ksh93u when running as a background child process of another ksh93u instance. i haven't had the time to do a full investigation yet, but it seems to be crashing inside the memory manager. i'm planning on running the script in question under valgrind to verify exactly what's going wrong, and then submit the logs here. is this a reasonable approach? anything else i should do? that would be a good approach if the script is small enough you could post that too ___ ast-users mailing list ast-users@lists.research.att.com http://lists.research.att.com/mailman/listinfo/ast-users