Hi! ----
Just curious: Does AST pax support POSIX.1-2001 compliant archives ? Joerg Schilling said about this: > Ast pax seems to lack support for extracting POSIX.1-2001 compliant archives. > This limits ast pax to files with a maximum size of 8 GB - 1 byte. It seems > that ast pax just throws away the POSIX.1-2001 extended headers. ---- Bye, Roland ---------- Forwarded message ---------- From: Joerg Schilling <[email protected]> Date: Mon, Aug 2, 2010 at 4:16 PM Subject: Re: [Illumos] Build instructions To: [email protected] Cc: [email protected] ????? ???????????? <[email protected]> wrote: > Joerg, remind that the default build of AST builds with the optimiser > disabled, if you compare performance do it equally, same compilers > with same optimiser flags. Hi Olga, after some research (the project uses an unusual build system that lacks a documentation for beginners at top level) I finally have been able to compile astpax with the same optimization level that is used by default with star. As expected, there is no significant performance win compared to the default compilation I did yesterday. The general tendency for performance tests (after running about a dozen different tests) is: star is the fastest gnu tar typically needs twice as much user CPU time than star but is not significantly slower in wallclock time in case the archive is not on a real tape. Sun pax typically needs 1.5 .. 2x as much user CPU time than gnu tar and is already noticable slower in wallclock time. Ast pax is the slowest. Ast pax typically needs 3x..5x the amount of user CPU time than star and typically needs 1.5x the wallclock time spend by star. Ast pax out of cousiosity needs more than 100x the amount of user CPU time when creating verbose listings from a POSIX.1-2001 extended tar archive with 5332 files (this archive type is sometimes called "pax"). In the latter cases, the wallclock time spend for listing the archive by ast pax is aprox. 19x the wallclock time spend by star. star -tv < /tmp/0a | head -10 Release star 1.5.1 (i386-pc-solaris2.11) Archtype exustar Dumpdate 1280745273.165657 (Mon Aug 2 12:34:33 2010) Volno 1 Blocksize 20 records 0 drwxr-xr-x 6 joerg/bs Aug 2 11:34 2010 ./ 1144 -rw-r--r-- 1 joerg/bs Oct 14 17:10 2008 README 0 drwxr-xr-x 2 joerg/bs Aug 2 11:38 2010 bin/ 1797 -rwxr-xr-x 1 joerg/bs Aug 2 11:38 2010 bin/ignore 6393 -rwxr-xr-x 1 joerg/bs Aug 2 11:38 2010 bin/mamprobe star -tv < /tmp/0a | wc star: 34418 blocks + 1536 bytes (total of 352441856 bytes = 344181.50k). 5337 48120 436071 star -tv < /tmp/0a > /dev/null star: 34418 blocks + 1536 bytes (total of 352441856 bytes = 344181.50k). 0.471r 0.060u 0.450s 127% 0M 0+0k 0st 0+0io 0pf+0w /usr/bin/pax -v < /tmp/0a > /dev/null USTAR format archive extended 0.556r 0.130u 0.280s 82% 0M 0+0k 0st 0+0io 0pf+0w arch/sol11.i386/src/cmd/pax/pax -v < /tmp/0a > /dev/null /dev/stdin in pax format 5332 files, 688363 blocks 8.900r 8.770u 0.040s 98% 0M 0+0k 0st 0+0io 0pf+0w This is 18.89x the wallclock time star needs and 146.16x the user CPU time star needs. Star typically needs a bit more system CPU than other archivers as it forks and creates a shared memory segment that is used to let data flow between the archiver process and the I/O process. This was introduced in 1988 (with the availability of shared memory on SunOS-4.0) in order to keep QIC-24 tapes streaming and still helps to gain performance and to reduce the wear-out on tape media. Ast pax seems to lack support for extracting POSIX.1-2001 compliant archives. This limits ast pax to files with a maximum size of 8 GB - 1 byte. It seems that ast pax just throws away the POSIX.1-2001 extended headers. Here are the tests (the times meter mainly bzip2 performance and thus are not relevant): star -tv < testscripts/pax-big-10g.tar.bz2 star: WARNING: Archive is 'bzip2' compressed, trying to use the -bz option. 10737418240 -rw------- user/group Jun 15 23:18 2002 10g 0 -rw-r--r-- user/group Jun 15 16:53 2002 file star: 1048576 blocks + 3072 bytes (total of 10737421312 bytes = 10485763.00k). 1:44.282r 85.790u 33.440s 114% 0M 0+0k 0st 0+0io 0pf+0w gtar -tvjf - < testscripts/pax-big-10g.tar.bz2 -rw------- user/group 10737418240 2002-06-15 23:18 10g -rw-r--r-- user/group 0 2002-06-15 16:53 file 1:45.555r 86.700u 25.570s 106% 0M 0+0k 0st 0+0io 0pf+0w bzip2 -d < testscripts/pax-big-10g.tar.bz2 | /usr/bin/pax -v USTAR format archive extended -rw------- 0 486 1060 10737418240 Jun 15 2002 10g -rw-r--r-- 0 486 1060 0 Jun 15 2002 file 1:42.928r 1.400u 11.760s 12% 0M 0+0k 0st 0+0io 0pf+0w 1:42.928r 83.940u 17.970s 99% 0M 0+0k 0st 0+0io 0pf+0w arch/sol11.i386/src/cmd/pax/pax -v < testscripts/pax-big-10g.tar.bz2 /dev/stdin in bzip pax format -rw------- 1 user group 0 Jun 15 2002 10g pax: warning: /dev/stdin: junk data after volume 1 1 file, 20971523 blocks 1:44.258r 110.360u 29.490s 134% 0M 0+0k 0st 0+0io 0pf+0w As you see, ast pax seems to be the only archiver that does not yet fully support the POSIX.1-2001 standard. but this is what users expect from pax on a POSIX.1-2001 compliant OS (well, it did at least correctly detect the archive type and it did not complain about the POSIX.1-2001 extended headers). I believe that astpax offers the interesting feature to support 25 different archive types (including MS CAB IIRC) and thus it would be nice to get astpax included. I am still not convinced that astpax is the implementation we should use to replace the closed source /usr/bin/pax. Jörg -- EMail:[email protected] (home) Jörg Schilling D-13353 Berlin [email protected] (uni) [email protected] (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily _______________________________________________ Illumos mailing list [email protected] http://mexico.purplecow.org/m/listinfo/illumos -- __ . . __ (o.\ \/ /.o) [email protected] \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer /O /==\ O\ TEL +49 641 3992797 (;O/ \/ \O;) _______________________________________________ ast-developers mailing list [email protected] https://mailman.research.att.com/mailman/listinfo/ast-developers
