I'll check the standard again given a corpus of test archive data I can also look into timing diffs
On Fri, 6 Aug 2010 00:51:49 +0200 Roland Mainz wrote: > Hi! > ---- > Just curious: > Does AST pax support POSIX.1-2001 compliant archives ? > Joerg Schilling said about this: > > Ast pax seems to lack support for extracting POSIX.1-2001 compliant > > archives. > > This limits ast pax to files with a maximum size of 8 GB - 1 byte. It seems > > that ast pax just throws away the POSIX.1-2001 extended headers. > ---- > Bye, > Roland > ---------- Forwarded message ---------- > From: Joerg Schilling <[email protected]> > Date: Mon, Aug 2, 2010 at 4:16 PM > Subject: Re: [Illumos] Build instructions > To: [email protected] > Cc: [email protected] > ????? ???????????? <[email protected]> wrote: > > Joerg, remind that the default build of AST builds with the optimiser > > disabled, if you compare performance do it equally, same compilers > > with same optimiser flags. > Hi Olga, after some research (the project uses an unusual build system that > lacks a documentation for beginners at top level) I finally have been able to > compile astpax with the same optimization level that is used by default with > star. As expected, there is no significant performance win compared to the > default compilation I did yesterday. > The general tendency for performance tests (after running about a dozen > different tests) is: > star is the fastest > gnu tar typically needs twice as much user CPU time than star but is not > significantly slower in wallclock time in case the archive is not on a real > tape. > Sun pax typically needs 1.5 .. 2x as much user CPU time than gnu tar and is > already noticable slower in wallclock time. > Ast pax is the slowest. Ast pax typically needs 3x..5x the amount of user CPU > time than star and typically needs 1.5x the wallclock time spend by star. > Ast pax out of cousiosity needs more than 100x the amount of user CPU time > when creating verbose listings from a POSIX.1-2001 extended tar archive with > 5332 files (this archive type is sometimes called "pax"). In the latter cases, > the wallclock time spend for listing the archive by ast pax is aprox. 19x the > wallclock time spend by star. > star -tv < /tmp/0a | head -10 > Release star 1.5.1 (i386-pc-solaris2.11) > Archtype exustar > Dumpdate 1280745273.165657 (Mon Aug 2 12:34:33 2010) > Volno 1 > Blocksize 20 records > 0 drwxr-xr-x 6 joerg/bs Aug 2 11:34 2010 ./ > 1144 -rw-r--r-- 1 joerg/bs Oct 14 17:10 2008 README > 0 drwxr-xr-x 2 joerg/bs Aug 2 11:38 2010 bin/ > 1797 -rwxr-xr-x 1 joerg/bs Aug 2 11:38 2010 bin/ignore > 6393 -rwxr-xr-x 1 joerg/bs Aug 2 11:38 2010 bin/mamprobe > star -tv < /tmp/0a | wc > star: 34418 blocks + 1536 bytes (total of 352441856 bytes = 344181.50k). > 5337 48120 436071 > star -tv < /tmp/0a > /dev/null > star: 34418 blocks + 1536 bytes (total of 352441856 bytes = 344181.50k). > 0.471r 0.060u 0.450s 127% 0M 0+0k 0st 0+0io 0pf+0w > /usr/bin/pax -v < /tmp/0a > /dev/null > USTAR format archive extended > 0.556r 0.130u 0.280s 82% 0M 0+0k 0st 0+0io 0pf+0w > arch/sol11.i386/src/cmd/pax/pax -v < /tmp/0a > /dev/null > /dev/stdin in pax format > 5332 files, 688363 blocks > 8.900r 8.770u 0.040s 98% 0M 0+0k 0st 0+0io 0pf+0w > This is 18.89x the wallclock time star needs and > 146.16x the user CPU time star needs. > Star typically needs a bit more system CPU than other archivers as it forks > and creates a shared memory segment that is used to let data flow between > the archiver process and the I/O process. This was introduced in 1988 (with > the availability of shared memory on SunOS-4.0) in order to keep QIC-24 tapes > streaming and still helps to gain performance and to reduce the wear-out on > tape media. > Ast pax seems to lack support for extracting POSIX.1-2001 compliant archives. > This limits ast pax to files with a maximum size of 8 GB - 1 byte. It seems > that ast pax just throws away the POSIX.1-2001 extended headers. > Here are the tests (the times meter mainly bzip2 performance and thus are not > relevant): > star -tv < testscripts/pax-big-10g.tar.bz2 > star: WARNING: Archive is 'bzip2' compressed, trying to use the -bz option. > 10737418240 -rw------- user/group Jun 15 23:18 2002 10g > 0 -rw-r--r-- user/group Jun 15 16:53 2002 file > star: 1048576 blocks + 3072 bytes (total of 10737421312 bytes = 10485763.00k). > 1:44.282r 85.790u 33.440s 114% 0M 0+0k 0st 0+0io 0pf+0w > gtar -tvjf - < testscripts/pax-big-10g.tar.bz2 > -rw------- user/group 10737418240 2002-06-15 23:18 10g > -rw-r--r-- user/group 0 2002-06-15 16:53 file > 1:45.555r 86.700u 25.570s 106% 0M 0+0k 0st 0+0io 0pf+0w > bzip2 -d < testscripts/pax-big-10g.tar.bz2 | /usr/bin/pax -v > USTAR format archive extended > -rw------- 0 486 1060 10737418240 Jun 15 2002 10g > -rw-r--r-- 0 486 1060 0 Jun 15 2002 file > 1:42.928r 1.400u 11.760s 12% 0M 0+0k 0st 0+0io 0pf+0w > 1:42.928r 83.940u 17.970s 99% 0M 0+0k 0st 0+0io 0pf+0w > arch/sol11.i386/src/cmd/pax/pax -v < testscripts/pax-big-10g.tar.bz2 > /dev/stdin in bzip pax format > -rw------- 1 user group 0 Jun 15 2002 10g > pax: warning: /dev/stdin: junk data after volume 1 > 1 file, 20971523 blocks > 1:44.258r 110.360u 29.490s 134% 0M 0+0k 0st 0+0io 0pf+0w > As you see, ast pax seems to be the only archiver that does not yet fully > support the POSIX.1-2001 standard. but this is what users expect from pax on a > POSIX.1-2001 compliant OS (well, it did at least correctly detect the > archive type and it did not complain about the POSIX.1-2001 extended headers). > I believe that astpax offers the interesting feature to support 25 different > archive types (including MS CAB IIRC) and thus it would be nice to get astpax > included. > I am still not convinced that astpax is the implementation we should use to > replace the closed source /usr/bin/pax. > Jörg > -- > EMail:[email protected] (home) Jörg Schilling D-13353 Berlin > [email protected] (uni) > [email protected] (work) Blog: > http://schily.blogspot.com/ > URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily > _______________________________________________ > Illumos mailing list > [email protected] > http://mexico.purplecow.org/m/listinfo/illumos > -- > __ . . __ > (o.\ \/ /.o) [email protected] > \__\/\/__/ MPEG specialist, C&&JAVA&&Sun&&Unix programmer > /O /==\ O\ TEL +49 641 3992797 > (;O/ \/ \O;) _______________________________________________ ast-developers mailing list [email protected] https://mailman.research.att.com/mailman/listinfo/ast-developers
