pax format keywords have 7 precedence levels depending on the source (command line, global header, extended header) and the type of value assignment (= vs :=)
there was a typo for the precedence of the "size" keyword that caused it to be ignored (write generated "size' keys, read recognized "size" keys, but ignored them because of the precedence typo) I had also coded the global+extended header parse to use the optstr() wrapper for optget() -- this turned out to be a gross misuse of optget() -- its not intended to be used in the main inner loop that might see a few orders of magnitude repititions ast pax codes options and header keywords in a hash table this table is used to generate the optget() usage[], but is also used directly in some places -- I changed the global+extended header keyword parse to use direct access and it brought the times down to the star neighborhood I was amused by the star source comments about users not reading build system manuals vs the comments below about the ast build system (which has first time user documentation links on the main download page) re the next message about non-standard file type '8' (SOKTYPE) it looks like a vestige of unknown origin -- it could have been from a formative standard discussion, or a non-ast pax/tar/cpio implementation -- anyway, J is correct that it is non-standard, so I commented it out On Fri, 6 Aug 2010 00:51:49 +0200 Roland Mainz wrote: > ---- > Just curious: > Does AST pax support POSIX.1-2001 compliant archives ? > Joerg Schilling said about this: > > Ast pax seems to lack support for extracting POSIX.1-2001 compliant > > archives. > > This limits ast pax to files with a maximum size of 8 GB - 1 byte. It seems > > that ast pax just throws away the POSIX.1-2001 extended headers. > ---- > Bye, > Roland > ---------- Forwarded message ---------- > From: Joerg Schilling <[email protected]> > Date: Mon, Aug 2, 2010 at 4:16 PM > Subject: Re: [Illumos] Build instructions > To: [email protected] > Cc: [email protected] > ????? ???????????? <[email protected]> wrote: > > Joerg, remind that the default build of AST builds with the optimiser > > disabled, if you compare performance do it equally, same compilers > > with same optimiser flags. > Hi Olga, after some research (the project uses an unusual build system that > lacks a documentation for beginners at top level) I finally have been able to > compile astpax with the same optimization level that is used by default with > star. As expected, there is no significant performance win compared to the > default compilation I did yesterday. > The general tendency for performance tests (after running about a dozen > different tests) is: > star is the fastest > gnu tar typically needs twice as much user CPU time than star but is not > significantly slower in wallclock time in case the archive is not on a real > tape. > Sun pax typically needs 1.5 .. 2x as much user CPU time than gnu tar and is > already noticable slower in wallclock time. > Ast pax is the slowest. Ast pax typically needs 3x..5x the amount of user CPU > time than star and typically needs 1.5x the wallclock time spend by star. > Ast pax out of cousiosity needs more than 100x the amount of user CPU time > when creating verbose listings from a POSIX.1-2001 extended tar archive with > 5332 files (this archive type is sometimes called "pax"). In the latter cases, > the wallclock time spend for listing the archive by ast pax is aprox. 19x the > wallclock time spend by star. > star -tv < /tmp/0a | head -10 > Release star 1.5.1 (i386-pc-solaris2.11) > Archtype exustar > Dumpdate 1280745273.165657 (Mon Aug 2 12:34:33 2010) > Volno 1 > Blocksize 20 records > 0 drwxr-xr-x 6 joerg/bs Aug 2 11:34 2010 ./ > 1144 -rw-r--r-- 1 joerg/bs Oct 14 17:10 2008 README > 0 drwxr-xr-x 2 joerg/bs Aug 2 11:38 2010 bin/ > 1797 -rwxr-xr-x 1 joerg/bs Aug 2 11:38 2010 bin/ignore > 6393 -rwxr-xr-x 1 joerg/bs Aug 2 11:38 2010 bin/mamprobe > star -tv < /tmp/0a | wc > star: 34418 blocks + 1536 bytes (total of 352441856 bytes = 344181.50k). > 5337 48120 436071 > star -tv < /tmp/0a > /dev/null > star: 34418 blocks + 1536 bytes (total of 352441856 bytes = 344181.50k). > 0.471r 0.060u 0.450s 127% 0M 0+0k 0st 0+0io 0pf+0w > /usr/bin/pax -v < /tmp/0a > /dev/null > USTAR format archive extended > 0.556r 0.130u 0.280s 82% 0M 0+0k 0st 0+0io 0pf+0w > arch/sol11.i386/src/cmd/pax/pax -v < /tmp/0a > /dev/null > /dev/stdin in pax format > 5332 files, 688363 blocks > 8.900r 8.770u 0.040s 98% 0M 0+0k 0st 0+0io 0pf+0w > This is 18.89x the wallclock time star needs and > 146.16x the user CPU time star needs. > Star typically needs a bit more system CPU than other archivers as it forks > and creates a shared memory segment that is used to let data flow between > the archiver process and the I/O process. This was introduced in 1988 (with > the availability of shared memory on SunOS-4.0) in order to keep QIC-24 tapes > streaming and still helps to gain performance and to reduce the wear-out on > tape media. > Ast pax seems to lack support for extracting POSIX.1-2001 compliant archives. > This limits ast pax to files with a maximum size of 8 GB - 1 byte. It seems > that ast pax just throws away the POSIX.1-2001 extended headers. > Here are the tests (the times meter mainly bzip2 performance and thus are not > relevant): > star -tv < testscripts/pax-big-10g.tar.bz2 > star: WARNING: Archive is 'bzip2' compressed, trying to use the -bz option. > 10737418240 -rw------- user/group Jun 15 23:18 2002 10g > 0 -rw-r--r-- user/group Jun 15 16:53 2002 file > star: 1048576 blocks + 3072 bytes (total of 10737421312 bytes = 10485763.00k). > 1:44.282r 85.790u 33.440s 114% 0M 0+0k 0st 0+0io 0pf+0w > gtar -tvjf - < testscripts/pax-big-10g.tar.bz2 > -rw------- user/group 10737418240 2002-06-15 23:18 10g > -rw-r--r-- user/group 0 2002-06-15 16:53 file > 1:45.555r 86.700u 25.570s 106% 0M 0+0k 0st 0+0io 0pf+0w > bzip2 -d < testscripts/pax-big-10g.tar.bz2 | /usr/bin/pax -v > USTAR format archive extended > -rw------- 0 486 1060 10737418240 Jun 15 2002 10g > -rw-r--r-- 0 486 1060 0 Jun 15 2002 file > 1:42.928r 1.400u 11.760s 12% 0M 0+0k 0st 0+0io 0pf+0w > 1:42.928r 83.940u 17.970s 99% 0M 0+0k 0st 0+0io 0pf+0w > arch/sol11.i386/src/cmd/pax/pax -v < testscripts/pax-big-10g.tar.bz2 > /dev/stdin in bzip pax format > -rw------- 1 user group 0 Jun 15 2002 10g > pax: warning: /dev/stdin: junk data after volume 1 > 1 file, 20971523 blocks > 1:44.258r 110.360u 29.490s 134% 0M 0+0k 0st 0+0io 0pf+0w > As you see, ast pax seems to be the only archiver that does not yet fully > support the POSIX.1-2001 standard. but this is what users expect from pax on a > POSIX.1-2001 compliant OS (well, it did at least correctly detect the > archive type and it did not complain about the POSIX.1-2001 extended headers). > I believe that astpax offers the interesting feature to support 25 different > archive types (including MS CAB IIRC) and thus it would be nice to get astpax > included. > I am still not convinced that astpax is the implementation we should use to > replace the closed source /usr/bin/pax. _______________________________________________ ast-developers mailing list [email protected] https://mailman.research.att.com/mailman/listinfo/ast-developers
