On November 12, 1998 at 16:25, Jason L Tibbitts III wrote:
> I'm trying to improve the speed at which Wilma indexes. Right now the real
> bottleneck is that we pass every MHonArc-generated page through the
> striphtml program, which is written in Perl. The time to load the Perl
> interpreter tens or hundred of thousands of times is pretty harsh, and
> occasionally we've seen HTML that the simple regexp-based approach freaks
> out on, causing it to take near infinite time to process.
>
> Does anyone know of any free (i.e. we can incorporate it into something
> under the Artistic License) C code, or a small utility that we can call,
> which will do this?
I have the SGML::StripParser as part of the perlSGML[1] package.
It is in Perl, so the performance is not as good as a C program.
But since it is a module, you can write a Perl program to iterate
through a list of files and use SGML::StripParser on each file to
avoid lauching perl for each file separately.
perlSGML is under the GPL, so if that will not work for you, I
can redistribute it under the Artistic License for your needs.
Another option may be to use James Clark's SP[2] package.
--ewh
[1] <URL:http://www.oac.uci.edu/indiv/ehood/perlSGML.html>
[2] <URL:http://www.jclark.com/sp/index.htm>
