> -----Original Message----- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On > Behalf Of Gisle Aas > Sent: 06 September 2005 17:30 > To: Craig Cardimon > Cc: ActivePerl > Subject: Re: HTML::TokeParser and tags split between lines > > Craig Cardimon <[EMAIL PROTECTED]> writes: > > > I'm passing to the parser the result gleaned from using > > Text::Context::EitherSide(). > > > > I am unable to pass the entire document toe the parser anyway. The > > files I am working with can be 60 MB in size and larger. My system > > freezes up and crashes. > > If what you want is to feed the parser the document in chunks > until you have extracted enough text, then I suggest you use > the HTML::Parser class directly. The HTML::TokeParser > assumes you are able to give it the whole document when it is created.
But isn't that what pretty much HTML::PullParser (from which HTML::TokeParser is derived) actually does? Its get_token method calls HTML::Parser->parse on 512 byte chunks of the input document, using read() or substr() as appropriate, until it has collected one or more tokens. HTH -- Brian Raven ----------------------------------------------------------------------- The information contained in this e-mail is confidential and solely for the intended addressee(s). Unauthorised reproduction, disclosure, modification, and/or distribution of this email may be unlawful. If you have received this email in error, please notify the sender immediately and delete it from your system. The views expressed in this message do not necessarily reflect those of LIFFE Holdings Plc or any of its subsidiary companies. ----------------------------------------------------------------------- _______________________________________________ ActivePerl mailing list [email protected] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
