Making tokenize stream would be really difficult. The reason being that
the pattern can be an arbitrary regular expression which is a lot of
cases would require to materialize the entire stream anyway.
Hence, I'm suggestion that we define our own string:tokenize function
which accepts only separators instead of regular expressions and make
this work in a streaming fashion. Would that also do the job?
** Changed in: zorba
Milestone: None => 2.2
You received this bug notification because you are a member of Zorba
Coders, which is the registrant for Zorba.
fn:tokenize() doesn't stream
Status in Zorba - The XQuery Processor:
The following query:
let $content := file:read-text("huge_file.txt")
return tokenize($content, "\s")
doesn't stream the result.
To manage notifications about this bug go to:
Mailing list: https://launchpad.net/~zorba-coders
Post to : email@example.com
Unsubscribe : https://launchpad.net/~zorba-coders
More help : https://help.launchpad.net/ListHelp