Making tokenize stream would be really difficult. The reason being that
the pattern can be an arbitrary regular expression which is a lot of
cases would require to materialize the entire stream anyway.

Hence, I'm suggestion that we define our own string:tokenize function
which accepts only separators instead of regular expressions and make
this work in a streaming fashion. Would that also do the job?

** Changed in: zorba
    Milestone: None => 2.2

-- 
You received this bug notification because you are a member of Zorba
Coders, which is the registrant for Zorba.
https://bugs.launchpad.net/bugs/898074

Title:
  fn:tokenize() doesn't stream

Status in Zorba - The XQuery Processor:
  New

Bug description:
  The following query:
  let $content := file:read-text("huge_file.txt")
  return tokenize($content, "\s")

  doesn't stream the result.

To manage notifications about this bug go to:
https://bugs.launchpad.net/zorba/+bug/898074/+subscriptions

-- 
Mailing list: https://launchpad.net/~zorba-coders
Post to     : zorba-coders@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zorba-coders
More help   : https://help.launchpad.net/ListHelp

Reply via email to