Making tokenize stream would be really difficult. The reason being that
the pattern can be an arbitrary regular expression which is a lot of
cases would require to materialize the entire stream anyway.

Hence, I'm suggestion that we define our own string:tokenize function
which accepts only separators instead of regular expressions and make
this work in a streaming fashion. Would that also do the job?

** Changed in: zorba
    Milestone: None => 2.2

You received this bug notification because you are a member of Zorba
Coders, which is the registrant for Zorba.

  fn:tokenize() doesn't stream

Status in Zorba - The XQuery Processor:

Bug description:
  The following query:
  let $content := file:read-text("huge_file.txt")
  return tokenize($content, "\s")

  doesn't stream the result.

To manage notifications about this bug go to:

Mailing list:
Post to     :
Unsubscribe :
More help   :

Reply via email to