*** This bug is a duplicate of bug 898168 ***

To circumvent the problem, you could try doing it as follows:

for $i at $y in parse-xml:parse(
        <opt:parse-external-parsed-entity opt:skip-root-nodes="1" />
return {
  if ($i mod 1000 eq 0)
    cdml:apply-insert-nodes( $def:articles, $w[fn:last()] );

** This bug has been marked a duplicate of bug 898168
   Windowing doesn't stream

** Changed in: zorba
     Assignee: Matthias Brantner (matthias-brantner) => Markos Zaharioudakis 

You received this bug notification because you are a member of Zorba
Coders, which is the registrant for Zorba.

  Memory overload when streaming large file

Status in Zorba - The XQuery Processor:

Bug description:
  We tried to put every 1000th wikipedia article into a Zorba
  collection. Unfortunately, the computer's main memory quickly fills up
  and swapping starts at ~93% memory consumption for the zorba process.

  Code is attached. The wiki.xml file (36GB) is a recent Wikipedia Dump
  and can be obtained at http://download.wikimedia.org/enwiki/latest
  /enwiki-latest-pages-articles.xml.bz2 (~7.8GB when compressed). There
  are currently around 4,000,000 articles in the English Wikipedia.

To manage notifications about this bug go to:

Mailing list: https://launchpad.net/~zorba-coders
Post to     : zorba-coders@lists.launchpad.net
Unsubscribe : https://launchpad.net/~zorba-coders
More help   : https://help.launchpad.net/ListHelp

Reply via email to