Hi,
I'm having trouble understanding just how to make write_chunk work for me.
My issue is that I'm querying a large database and emitting JSON. Sometimes
there are hundreds of thousands of records to write.
My algorithm looks like:
// $query = DBIx::Simple query that return a DB row as a hash
while (my $row = $query->hash) {
my $txt = ... # JSON encode hash
$controller->write_chunk($txt);
$count++;
$self->debug("Written $count/$row_count records") if ($count % 1000) ==
0;
}
But that buffers the JSON strings and never seems to drain so my Perl
process grows to a massive size. What I cannot understand is just how to
restructure the code above to use a callback which is supposed to drain the
buffer and prevent the huge memory usage.
The documentation shows how to write "Hello world" using write_chunk with a
callback but it's just a bit too terse for me to work out how to make use
of it in the context above. I'm sure it's a simple concept but I just
cannot grasp it at the moment.
Thanks,
Allan
--
You received this message because you are subscribed to the Google Groups
"Mojolicious" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
Visit this group at http://groups.google.com/group/mojolicious.
For more options, visit https://groups.google.com/d/optout.
