How to do slow stuff with -async

[Matt Sergeant]

Using gearman, that's how.

http://www.danga.com/gearman/
Example plugin:

#!/usr/bin/perl -w

use Gearman::Client::Async;
use Storable qw(thaw);

sub register {
    my ($self, $qp, @args) = @_;
    my @gearmen = $qp->config('gearmand_hosts');
    if ([EMAIL PROTECTED]) {
        $gearmen[0] = 'localhost';
    }
    $self->{client} = Gearman::Client::Async->new(
        job_servers => [EMAIL PROTECTED],
    );

    $self->{timeout}     = $qp->config('gearman.timeout') || 5;
    $self->{retry_count} = $qp->config('gearman.retry_count') || 0;

    $self->register_hook("data_post", "check_spam");
    $self->register_hook("data_post", "check_spam_done");
}

sub check_spam {
    my ($self, $transaction) = @_;

    $self->log(LOGDEBUG, "check_spam");

    my $qp = $self->qp;

    my $mail = $transaction->header->as_string .
               $transaction->body_as_string;

    my $task = Gearman::Task->new("scan", \$mail, {
            on_complete => sub {
                my $output = shift();
                if (!$output) {
                    $qp->log(LOGALERT, "Nothing returned from  
Gearman");
                    $qp->finish_continuation;
                    return;
                }
                $output = thaw($$output);
                # do something with $output here

                $qp->finish_continuation;
            },
            on_fail => sub {
                $qp->log(LOGALERT, "Gearman failed");
                $qp->finish_continuation;
            },
            on_retry => sub {
                $qp->log(LOGALERT, "Gearman failed but retrying for  
$_[0] time");
            },
            timeout     => $self->{timeout},
            retry_count => $self->{retry_count},
    });
    $self->{client}->add_task($task);
    return CONTINUATION;
}

And the worker just returns freeze($struct) and it all "just works".

The advantage of this is you can add in extra workers if you get too  
many timeouts, and gearman smoothes over any lumpiness in connections  
for you.

(I think this is a bit like what MailChannels are doing but using  
mod_perl instead of gearman).

Matt.