On Wed, Feb 14, 2007 at 03:04:53AM -0800, Ask Bjorn Hansen wrote:
> Hi,
>
> To give DBD::Gofer a quick spin I made a prototype gearman
> transport. I was surprised how easy it was. Very neat!
Yeah! Thanks. Gofer is going to be hot.
I've implemented an http transport on the client side and a
corresponding mod_perl transport on the server side. Also easy, and fun.
The 'use case' driving this development a particular part of Shopzilla.com's
system which has ~150 web servers each with 40 httpd children all
talking to a Sybase database.
The http transport will enable not just true connection pooling via
apache's acting as 'middleware', but also load balancing and response
caching via hardware web accelerators. (Or Squid or whatever caching /
reverse proxy / load balancing technology you want to use.)
> To try it,
>
> 1) start a gearmand process
> 2) start the worker process
> perl -w -Ilib -MDBI::Gofer::Transport::gearman -e
> DBI::Gofer::Transport::gearman::run
> 3) change the $dsn in 85-gopher.t to
> dbi:Gofer:transport=gearman;url=127.0.0.1;dsn=...
Better than that... just set the DBI_AUTOPROXY env var to
dbi:Gofer:transport=gearman;url=127.0.0.1
and do "make test", then the *whole DBI test suite* will be run through it!
> Index: lib/DBD/Gofer/Transport/gearman.pm
Ask, go ahead and check it in to the repository. With a few changes...
> ===================================================================
> --- lib/DBD/Gofer/Transport/gearman.pm (revision 0)
> +++ lib/DBD/Gofer/Transport/gearman.pm (revision 0)
> @@ -0,0 +1,70 @@
> +package DBD::Gofer::Transport::gearman;
> +
> +use lib '/Users/ask/src/gearman/api/perl/Gearman/lib';
*cough*
> Index: lib/DBD/Gofer/Transport/stream.pm
> ===================================================================
> --- lib/DBD/Gofer/Transport/stream.pm (revision 9102)
> +++ lib/DBD/Gofer/Transport/stream.pm (working copy)
> @@ -32,7 +32,7 @@
> if (not $connection || ($connection->{pid} && not kill 0,
> $connection->{pid})) {
> my $cmd = [qw(perl -MDBI::Gofer::Transport::stream -e
> run_stdio_hex)];
> #push @$cmd, "DBI_TRACE=2=/tmp/goferstream.log", "sh", "-c";
> - if (my $url = $self->go_url) {
> + if (my $url = $self->go_job_servers) {
> die "Only 'ssh:[EMAIL PROTECTED]' style url supported by
> this transport"
> unless $url =~ s/^ssh://;
Did you mean to edit DBD/Gofer/Transport/stream.pm?
> Index: lib/DBI/Gofer/Transport/gearman.pm
> ===================================================================
> --- lib/DBI/Gofer/Transport/gearman.pm (revision 0)
> +++ lib/DBI/Gofer/Transport/gearman.pm (revision 0)
> @@ -0,0 +1,37 @@
> +package DBI::Gofer::Transport::gearman;
> +sub run {
> + my $worker = Gearman::Worker->new;
> + $worker->job_servers('127.0.0.1');
> + $worker->register_function
> + ( 'dbi_gofer_execute' =>
> + sub {
> + my $job = shift;
> + my $executor = DBI::Gofer::Execute->new();
> + my $request = $transport->thaw_data($job->arg);
> + my $response = $executor->execute_request( $request );
> + my $frozen_response = $transport->freeze_data($response);
> + return $frozen_response;
> + });
> + $worker->work while 1;
> +}
You could optimize that slighly by reusing the same $executor object.
> Index: t/86gopher-gearman.t
> ===================================================================
> --- t/86gopher-gearman.t (revision 0)
> +++ t/86gopher-gearman.t (revision 0)
> @@ -0,0 +1,72 @@
> +#!perl -w # -*- perl -*-
> +# vim:sw=4:ts=8
> +$|=1;
> +
> +use strict;
> +use warnings;
> +
> +use Test::More 'no_plan';
> +
> +use DBI;
> +
> +use lib "/Users/timbo/dbi/trunk/lib";
*cough* - I've removed that now from t/85gofer.t (which you based this one on).
> +# so users can try others from the command line
> +my $dbm = $ARGV[0] || "SDBM_File";
> +
> +# use DBD::Gofer directly.
> +# when combined with DBI_AUTOPROXY this means we have DBD::Gofer =>
> DBD::Gofer => DBD::DBM!
> +#
> +my $dsn =
> "dbi:Gofer:transport=gearman;url=127.0.0.1;dsn=dbi:DBM:dbm_type=$dbm;lockfile=0";
> +my $dbh = DBI->connect($dsn);
Needs a skip_all if Gearman isn't installed and running.
Thanks!
Tim.
