Okay, I'm elated at the progress this morning. Once again I can thank
you guys enough for pointing me in the right direction. Take a look at
this quick and dirty example of my use of threads and Thread::Queue
It worked fairly well however my memory usage was on a steady climb as
well as my processor hitting that 100% mark.
This is simulation of what My program is trying to do using threads with
a set number of resources but and unknown number of jobs.
use strict;
use threads;
use Thread::Queue;
my ($thread,$resource,$wanPending);
my $wanQ = new Thread::Queue;
my $modemQ = new Thread::Queue;
# WAN Resources
my $wan1 = 'O:';
my $wan2 = 'P:';
my $wan3 = 'Q:';
my $wan4 = 'R:';
my $wan5 = 'S:';
# Modem Resources
my $modem1 = 'U:';
my $modem2 = 'V:';
my $modem3 = 'W:';
my $modem4 = 'X:';
my $modem5 = 'Y:';
# Build the Queue
$wanQ ->enqueue($wan1,$wan2,$wan3,$wan4,$wan5);
$modemQ ->enqueue($modem1,$modem2,$modem3,$modem4,$modem5);
# Number of Jobs to kick off...
my @wanList = (1..50);
# Create a new Thread with whats in the queue.
foreach my $numFilesToDo (@wanList) {
$wanPending = $wanQ->pending;
print $wanPending." wan resources left\n";
# If we have a resource available in the queue
# then create a thread to doJob
if ($wanQ->pending != 0) {
$resource = $wanQ->dequeue;
$thread = threads->create("doJob","$numFilesToDo","$resource");
$thread->detach();
} else {
sleep(10);
}
}
sub doJob {
my ($jobId,$resource) = @_;
my $random = int( rand (10)) + 2;
print " :: Job ID number $jobId has resource $resource in use\n";
# simulate performing job
sleep($random);
# Requeue it after we are done
$wanQ->enqueue($resource);
return;
}
-----Original Message-----
From: Mike Pomraning [mailto:[EMAIL PROTECTED]
Sent: Wednesday, April 19, 2006 9:27 AM
To: Daniel Rychlik
Cc: [email protected]; [email protected]
Subject: RE: iThreads troubles
On Wed, 19 Apr 2006, Daniel Rychlik wrote:
> Some of you have asked for code examples.
>
> One of issues that I am having is when the thread enters into
> getAvailableResource routine its not seeing that the
> $Resources{'WAN1'}->{INUSE} = 1; has been made unavailable. The idea
is
> for the next thread to pick up the next unavailable resource. I tried
> to sleep(1) between thread creation, and also used threads::shared
> share(%Resources); which also failed.
Dan,
A few quick remarks/suggestions.
First, please post the smallest possible code that reproduces your
symptoms:
pare down your problematic program, stuff it with stubs until you've got
a
cogent proof-of-problem.
Second, &share()ing your hash clears it [0], so populate it /after/ you
mark
it shared. Additionally, the hash members themselves have to be
explicitly
shared, IIUC. [1]
Third, your 'INUSE' check looks to have a race. You'll need to &lock(),
use
a semaphore, etc. As others have suggested, Thread::Queue makes these
kinds
of problems a lot easier. You could, perhaps, manage the "wan" and
"modem"
resources themselves in a queue -- pop off the next available one, use
it,
then re-queue it for another job later on. TMTOWTDI.
[0] "When share is used on arrays, hashes, array refs or hash refs, any
data they contain will be lost."
http://search.cpan.org/~jdhedden/threads-shared-1.01/shared.pm#BUGS_AND_
LIMITATIONS
[1] "This places restrictions on what may be assigned to shared array
and
hash elements: only simple values or references to shared variables
are
allowed..."
http://www.perldoc.com/perl5.8.4/pod/perlthrtut.html#Shared-And-Unshared
-Data
Good luck,
Mike
--
cat: .signature: Message too long
