Re: [Perl-unix-users] Tie a large list of DB_Files
Antonio Gulli wrote:
> Dear All,
> i hope that someone can help me.
> I have a large list of DB_Files stored in a directory. I would like to
> tie all of them to an array of hashes and to search them for key,
> using the hashes.
>
> My idea is rather simple:
>
> my (@listOfHashes);
> my ($numOfHashes);
>
> sub loadDBs{
>
> my ($dbDIR) = @_;
> my ($numFiles) = 0;
> my ($b) = new DB_File::BTREEINFO;
> my ($file);
>
> $numOfHashes = 0;
>
> opendir(DIR, $dbDIR) || die "cannot opendir $dbDIR: $!";
> while (defined($file = readdir(DIR))){
>
>if ($file =~ /db\.*/){
> tie %{ $listOfHashes[$numOfHashes] }, "DB_File", $file,
You might try : "$dbDIR/$file" instead of $file to pick up the full
path to the file.
> O_RDONLY, 0666, $b
> || die ("Cannot open $file");
> $numOfHashes++;
> print "Tied $file $numOfHashes\n";
>}
> }
> }
>
> sub dumpAll_Key_Value{
>
> my ($key, $value);
> for (my $i = 0; $i < $numOfHashes; $i++){
> print "Load $i";
> while (($key, $value) = keys %{ $listOfHashes[$i] }){
> print ">$key< >$value<\n";
>}
> }
> }
>
> sub searchKey{
>
> my ($key) = @_;
> my ($fetched);
> for (my $i = 0; $i < $numOfHashes; $i++){
> if (defined ($fetched = $listOfHashes[$i]{$key})){
> print "found $key on $i DB";
> return $fetched;
> }
> }
> return undef;
> }
>
> But neither searchKey() nor dumAll_Key_Value() seems to work,
> while loadDBs() seems to tie the hashes...
>
> Am i doing a trivial error or what? Any suggestion?
You could also try dumping listOfHashes using Data::Dumper and
see what you get.
--
,-/- __ _ _ $Bill LuebkertMailto:[EMAIL PROTECTED]
(_/ / )// // DBE CollectiblesMailto:[EMAIL PROTECTED]
/ ) /--< o // // Castle of Medieval Myth & Magic http://www.todbe.com/
-/-' /___/_<_http://dbecoll.tripod.com/ (My Perl/Lakers stuff)
___
Perl-Unix-Users mailing list
[EMAIL PROTECTED]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: [Perl-unix-users] Tie a large list of DB_Files
You might try : "$dbDIR/$file" instead of $file to pick up the full path to the file. Well, i provided the full path in the $file outside. You could also try dumping listOfHashes using Data::Dumper andsee what you get. I-ll try this. ___ Perl-Unix-Users mailing list [EMAIL PROTECTED] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: [Perl-unix-users] Tie a large list of DB_Files
My idea is rather simple:
my (@listOfHashes);
my ($numOfHashes);
sub loadDBs{
my ($dbDIR) = @_;
my ($numFiles) = 0;
my ($b) = new DB_File::BTREEINFO;
my ($file);
$numOfHashes = 0;
opendir(DIR, $dbDIR) || die "cannot opendir $dbDIR: $!";
while (defined($file = readdir(DIR))){
if ($file =~ /db\.*/){
tie %{ $listOfHashes[$numOfHashes] }, "DB_File", $file,
You might try : "$dbDIR/$file" instead of $file to pick up the full
path to the file.
O_RDONLY, 0666, $b
|| die ("Cannot open $file");
$numOfHashes++;
print "Tied $file $numOfHashes\n";
}
}
Added print Dumper([EMAIL PROTECTED]); here
}
sub dumpAll_Key_Value{
my ($key, $value);
for (my $i = 0; $i < $numOfHashes; $i++){
print "Load $i";
while (($key, $value) = keys %{ $listOfHashes[$i] }){
print ">$key< >$value<\n";
}
}
}
sub searchKey{
my ($key) = @_;
my ($fetched);
for (my $i = 0; $i < $numOfHashes; $i++){
if (defined ($fetched = $listOfHashes[$i]{$key})){
print "found $key on $i DB";
return $fetched;
}
}
return undef;
}
But neither searchKey() nor dumAll_Key_Value() seems to work,
while loadDBs() seems to tie the hashes...
Am i doing a trivial error or what? Any suggestion?
You could also try dumping listOfHashes using Data::Dumper and
see what you get.
And obtained (this is a directory with just a DB file, so i have
just one hash inside, the same happens with directory with many files).
$VAR1 = [
{}
];
So the hash is empty, but why the tie operation does not fail ?
Should the hash be empty when i tie it to a DB_File?
Re: [Perl-unix-users] Tie a large list of DB_Files
Antonio Gulli wrote:
>
>>>My idea is rather simple:
>>>
>>>my (@listOfHashes);
>>>my ($numOfHashes);
>>>
>>>sub loadDBs{
>>>
>>>my ($dbDIR) = @_;
>>>my ($numFiles) = 0;
>>>my ($b) = new DB_File::BTREEINFO;
>>>my ($file);
>>>
>>>$numOfHashes = 0;
>>>
>>>opendir(DIR, $dbDIR) || die "cannot opendir $dbDIR: $!";
>>>while (defined($file = readdir(DIR))){
>>>
>>> if ($file =~ /db\.*/){
>>> tie %{ $listOfHashes[$numOfHashes] }, "DB_File", $file,
>>>
>>>
>>
>>You might try : "$dbDIR/$file" instead of $file to pick up the full
>>path to the file.
>>
>>
>>
>>>O_RDONLY, 0666, $b
>>> || die ("Cannot open $file");
>>> $numOfHashes++;
>>> print "Tied $file $numOfHashes\n";
>>> }
>>>}
>>>
>>>
> Added print Dumper([EMAIL PROTECTED]); here
Try dumping $file and $dbDIR - I want to verify they're correct.
And what dir are you running it from ?
If that doesn't help, use a simpler construct and see what you get.
>>>}
>>>
>>>sub dumpAll_Key_Value{
>>>
>>>my ($key, $value);
>>>for (my $i = 0; $i < $numOfHashes; $i++){
>>> print "Load $i";
>>> while (($key, $value) = keys %{ $listOfHashes[$i] }){
>>>print ">$key< >$value<\n";
>>> }
>>>}
>>>}
>>>
>>>sub searchKey{
>>>
>>>my ($key) = @_;
>>>my ($fetched);
>>>for (my $i = 0; $i < $numOfHashes; $i++){
>>> if (defined ($fetched = $listOfHashes[$i]{$key})){
>>>print "found $key on $i DB";
>>>return $fetched;
>>> }
>>>}
>>> return undef;
>>>}
>>>
>>>But neither searchKey() nor dumAll_Key_Value() seems to work,
>>>while loadDBs() seems to tie the hashes...
>>>
>>>Am i doing a trivial error or what? Any suggestion?
>>>
>>>
>>
>>You could also try dumping listOfHashes using Data::Dumper and
>>see what you get.
>>
>>
> And obtained (this is a directory with just a DB file, so i have
> just one hash inside, the same happens with directory with many files).
>
> $VAR1 = [
> {}
> ];
>
> So the hash is empty, but why the tie operation does not fail ?
> Should the hash be empty when i tie it to a DB_File?
I didn't test the BTREE part, the the rest seems ok using a regular
DB_FILE file :
use Fcntl;
use Tie::Hash;
use DB_File;
my @hash_names;
my @listOfHashes;
my $numOfHashes;
loadDBsu (shift || 'data');
# print Data::Dumper->Dump([EMAIL PROTECTED], [qw([EMAIL PROTECTED])]) if $debug;
dumpAll_Key_Value ();
my $ret = searchKey (shift || 'Dan');
print "\n$ret\n" if $ret;
exit;
sub loadDBsu {
my $dbDIR = shift;
my $b = new DB_File::BTREEINFO;
my $numFiles = 0;
my $file;
opendir DIR, $dbDIR or die "cannot opendir $dbDIR: $!";
while (defined ($file = readdir DIR)) {
if ($file =~ /db/i) {
my %hash;
tie %hash, 'DB_File', "$dbDIR/$file", O_RDONLY, 0666, $b or do {
warn "Cannot tie $dbDIR/$file";
next;
};
push @hash_names, "$dbDIR/$file";
push @listOfHashes, \%hash;
$numOfHashes++;
print "Tied $file $numOfHashes\n";
}
}
closedir DIR;
}
sub dumpAll_Key_Value {
for (my $ii = 0; $ii < @listOfHashes; $ii++) {
print "Loading $hash_names[$ii]\n";
while (my ($key, $value) = each %{$listOfHashes[$ii]}) {
print "'$key' => '$value'\n";
}
}
}
sub searchKey {
my $key = shift;
for (my $ii = 0; $ii < @listOfHashes; $ii++) {
my $fetched = $listOfHashes[$ii]->{$key};
if (defined $fetched) {
print "found $key in $hash_names[$ii]\n";
return $fetched;
}
}
return undef;
}
__END__
--
,-/- __ _ _ $Bill LuebkertMailto:[EMAIL PROTECTED]
(_/ / )// // DBE CollectiblesMailto:[EMAIL PROTECTED]
/ ) /--< o // // Castle of Medieval Myth & Magic http://www.todbe.com/
-/-' /___/_<_http://dbecoll.tripod.com/ (My Perl/Lakers stuff)
___
Perl-Unix-Users mailing list
[EMAIL PROTECTED]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
Re: [Perl-unix-users] Tie a large list of DB_Files
opendir(DIR, $dbDIR) || die "cannot opendir $dbDIR: $!";
while (defined($file = readdir(DIR))){
if ($file =~ /db\.*/){
tie %{ $listOfHashes[$numOfHashes] }, "DB_File", $file,
O_RDONLY, 0666, $b
|| die ("Cannot open $file");
$numOfHashes++;
print "Tied $file $numOfHashes\n";
}
}
Gosh Gosh Gosh,
Bill you are quite right! It works fine just like it should. I tested it
with 80 different DB_files (2Gb each) BTREE. A huge amount of data, I
did a trivial mistake and $file DO NOT contains the path. O poor me.
Thank you so much. What a trivial mistake.
But what is the reason why tie succeded and not die. I saw the printed
message "Tied file ..." and this makes me erroneously trust the code.
Again thank you so much!
___
Perl-Unix-Users mailing list
[EMAIL PROTECTED]
To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
[Perl-unix-users] Duane Ebron/HNB/HBI/US is out of the office 12/18/02
I will be out of the office starting 12/22/2003 and will not return until 01/06/2004. I will be out of the office starting 12/22/2003 and not return untilI 01/06/2003. ___ Perl-Unix-Users mailing list [EMAIL PROTECTED] To unsubscribe: http://listserv.ActiveState.com/mailman/mysubs
