Re: DB information does not match filesystem
Hi again, Does anyone have an hint on this subject? dev7 was added after dev6 to the same storage node while the platform was already being used. The database indicates that we have 7862 files on dev7 but there are in fact more than 50.000 files under /var/mogdata/dev7 Actual data for these two devices goes here: Statistics for devices... device hostfiles status -- --- -- -- dev6 mogilestorage2 46267 alive dev7 mogilestorage2 7862 alive ls dev6 -R -lha | grep fid | wc 2582 23238 147174 ls dev7 -R -lha | grep fid | wc 51576 464184 2939832 Thanks, Sergio Bessa Sergio Bessa wrote: Hi, I seem to have a strange problem with one of my storage nodes. It is storing the files on one of the devices but registering the information on the DB for another device. The information I could retrieve goes below. mogadm --lib=/usr/local/share/perl/5.8.4 --trackers=192.168.0.123:6001 stats Fetching statistics... Statistics for devices... device hostfiles status -- --- -- -- dev3 mogilestorage1 23894 alive dev4 mogilestorage2 24464 alive dev5 mogilestorage1 53882 alive dev6 mogilestorage2 46132 alive dev7 mogilestorage2 6995 alive -- --- -- -- Statistics for file ids... Max file id: 98320 Statistics for files... domain class files --- -- mingle photos 77569 --- -- Statistics for replication... domain classdevcount files --- -- -- mingle photos 1 1 mingle photos 2 77386 mingle photos 3182 --- -- -- Files in DEV3 ls dev3 -R -lha | grep fid | wc 23897 215073 1386026 Files in DEV4 ls dev4 -R -lha | grep fid | wc 24464 220176 1419212 Files in DEV5 ls dev5 -R -lha | grep fid | wc 53694 483246 3061239 Files in DEV6 ls dev6 -R -lha | grep fid | wc 2443 21987 139251 Files in DEV7 ls dev7 -R -lha | grep fid | wc 50700 456300 2889900 Thanks for any advice. Regards, Sergio Bessa -- Sérgio Bessa *Ideavity, Lda.* *www*: http://www.ideavity.com *email*: [EMAIL PROTECTED] *tel.* +351 222 012 393
Re: MogileFS Perl initialisation / performance issues.
On Fri, 2008-04-18 at 11:49 +0200, Ferdy Galema wrote: Hello, Let's say I'm running the following snippet in apache2/mod_perl, all with the default configuration: ## #!/usr/bin/perl use strict; use CGI qw(:standard); use MogileFS::Client; snip #just serve the image directly for now print Content-type: image/jpeg\n\n; print $$image; ## That doesn't look like mod_perl to me? Looks like an old style CGI. -Does it get initialised everytime a request is made? Is that okay performance wise? Yes it does. That's not too bad performance wise, but not the best if you're trying to shave your request down to ms.. However loading perl (and loading / compiling the modules) on each request (as you appear to be a CGI) will be a much bigger overhead than connecting to mogileFS here as you're a CGI. If not, can it be made global in a way? Yes, just use a closure to hold the mogile object, and it'll stay connected. Cheers Tom
Re: Using md5 sum of file contents as key
On Tue, 2008-04-15 at 12:30 +0100, Ian Sherratt wrote: Heya, [...] We're considering using mogilefs as a storage solution, using a md5 sum (or SHA-xx) of the file's contents as the key. This key would be stored by each application in their own databases along with all the metainformation which is application dependent. [...] Has any body used a function of the file contents as the key before? Good idea/bad idea? Storing the checksum of the file also lets you verify the files contents. If mogilefs did this automatically, it could easily have a 'scrub' feature in addition to fsck. -- -- Justin Azoff -- Network Performance Analyst
