Thanks for the responces After discussing it with the cluster administrators, we discovered that is was due to the fasta file being located on an external drive, and the USB connection to the drive slowing down the whole process.
On Fri, Feb 10, 2012 at 10:41 PM, Eric Deutsch <[email protected]> wrote: > Hi Shaun, I just wanted to echo Jimmy that we do this all the time, Linux > cluster with NFS, and don't have an issue. NFS caches pretty well so it > shouldn't cause that much of a problem. What could possibly more of an > issue is 100+ nodes beating on one file server to search many GB of data. > We've had some issues like that in the past. But it seems like the FASTA > shouldn't be the issue. > > Note also that X!Tandem has a setting to run up to 8 threads, so it will > be more efficient to run one or two xtandem processes with threads set to > 8, rather than running many processes of X!Tandem per node with each just > a single thread. > > Regards, > Eric > > >> -----Original Message----- >> From: [email protected] [mailto:spctools- >> [email protected]] On Behalf Of Jimmy Eng >> Sent: Friday, February 10, 2012 9:57 AM >> To: [email protected] >> Subject: Re: [spctools-discuss] running xtandem in parallel on the same >> fasta file >> >> Shaun, >> >> Multiple processing reading the same fasta file off of an NFS >> partition has never really been a performance issue that I've ever >> noticed through the years on different clusters (even pushing a >> hundred processes accessing the same fasta). If this really is a >> problem for you, the obvious solution is to store the database on >> local disk on each node; I can't imagine any other 'simple' fix. >> >> On Fri, Feb 10, 2012 at 9:46 AM, naturofix <[email protected]> >> wrote: >> > I've installed TPP 4.5.1 on a linux computer cluster, which enables >> > parallel processing of numerous files. >> > >> > Unfortunately this does not work with xtandem since all instances are >> > trying to access the same fasta file. >> > Only very few of the processes can run at any one time. >> > >> > Is there a simple way around this. >> > >> > The cluster uses a NFS file system, which cannot be changed at the >> > moment. >> > >> > Thanks >> > Shaun >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "spctools-discuss" group. >> > To post to this group, send email to spctools- >> [email protected]. >> > To unsubscribe from this group, send email to spctools- >> [email protected]. >> > For more options, visit this group at >> http://groups.google.com/group/spctools-discuss?hl=en. >> > >> >> -- >> You received this message because you are subscribed to the Google >> Groups "spctools-discuss" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to spctools- >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/spctools-discuss?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "spctools-discuss" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/spctools-discuss?hl=en. > -- Blackburn Lab IIDMM University of Cape Town South Africa -- You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/spctools-discuss?hl=en.
