I posted the BSA data yesterday to pub/incoming, but I see it isn't there. I tried uploading again but got a permission denied (overwrite) error. So I have just uploaded agian to pub/incoming/mdharsee.
Let me know if you get it. Thanks, Moyez On Mar 17, 7:04 pm, Natalie Tasman <[email protected]> wrote: > Thank you, Moyez. We appreciate your work! By the way, I still > haven't received your file for testing. > > Thanks, > > Natalie > > > > On Tue, Mar 17, 2009 at 2:38 PM, Moe <[email protected]> wrote: > > > The memory leak appears to be isolated in class AnalystQS20Interface, > > method processMSSpecData, in the call to "m_ipGetRawDataSpec- > >>GetRawScanXYPoints()" when requesting the product spectrum's entire > > set of data points (origical code snip below). > > > /*** ORIGINAL CODE FROM RELEASE 4.2.0 ***/ > > > inline void AnalystQS20Interface::processMSSpecData(Scan *pScan) { > > #if 1 > > ... > > long lSpecDataPoints; > > ... > > VARIANT dXMass; VariantInit(&dXMass); > > VARIANT dYInts; VariantInit(&dYInts); > > ... > > HRESULT hr = m_ipGetRawDataSpec->GetRawScanXYPoints > > (&lSpecDataPoints, &dXMass, &dYInts); > > ... > > long lPeaksCount=0; > > ... > > for (long dpi=1; dpi<=lSpecDataPoints; dpi++, pXMass++, pYInts++) { > > if (*pYInts>0.0) { > > pScan->mzArray_[lPeaksCount] = *pXMass; > > pScan->totalIonCurrent_ += > > (pScan->intensityArray_[lPeaksCount] > > =*pYInts); > > lPeaksCount++; > > > if (pScan->basePeakIntensity_<*pYInts) { > > pScan->basePeakMZ_ = *pXMass; > > pScan->basePeakIntensity_ = *pYInts; > > } > > } > > } > > ... > > > It appears that the memory occupied at &dXMass and &xYInts VARIANTs > > does not get freed even after descoping, probably because it is being > > written externally to mzWiff by the AnalystService process. It does > > not seem possible to free this memory internally, but I have not > > explored all options here. > > > Perhaps an ultimate call to "m_ipGetRawDataSpec->Release" may induce > > the freeing, but I suppose this would render the IGetRawData structure > > prematurely unusable. > > > As a workaround, I avoid the call to GetRawScanXYPoints() altogether, > > instead calling m_ipFMANSpecData->GetDataPointXValue() and - > >>GetDataPointYValue() to request an individual datapoint, as per code > > below. As this call must be made for each data point, there is a > > performance hit. I estimated it to be about 30% longer execution time > > for a 500 MB input, which is still reasonable for me at this time. > > > /*** WORKAROUND CODE ***/ > > ... > > long lSpecDataPoints = m_ipFMANSpecData->GetNumberOfDataPoints(); > > ... > > double dblX, dblY; > > long lPeaksCount=0; > > for (long dpi=1; dpi<=lSpecDataPoints; dpi++) { > > dblX = m_ipFMANSpecData->GetDataPointXValue(dpi); > > dblY = m_ipFMANSpecData->GetDataPointYValue(dpi); > > if (dblY>0) { > > pScan->mzArray_[lPeaksCount] = dblX; > > pScan->intensityArray_[lPeaksCount] = dblY; > > pScan->totalIonCurrent_ += dblY; > > lPeaksCount++; > > > if (pScan->basePeakIntensity_ < dblY) { > > pScan->basePeakMZ_ = dblX; > > pScan->basePeakIntensity_ = dblY; > > } > > } > > } > > > On Mar 16, 6:09 pm, Moe <[email protected]> wrote: > >> I have uploaded the files bsa-20fmol-2.wiff/.scan. Although the > >> converter doesn't crash on this file, it did reach > 300 MB memory > >> usage in my attempt. > > >> Let me know if you need anything else for your debugging. > > >> Thanks, > >> Moyez > > >> On Mar 16, 5:46 pm, Natalie Tasman <[email protected]> wrote: > > >> >http://groups.google.com/group/spctools-discuss/web/how-to-upload-fil... > > >> > On Mon, Mar 16, 2009 at 2:43 PM, Moe <[email protected]> wrote: > > >> > > Thanks for the VC tips. Where can I upload the BSA file? > > >> > > Moyez > > >> > > On Mar 16, 5:25 pm, Natalie Tasman <[email protected]> wrote: > >> > >> Hi Moyez, > > >> > >> Of course we have our own files to test, but yes, please upload your > >> > >> file, as we've never observed the behavior you're reporting. > > >> > >> And also please rerun your conversions with the "-v" (verbose) option > >> > >> and copy and paste the output up to the "sha-1" calculation. > > >> > >> You're welcome to rebuild and debug the converter on your own. Just > >> > >> open the TPP.sln file and build the mzWiff project from there. Note > >> > >> you'll also need the "win_lib" project to be checked out at the same > >> > >> level as the TPP code for boost, zlib, etc. > > >> > >> -Natalie > > >> > >> On Mon, Mar 16, 2009 at 2:07 PM, Moe <[email protected]> wrote: > > >> > >> > Hi Natalie, > > >> > >> > The leak is apparent in QStar WIFF files of any size. For a 30 > >> > >> > minute > >> > >> > BSA QC run (25 MB file size), peak memory usage reached 314 MB (to > >> > >> > produce ~195 MB mzXML output), which doesn't seem reasonable. Do you > >> > >> > have your own data files that you can debug with? If not, I can > >> > >> > upload > >> > >> > the BSA run to your FTP site (need the details). > > >> > >> > The data I'm trying to analyze consists of 2 hour runs at about 500 > >> > >> > MB > >> > >> > each. mzWiff crashes about halfway through a data file, leaving a > >> > >> > 1.3 > >> > >> > GB (incomplete) mzXML file which ends at an elution time of about 60 > >> > >> > minutes (when memory reaches slightly > 2 GB). > > >> > >> > I have tried older releases of the executable up to 4.0.0 and all > >> > >> > behave the same. I am considering recompiling a 64-bit executable, > >> > >> > or > >> > >> > trying to locate the leak. I'll check out the code from SourceForge, > >> > >> > but if you have additional instructions for setting up the VC > >> > >> > project, > >> > >> > please let me know. > > >> > >> > Thanks, > >> > >> > Moyez > > >> > >> > On Mar 16, 1:41 pm, Natalie Tasman <[email protected]> > >> > >> > wrote: > >> > >> >> Hello Moyez, > > >> > >> >> Thank you for the detailed bug report. I haven't heard of issues > >> > >> >> with > >> > >> >> mzWiff. Thanks for the MSVC hint-- I'm pretty sure this is not > >> > >> >> turned > >> > >> >> on yet, but actually probably shouldn't be necessary. I agree with > >> > >> >> you that that much memory usage sounds like a memory leak. The > >> > >> >> converters are designed to process data scan-by-scan with a > >> > >> >> relatively > >> > >> >> low memory overhead. > > >> > >> >> If you can upload a sample file we can take a look and see if we > >> > >> >> can > >> > >> >> replicate behavior and then fix it. > > >> > >> >> Thanks, > > >> > >> >> Natalie > > >> > >> >> On Mon, Mar 16, 2009 at 9:11 AM, Moe <[email protected]> wrote: > > >> > >> >> > I'm trying to convert Analyst QStar Elite data acquired with > >> > >> >> > Analyst > >> > >> >> > QS 2.0, with mzWiff version 4.2.0(build Feb 19 2009 09:48:08), on > >> > >> >> > Windows XP Pro SP3 and Windows Server 2003 R2 x64 Standard SP2. > > >> > >> >> > There appears to be a memory leak causing process memory to > >> > >> >> > accumulate > >> > >> >> > to > 2GB, shortly after which the program crashes with error: > > >> > >> >> > ERROR: COM error 80004005 while processing sample#2 > >> > >> >> > INFO: error message: Unspecified error > > >> > >> >> > This happens both on 32-bit XP and 64-bit Server 2003, both > >> > >> >> > systems > >> > >> >> > have > 4GB physical memory. The per-process addressable limit > >> > >> >> > for a 32- > >> > >> >> > bit process is 2GB unless compiled with the /LARGEADDRESSAWARE > >> > >> >> > Visual C > >> > >> >> > ++ linker flag. > > >> > >> >> > Is anyone aware of this problem? Is this in fact a leak that can > >> > >> >> > be > >> > >> >> > avoided? > > >> > >> >> > Thanks, > >> > >> >> > Moyez- Hide quoted text - > > >> > >> >> - Show quoted text -- Hide quoted text - > > >> > >> - Show quoted text -- Hide quoted text - > > >> > - Show quoted text -- Hide quoted text - > > >> - Show quoted text -- Hide quoted text - > > - Show quoted text - --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/spctools-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
