As Chee-Hong said today, he just caught that he forgot to check in his macro change. It will be fixed in the next bugfix point release.
Natalie On Tue, Mar 17, 2009 at 4:59 PM, Moe <[email protected]> wrote: > > I see the #else code now, which is pretty much the same as my fix. Too > bad I didn't notice it earlier. > > However, the latest TPP release still contains this "unstable" > version. > > Moyez > > > On Mar 17, 7:17 pm, Chee-Hong Wong <[email protected]> wrote: >> Hi Moe, >> Thanks for pinpointing the cause. >> >> Please just change the macro of #if 1 to #if 0 and you will get the old >> behaviour exactly like what you have program. >> >> I have probably committed this code with the wrong macro by accident. It is >> based on a undocumented "API". The motivation is that mzWiff can reduce it >> execution time by 30-60%. But my testing has revealed that it is very >> unstable depending on the version of Analyst / AnalystQS. As I wish to >> return to it some days and thus have not removed the chunk of code from the >> check in. >> >> Sorry for causing you the problem. >> >> Note: AnalystQS 1.1, Analyst 1.4 and Analyst 1.5 are not affected by this >> "bug". >> >> CH >> >> >> >> On Tue, Mar 17, 2009 at 2:38 PM, Moe <[email protected]> wrote: >> >> > The memory leak appears to be isolated in class AnalystQS20Interface, >> > method processMSSpecData, in the call to "m_ipGetRawDataSpec- >> > >GetRawScanXYPoints()" when requesting the product spectrum's entire >> > set of data points (origical code snip below). >> >> > /*** ORIGINAL CODE FROM RELEASE 4.2.0 ***/ >> >> > inline void AnalystQS20Interface::processMSSpecData(Scan *pScan) { >> > #if 1 >> > ... >> > long lSpecDataPoints; >> > ... >> > VARIANT dXMass; VariantInit(&dXMass); >> > VARIANT dYInts; VariantInit(&dYInts); >> > ... >> > HRESULT hr = m_ipGetRawDataSpec->GetRawScanXYPoints >> > (&lSpecDataPoints, &dXMass, &dYInts); >> > ... >> > long lPeaksCount=0; >> > ... >> > for (long dpi=1; dpi<=lSpecDataPoints; dpi++, pXMass++, pYInts++) { >> > if (*pYInts>0.0) { >> > pScan->mzArray_[lPeaksCount] = *pXMass; >> > pScan->totalIonCurrent_ += >> > (pScan->intensityArray_[lPeaksCount] >> > =*pYInts); >> > lPeaksCount++; >> >> > if (pScan->basePeakIntensity_<*pYInts) { >> > pScan->basePeakMZ_ = *pXMass; >> > pScan->basePeakIntensity_ = *pYInts; >> > } >> > } >> > } >> > ... >> >> > It appears that the memory occupied at &dXMass and &xYInts VARIANTs >> > does not get freed even after descoping, probably because it is being >> > written externally to mzWiff by the AnalystService process. It does >> > not seem possible to free this memory internally, but I have not >> > explored all options here. >> >> > Perhaps an ultimate call to "m_ipGetRawDataSpec->Release" may induce >> > the freeing, but I suppose this would render the IGetRawData structure >> > prematurely unusable. >> >> > As a workaround, I avoid the call to GetRawScanXYPoints() altogether, >> > instead calling m_ipFMANSpecData->GetDataPointXValue() and - >> > >GetDataPointYValue() to request an individual datapoint, as per code >> > below. As this call must be made for each data point, there is a >> > performance hit. I estimated it to be about 30% longer execution time >> > for a 500 MB input, which is still reasonable for me at this time. >> >> > /*** WORKAROUND CODE ***/ >> > ... >> > long lSpecDataPoints = m_ipFMANSpecData->GetNumberOfDataPoints(); >> > ... >> > double dblX, dblY; >> > long lPeaksCount=0; >> > for (long dpi=1; dpi<=lSpecDataPoints; dpi++) { >> > dblX = m_ipFMANSpecData->GetDataPointXValue(dpi); >> > dblY = m_ipFMANSpecData->GetDataPointYValue(dpi); >> > if (dblY>0) { >> > pScan->mzArray_[lPeaksCount] = dblX; >> > pScan->intensityArray_[lPeaksCount] = dblY; >> > pScan->totalIonCurrent_ += dblY; >> > lPeaksCount++; >> >> > if (pScan->basePeakIntensity_ < dblY) { >> > pScan->basePeakMZ_ = dblX; >> > pScan->basePeakIntensity_ = dblY; >> > } >> > } >> > } >> >> > On Mar 16, 6:09 pm, Moe <[email protected]> wrote: >> > > I have uploaded the files bsa-20fmol-2.wiff/.scan. Although the >> > > converter doesn't crash on this file, it did reach > 300 MB memory >> > > usage in my attempt. >> >> > > Let me know if you need anything else for your debugging. >> >> > > Thanks, >> > > Moyez >> >> > > On Mar 16, 5:46 pm, Natalie Tasman <[email protected]> wrote: >> >> > > >http://groups.google.com/group/spctools-discuss/web/how-to-upload-fil. >> > .. >> >> > > > On Mon, Mar 16, 2009 at 2:43 PM, Moe <[email protected]> wrote: >> >> > > > > Thanks for the VC tips. Where can I upload the BSA file? >> >> > > > > Moyez >> >> > > > > On Mar 16, 5:25 pm, Natalie Tasman <[email protected]> >> > wrote: >> > > > >> Hi Moyez, >> >> > > > >> Of course we have our own files to test, but yes, please upload your >> > > > >> file, as we've never observed the behavior you're reporting. >> >> > > > >> And also please rerun your conversions with the "-v" (verbose) >> > option >> > > > >> and copy and paste the output up to the "sha-1" calculation. >> >> > > > >> You're welcome to rebuild and debug the converter on your own. Just >> > > > >> open the TPP.sln file and build the mzWiff project from there. Note >> > > > >> you'll also need the "win_lib" project to be checked out at the same >> > > > >> level as the TPP code for boost, zlib, etc. >> >> > > > >> -Natalie >> >> > > > >> On Mon, Mar 16, 2009 at 2:07 PM, Moe <[email protected]> wrote: >> >> > > > >> > Hi Natalie, >> >> > > > >> > The leak is apparent in QStar WIFF files of any size. For a 30 >> > minute >> > > > >> > BSA QC run (25 MB file size), peak memory usage reached 314 MB (to >> > > > >> > produce ~195 MB mzXML output), which doesn't seem reasonable. Do >> > you >> > > > >> > have your own data files that you can debug with? If not, I can >> > upload >> > > > >> > the BSA run to your FTP site (need the details). >> >> > > > >> > The data I'm trying to analyze consists of 2 hour runs at about >> > 500 MB >> > > > >> > each. mzWiff crashes about halfway through a data file, leaving a >> > 1.3 >> > > > >> > GB (incomplete) mzXML file which ends at an elution time of about >> > 60 >> > > > >> > minutes (when memory reaches slightly > 2 GB). >> >> > > > >> > I have tried older releases of the executable up to 4.0.0 and all >> > > > >> > behave the same. I am considering recompiling a 64-bit executable, >> > or >> > > > >> > trying to locate the leak. I'll check out the code from >> > SourceForge, >> > > > >> > but if you have additional instructions for setting up the VC >> > project, >> > > > >> > please let me know. >> >> > > > >> > Thanks, >> > > > >> > Moyez >> >> > > > >> > On Mar 16, 1:41 pm, Natalie Tasman <[email protected]> >> > wrote: >> > > > >> >> Hello Moyez, >> >> > > > >> >> Thank you for the detailed bug report. I haven't heard of issues >> > with >> > > > >> >> mzWiff. Thanks for the MSVC hint-- I'm pretty sure this is not >> > turned >> > > > >> >> on yet, but actually probably shouldn't be necessary. I agree >> > with >> > > > >> >> you that that much memory usage sounds like a memory leak. The >> > > > >> >> converters are designed to process data scan-by-scan with a >> > relatively >> > > > >> >> low memory overhead. >> >> > > > >> >> If you can upload a sample file we can take a look and see if we >> > can >> > > > >> >> replicate behavior and then fix it. >> >> > > > >> >> Thanks, >> >> > > > >> >> Natalie >> >> > > > >> >> On Mon, Mar 16, 2009 at 9:11 AM, Moe <[email protected]> wrote: >> >> > > > >> >> > I'm trying to convert Analyst QStar Elite data acquired with >> > Analyst >> > > > >> >> > QS 2.0, with mzWiff version 4.2.0(build Feb 19 2009 09:48:08), >> > on >> > > > >> >> > Windows XP Pro SP3 and Windows Server 2003 R2 x64 Standard SP2. >> >> > > > >> >> > There appears to be a memory leak causing process memory to >> > accumulate >> > > > >> >> > to > 2GB, shortly after which the program crashes with error: >> >> > > > >> >> > ERROR: COM error 80004005 while processing sample#2 >> > > > >> >> > INFO: error message: Unspecified error >> >> > > > >> >> > This happens both on 32-bit XP and 64-bit Server 2003, both >> > systems >> > > > >> >> > have > 4GB physical memory. The per-process addressable limit >> > for a 32- >> > > > >> >> > bit process is 2GB unless compiled with the /LARGEADDRESSAWARE >> > Visual C >> > > > >> >> > ++ linker flag. >> >> > > > >> >> > Is anyone aware of this problem? Is this in fact a leak that >> > can be >> > > > >> >> > avoided? >> >> > > > >> >> > Thanks, >> > > > >> >> > Moyez- Hide quoted text - >> >> > > > >> >> - Show quoted text -- Hide quoted text - >> >> > > > >> - Show quoted text -- Hide quoted text - >> >> > > > - Show quoted text -- Hide quoted text - >> >> > > - Show quoted text -- Hide quoted text - >> >> - Show quoted text - > > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "spctools-discuss" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/spctools-discuss?hl=en -~----------~----~----~----~------~----~------~--~---
