The memory leak appears to be isolated in class AnalystQS20Interface,
method processMSSpecData, in the call to "m_ipGetRawDataSpec-
>GetRawScanXYPoints()" when requesting the product spectrum's entire
set of data points (origical code snip below).
/*** ORIGINAL CODE FROM RELEASE 4.2.0 ***/
inline void AnalystQS20Interface::processMSSpecData(Scan *pScan) {
#if 1
...
long lSpecDataPoints;
...
VARIANT dXMass; VariantInit(&dXMass);
VARIANT dYInts; VariantInit(&dYInts);
...
HRESULT hr = m_ipGetRawDataSpec->GetRawScanXYPoints
(&lSpecDataPoints, &dXMass, &dYInts);
...
long lPeaksCount=0;
...
for (long dpi=1; dpi<=lSpecDataPoints; dpi++, pXMass++, pYInts++) {
if (*pYInts>0.0) {
pScan->mzArray_[lPeaksCount] = *pXMass;
pScan->totalIonCurrent_ +=
(pScan->intensityArray_[lPeaksCount]
=*pYInts);
lPeaksCount++;
if (pScan->basePeakIntensity_<*pYInts) {
pScan->basePeakMZ_ = *pXMass;
pScan->basePeakIntensity_ = *pYInts;
}
}
}
...
It appears that the memory occupied at &dXMass and &xYInts VARIANTs
does not get freed even after descoping, probably because it is being
written externally to mzWiff by the AnalystService process. It does
not seem possible to free this memory internally, but I have not
explored all options here.
Perhaps an ultimate call to "m_ipGetRawDataSpec->Release" may induce
the freeing, but I suppose this would render the IGetRawData structure
prematurely unusable.
As a workaround, I avoid the call to GetRawScanXYPoints() altogether,
instead calling m_ipFMANSpecData->GetDataPointXValue() and -
>GetDataPointYValue() to request an individual datapoint, as per code
below. As this call must be made for each data point, there is a
performance hit. I estimated it to be about 30% longer execution time
for a 500 MB input, which is still reasonable for me at this time.
/*** WORKAROUND CODE ***/
...
long lSpecDataPoints = m_ipFMANSpecData->GetNumberOfDataPoints();
...
double dblX, dblY;
long lPeaksCount=0;
for (long dpi=1; dpi<=lSpecDataPoints; dpi++) {
dblX = m_ipFMANSpecData->GetDataPointXValue(dpi);
dblY = m_ipFMANSpecData->GetDataPointYValue(dpi);
if (dblY>0) {
pScan->mzArray_[lPeaksCount] = dblX;
pScan->intensityArray_[lPeaksCount] = dblY;
pScan->totalIonCurrent_ += dblY;
lPeaksCount++;
if (pScan->basePeakIntensity_ < dblY) {
pScan->basePeakMZ_ = dblX;
pScan->basePeakIntensity_ = dblY;
}
}
}
On Mar 16, 6:09 pm, Moe <[email protected]> wrote:
> I have uploaded the files bsa-20fmol-2.wiff/.scan. Although the
> converter doesn't crash on this file, it did reach > 300 MB memory
> usage in my attempt.
>
> Let me know if you need anything else for your debugging.
>
> Thanks,
> Moyez
>
> On Mar 16, 5:46 pm, Natalie Tasman <[email protected]> wrote:
>
>
>
> >http://groups.google.com/group/spctools-discuss/web/how-to-upload-fil...
>
> > On Mon, Mar 16, 2009 at 2:43 PM, Moe <[email protected]> wrote:
>
> > > Thanks for the VC tips. Where can I upload the BSA file?
>
> > > Moyez
>
> > > On Mar 16, 5:25 pm, Natalie Tasman <[email protected]> wrote:
> > >> Hi Moyez,
>
> > >> Of course we have our own files to test, but yes, please upload your
> > >> file, as we've never observed the behavior you're reporting.
>
> > >> And also please rerun your conversions with the "-v" (verbose) option
> > >> and copy and paste the output up to the "sha-1" calculation.
>
> > >> You're welcome to rebuild and debug the converter on your own. Just
> > >> open the TPP.sln file and build the mzWiff project from there. Note
> > >> you'll also need the "win_lib" project to be checked out at the same
> > >> level as the TPP code for boost, zlib, etc.
>
> > >> -Natalie
>
> > >> On Mon, Mar 16, 2009 at 2:07 PM, Moe <[email protected]> wrote:
>
> > >> > Hi Natalie,
>
> > >> > The leak is apparent in QStar WIFF files of any size. For a 30 minute
> > >> > BSA QC run (25 MB file size), peak memory usage reached 314 MB (to
> > >> > produce ~195 MB mzXML output), which doesn't seem reasonable. Do you
> > >> > have your own data files that you can debug with? If not, I can upload
> > >> > the BSA run to your FTP site (need the details).
>
> > >> > The data I'm trying to analyze consists of 2 hour runs at about 500 MB
> > >> > each. mzWiff crashes about halfway through a data file, leaving a 1.3
> > >> > GB (incomplete) mzXML file which ends at an elution time of about 60
> > >> > minutes (when memory reaches slightly > 2 GB).
>
> > >> > I have tried older releases of the executable up to 4.0.0 and all
> > >> > behave the same. I am considering recompiling a 64-bit executable, or
> > >> > trying to locate the leak. I'll check out the code from SourceForge,
> > >> > but if you have additional instructions for setting up the VC project,
> > >> > please let me know.
>
> > >> > Thanks,
> > >> > Moyez
>
> > >> > On Mar 16, 1:41 pm, Natalie Tasman <[email protected]> wrote:
> > >> >> Hello Moyez,
>
> > >> >> Thank you for the detailed bug report. I haven't heard of issues with
> > >> >> mzWiff. Thanks for the MSVC hint-- I'm pretty sure this is not turned
> > >> >> on yet, but actually probably shouldn't be necessary. I agree with
> > >> >> you that that much memory usage sounds like a memory leak. The
> > >> >> converters are designed to process data scan-by-scan with a relatively
> > >> >> low memory overhead.
>
> > >> >> If you can upload a sample file we can take a look and see if we can
> > >> >> replicate behavior and then fix it.
>
> > >> >> Thanks,
>
> > >> >> Natalie
>
> > >> >> On Mon, Mar 16, 2009 at 9:11 AM, Moe <[email protected]> wrote:
>
> > >> >> > I'm trying to convert Analyst QStar Elite data acquired with Analyst
> > >> >> > QS 2.0, with mzWiff version 4.2.0(build Feb 19 2009 09:48:08), on
> > >> >> > Windows XP Pro SP3 and Windows Server 2003 R2 x64 Standard SP2.
>
> > >> >> > There appears to be a memory leak causing process memory to
> > >> >> > accumulate
> > >> >> > to > 2GB, shortly after which the program crashes with error:
>
> > >> >> > ERROR: COM error 80004005 while processing sample#2
> > >> >> > INFO: error message: Unspecified error
>
> > >> >> > This happens both on 32-bit XP and 64-bit Server 2003, both systems
> > >> >> > have > 4GB physical memory. The per-process addressable limit for a
> > >> >> > 32-
> > >> >> > bit process is 2GB unless compiled with the /LARGEADDRESSAWARE
> > >> >> > Visual C
> > >> >> > ++ linker flag.
>
> > >> >> > Is anyone aware of this problem? Is this in fact a leak that can be
> > >> >> > avoided?
>
> > >> >> > Thanks,
> > >> >> > Moyez- Hide quoted text -
>
> > >> >> - Show quoted text -- Hide quoted text -
>
> > >> - Show quoted text -- Hide quoted text -
>
> > - Show quoted text -- Hide quoted text -
>
> - Show quoted text -
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"spctools-discuss" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/spctools-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---
