The cdf file mentioned in my original post is called HG-U133_Plus_2.cdf, and
is downloadable from Affymetrix. It is 97 MB.
Here is the Stack trace provided through the BASE CDF ProbeSet Importer (not
using the custom reporter importer here, just the generic one that comes
with base), accessible from the array design page, as "verify Reporters"
link.
Thanks a bunch for your help. Let me know if I can be more specific.
Kind Regards,
matthew
java.lang.OutOfMemoryError: Java heap space
at java.lang.String.substring(String.java:1770)
at java.lang.String.subSequence(String.java:1803)
at java.util.regex.Pattern.split(Pattern.java:993)
at java.lang.String.split(String.java:2103)
at java.lang.String.split(String.java:2145)
at affymetrix.gcos.cdf.CDFFileData.readTextProbeSets(Unknown Source)
at affymetrix.gcos.cdf.CDFFileData.readTextFormat(Unknown Source)
at affymetrix.gcos.cdf.CDFFileData.open(Unknown Source)
at affymetrix.gcos.cdf.CDFFileData.read(Unknown Source)
at affymetrix.fusion.cdf.FusionCDFData.read(Unknown Source)
at net.sf.basedb.core.Affymetrix.checkIfReportersExists(Affymetrix.java:416)
at net.sf.basedb.plugins.CdfFileReporterImporter.importFromCdf(
CdfFileReporterImporter.java:460)
at net.sf.basedb.plugins.CdfFileReporterImporter.run(
CdfFileReporterImporter.java:183)
at net.sf.basedb.core.PluginExecutionRequest.invoke(
PluginExecutionRequest.java:89)
at net.sf.basedb.core.InternalJobQueue$JobRunner.run(InternalJobQueue.java
:421)
at java.lang.Thread.run(Thread.java:595)
On Dec 3, 2007 11:02 PM , Nicklas Nordborg <[EMAIL PROTECTED]> wrote:
>
>
> ---------- Forwarded message ----------
> From: Nicklas Nordborg <[EMAIL PROTECTED]>
> Date: Dec 3, 2007 11:02 PM
> Subject: Re: [base] Java memory error
> To: BASE ML <basedb-users@lists.sourceforge.net>
> Cc: Joseph Fass <[EMAIL PROTECTED] >, [EMAIL PROTECTED]
>
>
> Matthew Lange wrote:
> > I am using the latest Base, and trying to load an initial data set. My
> > initial effort was to modify the reporter importer to load some affy
>
> Why have you modified the reporter importer? What modifications have you
> made? The regular reporter importer can import Affymetrix csv files
> without problems. The hard part may be to come up with regular
> expressions that works. Sample configurations can be downloaded from
> http://base.thep.lu.se/chrome/site/doc/admin/plugin_configuration/coreplugins.html
>
>
>
> > probes from a csv file. This worked fine for a small list, but as the
> > list approached >500 individuals, I started receiving out of memory
>
> What do you mean with "individuals"? We regularly test with importing
> 50000+ reporters/probesets. It works fine with 500MB memory with room to
> spare.
>
> > errors from the Java Heap...and even brought Tomcat to its knees on a
> > couple of occassions. As a workaround, I uploaded the an 80ish MB CDF
> > file from Affy, which contains the array design. This went in fine.
> > Then I tried to use the feature that allows me to import reporters from
> > the CDF file--again Java OOM errors, and tomcat whining.
>
> We use the Affymetrix Fusion SDK for reading CDF files. I don't know
> about the memory requirements. I think the binary format uses less
> memory since data is read from the file at demand. If the file is in
> text format everything is parses and stored in memory. The biggest test
> file we have is ~40MB text format. Memory usage stays below 100MB while
> parsing that file.
>
>
> > We have given Tomcat 2GB to play with, and this seems like it should be
> > more than enough. Are there known issues with this utility that I
> > should be aware of, or is there a rule-of-thumb to follow when
> > allocating memory to tomcat, based on the size of files that need to be
> > parsed? Or maybe another memory setting somewhere that I must have
> > overlooked?
>
> Are you really sure that you have given Tomcat that much memory. You can
> check various configuration options including memory usage on the Help
> -> About menu. Switch to the "Configuration" tab.
>
> > I will be happy to post more specific error messages if that will
> > help the troubleshooting process.
>
> I am not even close to reproduce this so more information is certainly
> needed.
>
> /Nicklas
>
>
>
> --
> Joseph Fass
> joseph.fass -at- gmail.com || joefass -at- hotmail.com
> 970.227.5928 (c) || 530.754.7978 (w)
-------------------------------------------------------------------------
SF.Net email is sponsored by: The Future of Linux Business White Paper
from Novell. From the desktop to the data center, Linux is going
mainstream. Let it simplify your IT future.
http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4
_______________________________________________
The BASE general discussion mailing list
basedb-users@lists.sourceforge.net
unsubscribe: send a mail with subject "unsubscribe" to
[EMAIL PROTECTED]
