Matthew Lange wrote: > I am using the latest Base, and trying to load an initial data set. My > initial effort was to modify the reporter importer to load some affy
Why have you modified the reporter importer? What modifications have you made? The regular reporter importer can import Affymetrix csv files without problems. The hard part may be to come up with regular expressions that works. Sample configurations can be downloaded from http://base.thep.lu.se/chrome/site/doc/admin/plugin_configuration/coreplugins.html > probes from a csv file. This worked fine for a small list, but as the > list approached >500 individuals, I started receiving out of memory What do you mean with "individuals"? We regularly test with importing 50000+ reporters/probesets. It works fine with 500MB memory with room to spare. > errors from the Java Heap...and even brought Tomcat to its knees on a > couple of occassions. As a workaround, I uploaded the an 80ish MB CDF > file from Affy, which contains the array design. This went in fine. > Then I tried to use the feature that allows me to import reporters from > the CDF file--again Java OOM errors, and tomcat whining. We use the Affymetrix Fusion SDK for reading CDF files. I don't know about the memory requirements. I think the binary format uses less memory since data is read from the file at demand. If the file is in text format everything is parses and stored in memory. The biggest test file we have is ~40MB text format. Memory usage stays below 100MB while parsing that file. > We have given Tomcat 2GB to play with, and this seems like it should be > more than enough. Are there known issues with this utility that I > should be aware of, or is there a rule-of-thumb to follow when > allocating memory to tomcat, based on the size of files that need to be > parsed? Or maybe another memory setting somewhere that I must have > overlooked? Are you really sure that you have given Tomcat that much memory. You can check various configuration options including memory usage on the Help -> About menu. Switch to the "Configuration" tab. > I will be happy to post more specific error messages if that will > help the troubleshooting process. I am not even close to reproduce this so more information is certainly needed. /Nicklas ------------------------------------------------------------------------- SF.Net email is sponsored by: The Future of Linux Business White Paper from Novell. From the desktop to the data center, Linux is going mainstream. Let it simplify your IT future. http://altfarm.mediaplex.com/ad/ck/8857-50307-18918-4 _______________________________________________ The BASE general discussion mailing list basedb-users@lists.sourceforge.net unsubscribe: send a mail with subject "unsubscribe" to [EMAIL PROTECTED]
