when code will not handle real world data, it is broken. you and I discussed this when importing xml file and I choose to parse them manually instead of using DOM. So what term is there besides bug for broken design.
========================= BJ Freeman http://bjfreeman.elance.com Strategic Power Office with Supplier Automation <http://www.businessesnetwork.com/automation/viewforum.php?f=93> Specialtymarket.com <http://www.specialtymarket.com/> Systems Integrator-- Glad to Assist Chat Y! messenger: bjfr33man Linkedin <http://www.linkedin.com/profile?viewProfile=&key=1237480&locale=en_US&trk=tab_pro> David E Jones sent the following on 3/15/2010 6:05 PM: > On Mar 15, 2010, at 6:58 PM, Adam Heath wrote: > >> BJ Freeman wrote: >>> one of my smaller import files (8mb) is taking forever to be read in and >>> there is no output. >>> I am seeing the memory rail against the max setting. >>> got this error. >> The datafile set of classes is very broken for large files. It has a >> List<Record>, which means it will copy the entire file into memory >> before doing anything with it. The datafile code is not designed to >> handle large files. >> >> I consider this a bug that needs to be fixed. Checking... > > This should be possible, but may require API changes. The problem is that, > like XML, data files can be hierarchical and a "node" can have header and > footer lines in the file. > > I wouldn't consider this a bug, just like XML DOM parsing is not a "bug". Of > course, you're certainly entitled to your opinion. > > -David > >
