To avoid having 300,000 records sitting in memory I generally have my queries return record id's. My DAO returns an int[] or ResultObjects collection containing record id's that I store in some scope(session) as it takes little mem. The records are not generally all displayed at once so I only fetch the records that will be displayed on a single page and use a pager tag lib to provide paging functionality. The actual recordObjects currently being viewed are put into request scope to reduce mem consumption.
Not sure of your iBatis plug-in but maybe you could still page the records and only pass one page of records at a time and use the strategy above. HTH, Greg > -----Original Message----- > From: Jerry Jalenak [mailto:[EMAIL PROTECTED] > Sent: Thursday, November 06, 2003 9:36 AM > To: '[EMAIL PROTECTED]' > Subject: [OT - Design] Needing Fast Access to 300,000 Records > > Hi All - > > I've been trying to figure out a good way of handling something, and just > can't quite seem to get a grip on the best approach. Here's what I've > got: > > Two database tables - Table 1 has a BILLING_CODE and an ACCOUNT_CODE. > Table > 2 has the ACCOUNT_CODE and other account information (name, address, etc.) > The tables are linked by ACCOUNT_CODE. > > Table 1: > Table 2: > BILLING_CODE ACCOUNT_CODE > ACCOUNT_CODE NAME ADDRESS ... > 1234 ABC1 > ABC1 blah blah > 1234 ABC2 > ABC2 blah blah > 1234 ABC3 > ABC3 blah blah > 5678 DEF1 > DEF1 blah blah > 5678 DEF2 > DEF2 blah blah > > I need to be able to rapidly access the information in table 2 either > through the BILLING_CODE, or directly through the ACCOUNT_CODE. I can > create a POJO containing the BILLING_CODE and a List object to hold a > second > POJO for the table 2 info. Or I can use a Map. Either way doesn't give > me > a good method of accessing the table 2 information based on ACCOUNT_CODE. > > Two other bits of info - the combined number of records exceeds 300,000, > so > I have a scaling issue. Second, I'd like to load everything in a plug-in > using iBatis dbLayer and store it in application scope (to eliminate db > calls as the webapp is used.) > > Does anyone have any experience in handling something like this? How did > you do it? > > Thanks... > > Jerry Jalenak > Development Manager, Web Publishing > LabOne, Inc. > 10101 Renner Blvd. > Lenexa, KS 66219 > (913) 577-1496 > > [EMAIL PROTECTED] > > > This transmission (and any information attached to it) may be confidential > and > is intended solely for the use of the individual or entity to which it is > addressed. If you are not the intended recipient or the person responsible > for > delivering the transmission to the intended recipient, be advised that you > have received this transmission in error and that any use, dissemination, > forwarding, printing, or copying of this information is strictly > prohibited. > If you have received this transmission in error, please immediately notify > LabOne at the following email address: > [EMAIL PROTECTED] > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
