Any chance to have the database engine do all that record logic? That would be the first thing I would try. Stored procedures are great for things like this. I don't know what RDMS you are using, but most have SQL that can do this. Complex database stuff is usually better off doing within the database as it can scale.
1 store the four Excel fields in a temporary table in the database 2 Use SQL to weed out all the records against the current tables. Google for examples. 3 Dump the parsed records into the production table. 2 and 3 probably are one step in the database engine, but the SQL might be tedious so my first thought is to separate it. Good luck, Roger ---- Les Mizzell <lesm...@bellsouth.net> wrote: > > Evening, > > I'm still fighting with my Excel Import application a bit. Basically, it > works pretty well, as long as there's a limited number of records being > imported. > Problem is, the client tried to import a list with almost 15,000 > addresses today, and it eventually timed out or the server reset the > connection before it finished the import. Then my phone rings... > > The app does several things: > 1. Import 4 columns from an excel sheet [email][first name][last > name][company] into an array > 2. Check the email address to see if it's valid. If not, delete that row > from the array > 3. Check the email address to see if it's been previously unsubscribed > from the list and delete that row from the array > 4. Check the list against other email addresses already in the database > for that list, and delete that row so as to not import it if it's > already there. > 5. After all that, write the record > > Running some test with the 15,000 address sheet > Reading the Excel sheet in and running steps 1 through 4 above is taking > well over 10 minutes by itself. > Writing the record after that, enough extra time to be sure it borks up. > Smaller list (several hundred addresses) seem to import fine - MUCH > faster than the former POI based app. -- LinkedIn: http://www.linkedin.com/pub/roger-austin/8/a4/60 Twitter: http://twitter.com/RogerTheGeek Blog: http://RogerTheGeek.wordpress.com/ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Order the Adobe Coldfusion Anthology now! http://www.amazon.com/Adobe-Coldfusion-Anthology/dp/1430272155/?tag=houseoffusion Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:359649 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
