I've done a large amount of data import routines over the years. To this day the best method is still server side scripting with PERL or a native PL (procedural language). I have written both PERL and ColdFusion based imports in the past. With ColdFusion (well either really) I would stay away from objects or other complex data types and concentrate the efforts on getting the data into the database as quickly and painlessly as possible. I've even built a system where the client uploaded their CSV,Excel, or TAB file and then CF simply sent the file to the DB server via FTP or some other backend network connection. The DB server had a CRON job that would look for a file to import and did so when ever it was found. That worked really slick.
You have options to work with and I hope I just gave you some useful ideas. For your situation to determine the best method, I think that if you ask yourself this question you may get the solution you need. Is this a one time import or a regular import on a reoccurring basis? If one time, then CF may be the best answer since the code is written. If this is a data import scheduled on a regular basis then maybe a scripting solution with PERL or other PL language would be a better solution. The more frequent the imports the more likely a PERL or PL scripting solution would be beneficial. Good Luck, Wil Genovese One man with courage makes a majority. -Andrew Jackson A fine is a tax for doing wrong. A tax is a fine for doing well. On Apr 20, 2010, at 9:41 AM, Donnie Carvajal wrote: > > I have a question about preferences for importing data and filtering the data > through CFCs. We have functionality in our app to upload a csv file to make > mass adds/updates. From the file, we are loading the data into a database > table. The app has many great CFCs that create beans and DAOs that manage > all of the business logic for adding/updating through the GUI of the app. > Would you suggest taking each record in the table the data was imported to > and creating an object for each record and using the normal add/update logic > that the CFCs already handle or would you suggest writing a single UPDATE > query and an INSERT with SELECT query to get the data into the system? It > seems the greatest reason for using CFCs is to encapsulate all of the > business logic in one spot to avoid having to change code in many different > places, so creating objects would be the wiser choice; however, some of these > imports will have thousands if not tens of thousands of records and CF is > very slow at creating objects, so it seems it would be more beneficial to > simply write queries to translate the data and duplicate the logic. > > Thoughts? > > Thanks, > > Donnie > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~| Want to reach the ColdFusion community with something they want? Let them know on the House of Fusion mailing lists Archive: http://www.houseoffusion.com/groups/cf-talk/message.cfm/messageid:333037 Subscription: http://www.houseoffusion.com/groups/cf-talk/subscribe.cfm Unsubscribe: http://www.houseoffusion.com/groups/cf-talk/unsubscribe.cfm
