RE: Witango-Talk: best practice for uploading from file to db

[Dave Shelley]

Roland,   I wouldn’t recommend parsing the file into an array. It would be better to use the MySQL command LOAD DATA INFILE to import the data. If you need to massage the data, you can load it into a temporary table, process it, then move it to your production table.   I did this on a project recently. Importing >40K records, massaging the data, updating changed records, inserting new records and deleting old ones. It all ran in under a minute from a Witango cron job.   You will also want to use a delimiter that does not appear anywhere in the data. A vertical bar is a good option.   Dave Shelley   -----Original Message----- From: Roland Dumas [mailto:[EMAIL PROTECTED] Sent: May 14, 2004 3:46 PM To: [EMAIL PROTECTED] Subject: Re: Witango-Talk: best practice for uploading from file to db   Perhaps my problem is not being able to make the delimiters change from the defaults. No matter what I set the delimiters to be (I'm trying a pipe delimiter here), the array attributes don't override the defaults. <@ASSIGN request$newarray value="<@ARRAY value='@@request$fromFile'>" CDELIM=| RDELIM=;> On May 14, 2004, at 11:26 AM, Roland Dumas wrote: System A exports a file that has many records, 10K to 15K. Records have fields that are text and fields that are numeric. That file will be read by witango and turned into records to add to a database. I can have the exported file in any text format. (CSV, different delimiters, quoted values, whatever) I did a couple of quick attempts to read the file, turn it into an array, and ran into issues. No matter what I set the delimiters to, a comma in a field triggers errors when creating the array. So, rather than stumble into the right approach, I thought I'd ask the list for what the best practices are for creating uploaders like this. ________________________________________________________________________ TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf ________________________________________________________________________ TO UNSUBSCRIBE: Go to http://www.witango.com/developer/maillist.taf