I totally agree, and have used this technique many times for data imports. Data scrubbing can be messy, but once it's in a table, SQL is so powerful it can make an initially daunting-in-appearance task very manageable.
As a worst case scenario, you may need to use sed/awk/perl/(or gasp, maybe php) to filter the file if it's not properly delimited. But it's often _much_ faster to filter the file on disk first and then use a LOAD DATA INFILE than to even batch it in via INSERT queries. I haven't tried this, but you might be able to save yourself the extra copy if you need to filter it this way by writing the filter output to a named pipe and LOAD DATA INFILE on the pipe... /* PLUG: http://plug.org, #utah on irc.freenode.net Unsubscribe: http://plug.org/mailman/options/plug Don't fear the penguin. */
