tl;dr: CSV is a useful standard which, like most things in computing, has a 
couple of 'gotchas'.  Not all '.CSV' files are CSV files.

Can't give details but I just identified a problem which had caused the SQL 
database owner £100,000 == US$130,000 == €117,000 of damage, mostly because 
end-of-day crashed three days in a row.  Plus my company's fees.

The cause was some or all of the following, depending on your philosophy.

A) SQL schema had constraints defined for columns, but the person who did it 
didn't think of all the possible things that might go wrong, or didn't think 
they were all worth guarding against.
B) A CSV file was imported by someone who didn't understand the concepts of 
escaping quote characters and how missing values are handled.
C) Excel's CSV exporter does not conform to RFC 4180, the standard for CSV 
files.
D) An organisational culture which allows one geek to do a special one-off 
operation, outside his specialist area, without another geek casting a 
skeptical eye over the procedure or results.
E) Software which sanitises the data it put into the database very carefully, 
but didn't expect anything else to mess with the database.
F) Original writers of the software no longer work for the company.  New 
maintainers have never had anything serious go wrong before and didn't know how 
to debug serious problems.  Good enough to do their regular job, not good 
enough to deal with emergencies.

Situation was complicated by happening just after the Christmas holiday, so 
some people thought that Christmas disruption or an end-of-year run was part of 
the problem.
_______________________________________________
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Reply via email to