Re: [sqlite] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On 29 Jun 2017 at 08:01, Eric Grange wrote: >> The sender, however, could be lying, and this needs to be considered > > This is an orthogonal problem: if the sender is sending you data that is > not what it should be, then he could just as well be sending you > well-encoded

Re: [sqlite] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

> The sender, however, could be lying, and this needs to be considered This is an orthogonal problem: if the sender is sending you data that is not what it should be, then he could just as well be sending you well-encoded and well-formed but invalid data, or malware, or confidential/personal data

Re: [sqlite] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On 28 Jun 2017 at 14:20, Rowan Worth wrote: > On 27 June 2017 at 18:42, Eric Grange wrote: > >> So while in theory all the scenarios you describe are interesting, in >> practice seeing an utf-8 BOM provides an extremely >> high likeliness that a file will