Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On Tue, Jun 27, 2017 at 4:02 PM, Keith Medcalf wrote: > > > If an implementation "uses" 8 bits for ASCII text (as opposed to > > hardware storage which is never less than 8 bits for a single C char, > > AFAIK), then it is not a valid ASCII implementation, i.e. does not > >

[sqlite] CSV Extension Typo ...

sqlite3x.c:206636:18: warning: implicit declaration of function 'csv_read_one_field'; did you mean 'csv_read_one_field_ext'? [-Wimplicit-function-declaration] return csv_read_one_field(p); ^~ csv_read_one_field_ext

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

> If an implementation "uses" 8 bits for ASCII text (as opposed to > hardware storage which is never less than 8 bits for a single C char, > AFAIK), then it is not a valid ASCII implementation, i.e. does not > interpret ASCII according to its definition. The whole point of > specifying a format

Re: [sqlite] INSERT ... VALUES / want to "skip" default values

If you have to provide 4 values then the way you can use null to do that is to add in a trigger to set the default, since NULL _is_ a value and _is_ legal for that field. CREATE TRIGGER test_populate_b AFTER INSERT ON test WHEN new.b is null BEGIN UPDATE test SET b = '-' WHERE

Re: [sqlite] INSERT ... VALUES / want to "skip" default values

Single quotes should be used for strings, so DEFAULT '-' Not quite sure what you're asking. Do you mean how to insert defaults in general? INSERT INTO test (a, c, d) VALUES ('field a', 'field c', 'field d'); will get you a b c d -- -- --

Re: [sqlite] INSERT ... VALUES / want to "skip" default values

On 27 Jun 2017, at 8:13pm, Robert M. Münch wrote: > CREATE TABLE test(a, b DEFAULT "-", c, d) > > Now I would like to use > > INSERT VALUES(a,?,c,d) > > Where ? is something that the default value is used and not the provided > value. Is this possible at all?

Re: [sqlite] UTF8-BOM not disregarded in CSV import

On Tue, Jun 27, 2017 at 4:18 AM, Richard Hipp wrote: > The CSV import feature of the SQLite command-line shell expects to > find UTF-8. It does not understand other encodings, and I have no > plans to add converters for alternative encodings any time soon. > > The latest version

[sqlite] INSERT ... VALUES / want to "skip" default values

Hi, I have a table like: CREATE TABLE test(a, b DEFAULT "-", c, d) Now I would like to use INSERT VALUES(a,?,c,d) Where ? is something that the default value is used and not the provided value. Is this possible at all? Viele Grüsse. -- Robert M. Münch, CEO M: +41 79 65 11 49 6 Saphirion

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On Tue, 2017-06-27 at 16:38 +0200, Eric Grange wrote: > > > > ASCII / ANSI is a 7-bit format. > ASCII is a 7 bit encoding, but uses 8 bits in just about any > implementation > out there. I do not think there is any 7 bit implementation still > alive > outside of legacy mode for low-level wire

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

> ASCII / ANSI is a 7-bit format. ASCII is a 7 bit encoding, but uses 8 bits in just about any implementation out there. I do not think there is any 7 bit implementation still alive outside of legacy mode for low-level wire protocols (RS232 etc.). I personally have never encountered a 7 bit ASCII

Re: [sqlite] UTF8-BOM not disregarded in CSV import

Thank you. From: sqlite-users on behalf of Richard Hipp Sent: Tuesday, June 27, 2017 5:18:51 AM To: SQLite mailing list Subject: Re: [sqlite] UTF8-BOM not disregarded in CSV import The CSV import

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On 27 Jun 2017, at 7:12am, Rowan Worth wrote: > In fact using this assumption we could dispense with the BOM entirely for > UTF-8 and drop case 5 from the list. If you do that, you will try to process the BOM at the beginning of a UTF-8 stream as if it is characters. > So my

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On Tue, 2017-06-27 at 12:42 +0200, Eric Grange wrote: > In the real world, text files are heavily skewed towards 8 bit > formats, > meaning just three cases dominate the debate: > - ASCII / ANSI > - utf-8 with BOM > - utf-8 without BOM ASCII / ANSI is a 7-bit format.

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

> In case 7 we have little choice but to invoke heuristics or defer to the > user, yes? Yes in theory, but "no" in the real world, or rather "not in any way that matters" In the real world, text files are heavily skewed towards 8 bit formats, meaning just three cases dominate the debate: - ASCII

Re: [sqlite] UTF8-BOM not disregarded in CSV import

The CSV import feature of the SQLite command-line shell expects to find UTF-8. It does not understand other encodings, and I have no plans to add converters for alternative encodings any time soon. The latest version of trunk skips over a UTF-8 BOM at the beginning of the input file. -- D.

Re: [sqlite] UTF8-BOM not disregarded in CSV import

Hello, On 2017-06-26 17:26, Scott Robison wrote: +1 FAQ quote: Q: When a BOM is used, is it only in 16-bit Unicode text? A: No, a BOM can be used as a signature no matter how the Unicode text is transformed: UTF-16, UTF-8, or UTF-32. Q: How I should deal with BOMs? A: Here are some

Re: [sqlite] UTF8-BOM not disregarded in CSV import

On 2017-06-26 15:01, jose isaias cabrera wrote: I have made a desicion to always include the BOM in all my text files whether they are UTF8, UTF16 or UTF32 little or big endian. I think all of us should also. I'm sorry, if I introduced ambiguity, but I had described SQLite's and SQLite

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On Tue, 2017-06-27 at 01:14 -0600, Scott Robison wrote: > The original issue was two of the largest companies in the world > output the > Byte Encoding Mark(TM)(Patent Pending) (or BOM) at the beginning of > UTF-8 > encoded text streams, and it would be friendly for the SQLite3 shell > to > skip

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On Tue, 2017-06-27 at 01:14 -0600, Scott Robison wrote: > On Jun 27, 2017 12:13 AM, "Rowan Worth" wrote: > > I'm sure I've simplified things with this description - have I missed > something crucial? Is the BOM argument about future proofing? Are we > worried about EBCDIC? Is my

[sqlite] Any reason for sqlite3changeset_concat not using const parameters ?

Hello ! I'm trying to use sqlite3 session extension with C++ and I'm getting errors because sqlite3changeset_concat aren't using const parameters for input parameters like sq_sqlite3_session_invert does, looking at the code it doesn't seem that it modifies it's input parameters. Cheers !

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On Jun 27, 2017 12:13 AM, "Rowan Worth" wrote: I'm sure I've simplified things with this description - have I missed something crucial? Is the BOM argument about future proofing? Are we worried about EBCDIC? Is my perspective too anglo-centric? The original issue was two of the

Re: [sqlite] [OT] UTF8-BOM and text encoding detection (was: UTF8-BOM not disregarded in CSV import)

On 26 June 2017 at 19:03, Eric Grange wrote: > No BOM = you have to fire a whole suite of heuristics or present the user > with choices he/she will not understand. > Requiring heuristics to determine text encoding/codepage exists regardless of whether BOM is used since the