[sqlite] shell edit() trips on Windows newline kink
Upon thinking further about this bug, I realize that "correct" behavior is not obvious and might warrant some discussion. At first, I questioned whether it is appropriate to handle data differently depending upon whether it is typed as BLOB or text. Maybe the edit() function needed a flag for this. However, since this can always be affected, either way, by a type cast, there is no such need. And, given this (effective) option, it seems that treating a text-typed value as being subject to the usual text transformations [a], is perfectly fine, and convenient when they may be needed in order for a user's favored text-editing program to be used. [b] [a. The CRLF <-> "\n" translation is just one such transformation. Character coding could be another. ] [b. On Windows, notepad.exe is always available and "on the PATH", which might make it favored sometimes. But it handles LF as line boundaries very badly. ] I notice an upcoming change to the edit() function, checked in a few days ago, which appears to make the allow-text-transformation decision based upon whether the original input contained any CRLF pairs. I would urge not adopting this approach because it presupposes something not necessarily true, that the input to edit() reliably signifies what should happen to its output. That would not be true if, for example, a single "line" of text without a terminating CRLF were to be edited to become multiples "lines" containing CRLF pairs (which might number one less than "lines".) I submit that a better approach is to embrace the possibility of newline transformation for text-typed edit() input, leaving in the user's hands whether to block that by type-casting to BLOB. That is an option that is eliminated with the recent check-in. Best regards, -- Larry ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] shell edit() trips on Windows newline kink
I was experimenting today with a v3.25.0 variant and encountered this bug,
on my Windows 10 system, when I used the SQLite CLI shell's edit()
function. Because I had put a newline in the text with the invoked editor,
and it was written and read back as a text file, the following code got
unhappy:
fseek(f, 0, SEEK_END);
sz = ftell(f);
...
x = fread(p, 1, sz, f);
...
if( x!=sz ){
sqlite3_result_error(context, "could not read back the whole file", -1);
goto edit_func_end;
}
The problem is that, on the Windows platform, newlines are stored on disk
as CR LF character pairs but, for text mode file I/O, are translated to a
single LF character in the C buffered file I/O library. In the above
check, this causes x to be less than sz by the number of newlines so
translated. There is an additional related problem whereby the 0
terminator on the read-in string (in text mode) is put in the wrong place.
Here is fossil diff output showing (what I believe to be) an effective fix,
which I have tested:
===
if( bBin ){
x = fread(p, 1, sz, f);
}else{
x = fread(p, 1, sz, f);
-p[sz] = 0;
+p[x] = 0;
}
- fclose(f);
- f = 0;
- if( x!=sz ){
+ if( ftell(f)!=sz ){
sqlite3_result_error(context, "could not read back the whole file",
-1);
goto edit_func_end;
}
===
(I omit line number marking because my unrelated changes make them
inapplicable to source in the SQLite sources.)
Cheers,
--
Larry Brasfield
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
