On 07/15/2015 12:05 AM, Richard Hipp wrote:
> The plan is to release SQLite version 3.8.11 on or about the end of July.
>
> The current code is passing all tests that we have run against it.
> Some soak tests are still running. There are quite a few
> cross-platform tests (running on PPC, Sparc, etc) that have yet to be
> started, but which should not offer any trouble. The current code is
> stable and perfectly appropriate for beta testing.
>
> Please test the latest SQLite snapshot in your products and report any
> problems to this list, or directly to me.
I've run the address and undefined behaviour sanitizer (+ usual hardening and
bug finding flags from Debian) from GCC 4.9.2 on Debian Jessie on this fossil
checkout: a73d7128fbca8dde5e90bd46ee915e39ae07dd1f 2015-07-14 22:43:37 UTC
(the snapshots tarballs don't seem to include the tests).
I found some issues, but they look more like bugs in the sanitizer or the test
runner than bugs in sqlite, but I'm posting it here just to double-check:
$ ./configure CFLAGS="-g -O2 -Werror=array-bounds -Werror=clobbered
-Werror=volatile-register-var -Werror=implicit-function-declaration -fPIE
-fstack-protector-strong -Wformat -Werror=format-security -fsanitize=address
-fsanitize=undefined -fno-omit-frame-pointer" CPPFLAGS="-D_FORTIFY_SOURCE=2"
LDFLAGS="-fPIE -pie -Wl,-z,relro -Wl,-z,now -fsanitize=address
-fsanitize=undefined -pthread" --enable-debug --enable-threadsafe
$ make clean
$ make -j10
$ make test -j10
1) unknown-crash (might be due to some alignment requirements in asan):
fuzzdata3.db: Database fuzz as of 2015-06-24
fuzzdata3.db: 0%
10%=================================================================
==1050==ERROR: AddressSanitizer: unknown-crash on address 0x6150000abb41 at pc
0x7fa3dd350ec9 bp 0x7ffd7b8ec180 sp 0x7ffd7b8ec178
READ of size 385 at 0x6150000abb41 thread T0
#0 0x7fa3dd350ec8 in memcpy /usr/include/x86_64-linux-gnu/bits/string3.h:51
#1 0x7fa3dd350ec8 in rebuildPage /home/edwin/skylable/sqlite/sqlite3.c:60141
#2 0x7fa3dd3f28b3 in editPage /home/edwin/skylable/sqlite/sqlite3.c:60370
#3 0x7fa3dd3f28b3 in balance_nonroot
/home/edwin/skylable/sqlite/sqlite3.c:61299
#4 0x7fa3dd3f486e in balance /home/edwin/skylable/sqlite/sqlite3.c:61547
#5 0x7fa3dd40842f in sqlite3BtreeInsert
/home/edwin/skylable/sqlite/sqlite3.c:61737
#6 0x7fa3dd48c765 in sqlite3VdbeExec
/home/edwin/skylable/sqlite/sqlite3.c:76236
#7 0x7fa3dd4c4746 in sqlite3Step /home/edwin/skylable/sqlite/sqlite3.c:70639
#8 0x7fa3dd4c4746 in sqlite3_step
/home/edwin/skylable/sqlite/sqlite3.c:70700
#9 0x7fa3dd2665f1 in runSql /home/edwin/skylable/sqlite/test/fuzzcheck.c:617
#10 0x7fa3dd262bb6 in main /home/edwin/skylable/sqlite/test/fuzzcheck.c:975
#11 0x7fa3da929b44 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x3452621b44)
#12 0x7fa3dd264343 (/home/edwin/skylable/sqlite/fuzzcheck+0x3bc343)
0x6150000abb80 is located 0 bytes to the right of 512-byte region
[0x6150000ab980,0x6150000abb80)
allocated by thread T0 here:
#0 0x7fa3dbdfc73f in malloc
(/usr/lib/x86_64-linux-gnu/libasan.so.1+0x3f2205473f)
#1 0x7fa3dd351631 in sqlite3MemMalloc
/home/edwin/skylable/sqlite/sqlite3.c:17235
#2 0x7fa3dd2d4e98 in mallocWithAlarm
/home/edwin/skylable/sqlite/sqlite3.c:20909
#3 0x7fa3dd2d4e98 in sqlite3Malloc
/home/edwin/skylable/sqlite/sqlite3.c:20940
#4 0x7fa3dd2ea741 in pcache1Alloc
/home/edwin/skylable/sqlite/sqlite3.c:40705
#5 0x7fa3dd2eab62 in sqlite3PageMalloc
/home/edwin/skylable/sqlite/sqlite3.c:40843
#6 0x7fa3dd2eab62 in sqlite3PagerSetPagesize
/home/edwin/skylable/sqlite/sqlite3.c:45907
#7 0x7fa3dd4196d2 in sqlite3BtreeOpen
/home/edwin/skylable/sqlite/sqlite3.c:56012
#8 0x7fa3dd52fe42 in openDatabase
/home/edwin/skylable/sqlite/sqlite3.c:132083
#9 0x7fa3dd262b64 in main /home/edwin/skylable/sqlite/test/fuzzcheck.c:965
#10 0x7fa3da929b44 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x3452621b44)
SUMMARY: AddressSanitizer: unknown-crash
/usr/include/x86_64-linux-gnu/bits/string3.h:51 memcpy
Shadow bytes around the buggy address:
0x0c2a8000d710: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2a8000d720: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2a8000d730: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2a8000d740: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c2a8000d750: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x0c2a8000d760: 00 00 00 00 00 00 00 00[00]00 00 00 00 00 00 00
0x0c2a8000d770: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2a8000d780: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2a8000d790: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2a8000d7a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c2a8000d7b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Contiguous container OOB:fc
ASan internal: fe
==1050==ABORTING
Makefile:1047: recipe for target 'fuzztest' failed
2) heap-use-after-free
This might be just the test runner and not sqlite itself, I'm not sure:
Time: capi2.test 25 ms
=================================================================
==2330==ERROR: AddressSanitizer: heap-use-after-free on address 0x6180003b58dc
at pc 0x7f5bb4894d49 bp 0x7ffd1e988d20 sp 0x7ffd1e988d18
READ of size 4 at 0x6180003b58dc thread T0
#0 0x7f5bb4894d48 in sqlite3SafetyCheckSickOrOk
/home/edwin/skylable/sqlite/sqlite3.c:25082
#1 0x7f5bb49b1174 in sqlite3Close
/home/edwin/skylable/sqlite/sqlite3.c:130253
#2 0x7f5bb468f520 in sqlite_test_close
/home/edwin/skylable/sqlite/src/test1.c:715
#3 0x7f5bb2d30693 in TclInvokeStringCommand
(/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f21440693)
#4 0x7f5bb2d32a86 in TclNRRunCallbacks
(/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f21442a86)
#5 0x7f5bb2dd5c41 (/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f214e5c41)
#6 0x7f5bb2dd330e (/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f214e330e)
#7 0x7f5bb2d32a86 in TclNRRunCallbacks
(/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f21442a86)
#8 0x7f5bb2d4ed84 (/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f2145ed84)
#9 0x7f5bb2d32a86 in TclNRRunCallbacks
(/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f21442a86)
#10 0x7f5bb2d337ba (/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f214437ba)
#11 0x7f5bb2debf8f in Tcl_FSEvalFileEx
(/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f214fbf8f)
#12 0x7f5bb2dea996 in Tcl_EvalFile
(/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f214fa996)
#13 0x7f5bb467f245 in main /home/edwin/skylable/sqlite/src/tclsqlite.c:3885
#14 0x7f5bb1871b44 in __libc_start_main
(/lib/x86_64-linux-gnu/libc.so.6+0x3452621b44)
#15 0x7f5bb467f4b3 (/home/edwin/skylable/sqlite/testfixture+0x4d74b3)
0x6180003b58dc is located 92 bytes inside of 816-byte region
[0x6180003b5880,0x6180003b5bb0)
freed by thread T0 here:
#0 0x7f5bb30fc527 in __interceptor_free
(/usr/lib/x86_64-linux-gnu/libasan.so.1+0x3f22054527)
#1 0x7f5bb479a9da in sqlite3_free
/home/edwin/skylable/sqlite/sqlite3.c:21118
#2 0x7f5bb49b141b in sqlite3Close
/home/edwin/skylable/sqlite/sqlite3.c:130290
#3 0x7f5bb468f520 in sqlite_test_close
/home/edwin/skylable/sqlite/src/test1.c:715
#4 0x7f5bb2d30693 in TclInvokeStringCommand
(/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f21440693)
previously allocated by thread T0 here:
#0 0x7f5bb30fc73f in malloc
(/usr/lib/x86_64-linux-gnu/libasan.so.1+0x3f2205473f)
#1 0x7f5bb4894c81 in sqlite3MemMalloc
/home/edwin/skylable/sqlite/sqlite3.c:17235
#2 0x7f5bb47a012a in mallocWithAlarm
/home/edwin/skylable/sqlite/sqlite3.c:20909
#3 0x7f5bb47a012a in sqlite3Malloc
/home/edwin/skylable/sqlite/sqlite3.c:20940
#4 0x7f5bb47ac01e in sqlite3MallocZero
/home/edwin/skylable/sqlite/sqlite3.c:21238
#5 0x7f5bb49bcac7 in openDatabase
/home/edwin/skylable/sqlite/sqlite3.c:131996
#6 0x7f5bb468a09c in test_open /home/edwin/skylable/sqlite/src/test1.c:3875
#7 0x7f5bb2d32a86 in TclNRRunCallbacks
(/usr/lib/x86_64-linux-gnu/libtcl8.6.so+0x3f21442a86)
SUMMARY: AddressSanitizer: heap-use-after-free
/home/edwin/skylable/sqlite/sqlite3.c:25082 sqlite3SafetyCheckSickOrOk
Shadow bytes around the buggy address:
0x0c308006eac0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c308006ead0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c308006eae0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c308006eaf0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
0x0c308006eb00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x0c308006eb10: fd fd fd fd fd fd fd fd fd fd fd[fd]fd fd fd fd
0x0c308006eb20: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c308006eb30: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c308006eb40: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c308006eb50: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c308006eb60: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Heap right redzone: fb
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack partial redzone: f4
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Contiguous container OOB:fc
ASan internal: fe
==2330==ABORTING
3) possible undefined behaviour in date conversion
Running 'make test' reports one undefined behaviour in fuzzcheck:
sqlite3.c:15778:15: runtime error: signed integer overflow: 3328620 * 36525
cannot be represented in type 'int'
4) array-bounds warning from GCC
I got some (probably false positives) from the array-bounds check, which can be
fixed by adding another assert and changing the type of the index to unsigned
(GCC probably thinks that the ++ can cause the signed counter to overflow to
negative):
sqlite3.c:51032:32: error: array subscript is below array bounds
[-Werror=array-bounds]
struct Sublist *p = &aSub[iSub];
^
sqlite3.c:51021:32: error: array subscript is above array bounds
[-Werror=array-bounds]
struct Sublist *p = &aSub[iSub];
^
And on this line too:
pToplevel->cookieValue[iDb] = db->aDb[iDb].pSchema->schema_cookie;
I've attached a patch that fixes these warnings from GCC
Best regards,
--Edwin
