Some more information: A co-worker managed to get an copy of the db by as interpreted by jffs2dump of the file system, that was extracted by the jffs2dump python script (from git hub). It is interesting that it is also corrupt but in a different way.
sqlite> select * from config where id ='isp_de_mode'; (11) database corruption at line 70244 of [fd0a50f079] (11) statement aborts at 9: [select * from config where id ='isp_de_mode';] database disk image is malformed Error: database disk image is malformed sqlite> select id from config where id ='isp_de_mode'; isp_de_mode sqlite> select count(id) from config where id ='isp_de_mode'; 1 sqlite> select count(1), id from config group by id order by 1 desc limit 2; 2|isp_de_mode 1|audio_bitrate_s0 sqlite> .schema CREATE TABLE config( id varchar(255) PRIMARY KEY,file varchar(255),xpath varchar (255), value varchar(255), venc_switch_xpath varchar(255), apply_cmd varchar(255), cacheIsDirty integer default -1 ); sqlite> sqlite> pragma integrity_check; row 1275 missing from index sqlite_autoindex_config_1 row 1276 missing from index sqlite_autoindex_config_1 row 1277 missing from index sqlite_autoindex_config_1 row 1346 missing from index sqlite_autoindex_config_1 row 1347 missing from index sqlite_autoindex_config_1 row 1348 missing from index sqlite_autoindex_config_1 row 1349 missing from index sqlite_autoindex_config_1 row 1350 missing from index sqlite_autoindex_config_1 row 1351 missing from index sqlite_autoindex_config_1 row 1352 missing from index sqlite_autoindex_config_1 row 1353 missing from index sqlite_autoindex_config_1 row 1354 missing from index sqlite_autoindex_config_1 row 1367 missing from index sqlite_autoindex_config_1 row 1372 missing from index sqlite_autoindex_config_1 wrong # of entries in index sqlite_autoindex_config_1 regards, Adam On Tue, Jan 12, 2016 at 12:01 PM, Adam Devita <adevita at verifeye.com> wrote: > Good day, > Thank you for some avenues of investigation. > > Q: Does your program examine the codes returned by SQLite3 calls and > check to see that they are all returning SQLITE_OK ? > > A1: The upgrade process is done by a script. It isn't error checking & > executes queries via the shell tool. I followed up with the script > folks and they got a pull of the file system log from the failed > unit. There are 2 queries that run to update new.db from data in > old.db, and appear to execute and update multiple rows each. > > A2: From inspecting the normal op program (at the version of the > firmware tag) It looks fairly good. > The only return code not checked is a on commit/rollback that is > after a failed prepare statement or one particular function that takes > a checked input, performs bind, step, reset at one point. > There may be a vulnerability at one point to sqlite bind text that has > a null input for the string. The docs say such an oversight would > return SQLITE_MISUSE so the step would not happen. > > Q: Does the script shut down the program and wait for the program to > quit before it starts running its own commands, or are the two things > done independently ? > > A: The script does not check for a response but it waits. It would be > very unusual for their to be pending writes to the old db (or user > requests) while new db is extracting the information. I suppose WAL > mode would be safer here. > > The upgrade Script doesn't check values of the returned by the shell > tool. That said. Since the alg is :update new.db from old.db, replace > old.db file with new.db. New.db's default value for cache is "reload > value from config files", the script failure would leave new.db in a > state that, as long as not corrupt, would simply reload from the > config files. > > > Q: Are both programs running on the computer with the database stored > on a hard disk, or is anything accessing the database across a network > ? > A: All operations happen on locally stored flash or memory only. (The > new.db in an update is uncompressed to ram, once updates are complete, > then it gets copied to the local flash.) > The only network ops are "upload upgrade package to remote device, > then tell it to use it". There are no sqlite operations over a > network. > > > --As per DRH's instruction---------- > f:\Users\Adam>sqlite3.exe system.bad > SQLite version 3.10.0 2016-01-06 11:01:07 > Enter ".help" for usage hints. > sqlite> .log stdout > sqlite> pragma integrity_check; > (11) database corruption at line 58034 of [fd0a50f079] > (11) database disk image is malformed > Error: database disk image is malformed > sqlite> > --------------------- > > showdb: I assume this is a linux tool? Where would I pull that from? > Our device doesn't have all the utilities but I can put a copy of the > bad db on a development linux environment for further tests. > > On Tue, Jan 12, 2016 at 10:55 AM, Richard Hipp <drh at sqlite.org> wrote: >> On 1/12/16, Adam Devita <adevita at verifeye.com> wrote: >>> >>> Shell Tool Observations: >>> SQLite version 3.8.4.3 2014-04-03 16:53:12 >>> Enter ?.help? for usage hints. >> >> If you first do: ".log stdout" before doing the "PRAGMA >> integrity_check", you might get some better diagnostics. Or maybe >> not. In any event, it doesn't hurt to try. >> >>> sqlite> pragma integrity_check; >>> Error: database disk image is malformed >>> >> >> Other things to try: >> >> ./configure; make showdb; >> ./showdb your-corrupt-db-file.db dbheader >> ./showdb your-corrupt-db-file.db pgidx >> >> There is a lot of other things you can do with the showdb program. >> Type "./showdb" with no argument for a very terse summary. I, for >> one, would be very interested in seeing the output of the above two >> commands. >> >> -- >> D. Richard Hipp >> drh at sqlite.org >> _______________________________________________ >> sqlite-users mailing list >> sqlite-users at mailinglists.sqlite.org >> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > > > > -- > -------------- > VerifEye Technologies Inc. > 151 Whitehall Dr. Unit 2 > Markham, ON > L3R 9T1 -- -------------- VerifEye Technologies Inc. 151 Whitehall Dr. Unit 2 Markham, ON L3R 9T1
