Re: [sqlite] How to detect "database disk image malformed" in advance?

[Mario M. Westphal]

> If you have a contrary example, please send me a
> copy of the database file via private email.

I take it from your reply that the integrity_check indeed should reveal
problems in the database file which cause the "disk image malformed" return
code so my logic is OK as it is.

Unfortunately, the database is at a customers site and has about 4 or 5 GB
in size. I doubt I can get this DB to you somehow. 

I asked the user to send me the log files my application automatically
writes. Now the problem looks slightly different.

The diagnosis with the "OK" result was on 11/7 but the error happened on
11/8. So there most likely where a lot of database activity in-between,
although no disk problems or other issues. If integrity_check would have
revealed problem but did not, the database seems to have become corrupted
between these two events. I'll ask him to run the diagnosis again and check
if the integrity_check reports something.

I test the result for the "OK" result and is something different is
returned, I write the first 100 rows into a log file. Maybe we see something
there.

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] How to detect "database disk image malformed" in advance?

[Simon Slavin]

On 8 Nov 2013, at 3:06pm, Mario M. Westphal  wrote:

> The first phase in my analysis checks the data in the database for logical
> problems, business logic issues etc.
> After this phase has been completed, my diagnosis function runs:
> 
> 1. PRAGMA integrity_check(100)
> 2. VACUUM
> 3. ANALYZE

I would move just the integrity_check to before your business-logic check.  And 
if that stage notices any problems, your program should abort there, since 
doing anything else to the database at that point will only mess it up further.

If you want the ultimate in cleanness and neatness, ANALYZE should be before 
VACUUM, but it won't make that much difference.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] How to detect "database disk image malformed" in advance?

[Richard Hipp]

On Fri, Nov 8, 2013 at 10:06 AM, Mario M. Westphal  wrote:

>
> 1. PRAGMA integrity_check(100)
> 2. VACUUM
> 3. ANALYZE
>
> My idea was that these routines should reveal problems reading from or
> writing to the database file.
>
> Today I had a case where my diagnosis considered a database as OK (all
> SQLite functions used in the diagnosis returned SQLITE_OK), but the
> database
> still runs into the dreaded "disk image malformed" problem shortly
> afterwards :-(
>

You should never get an SQLITE_CORRUPT error after querying a database file
that is unchanged since a successful PRAGMA integrity_check.  If you have a
contrary example, please send me a copy of the database file via private
email.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] How to detect "database disk image malformed" in advance?

[Mario M. Westphal]

I have implemented diagnostic routines which allow my users to check the
database for problems.
If something bad happens to a database (which may be not the fault of SQLite
at all, e.g. a network problem or disk problem) I want to detect this as
early as possible in order to inform the user. This allows them to restore
their backups before too much data is lost.

The first phase in my analysis checks the data in the database for logical
problems, business logic issues etc.
After this phase has been completed, my diagnosis function runs:

1. PRAGMA integrity_check(100)
2. VACUUM
3. ANALYZE

My idea was that these routines should reveal problems reading from or
writing to the database file.

Today I had a case where my diagnosis considered a database as OK (all
SQLite functions used in the diagnosis returned SQLITE_OK), but the database
still runs into the dreaded "disk image malformed" problem shortly
afterwards :-(

My question: Is there a API function etc. which performs a reliable test of
all database pages and other structures? Something that will detect that
"disk image malformed" will be returned when some areas of the database are
updated?


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users