Re: [whatwg] SQL API error handling
On Thu, 18 Oct 2007, Scott Hess wrote: We don't have error codes for the DOM has become corrupted or for the Window object's properties have become corrupted, why do we need one for the Database's contents are corrupted? Why can't the database contents simply not be corrupted in the first place? One difference is that the database is persistent. If the DOM became corrupted, and you refresh the page or restart the browser, there's a good chance that your DOM will no longer be corrupted. If your Database is corrupted and you refresh the page or restart the browser, your Database is still corrupted. I think the UA would be doing a poor job if it detected a database was corrupt and didn't do something about it before the next time it started. You almost need an oncorruption handler (or maybe something more general). Detecting that a database is corrupt from within an individual statement's callback really isn't all that interesting, there's nothing statement-specific that you'd want to do at that point. But your oncorruption handler could do things like delete the corrupt database and the three other databases which no longer make sense without it, and reload the page. The default handler could just delete the database in question. Yeah. We'll look into this in the next version, I guess. On Thu, 18 Oct 2007, K�i�tof �elechovski wrote: It is normal that the disk gets full; the probability of this event is 1 for a consumer disk. (Admittedly, the operating system can cry aloud and refuse to do anything when the startup volume is about to overflow but the database could be stored on another volume that is not protected that way). OTOH, it is an exceptional situation that data become corrupt. Therefore these two situations are not equivalent. Indeed. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] SQL API error handling
On 10/17/07, Ian Hickson [EMAIL PROTECTED] wrote: On Mon, 15 Oct 2007, Brady Eidson wrote: In some embedded (and client-server) database implementations - including SQLite - continuing to operate on a database that is known to be corrupt can lead to the process crashing. Unlike the CPU core just overheated case, it is a dangerous state software can help avoid. Ok... but why can't the software simply avoid corrupting the database in the first place? We don't have error codes for the DOM has become corrupted or for the Window object's properties have become corrupted, why do we need one for the Database's contents are corrupted? Why can't the database contents simply not be corrupted in the first place? One difference is that the database is persistent. If the DOM became corrupted, and you refresh the page or restart the browser, there's a good chance that your DOM will no longer be corrupted. If your Database is corrupted and you refresh the page or restart the browser, your Database is still corrupted. On Tue, 16 Oct 2007, Scott Hess wrote: I think that if the user agent did detect corruption and nuke the database from orbit, then it would be reasonable for the user agent to invalidate all outstanding database handles. But that kind of thing would seem to be something really beyond the spec to deal with. It seems like at that point the most appropriate action to take would be to refresh the page and start from scratch, rather than expecting the app to somehow handle the problem. I agree. You almost need an oncorruption handler (or maybe something more general). Detecting that a database is corrupt from within an individual statement's callback really isn't all that interesting, there's nothing statement-specific that you'd want to do at that point. But your oncorruption handler could do things like delete the corrupt database and the three other databases which no longer make sense without it, and reload the page. The default handler could just delete the database in question. -scott
Re: [whatwg] SQL API error handling
It is normal that the disk gets full; the probability of this event is 1 for a consumer disk. (Admittedly, the operating system can cry aloud and refuse to do anything when the startup volume is about to overflow but the database could be stored on another volume that is not protected that way). OTOH, it is an exceptional situation that data become corrupt. Therefore these two situations are not equivalent. Best regards Chris -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brady Eidson Sent: Thursday, October 18, 2007 4:51 AM To: Ian Hickson Cc: WHATWG Subject: Re: [whatwg] SQL API error handling Corruption in the database isn't the fault of the user agent. I consider it at the same level as corruption on disk - or even a full disk! As I consider these to be similar, I assert that database corruption is an external force the user agent - and potentially the application it hosts - needs to be ready to handle.
Re: [whatwg] SQL API error handling
Aside: the net gain from shortening INVALID_STATE_ERROR to INVALID_STATE_ERR is 2/19; is it worth sacrificing readability for 11% length? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Brady Eidson Sent: Wednesday, October 17, 2007 12:08 AM To: Geoffrey Garen Cc: Scott Hess; Maciej Stachowiak; Ian Hickson; WHATWG Subject: Re: [whatwg] SQL API error handling INVALID_STATE_ERR is already overloaded and I think the corruption case is a particularly problematic one - I guess what I'm after is a special condition for the corruption case built in to the spec.
Re: [whatwg] SQL API error handling
On Wed, 17 Oct 2007, Křištof Želechovski wrote: Aside: the net gain from shortening INVALID_STATE_ERROR to INVALID_STATE_ERR is 2/19; is it worth sacrificing readability for 11% length? The exception names are decided by the W3C's DOM working group(s), I recommend raising the issue with them. HTML5 doesn't define any new exception codes. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] SQL API error handling
On Mon, 15 Oct 2007, Brady Eidson wrote: In some embedded (and client-server) database implementations - including SQLite - continuing to operate on a database that is known to be corrupt can lead to the process crashing. Unlike the CPU core just overheated case, it is a dangerous state software can help avoid. Ok... but why can't the software simply avoid corrupting the database in the first place? We don't have error codes for the DOM has become corrupted or for the Window object's properties have become corrupted, why do we need one for the Database's contents are corrupted? Why can't the database contents simply not be corrupted in the first place? As I see it there are two ways the database could get corrupted -- software error (bugs, spec-wise illegal behavious) and uncontrollable external factors (cosmic rays, hardware failure, overheating, the user hexediting the database file...). The uncontrollable external factors can affect _anything_. The DOM could get corrupted in RAM by cosmic rays while the browser is running. A chip could overheat and return bogus data randomly for a canvas getPixelData() call. The drive could intermittently fail, preventing the cookies from being read completely. We don't have any sort of handling for _any_ of these problems right now, and I don't see why the database would be a special case. If we want to handle these problems, we should find a generic solution, IMHO (though it might involve errorCode values for the Database object as well). So this leaves the software error case. Specifying what should happen in the case of a software error seems to be about as pointless as regulating how outlaws should behave. By definition, the rules aren't being followed -- how can we provide rules to decide how things work? How the user agent handles the problem long term (ask the user what to do, silently delete and recreate it, let the database continue to be corrupt, etc) may not need to be specified, but perhaps it would be prudent to change the spec to at least suggest that if a database becomes known to be corrupt, operations on all open handles to that database should start throwing INVALID_STATE_ERR exceptions. I think it would be reasonable for the UA to start raising UA-specific exceptions in this case, but I don't see why it's any different than the UA detecting that the network stack is corrupted and raising exceptions on calls to XMLHttpRequest. On Tue, 16 Oct 2007, Brady Eidson wrote: Say corruption is detected and we present this error to the script somehow - an exception for example. Then we nuke the database and recreate it from scratch. Unless the error we presented to the script was explicit about the condition, the script might not know that we just deleted the entire database and it is now empty. They might execute some query that will succeed on the new, fresh database and they didn't even realize we reset the database behind their back. They might think that some cached data they have in memory is still persistent in the database, instead of knowing they could recover a little by writing it back out. It would be nice to have a way to indicate to the script There was a catastrophic event and we reset your database, assume you're starting over from scratch. But what if the catastrophic failure was that the Window object's properties got wiped? Should we fire an event for that case and let the script repopulate the Window object from the database? I guess I just don't understand the expected scenarios where failure is to occur in such a localised case as to make this useful. On Tue, 16 Oct 2007, Brady Eidson wrote: You are (rightly) reading very specifically into what I am saying, whereas what I'm trying to get at is still vague and general. I'm also trying to read very specifically into what you're saying, since I have to write very specifically into the spec. :-) Let me take a step back and try to frame it at a higher level - A page opens a valid database handle. - Some script uses that database handle - successfully - Some external event happens on the client machine - database corruption, the user deletes the database from the user agent's database management mode, gamma rays corrupt a single bit on the disk, or whatever. This event renders the database unusable. - Some action is taken to reset the database so that it is usable - lets pretend the resolution is always recreating an empty database from scratch - The script knows something wrong happened - it has a completely 100% generic error on its query. But it is unaware of the nature of this event and its resolution. It decides to execute a new sql statement, and the value of this statement (from the script's perspective) is reliant on previously established values in the database. The statement coincidentally succeeds even with the new empty database. For further argument, lets
Re: [whatwg] SQL API error handling
I don't feel quite passionately enough about this issue to pursue it with much more vigor, but will make one remaining point: On Oct 17, 2007, at 6:58 PM, Ian Hickson wrote: On Mon, 15 Oct 2007, Brady Eidson wrote: In some embedded (and client-server) database implementations - including SQLite - continuing to operate on a database that is known to be corrupt can lead to the process crashing. Unlike the CPU core just overheated case, it is a dangerous state software can help avoid. Ok... but why can't the software simply avoid corrupting the database in the first place? Let's make the assumption that all User Agents are infallible. There will never be a software bug in code that is specifically part of the browser that is exposed to the hosted application. This is an unrealistic assumption, but lets make it for the sake of argument. However, some things are outside of the user agent's complete control. Say disk space, for example. We just added an error code that means the disk is full. The disk is an external entity the user agent doesn't have complete control over, and therefore the user agent - and potentially applications it hosts - have to be ready to handle these external forces. Corruption in the database isn't the fault of the user agent. I consider it at the same level as corruption on disk - or even a full disk! As I consider these to be similar, I assert that database corruption is an external force the user agent - and potentially the application it hosts - needs to be ready to handle. There's a line between the user agent and application. *Obviously* the user agent has to gracefully handle corruption, but I seem to be the only person on the side of the line where the hosted application gets to participate in handling the corruption case. You've already alluded to a list of concerns for version 2 of the spec that can be addressed based on real world experience with version 1 in the wild. Perhaps we can put this concern on the v2 list, and put this thread to rest. ;) Thanks, ~Brady
Re: [whatwg] SQL API error handling
On 10/15/07, Ian Hickson [EMAIL PROTECTED] wrote: On Mon, 15 Oct 2007, Scott Hess wrote: Whoa! I just realized that there's another group, constraint failures. These are statements which will sometimes succeed, sometimes fail. As currently spec'ed, it looks like a constraint failure will cause rollback. This is probably not appropriate, constraint failures are generally expected to be benign and detectable. Arguing against myself, I've seen plenty of code which just treats any errors from something which allows for a constraint failure as a constraint failure, so maybe this also can be lumped under a single big error code. Could you elaborate on this? What would code running into this situation look like? How should we expose it? A common case is wanting to update a row, if present, or insert it, if not. You can handle this in three ways. You can use a transaction to keep things consistent: BEGIN; SELECT COUNT(*) FROM t WHERE id = ?; -- if == 0 INSERT INTO t VALUES (?, ...); -- if == 1 UPDATE t SET c = ?, ... WHERE id = ?; END; [Note that the above gets you the read lock upgraded to write lock case.] This style is generally avoided, because in a server environment, you have four round trips from when the transaction is opened to when it's closed, plus whatever contention for CPU is present at both ends, so it is not great for concurrency. Instead, you can just try the insert and rely on a unique or primary key to cause a constraint violation: INSERT INTO t VALUES (?, ...); -- if constraint violation on id UPDATE t SET c = ?, ... WHERE id = ?; That variant is best if you expect to usually succeed, and sometimes fall through to the update (for instance when inserting a new user record). Otherwise, you could do: UPDATE t SET c = ?, ... WHERE id = ?; -- if no rows affected INSERT INTO t VALUES (?, ...); That might be more appropriate for updating a user record where the user is known to exist but the record is not known to exist (say a record of the last time the user performed a particular operation). The first and third cases should work fine within the spec. --- Another example would be a table like: CREATE TABLE UniqueName ( id INTEGER PRIMARY KEY, name TEXT UNIQUE ); You might run: INSERT INTO UniqueName (id, name) VALUES (null, ?); If it succeeds, the id is in insertId. If there's a constraint failure, the programmer knows that it's because the value provided for name was already take. You can of course rewrite this like: BEGIN; SELECT count(*) FROM UniqueName WHERE name = ?; -- If != 0, fail. INSERT INTO UniqueName (id, name) VALUES (null, ?); END; Using the INSERT directly is the standard idiom for this type of thing, though. --- Unfortunately, offhand I'm not coming up with any cases which aren't fundamentally server-side. Part of the justification for using SQL in the spec was so that developers could use similar/identical code server-side and client-side, but I'll admit that the kinds of cases above are probably no more or less annoying to work around than the differences between any chosen server-side database and SQLite would be. -scott
Re: [whatwg] SQL API error handling
perhaps it would be prudent to change the spec to at least suggest that if a database becomes known to be corrupt, operations on all open handles to that database should start throwing INVALID_STATE_ERR exceptions. I think this is already specified: 3. If transaction has been marked as bad, then raise an INVALID_STATE_ERR exception. ... 7. If the statement execution fails for some reason, transaction must be rolled back and marked as bad. I think you can reasonably consider a statement on a corrupt database to have failed for some reason. Geoff
Re: [whatwg] SQL API error handling
On Oct 16, 2007, at 11:29 AM, Geoffrey Garen wrote: perhaps it would be prudent to change the spec to at least suggest that if a database becomes known to be corrupt, operations on all open handles to that database should start throwing INVALID_STATE_ERR exceptions. I think this is already specified: 3. If transaction has been marked as bad, then raise an INVALID_STATE_ERR exception. ... 7. If the statement execution fails for some reason, transaction must be rolled back and marked as bad. I think you can reasonably consider a statement on a corrupt database to have failed for some reason. After all active transactions are cleared, there is no context that remembers that the database is corrupt, and the next statement to be run would actually attempt to be executed. I suppose user agents can volunteer to remember this and automatically fail the next statement, but it's certainly not specified. ~Brady
Re: [whatwg] SQL API error handling
perhaps it would be prudent to change the spec to at least suggest that if a database becomes known to be corrupt, operations on all open handles to that database should start throwing INVALID_STATE_ERR exceptions. I think this is already specified: 3. If transaction has been marked as bad, then raise an INVALID_STATE_ERR exception. ... 7. If the statement execution fails for some reason, transaction must be rolled back and marked as bad. I think you can reasonably consider a statement on a corrupt database to have failed for some reason. After all active transactions are cleared, there is no context that remembers that the database is corrupt, and the next statement to be run would actually attempt to be executed. I suppose user agents can volunteer to remember this and automatically fail the next statement, but it's certainly not specified. Are you proposing that, once a database has been corrupted, all transactions executed on it should fail, raising an INVALID_STATE_ERR exception, for all time? Once all active transactions are cleared, there's no need to remember that the database was corrupt. The user agent should simply recover from the corruption in an implementation-defined way -- either by deleting the database, performing an error-recovering integrity check, asking the user to install cosmic ray shielding around the house, or something else. Geoff
Re: [whatwg] SQL API error handling
On Oct 16, 2007, at 2:08 PM, Geoffrey Garen wrote: After all active transactions are cleared, there is no context that remembers that the database is corrupt, and the next statement to be run would actually attempt to be executed. I suppose user agents can volunteer to remember this and automatically fail the next statement, but it's certainly not specified. Are you proposing that, once a database has been corrupted, all transactions executed on it should fail, raising an INVALID_STATE_ERR exception, for all time? No, I was proposing that once a database has been corrupted, all transactions executed on it should fail, raising an INVALID_STATE_ERR exception, until the corruption is resolved. But my intentions have changed (read below) Once all active transactions are cleared, there's no need to remember that the database was corrupt. The user agent should simply recover from the corruption in an implementation-defined way -- either by deleting the database, performing an error-recovering integrity check, asking the user to install cosmic ray shielding around the house, or something else. This is great and all, and I suspect its what most user agents would do. Nuke the database from orbit, and all. My concern is about any already-open-database handles. INVALID_STATE_ERR is already overloaded and I think the corruption case is a particularly problematic one - I guess what I'm after is a special condition for the corruption case built in to the spec. Say corruption is detected and we present this error to the script somehow - an exception for example. Then we nuke the database and recreate it from scratch. Unless the error we presented to the script was explicit about the condition, the script might not know that we just deleted the entire database and it is now empty. They might execute some query that will succeed on the new, fresh database and they didn't even realize we reset the database behind their back. They might think that some cached data they have in memory is still persistent in the database, instead of knowing they could recover a little by writing it back out. It would be nice to have a way to indicate to the script There was a catastrophic event and we reset your database, assume you're starting over from scratch. ~Brady
Re: [whatwg] SQL API error handling
It would be nice to have a way to indicate to the script There was a catastrophic event and we reset your database, assume you're starting over from scratch. In general, I'm not sure how useful it is to know that you're starting over from scratch, since any database query needs to check its result. Presumably, an app's behavior in the no data case is the same regardless of *why* there's no data. 99% of the time the behavior will be to reload the data from a server. More importantly, what constitutes a corrupt database, and how to recover from it, are serious implementation details. Some implementations may have error correction algorithms. Others may have backups they can restore. Others may have to wipe the database completely and start over. Still others may not be able to start anything. (For example, the storage medium might have gone bad, or been locked or disconnected.) So, imposing a start over from scratch requirement would hamper some implementations while requiring the impossible of others. Geoff
Re: [whatwg] SQL API error handling
On Oct 16, 2007, at 4:04 PM, Geoffrey Garen wrote: It would be nice to have a way to indicate to the script There was a catastrophic event and we reset your database, assume you're starting over from scratch. In general, I'm not sure how useful it is to know that you're starting over from scratch, since any database query needs to check its result. Presumably, an app's behavior in the no data case is the same regardless of *why* there's no data. 99% of the time the behavior will be to reload the data from a server. More importantly, what constitutes a corrupt database, and how to recover from it, are serious implementation details. Some implementations may have error correction algorithms. Others may have backups they can restore. Others may have to wipe the database completely and start over. Still others may not be able to start anything. (For example, the storage medium might have gone bad, or been locked or disconnected.) So, imposing a start over from scratch requirement would hamper some implementations while requiring the impossible of others. You are (rightly) reading very specifically into what I am saying, whereas what I'm trying to get at is still vague and general. Let me take a step back and try to frame it at a higher level - A page opens a valid database handle. - Some script uses that database handle - successfully - Some external event happens on the client machine - database corruption, the user deletes the database from the user agent's database management mode, gamma rays corrupt a single bit on the disk, or whatever. This event renders the database unusable. - Some action is taken to reset the database so that it is usable - lets pretend the resolution is always recreating an empty database from scratch - The script knows something wrong happened - it has a completely 100% generic error on its query. But it is unaware of the nature of this event and its resolution. It decides to execute a new sql statement, and the value of this statement (from the script's perspective) is reliant on previously established values in the database. The statement coincidentally succeeds even with the new empty database. For further argument, lets pretend the script is in an onunload handler and its writing some small piece of data out to the database before the user quits. It has a lot of other data in memory client- side that it *thinks* is in the database, but really isn't anymore. It *could* write this data out, preserving a lot of important user state. But it doesn't know to do so. One can certainly make the argument that if this were a native application saving data to disk, it would be prudent to verify its data on disk. But I think raw filesystem and database are different paradigms with different usage expectations. An error code along the lines of your database was just reset might fit the bill. This could be because of corruption, because the user agent database management was invoked and the database cleared, or a number of other reasons. This is a requested split from code 1 - The statement failed for reasons not covered by any other code. ~Brady
Re: [whatwg] SQL API error handling
On 10/16/07, Geoffrey Garen [EMAIL PROTECTED] wrote: It would be nice to have a way to indicate to the script There was a catastrophic event and we reset your database, assume you're starting over from scratch. In general, I'm not sure how useful it is to know that you're starting over from scratch, since any database query needs to check its result. Presumably, an app's behavior in the no data case is the same regardless of *why* there's no data. 99% of the time the behavior will be to reload the data from a server. More importantly, what constitutes a corrupt database, and how to recover from it, are serious implementation details. Some implementations may have error correction algorithms. Others may have backups they can restore. Either of those cases would presumably be handled transparently. Others may have to wipe the database completely and start over. Still others may not be able to start anything. (For example, the storage medium might have gone bad, or been locked or disconnected.) So, imposing a start over from scratch requirement would hamper some implementations while requiring the impossible of others. I agree that we probably can't say specifically what should happen, here. I think that if the user agent did detect corruption and nuke the database from orbit, then it would be reasonable for the user agent to invalidate all outstanding database handles. But that kind of thing would seem to be something really beyond the spec to deal with. It seems like at that point the most appropriate action to take would be to refresh the page and start from scratch, rather than expecting the app to somehow handle the problem. -scott
Re: [whatwg] SQL API error handling
On Fri, 5 Oct 2007, Scott Hess wrote: Reviewing SQLite's error list, the things that MAY have utility to report more finely might be: * LOCKED, where you failed because someone else has things locked. Presumably if a single thread of control tries to open the same database via two objects and start two transactions, one of them is going to lose. Having a transaction fail for this reason seems materially different from having it fail because the SQL was invalid or something of that nature, because the appropriate response might be to retry. Wouldn't we just want the transaction to wait for the lock to go away? * CORRUPT, insofar as the Database API lets you delete databases (it doesn't currently, but we've thought of adding that to Gears). Do we expect authors to actually test for this? Wouldn't the better behaviour upon finding that the database was corrupt just be to inform the user and wipe it clean? I don't think we want random sites dealing with user-side corruption, surely. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] SQL API error handling
On 10/15/07, Ian Hickson [EMAIL PROTECTED] wrote: On Fri, 5 Oct 2007, Scott Hess wrote: Reviewing SQLite's error list, the things that MAY have utility to report more finely might be: * LOCKED, where you failed because someone else has things locked. Presumably if a single thread of control tries to open the same database via two objects and start two transactions, one of them is going to lose. Having a transaction fail for this reason seems materially different from having it fail because the SQL was invalid or something of that nature, because the appropriate response might be to retry. Wouldn't we just want the transaction to wait for the lock to go away? Hmm. Right, this API is async, so we can spin for as long as necessary. [I notice that there is no way to cancel a long-running statement!] Under SQLite, there are cases where retrying might work, in which case you can retry. There are other cases where retry will never work, you need to rollback your transaction and start over. If you don't do so, you can cause a deadlock. Much of this can be addressed by using BEGIN IMMEDIATE rather than BEGIN DEFERRED (the default for BEGIN). I'm not certain we can address this kind of issue at the level of this API, if multiple connections to the same database are allowed. * CORRUPT, insofar as the Database API lets you delete databases (it doesn't currently, but we've thought of adding that to Gears). Do we expect authors to actually test for this? Wouldn't the better behaviour upon finding that the database was corrupt just be to inform the user and wipe it clean? I don't think we want random sites dealing with user-side corruption, surely. You may be correct that authors shouldn't be dealing with this. Guaranteeing the integrity of the database at open is prohibitive (you may have to scan the entire database), and no guarantee in practice, so it's possible that you can detect corruption at any arbitrary statement. I'm considering two classes of error, here. One the one hand are statements which are just incorrect, either syntactically or structurally. They will never execute, your app is broken. On the other hand are statement which fail, but are otherwise correct. I think these cases are reasonable to distinguish, but it may be that the author actions for either statement would be identical, making distinguishing them bootless. Whoa! I just realized that there's another group, constraint failures. These are statements which will sometimes succeed, sometimes fail. As currently spec'ed, it looks like a constraint failure will cause rollback. This is probably not appropriate, constraint failures are generally expected to be benign and detectable. Arguing against myself, I've seen plenty of code which just treats any errors from something which allows for a constraint failure as a constraint failure, so maybe this also can be lumped under a single big error code. -scott
Re: [whatwg] SQL API error handling
On Oct 15, 2007, at 2:07 PM, Ian Hickson wrote: On Fri, 5 Oct 2007, Scott Hess wrote: Reviewing SQLite's error list, the things that MAY have utility to report more finely might be: * LOCKED, where you failed because someone else has things locked. Presumably if a single thread of control tries to open the same database via two objects and start two transactions, one of them is going to lose. Having a transaction fail for this reason seems materially different from having it fail because the SQL was invalid or something of that nature, because the appropriate response might be to retry. Wouldn't we just want the transaction to wait for the lock to go away? With upgradeable read-write locks, this can lead to a deadlock. Consider two transactions that start with a statement that only needs a read lock. They each grab a non-exclusive read lock and proceed in parallel. For each, the second statement of the transaction is a write. Each would like to upgrade its read lock to an exclusive write lock, but neither can because a shared read lock is still held. At least one must fail and roll back to avoid deadlock. Thus, it must be possible for the first write statement in a formerly read-only transaction to fail. Regards, Maciej
Re: [whatwg] SQL API error handling
On Mon, 15 Oct 2007, Scott Hess wrote: Under SQLite, there are cases where retrying might work, in which case you can retry. There are other cases where retry will never work, you need to rollback your transaction and start over. If you don't do so, you can cause a deadlock. Much of this can be addressed by using BEGIN IMMEDIATE rather than BEGIN DEFERRED (the default for BEGIN). I'm not certain we can address this kind of issue at the level of this API, if multiple connections to the same database are allowed. See my reply to Maciej below. * CORRUPT, insofar as the Database API lets you delete databases (it doesn't currently, but we've thought of adding that to Gears). You may be correct that authors shouldn't be dealing with this. Guaranteeing the integrity of the database at open is prohibitive (you may have to scan the entire database), and no guarantee in practice, so it's possible that you can detect corruption at any arbitrary statement. Sure, but that problem occurs everywhere. I mean, there's no JS exception for your CPU core just overheated, but we don't guarentee that won't happen either. Database corruption will occur either for hardware reasons or due to software bugs. Hardware failures could cause all kinds of random stuff, including software bugs (through corruption of executables). Having an API to handle software bugs seems silly, since if we could assume that that API was bug free, why not assume the rest of the API is too. This just seems like a case we shouldn't worry about. I'm considering two classes of error, here. One the one hand are statements which are just incorrect, either syntactically or structurally. They will never execute, your app is broken. On the other hand are statement which fail, but are otherwise correct. I think these cases are reasonable to distinguish, but it may be that the author actions for either statement would be identical, making distinguishing them bootless. Things that will always fail should raise exceptions from the method. Things that might, but whose failure state is not immediately known, call the callback with an errorCode. Whoa! I just realized that there's another group, constraint failures. These are statements which will sometimes succeed, sometimes fail. As currently spec'ed, it looks like a constraint failure will cause rollback. This is probably not appropriate, constraint failures are generally expected to be benign and detectable. Arguing against myself, I've seen plenty of code which just treats any errors from something which allows for a constraint failure as a constraint failure, so maybe this also can be lumped under a single big error code. Could you elaborate on this? What would code running into this situation look like? How should we expose it? On Mon, 15 Oct 2007, Maciej Stachowiak wrote: With upgradeable read-write locks, this can lead to a deadlock. Consider two transactions that start with a statement that only needs a read lock. They each grab a non-exclusive read lock and proceed in parallel. For each, the second statement of the transaction is a write. Each would like to upgrade its read lock to an exclusive write lock, but neither can because a shared read lock is still held. At least one must fail and roll back to avoid deadlock. Thus, it must be possible for the first write statement in a formerly read-only transaction to fail. Ok, I've added a new error code (4), with the description: The statement failed because the transaction's first statement was a read-only statement, and a subsequent statement in the same transaction tried to modify the database, but the transaction failed to obtain a write lock before another transaction obtained a write lock and changed a part of the database that the former transaction was dependending upon. Is that what we want? -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] SQL API error handling
On Oct 15, 2007, at 8:37 PM, Ian Hickson wrote: * CORRUPT, insofar as the Database API lets you delete databases (it doesn't currently, but we've thought of adding that to Gears). You may be correct that authors shouldn't be dealing with this. Guaranteeing the integrity of the database at open is prohibitive (you may have to scan the entire database), and no guarantee in practice, so it's possible that you can detect corruption at any arbitrary statement. Sure, but that problem occurs everywhere. I mean, there's no JS exception for your CPU core just overheated, but we don't guarentee that won't happen either. Database corruption will occur either for hardware reasons or due to software bugs. Hardware failures could cause all kinds of random stuff, including software bugs (through corruption of executables). Having an API to handle software bugs seems silly, since if we could assume that that API was bug free, why not assume the rest of the API is too. This just seems like a case we shouldn't worry about. I agree with your principals here, but think databases are a different story. In some embedded (and client-server) database implementations - including SQLite - continuing to operate on a database that is known to be corrupt can lead to the process crashing. Unlike the CPU core just overheated case, it is a dangerous state software can help avoid. How the user agent handles the problem long term (ask the user what to do, silently delete and recreate it, let the database continue to be corrupt, etc) may not need to be specified, but perhaps it would be prudent to change the spec to at least suggest that if a database becomes known to be corrupt, operations on all open handles to that database should start throwing INVALID_STATE_ERR exceptions. Thanks, ~Brady
Re: [whatwg] SQL API error handling
On 9/24/07, Ian Hickson [EMAIL PROTECTED] wrote: On Thu, 20 Sep 2007, Anne van Kesteren wrote: The SQL API doesn't seem to define how to deal with errors, such as: snip * Database that is full This currently just reports an error with code 1, like everything else, but in due course we should get a useful set of error codes and define them. I'd love implementation feedback on what errors they are actually going to naturally end up wanting to report. In most database clients, errors can be separated into: - Things that the client can do something about. - Things that it cannot do anything about. In the first case, since we're working within the browser, there's generally really nothing to be done, so most stuff falls to the second case. At that point, it would be useful to have a human-readable error string generated by the library which can be logged, for debugging purposes. Reviewing SQLite's error list, the things that MAY have utility to report more finely might be: * LOCKED, where you failed because someone else has things locked. Presumably if a single thread of control tries to open the same database via two objects and start two transactions, one of them is going to lose. Having a transaction fail for this reason seems materially different from having it fail because the SQL was invalid or something of that nature, because the appropriate response might be to retry. * CORRUPT, insofar as the Database API lets you delete databases (it doesn't currently, but we've thought of adding that to Gears). -scott
Re: [whatwg] SQL API error handling
On 10/5/07, Scott Hess [EMAIL PROTECTED] wrote: In the first case, since we're working within the browser, there's generally really nothing to be done, so most stuff falls to the second case. At that point, it would be useful to have a human-readable error string generated by the library which can be logged, for debugging purposes. Sorry, I somehow didn't see the ResultSet.error attribute. Definitely ignore that part of my post. -scott
Re: [whatwg] SQL API error handling
On Thu, 20 Sep 2007, Anne van Kesteren wrote: The SQL API doesn't seem to define how to deal with errors, such as: * Bogus SQL statements Defined. * SQL statements that are not supported for security reasons Defined to be the same as bogus statements. * SQL statements that are not supported because they don't make sense Same. * SQL statements that fail to return anything Why is this an error? * Database that is not available for some reason If the database isn't available, it should be created. * Database that is full This currently just reports an error with code 1, like everything else, but in due course we should get a useful set of error codes and define them. I'd love implementation feedback on what errors they are actually going to naturally end up wanting to report. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
[whatwg] SQL API error handling
Hi, The SQL API doesn't seem to define how to deal with errors, such as: * Bogus SQL statements * SQL statements that are not supported for security reasons * SQL statements that are not supported because they don't make sense * SQL statements that fail to return anything * Database that is not available for some reason * Database that is full I think it should also be a bit more clear on how the user agent constructs the SQL statement. Cheers, -- Anne van Kesteren http://annevankesteren.nl/ http://www.opera.com/
