Re: [sqlite] SQLite iOS timestamp type mapping settings must be set to float to get correct data

[Guy Harris]

On Nov 20, 2018, at 12:41 PM, Jens Alfke  wrote:

> On Nov 20, 2018, at 11:44 AM, Bill Hashman  
> wrote:
> 
>> The timestamp from iOS systems is not compliant with ISO 8601/Unix or other 
>> common timestamps.  It appears apple has their start date offset 31 years.
> 
> Yes, the ‘epoch’ in Apple’s own APIs (CoreFoundation, Foundation) is 
> 1/1/2001, expressed as a double. But of course the POSIX APIs on Apple 
> platforms use the regular Unix epoch of 1/1/1970 as integer.

And this applies to all Apple Darwin-based OSes, including macOS, watchOS, and 
tvOS, not just iOS.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] ___fixunsdfdi

[Guy Harris]

On Jun 18, 2018, at 3:21 AM, x  wrote:

> I’m using c++ builder 10.2 Tokyo on windows 10 pro.
> In a console app I’m getting this error message
> 
> [ilink32 Error] Error: Unresolved external '___fixunsdfdi' referenced from 
> C:\...\PROJECTS\WIN32\DEBUG\SQLITE3.OBJ
> 
> I can’t find any mention of it in sqlite3.h or sqlite3.c.
> 
> Anyone know anything about it?

It's a support routine to which some compilers generate calls.  The Intel 
System Studio documentation:

https://software.intel.com/en-us/node/704849

says

These functions convert a to an unsigned 64-bit integer rounding toward 
zero. If the integral value cannot be represented by the integer type, zero is 
returned.

The incremental linker is probably not linking with whatever library the 
compiler that built sqlite3.obj expects the program to be linked with.  You'd 
probably have to check the C++ Builder documentation to see what library that 
might be and how to ensure that it gets linked with that library.

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Cannot initialize statically linked extension

[Guy Harris]

On Dec 4, 2017, at 3:42 PM, Keith Medcalf  wrote:

> On Monday, 4 December, 2017 15:44, Jens Alfke  wrote:
> 
>>> If one object is using, for example, the multithreaded runtime and
>>> the others are using the single threaded runtime (for example), and
>>> the third perhaps the subsystem runtime
> 
>> From the OP’s other thread here it looks like they’re developing for
>> iOS or macOS, neither of which have the sorts of multiple runtimes
>> you’re talking about (is that a Windows thing?)
> 
> I had believed it was Windows since "DLL" was mentioned

No, the original message said

> Btw, if I build my extension as a dynamic library and load it
> in a SQLite shell, all is fine.

which says "dynamic library", rather than "DLL"; the term "dynamic" is used 
with most UN*X implementations of shared libraries (and the file suffix used 
for shared libraries on Darwin-based systems such as macOS and iOS is ".dylib").

> and that is a Windows (or OS/2) contraction for dynamic load library.  I 
> don't think anything else uses that term in quite the same way.

The initialism "DLL" is a sign that the system being discussed is almost 
certainly Windows.  The word "dynamic", however, isn't - it may even be a sign 
that it's a UN*X, as they probably would have said "DLL" were it Windows.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 27, 2017, at 10:00 AM, Keith Medcalf <kmedc...@dessus.com> wrote:

> On Wednesday, 27 September, 2017 10:39, Guy Harris <g...@alum.mit.edu> wrote:
> 
>> On Sep 27, 2017, at 6:58 AM, Keith Medcalf <kmedc...@dessus.com> wrote:
> 
>>> Well, the terminology is correct.  These *ARE* I/O Errors.  The
>>> system attempted I/O.  It failed.  Hence the term I/O Error.
> 
>> Just don't call it a "disk I/O error".
> 
> Well, maybe.  However the I/O that had the error was associated with a disk 
> operation (as opposed to a "Video I/O Error", or a "Cardpunch I/O Error", 
> "Printer I/O Error", etc.).

Actually, if it had occurred on my machine, it wouldn't have been associated 
with a disk operation; should the application check where the data is stored 
and say "flash memory I/O" error if appropriate? :-)

The point is that the *disk* isn't particularly relevant to some possible 
errors - the problem isn't with the *disk*, which reported no error, the 
problem is with something in the *file system*, such as the amount of space 
available, the permissions on files, etc..

>>> It is irrelevant whether the error was caused because the heads on
>>> the tape drive need cleaning, access was denied to spool storage, the
>>> disk was full, someone yanked the cable out of the disk drive, or the
>>> card reader got jammed up.
> 
>> I.e., SQLITE_IOERR is equivalent to -1 as a return from various UN*X
>> system calls, so that, when a program sees it, it needs to get
>> further error information, such as an errno value, to deal with the
>> error and, if necessary, to report it.
> 
> Yes.  An I/O operation of some sort was attempted.  That I/O operation 
> involved some sort of "disk" access.  That operation failed with an error.

...and the next step is to determine what the exact error was.

>> So it *is* relevant to what to do next.
> 
> Well, in the same sort of way as the message from attempting to send Snail 
> mail "Mail Undeliverable" is relevant to what to do next.  You know that the 
> error was related to the delivery of the postal item just as the "Disk I/O 
> Error" indicates that an I/O operation that involved a disk operation failed 
> with an error.
> 
> In both cases you need to query for the underlying error condition in order 
> to determine what to do.

Well, in the first case, the postal service may well say more than just "Mail 
undeliverable", such as "no such person at that address", "no such address", 
etc..

> So in that sense it is relevant to what to do next -- you need to query for 
> more particulars.  This is opposed to say a "Syntax Error" in which it is 
> pretty clear that the error is a mis-formed statement.

Yes, but even in *that* case, it's often possible to say, for example, "there's 
no operator between the operands "foo" and "bar"" rather than just "syntax 
error".
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 27, 2017, at 6:58 AM, Keith Medcalf  wrote:

> Well, the terminology is correct.  These *ARE* I/O Errors.  The system 
> attempted I/O.  It failed.  Hence the term I/O Error.

Just don't call it a "disk I/O error".

> It is irrelevant whether the error was caused because the heads on the tape 
> drive need cleaning, access was denied to spool storage, the disk was full, 
> someone yanked the cable out of the disk drive, or the card reader got jammed 
> up.

I.e., SQLITE_IOERR is equivalent to -1 as a return from various UN*X system 
calls, so that, when a program sees it, it needs to get further error 
information, such as an errno value, to deal with the error and, if necessary, 
to report it.

So it *is* relevant to what to do next.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 26, 2017, at 3:11 PM, Simon Slavin <slav...@bigfraud.org> wrote:

> On 26 Sep 2017, at 10:53pm, Guy Harris <g...@alum.mit.edu> wrote:
>> 
>> I *would* suggests an additional API to get a *separate* extended error 
>> code, so that if, for example, a write() fails and that failure is turned 
>> into SQLITE_IOERR, you can get something that distinguishes EIO from other 
>> errors such as EFBIG, EDQUOT, etc..
> 
> You know about this, right ?
> 
> <https://www.sqlite.org/c3ref/extended_result_codes.html>
> 
> <https://www.sqlite.org/c3ref/c_abort_rollback.html>

Yes.  I do.

You know about this, right?

https://www.sqlite.org/rescode.html#ioerr_access

It shows a whole bunch of codes, none of which are "something that 
distinguishes EIO from other errors such as EFBIG, EDQUOT, etc.".

I'm not asking for something that indicates what xXYZZY method reported the 
error.  I'm asking for something that indicates what the underlying problem 
causing the I/O error is, to the extent that information is available from the 
OS, i.e. *why* did the I/O operation not succeed?

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 26, 2017, at 2:22 PM, Jens Alfke <j...@mooseyard.com> wrote:

>> On Sep 26, 2017, at 1:57 PM, Guy Harris <g...@alum.mit.edu> wrote:
>> 
>> Which means "for stuff that would be shown to the user, for the user to 
>> read, either localize your error messages, or make sure your API returns 
>> error codes that the application can turn into localized error messages".
> 
> Um, that’s what I said.
> 
>> And none of this argues against presenting to the user, in their native 
>> language, a message saying "you've exceeded your file system quota", if that 
>> is, in fact, what happened.
> 
> This thread isn’t about filesystem quotas. Why do you keep bringing them up 
> as an example?

Because the thread brings up the general question of folding multiple types of 
errors into a single error code, and because it's an example of an error you 
*would* want to show to the user, just as SQLITE_FULL is.

>> *don't* tell the user anything that might convince them that their disk is 
>> failing if you didn't get EIO or the equivalent on some other OS - and don't 
>> tell them something that, when relayed to tech support, would lead the 
>> support person to believe that, either.
> 
> As we’ve been saying, error messages produced by SQLite are not meant to be 
> shown to end users, for all the reasons previously discussed.
> 
> SQLite’s error numbers ought to be sufficiently detailed once you enable 
> extended error codes, and/or get the OS errno. The original set of error 
> codes is inadequate to be sure, for historical reasons, but compatibility 
> rules out breaking that API; that’s why the extended error codes exist.

Yes, which is why I wasn't suggesting changing the error codes.

I *would* suggests an additional API to get a *separate* extended error code, 
so that if, for example, a write() fails and that failure is turned into 
SQLITE_IOERR, you can get something that distinguishes EIO from other errors 
such as EFBIG, EDQUOT, etc..  I would also suggest that the documentation say 
that, if you don't have to run on a version of SQLite that doesn't support the 
new API, the new API be used by applications and libraries running atop SQLite 
in their error-reporting code, rather than, for example, just using 
sqlite3_errstr().
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 26, 2017, at 2:08 PM, Scott Robison  wrote:

> There are physical errors and there are logical errors. If an error is
> generated from write, it's not unreasonable to classify it as an
> "output error". From read as an "input error".

"Output error", yes, although it'd be useful to provide more information.

"Disk I/O error", no; it'd be unreasonable to classify "out of file system free 
space", "over quota", "permission error", "file bigger than 2GB-1 bytes", etc. 
as "disk I/O errors".

> There is a lot of sqlite source code that already exists and has been
> written to work with the current interface. That's probably one of the
> reasons why extended errors were created, to provide finer
> granularity. Regardless of whether it is ideal or not, changing sqlite
> in a way that would break existing code is unlikely to happen.

I was not suggesting that.  I didn't suggest adding SQLITE_OVERQUOTA or 
SQLITE_WRITE_PERMISSION_ERROR.

> Ultimately it doesn't matter when error codes were added to a given
> operating system or which predates what. A decision was made in the
> past. The options are to live with decisions that were made in the
> past (one I've seen espoused multiple times in this mailing list),
> come up with an approach that allows old code to work but exposes new
> information (probably the genesis of extended error codes), or break
> older code (which I've not seen done deliberately).

I'm advocating a better version of the second of those choices than the current 
"here's the raw operating system error code" version that's currently provided. 
 (sqlite3_system_errno() also has the problem that if SQLITE_IOERR is provided 
for something *other* than a failure that provides a system errno value, it 
doesn't do the job.)

> That being said, I don't know any non-technical users who are going to
> panic that IOERR means their hard drive is dying specifically because
> of that text being displayed. Panic perhaps, but not that a hard drive
> is about to die. Most people I know don't have that level of
> understanding to correlate IO / ERR / hard drive failure rates.

They don't treat "disk I/O error" as an indication that their disk is having a 
problem?  That doesn't need an understanding of hard drive failure rates.

I have no reason to dismiss the original writer's notion that "disk I/O error" 
might "[scare] the hell out of the poor sysadmin who suspects a filesystem 
corruption might be going on".

> They
> just think the stupid program is broken and not letting them get their
> work done. As for the experienced technical people I know (or at least
> me), their first thought would be to investigate the problem, not to
> assume their hard drive is failing.

Less investigative work is needed if the software gives a more detailed error 
report.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 26, 2017, at 2:16 PM, Simon Slavin <slav...@bigfraud.org> wrote:

> On 26 Sep 2017, at 9:57pm, Guy Harris <g...@alum.mit.edu> wrote:
> 
>> On Sep 26, 2017, at 1:37 PM, Jens Alfke <j...@mooseyard.com> wrote:
>> 
>>>> On Sep 26, 2017, at 1:17 PM, Guy Harris <g...@alum.mit.edu> wrote:
>>>> 
>>>> A user wouldn't know what to do with "you've exceeded your stored data 
>>>> quota”?
>>> 
>>> A Turkish or Chinese user likely wouldn’t. (SQLite’s error messages are not 
>>> localized.)
>> 
>> Which means "for stuff that would be shown to the user, for the user to 
>> read, either localize your error messages, or make sure your API returns 
>> error codes that the application can turn into localized error messages".
> 
> No.  It means that you should present /your/ error messages to your users, 
> not error messages generated by SQLite.  SQLite is a programmer’s tool.  Its 
> users are programmers, and that’s who its error messages are addressed to.  
> You should not be letting your users see error message intended for you, and 
> you should not be making your users worry about what to do about them.

"You" in "either localize your error messages, *or* make sure your API returns 
error codes that the application can turn into localized error messages", 
refers to SQLite.  It ultimately doesn't *need* have have error messages - it 
could leave that entirely up to the application - but it provides them 
nonetheless.

And there's an "or" in my statement; providing a way to get error codes more 
fine-grained than SQLITE_IOERR - so that you don't say "disk I/O error" for 
errors that have nothing to do with a disk reporting an I/O error - is 
something that the application would need in order to provide an appropriate 
error to end users and to the people to whom the end user might report an 
error.  And, no, "that error occurred on this operation" is not the sort of 
fine-grained to which I'm referring.

So just provide a way to get an indication of what *particular* type of error 
generated SQLITE_IOERR - permission error, quota error, actual disk I/O error, 
etc. - and recommend that this *always* be used for SQLITE_IOERR.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 26, 2017, at 1:43 PM, Simon Slavin <slav...@bigfraud.org> wrote:

> On 26 Sep 2017, at 9:17pm, Guy Harris <g...@alum.mit.edu> wrote:
> 
>> The *number* might annoy the support staff; right off the top of your head, 
>> what's the error number for "file system quota exceeded" or "I/O error"?  
>> (No cheating by looking it up in a man page or include file!)
> 
> My support staff are allowed to look things up.

Just don't force them to ask, *before* the look it up, whether the user's 
running Linux or macOS or FreeBSD or Solaris or Windows.

> My users, when faced with a result which means "permission error" will 
> probably grant all permissions to all apps and all users because that’s the 
> simplest way to make a permission error message go away.  My users don’t 
> understand the Posix permission model, because they’re not computer experts, 
> they are financial sector specialists, or psychologists, or tailors.  I don’t 
> want them thinking about computer problems.  If they knew enough about 
> computer problems to fix a permission problem the right way, they wouldn’t be 
> paying me.

And, when faced with a result that says "disk I/O error", your users will 
probably think their disk is broken and take it in to be fixed.

So:

for errors where the user *can* perhaps fix the problem, such as "out 
of file system space" (which already has its own error) and "out of disk quota" 
(which doesn't, and which is different from "out of file system space"), tell 
the user what the problem is (and, at the application level, offer a suggestion 
such as "delete some of those cat videos you've saved");

for errors where the user probably *can't* fix the problem, tell them 
that there's a problem for which they need to talk to support, and tell them 
what to say to the support staff so that the support staff knows that, for 
example, a disk hasn't gone bad.

(And there are places where "you don't have permission to do that" *is* the 
appropriate thing to tell the user, e.g. if they're trying to open a document 
to which they haven't been given read permission, or trying to write to a 
document to which they haven't been given write permission, etc..  I suspect 
your support staff have better things to do with their time than explain to a 
user that they're not allowed to read somebody else's private files.)
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 26, 2017, at 1:37 PM, Jens Alfke <j...@mooseyard.com> wrote:

>> On Sep 26, 2017, at 1:17 PM, Guy Harris <g...@alum.mit.edu> wrote:
>> 
>> A user wouldn't know what to do with "you've exceeded your stored data 
>> quota”?
> 
> A Turkish or Chinese user likely wouldn’t. (SQLite’s error messages are not 
> localized.)

Which means "for stuff that would be shown to the user, for the user to read, 
either localize your error messages, or make sure your API returns error codes 
that the application can turn into localized error messages".

And none of this argues against presenting to the user, in their native 
language, a message saying "you've exceeded your file system quota", if that 
is, in fact, what happened.

> And there are plenty of messages that are much less understandable to a lay 
> user than the one you picked out.

"I got a permission error trying to write to the journal" isn't something you'd 
directly say to the lay user, but *don't* tell the user anything that might 
convince them that their disk is failing if you didn't get EIO or the 
equivalent on some other OS - and don't tell them something that, when relayed 
to tech support, would lead the support person to believe that, either.

I.e., Richard Krekel is 100% correct when he says that "disk I/O error" is an 
inappropriate message for a permission error - the *disk* had no problem, the 
*OS* had a problem when the disk returned file system data that, among other 
things, indicated that the user didn't have permission to do something.  
Replacing the disk and restoring from a backup probably won't fix that problem 
(unless the user had that permission when the backup was done).

>> The *number* might annoy the support staff; right off the top of your head, 
>> what's the error number for "file system quota exceeded" or "I/O error"?  
>> (No cheating by looking it up in a man page or include file!)
> 
> On the contrary, error numbers are a lot easier for support. They’re 
> independent of locale,

But the error reported by sqlite3_system_errno() isn't independent of the OS on 
which the user is running, so *that* error wouldn't be easy for support.  You'd 
need a platform-independent error code, meaning, in this case, one supplied by 
SQLite, not by the OS.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 26, 2017, at 1:05 PM, Simon Slavin <slav...@bigfraud.org> wrote:

> On 26 Sep 2017, at 8:47pm, Guy Harris <g...@alum.mit.edu> wrote:
> 
>> On Sep 26, 2017, at 8:22 AM, Jens Alfke <j...@mooseyard.com> wrote:
>> 
>>> The basic error code is SQLITE_IOERR, which just means "Some kind of disk 
>>> I/O error occurred” according to the comment. Which is true in this case; 
>>> an I/O operation returned an error.
>> 
>> But the *disk* didn't - the *operating system* did, so if SQLITE_IOERR 
>> really means "Some kind of disk I/O error occurred", it's *not* the right 
>> error to return for a *permission* error.
> 
> Those error codes were devised in a day when OS error codes were more simple.

EDQUOT was introduced in 1982, with 4.2BSD; when was SQLITE_IOERR devised?

> Also please note that those error codes are addressed to programmers.  Your 
> users should never see the text explanation of the number.  Because your 
> users wouldn’t know what to do about them.

A user wouldn't know what to do with "you've exceeded your stored data quota"?  
If so, your site has failed to explain to the users that they've been given a 
quota, limiting the amount of space on the server that they can use, and that 
if they exceed their quota, they either need to delete stuff they no longer 
need, move stuff they might *someday* need but don't need *now* to some 
archival medium, or ask their system administrator to increase their quota?

> At most the user can be shown the number returned to they can quote it in a 
> support call.

The *number* might annoy the support staff; right off the top of your head, 
what's the error number for "file system quota exceeded" or "I/O error"?  (No 
cheating by looking it up in a man page or include file!)

And, yes, there needs to be *some* way to get the underlying problem reported 
to somebody in a position to do something about it - where "the underlying 
problem" includes "what did the OS say?" as much as it includes "what SQLite 
operation got the error?".
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] bug: failure to write journal reported as "disk I/O error"

[Guy Harris]

On Sep 26, 2017, at 8:22 AM, Jens Alfke  wrote:

> The basic error code is SQLITE_IOERR, which just means "Some kind of disk I/O 
> error occurred” according to the comment. Which is true in this case; an I/O 
> operation returned an error.

But the *disk* didn't - the *operating system* did, so if SQLITE_IOERR really 
means "Some kind of disk I/O error occurred", it's *not* the right error to 
return for a *permission* error.

And, on UN*X, a write() call can return ENOSPC; a write() is an I/O operation, 
and "returns -1 with errno set to ENOSPC" is an error, but that presumably gets 
reported as SQLITE_FULL, not as SQLITE_IOERR.

Sadly, the name chosen for that error code

1) suggests an "I/O error" in the sense of "a device reported an error 
trying to read or write it"

and

2) is probably part of the API and thus unchangeable.

However, if SQLITE_IOERR is returned for *anything* other than, on UN*X, an EIO 
errno:

1) The documentation should *really really really really really* avoid 
calling it an "I/O error", as "I/O error" has a connotation of "the device 
reported an error" (which is what EIO signifies) rather than "an I/O operation 
got some sort of error, not necessarily an error from the device from which we 
were trying to read data or to which we were trying to write data".

2) The documentation should tell people *always* to use 
sqlite3_system_errno() after an SQLITE_IOERR and report the error based on 
*that*, not just by reporting an "I/O error".  Yes, that means writing 
platform-dependent code; if you want to allow platform-independent code to be 
written atop SQLite, stuff the platform dependency inside SQLite, by providing 
some API to get errors such as, for example, "permission denied" or "disk quota 
exceeded" or "an actual disk I/O error occurred" rather than "write() got some 
error other than ENOSPC".  (Yes, you *can* get "permission denied", e.g. in an 
NFSv2/NFSv3 write to a file to which you had write permission when you opened 
it but to which you no longer have write permission, and, yes, if, for example, 
you're in the remote file system group at Apple, with a home directory on an 
NFS server, you can have an SQLite database being accessed over NFS.)

> If you want more detailed info, use extended error codes by calling 
> sqlite3_extended_result_codes() or sqlite3_extended_errcode(). Then you’ll 
> get a more specific error; in your situation probably SQLITE_IOERR_ACCESS.

Perhaps, in that particular code path, the permission problem would show up in 
an xAccess method call, so that this would happen to be able to give you a 
better error.

However, what matters isn't "what operation got the error?", it's "what 
non-file-system-full error did you get?", and the extended error code won't 
help for errors other than ENOSPC and EIO returned by write().
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Lemon-generated parser gives an assertion failure

[Guy Harris]

If you use the version of lemon.c and lempar.c in the Fossil repository for 
SQLite as of 2017-04-16 20:54:23 UTC, take the following Lemon parser, compile 
it, and run it, it fails with

Assertion failed: (yyruleno!=116), function yy_reduce, file 
mate_grammar.c, line 2165.

(It's a grammar for a domain-specific language in Wireshark, with the actions 
trimmed down to avoid having to drag in the entire MATE mechanism, and other 
changes to make it a stand-along program.  As such, you get a bunch of warnings 
from Lemon about unused labels; ignore them.)

%include {

/* mate_grammar.lemon
* MATE's configuration language grammar
*
* Copyright 2005, Luis E. Garcia Ontanon 
*
* Wireshark - Network traffic analyzer
* By Gerald Combs 
* Copyright 1998 Gerald Combs
*
* This program is free software; you can redistribute it and/or
* modify it under the terms of the GNU General Public License
* as published by the Free Software Foundation; either version 2
* of the License, or (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*/

#include 
#include 
#include 
#include 

#include "mate_grammar.h"

typedef enum _gop_tree_mode_t {
GOP_NULL_TREE,
GOP_BASIC_TREE,
GOP_FULL_TREE
} gop_tree_mode_t;

typedef enum _gop_pdu_tree {
GOP_NO_TREE,
GOP_PDU_TREE,
GOP_FRAME_TREE,
GOP_BASIC_PDU_TREE
} gop_pdu_tree_t;

typedef enum _accept_mode_t {
ACCEPT_MODE,
REJECT_MODE
} accept_mode_t;
typedef enum _avpl_match_mode {
AVPL_NO_MATCH,
AVPL_STRICT,
AVPL_LOOSE,
AVPL_EVERY
} avpl_match_mode;

typedef enum _avpl_replace_mode {
AVPL_NO_REPLACE,
AVPL_INSERT,
AVPL_REPLACE
} avpl_replace_mode;

#define DUMMY void*

#define MATE_PARSE(token_type, text) \
MateParser(pParser, (token_type), strdup(text));

int
main(void)
{
void MateParser(void*,int, char*);
void *MateParserAlloc(void *(*)(size_t));
void MateParserFree( void*, void(*)(void*) );
void MateParseTrace(FILE*,char*);
void* pParser;

pParser = MateParserAlloc(malloc);

/* Here come the tokens */
MATE_PARSE(TOKEN_PDU_KW, "Pdu");
MATE_PARSE(TOKEN_NAME, "sip_pdu");
MATE_PARSE(TOKEN_PROTO_KW, "Proto");
MATE_PARSE(TOKEN_NAME, "sip");
MATE_PARSE(TOKEN_TRANSPORT_KW, "Transport");
MATE_PARSE(TOKEN_NAME, "ip");
MATE_PARSE(TOKEN_OPEN_BRACE, "{");
MATE_PARSE(TOKEN_EXTRACT_KW, "Extract");
MATE_PARSE(TOKEN_NAME, "call_id");
MATE_PARSE(TOKEN_FROM_KW, "From");
MATE_PARSE(TOKEN_NAME, "sip.Call-Id");
MATE_PARSE(TOKEN_SEMICOLON, ";");
MATE_PARSE(TOKEN_EXTRACT_KW, "Extract");
MATE_PARSE(TOKEN_NAME, "method");
MATE_PARSE(TOKEN_FROM_KW, "From");
MATE_PARSE(TOKEN_NAME, "sip.Method");
MATE_PARSE(TOKEN_SEMICOLON, ";");
MATE_PARSE(TOKEN_CLOSE_BRACE, "}");
MATE_PARSE(TOKEN_GOP_KW, "Gop");
MATE_PARSE(TOKEN_NAME, "sip");
MATE_PARSE(TOKEN_ON_KW, "On");
MATE_PARSE(TOKEN_NAME, "sip_pdu");
MATE_PARSE(TOKEN_MATCH_KW, "Match");
MATE_PARSE(TOKEN_OPEN_PARENS, "(");
MATE_PARSE(TOKEN_NAME, "call_id");
MATE_PARSE(TOKEN_CLOSE_PARENS, ")");
MATE_PARSE(TOKEN_OPEN_BRACE, "{");
MATE_PARSE(TOKEN_START_KW, "Start");
MATE_PARSE(TOKEN_OPEN_PARENS, "(");
MATE_PARSE(TOKEN_NAME, "method");
MATE_PARSE(TOKEN_AVP_OPERATOR, "=");
MATE_PARSE(TOKEN_QUOTED, "SUBSCRIBE");
MATE_PARSE(TOKEN_CLOSE_PARENS, ")");
MATE_PARSE(TOKEN_SEMICOLON, ";");
MATE_PARSE(TOKEN_STOP_KW, "Stop");
MATE_PARSE(TOKEN_OPEN_PARENS, "(");
MATE_PARSE(TOKEN_NAME, "method");
MATE_PARSE(TOKEN_AVP_OPERATOR, "=");
MATE_PARSE(TOKEN_QUOTED, "NOTIFY");
MATE_PARSE(TOKEN_CLOSE_PARENS, ")");
MATE_PARSE(TOKEN_SEMICOLON, ";");
MATE_PARSE(TOKEN_CLOSE_BRACE, "}");
MATE_PARSE(TOKEN_DONE_KW, "Done");
MATE_PARSE(TOKEN_SEMICOLON, ";");

/* Inform parser that end of input has reached. */
MateParser(pParser, 0, NULL);

MateParserFree(pParser, free);
}

}

%name MateParser

%token_prefix TOKEN_

%token_type { char* }
%token_destructor {
if ($$) free($$);
}

%syntax_error {
fprintf(stderr, "Syntax Error before %s",yyminor);
}

%parse_failure {
fprintf(stderr, "Parse Error");
}

%type   transform_decl  { char* }
%type   

Re: [sqlite] Get rid of old-style function declarations/definitions in lemon.c

[Guy Harris]

On Apr 14, 2017, at 3:59 AM, Richard Damon  wrote:

> In C (as opposed to C++), it is the only way to provide a real prototype for 
> such a function. The empty parameter list means with an unspecified parameter 
> list in C.

Exactly.  Perhaps some future C standard will finally get rid of that support 
for legacy C, but, for now, if you want to say "this function takes no 
arguments", you have no choice but to declare it as noargs(void) - and I think 
explicitly declaring that is a Very Good Thing, as it means that the compiler 
will reject calls to the function that pass it arguments, as it should.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Get rid of old-style function declarations/definitions in lemon.c

[Guy Harris]

On Apr 13, 2017, at 2:36 AM, Dominique Devienne <ddevie...@gmail.com> wrote:

> On Thu, Apr 13, 2017 at 6:18 AM, Guy Harris <g...@alum.mit.edu> wrote:
> 
>> Here's a patch, against the current Fossil repository, including some
>> changes we've made to lemon.c in Wireshark, that gets rid of old-style K
>> function definitions/declarations, and also removes trailing white space
>> from some lines (the Wireshark pre-commit hook complains about them):
> 
> Attachments are stripped on this list.
> Paste the diff/patch as text in your message,
> if you want it to be considered. --DD

OK:

Index: tool/lemon.c
==
--- tool/lemon.c
+++ tool/lemon.c
@@ -166,16 +166,16 @@
 
 static struct action *Action_new(void);
 static struct action *Action_sort(struct action *);
 
 /** From the file "build.h" /
-void FindRulePrecedences();
-void FindFirstSets();
-void FindStates();
-void FindLinks();
-void FindFollowSets();
-void FindActions();
+void FindRulePrecedences(struct lemon *);
+void FindFirstSets(struct lemon *);
+void FindStates(struct lemon *);
+void FindLinks(struct lemon *);
+void FindFollowSets(struct lemon *);
+void FindActions(struct lemon *);
 
 /* From the file "configlist.h" */
 void Configlist_init(void);
 struct config *Configlist_add(struct rule *, int);
 struct config *Configlist_addbasis(struct rule *, int);
@@ -455,11 +455,11 @@
 int Configcmp(const char *, const char *);
 struct state *State_new(void);
 void State_init(void);
 int State_insert(struct state *, struct config *);
 struct state *State_find(struct config *);
-struct state **State_arrayof(/*  */);
+struct state **State_arrayof(void);
 
 /* Routines used for efficiency in Configlist_add */
 
 void Configtable_init(void);
 int Configtable_insert(struct config *);
@@ -559,12 +559,12 @@
 ** of the lookahead input, then the value of the action_number output is
 ** aAction[X].action.  If the lookaheads do not match then the
 ** default action for the state_number is returned.
 **
 ** All actions associated with a single state_number are first entered
-** into aLookahead[] using multiple calls to acttab_action().  Then the 
-** actions for that single state_number are placed into the aAction[] 
+** into aLookahead[] using multiple calls to acttab_action().  Then the
+** actions for that single state_number are placed into the aAction[]
 ** array with a single call to acttab_insert().  The acttab_insert() call
 ** also resets the aLookahead[] array in preparation for the next
 ** state number.
 */
 struct lookahead_action {
@@ -610,11 +610,11 @@
   }
   memset(p, 0, sizeof(*p));
   return p;
 }
 
-/* Add a new action to the current transaction set.  
+/* Add a new action to the current transaction set.
 **
 ** This routine is called once for each lookahead for a particular
 ** state.
 */
 void acttab_action(acttab *p, int lookahead, int action){
@@ -672,11 +672,11 @@
   p->aAction[i].lookahead = -1;
   p->aAction[i].action = -1;
 }
   }
 
-  /* Scan the existing action table looking for an offset that is a 
+  /* Scan the existing action table looking for an offset that is a
   ** duplicate of the current transaction set.  Fall out of the loop
   ** if and when the duplicate is found.
   **
   ** i is the index in p->aAction[] where p->mnLookahead is inserted.
   */
@@ -750,11 +750,11 @@
 ** Routines to construction the finite state machine for the LEMON
 ** parser generator.
 */
 
 /* Find a precedence symbol of every rule in the grammar.
-** 
+**
 ** Those rules which have a precedence symbol coded in the input
 ** grammar using the "[symbol]" construct will already have the
 ** rp->precsym field filled.  Other rules take as their precedence
 ** symbol the first RHS symbol with a defined precedence.  If there
 ** are not RHS symbols with a defined precedence, the precedence
@@ -1070,11 +1070,11 @@
   for(i=0; instate; i++){
 for(cfp=lemp->sorted[i]->cfp; cfp; cfp=cfp->next){
   cfp->status = INCOMPLETE;
 }
   }
-  
+
   do{
 progress = 0;
 for(i=0; instate; i++){
   for(cfp=lemp->sorted[i]->cfp; cfp; cfp=cfp->next){
 if( cfp->status==COMPLETE ) continue;
@@ -1101,11 +1101,11 @@
   struct config *cfp;
   struct state *stp;
   struct symbol *sp;
   struct rule *rp;
 
-  /* Add all of the reduce actions 
+  /* Add all of the reduce actions
   ** A reduce action is added for each element of the followset of
   ** a configuration which has its dot at the extreme right.
   */
   for(i=0; instate; i++){   /* Loop over all states */
 stp = lemp->sorted[i];
@@ -1218,11 +1218,11 @@
   apy->type = RD_RESOLVED;
 }else if( spx->precprec ){
   apx->type = RD_RESOLVED;
 }
   }else{
-assert( 
+assert(
   apx-

[sqlite] Get rid of old-style function declarations/definitions in lemon.c

[Guy Harris]

Here's a patch, against the current Fossil repository, including some changes 
we've made to lemon.c in Wireshark, that gets rid of old-style K function 
definitions/declarations, and also removes trailing white space from some lines 
(the Wireshark pre-commit hook complains about them):

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bug report + fix: SQLite 3.11+ broken on EBCDIC systems

[Guy Harris]

On Dec 11, 2016, at 4:57 PM, Richard Hipp  wrote:

> I agree with most of your changes.  But I wonder about moving the
> QUOTE2 (the '[' character) value from code 0xba over to 0xad.
> According to EBCDIC chart at https://en.wikipedia.org/wiki/EBCDIC the
> '[' character should be at 0xba.  Is wikipedia wrong?

If so, then some obscure company called the International Business Machines 
Corporation is also wrong:

http://publibfp.dhe.ibm.com/epubs/pdf/dz9zr010.pdf

The EBCDIC/ISO-8 chart in appendix I of version 10 of the z/Architecture 
Principles of Operation has [ at 0xba and ] at 0xbb for EBCDIC; it has an 
accented capital Y at 0xad.

However, it's apparently a bit more complicated; different EBCDIC code pages 
apparently give square brackets different code points:

http://www-01.ibm.com/support/docview.wss?uid=swg21515307

Apparently EBCDIC code page IBM 1047 has [ as 0xad and EBCDIC code page 037 has 
[ as 0xba.  Here's 037:

https://en.wikipedia.org/wiki/EBCDIC_037


ftp://ftp.software.ibm.com/software/globalization/gcoc/attachments/CP00037.pdf

and here's 1047:

https://en.wikipedia.org/wiki/EBCDIC_1047


ftp://ftp.software.ibm.com/software/globalization/gcoc/attachments/CP01047.pdf

The Wikipedia page on EBCDIC code pages:

https://en.wikipedia.org/wiki/EBCDIC_code_pages

says of 1047 "Open Systems (MVS C compiler)", so the issue may be that Brad 
Larsen is using that compiler and it's insisting that '[' is 0xad.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users