Re: [sqlite] Help with confirming a couple of error traces

[Richard Hipp]

On 1/31/17, Shaobo He  wrote:
> Hi there,
>
> My name is Shaobo He and I am a graduate student at University of Utah. I
> am applying a couple of static analysis tools to C projects. The tools I am
> using reports a few partial error traces about null pointer dereferences. I
> was wondering if you could help me to identify whether they (described
> below) were true bugs or just false positives. Your feedback is really
> appreciated.

They are both false-positives.

>
> 1) In function `statGet`, `sqlite3_value_blob` can return a null pointer.
> One possible case is that `ExpandBlob(p)` returns `SQLITE_OK` and the
> condition expression `p->n ? p->z : 0;` evaluates to null given `p->n ==
> 0`. I tried to figure out if `p->n` can be 0 by adding an assertion before
> the call site to `sqlite_value_blob` and running all regression tests. It
> seems it cannot be for these test cases. My question is that if the case
> described above can happen. Moreover, function `statPush` has a similar
> error trace.

The first parameter to statGet() and statPush() will always be a
sizeof(void*)-byte blob that is in fact a pointer to an object.  So
sqlite3_value_blob() will never return NULL there.

>
> 2) In function `walCleanupHash`, `aHash` is initialized to null and is
> probably updated by function `walHashGet`. However, the update may not
> happen if `walIndexPage` returns a status not equal to `SQLITE_OK`. So
> `aHash` remains null and got dereferenced.

In walCleanupHash(), the pages of the -shm file that contains the hash
have already been allocated and initialized - otherwise
walCleanupHash() would have never been called.  But if the -shm file
has already been allocated and initialized, then there is no way for
walHashGet() to fail and leave aHash uninitialized.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] BUG: Illegal initialization in icu.c : sqlite3IcuInit

[dandl]

From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On 
Behalf Of James K. Lowden

>>>Despite the fact that the Windows API is defined in terms of C, Microsoft 
>>>evidently and probably correctly has decided its market does not demand a 
>>>modern C compiler.  

The Microsoft Windows API is defined very much in terms of C89. As an interface 
standard, I approve.

But you are correct. I haven't heard of anyone using C for new Windows 
development for many years, since C++ is almost essential for accessing any of 
the newer Microsoft technologies (eg COM, DirectX), and provides a vastly 
superior programming environment. 

We need to be clear about the difference between a piece of software than can 
be built on every possible kind of hardware and O/S, as against API standards 
and programming convenience on a specific platform. As I said previously, there 
are trade-offs.

>>>Last year there was much rejoicing when Microsoft decided to bundle SQLite 
>>>with Windows.  That leaves me with a new question: if SQLite announced its 
>>>intention to move to C11 in 2018, would that perhaps influence Microsoft's 
>>>timeline to update its compiler?  

Hypothetical question, won't happen, no it wouldn't.

Regards
David M Bennett FACS

Andl - A New Database Language - andl.org





___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] BUG: Illegal initialization in icu.c : sqlite3IcuInit

[Scott Robison]

On Jan 31, 2017 6:25 PM, "James K. Lowden"  wrote:

On Tue, 31 Jan 2017 15:50:08 -0800
Nathan Bossett  wrote:

> Since this is the sqlite users list and not the dev's list, can I ask
> what your use case is that writing a thin wrapper around SQLITE
> doesn't solve your problem?

I've encountered no problem compiling SQLite header files as C11.

The OP encountered a problem compiling SQLite itself as C89 because
there was a C99 construct his compiler didn't support.  The fix was to
"update" the code to conform to a 17-year old standard.

I wondered aloud what system still exists that supports only C89.  On
review of this thread, there is indeed one such compiler in widespread
use: Microsoft Visual Studio.[1]


An older Visual C++ compiler. Not all versions. Anyone could download a
newer compiler for Windows, even Visual C++. See
http://stackoverflow.com/questions/9610747/which-c99-features-are-available-in-the-ms-visual-studio-compiler
for some commentary on feature support.

Windows doesn't come with a compiler, so telling someone on Windows they
need a newer compiler isn't the end of the world, necessarily. Not all
platforms have as much support as Windows. Not that one needs to support
all those platforms. Say cc65... Probably isn't going to ever work.

Despite the fact that the Windows API is defined in terms of C,


ANSI C. And C++. And C# / CLR. Just not C99.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] BUG: Illegal initialization in icu.c : sqlite3IcuInit

[James K. Lowden]

On Tue, 31 Jan 2017 15:50:08 -0800
Nathan Bossett  wrote:

> Since this is the sqlite users list and not the dev's list, can I ask
> what your use case is that writing a thin wrapper around SQLITE
> doesn't solve your problem?  

I've encountered no problem compiling SQLite header files as C11.  

The OP encountered a problem compiling SQLite itself as C89 because
there was a C99 construct his compiler didn't support.  The fix was to
"update" the code to conform to a 17-year old standard.  

I wondered aloud what system still exists that supports only C89.  On
review of this thread, there is indeed one such compiler in widespread
use: Microsoft Visual Studio.[1]

Despite the fact that the Windows API is defined in terms of C,
Microsoft evidently and probably correctly has decided its market does
not demand a modern C compiler.  

Last year there was much rejoicing when Microsoft decided to bundle
SQLite with Windows.  That leaves me with a new question: if SQLite
announced its intention to move to C11 in 2018, would that perhaps
influence Microsoft's timeline to update its compiler?  

--jkl

[1]
https://msdn.microsoft.com/en-us/library/hh409293.aspx#ISO%20C/C++%20Standards%20Support
  

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] DELETE when DB is full

[Simon Slavin]

On 31 Jan 2017, at 10:40pm, Warren Young  wrote:

> On Jan 31, 2017, at 2:03 PM, Ward WIllats  wrote:
> 
>> the delete sometimes (very rarely) fails with a 13 "disk or database full" 
>> error. I assume because the purger is late to the party and it needs pages 
>> in the WAL to be able to rollback if necessary.
> 
> Is there an especially good reason you have to do this in a single shot?
> 
> If you get this error, shrink the date range or pages-to-free value by half 
> and try again.  Repeat until it works, then repeat at that size until you’ve 
> deleted as much as you need to.

It’s possible to do a DELETE in chunks, like you would use LIMIT on a SELECT.  
Start by doing

SELECT rowid FROM MyTable
WHERE [whatever condition picks deletable rows]
OFFSET 1000 LIMIT 1

This gives you the 1000th row you would DELETE.  Then you DELETE all the rows 
up to that one:

DELETE FROM MyTable WHERE [whatever condition picks deletable rows] AND 
rowid <= [whatever was returned]

then you do the SELECT again.  Eventually you get nothing returned from the 
SELECT and then you do the DELETE you would have done in the first place to 
delete any surviving rows.

This keeps any transaction down to a maximum of 1000 rows, which puts an upper 
limit on the amount of space the journal should take up.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] BUG: Illegal initialization in icu.c : sqlite3IcuInit

[Nathan Bossett]

On Tue, Jan 31, 2017 at 11:48:10AM -0500, James K. Lowden wrote:
> On Mon, 30 Jan 2017 13:32:46 -0700
> Scott Robison  wrote:
> 
> > Basing source on "ANSI C" (as much as possible) just gives you the
> > biggest possible distribution / compatibility. 
> 
> Yes, but it also limits you to C as it stood 20 years ago.  And
> counting.  Is there no point at which a more recent standard should be
> adopted?  Among features of C11 I use:
> 
>   stdint.h
>   stdbool.h
>   VLA
>   designated initialization
> 
> SQLite would benefit from all of those, plus UTF-8 string constants.  

Since this is the sqlite users list and not the dev's list, can I ask
what your use case is that writing a thin wrapper around SQLITE doesn't
solve your problem?  

For example, I didn't like the amount of boilerplate code I was writing
for database calls and wanted classes plus exceptions, so I wrote
the thin wrapper sqLITER in very basic C++.  Ok, the name wasn't very 
creative and it's only a few pages of code, but it gave me the wide
support and even file format of SQLITE and the greater efficiency
and maintainability of a higher level language.
sqliter.org  (haven't pushed the exceptions code yet)

-Nathan
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Sqlite on C99/C11 (was: RE: BUG: Illegal initialization in icu.c : sqlite3IcuInit)

[dandl]

From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On 
Behalf Of James K. Lowden

 Basing source on "ANSI C" (as much as possible) just gives you the 
 biggest possible distribution / compatibility.
>>>
>>>Yes, but it also limits you to C as it stood 20 years ago.  And counting.  
>>>Is there no point at which a more recent standard should be adopted?  Among 
>>>features of C11 I use:

This is the trade-off. As with many of us I've been living this for decades. 
This is the genesis of the 'breaking change', which is a killer for those of us 
supporting older software.

>>>SQLite would benefit from all of those, plus UTF-8 string constants.  
>>>
>>>I full well understand the workarounds for integer sizes and Booleans are 
>>>baked into SQLite, and there's no real need to adopt the new (er,
>>>new-ish) forms.  VLAs and designated initializers should not be 
>>>underestimated, though.  There are many places in SQLite where VLAs could 
>>>replace local malloc/free pairs, and SQLite code is rife with structures 
>>>that can be more succintly initialized with the modern syntax.  

UTF-8 perhaps but I don't think VLA's are to be considered. Without going into 
the history, VLA's are (a) not part of C++ (b) not guaranteed to be supported 
by all compliant C compilers (c) optional in C11 (see _ _STDC_NO_VLA_ _) (d) 
not guaranteed to be required by future C standards. They're a lovely feature 
if you know exactly which compiler(s) you're working with.

>>>Nowawdays, once again making a living writing C code and having the daily 
>>>opportunity to experiment the the new language features, I've come to like 
>>>them.  C is a sharp-edged language, and has become a bit fussy as the 
>>>compilers writers have adopted, as we've discussed here, pedantic notions of 
>>>UB.  OTOH, it's an improvement to relieve the language of some of its 
>>>ancient constraints, such as static array sizes and the prohibition on 
>>>type-punning in unions (which began as SOP, then was UB, and is now OK 
>>>again).  

I would never use C if C++ is available. If I have to, I choose a conservative 
feature set. There may be good reasons for Sqlite to move to C99 (the main one 
being that it's already a 15yo standard), but it's not a decision to be taken 
lightly or piecemeal. C11 has barely taken off its training wheels.


Regards
David M Bennett FACS

Andl - A New Database Language - andl.org






___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Help with confirming a couple of error traces

[Shaobo He]

Hi there,

My name is Shaobo He and I am a graduate student at University of Utah. I
am applying a couple of static analysis tools to C projects. The tools I am
using reports a few partial error traces about null pointer dereferences. I
was wondering if you could help me to identify whether they (described
below) were true bugs or just false positives. Your feedback is really
appreciated.

1) In function `statGet`, `sqlite3_value_blob` can return a null pointer.
One possible case is that `ExpandBlob(p)` returns `SQLITE_OK` and the
condition expression `p->n ? p->z : 0;` evaluates to null given `p->n ==
0`. I tried to figure out if `p->n` can be 0 by adding an assertion before
the call site to `sqlite_value_blob` and running all regression tests. It
seems it cannot be for these test cases. My question is that if the case
described above can happen. Moreover, function `statPush` has a similar
error trace.

2) In function `walCleanupHash`, `aHash` is initialized to null and is
probably updated by function `walHashGet`. However, the update may not
happen if `walIndexPage` returns a status not equal to `SQLITE_OK`. So
`aHash` remains null and got dereferenced.

Thanks,
Shaobo
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] DELETE when DB is full

[Warren Young]

On Jan 31, 2017, at 2:03 PM, Ward WIllats  wrote:
> 
> the delete sometimes (very rarely) fails with a 13 "disk or database full" 
> error. I assume because the purger is late to the party and it needs pages in 
> the WAL to be able to rollback if necessary.

Is there an especially good reason you have to do this in a single shot?

If you get this error, shrink the date range or pages-to-free value by half and 
try again.  Repeat until it works, then repeat at that size until you’ve 
deleted as much as you need to.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] DELETE when DB is full

[Richard Hipp]

On 1/31/17, Ward WIllats  wrote:
> 1. The operational assumption (delete need pages) I've asserted here is
> correct, and

That is correct.  While the delete is underway, you need to have both
the old and new content of every modified database page stored on
disk, in case a rollback is required.

Actually, that is not 100% true.  The "new" value of pages added to
the freelist leaf pages is not saved since it does not matter if the
content of a freelist leaf page changes due to a power failure
followed by a recovery.  But even on a full table delete, not every
page becomes a freelist leaf page, so you still need some space.

That said, if you are in WAL mode, the new page content might be
stored in WAL pages that have already been allocated, if space is
available in the WAL file.  You can increase the chance that space
will be available in the WAL file if you run "PRAGMA wal_checkpoint"
just prior to running your big DELETE.

>
> This is sqlite 3.10.1.

Version 3.16.2 is faster, fully compatible, and contains new features.  :-)
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] DELETE when DB is full

[Ward WIllats]

Hello.

We have an embedded system out in the wild with a DB in WALL mode that we set a 
max_pages value on to keep its size constrained. The system is more or less a 
data logger. We run a "purger" thread at intervals to DELETE records when it 
discovers free space is running below a threshold. 

The purger starts a transaction, does a query to figure out the time range to 
delete, and then does a delete with that time range. (I realize now I could 
combine the DELETE and SELECT into one statement) and then ends the transaction.

Thing is, the delete sometimes (very rarely) fails with a 13 "disk or database 
full" error. I assume because the purger is late to the party and it needs 
pages in the WAL to be able to rollback if necessary.

I figure I can run the purger more often, or raise my "must keep free" 
threshold, but it is hard to know the rate at which data will be generated and 
it can be really "bursty" -- so such fixes are a bit hand-wavy and unsatisfying.

So, I thought I'd check with you folks and see if: 

1. The operational assumption (delete need pages) I've asserted here is 
correct, and
2. There is some trick I can use to force a delete of records when the DB pages 
are already maxed out -- esp. if I don't need to roll this back.

This is sqlite 3.10.1.

Thanks

-- Ward

 
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bitten by lack of isolation between SELECT and UPDATE on the same connection

[Scott Robison]

On Tue, Jan 31, 2017 at 12:15 PM, Jens Alfke  wrote:

>
> > On Jan 31, 2017, at 9:39 AM, James K. Lowden 
> wrote:
> >
> > According the SQL standard, every SQL statement is atomic.  SELECT has
> > no beginning and no end: the results it returns reflect the state of
> > the database as of the moment the statement was executed.  If you fetch
> > the last row six days after the first, it still belongs to the database
> > as it stood when you began.
>
> That is the behavior I was assuming and desiring, but it’s not what
> actually occurs. If there are concurrent mutations in the same connection,
> the rows returned by SELECT do _not_ reflect the prior state of the
> database, but suffer from “undefined” behavior. In other words, there is a
> lack of isolation between the SELECT and the concurrent UPDATEs.
>
> It’s possible I’m misunderstanding your point, though!
>
> My immediate workaround (implemented last night) is to iterate over the
> statement at the moment the query is run, saving all the rows in memory.
> Our enumerator object then just reads and returns successive rows from that
> list.
>
> In the medium term I have ideas for optimizations that can let us avoid
> this memory hit in most circumstances (since most queries are not made at
> the same time as mutations.) For example, I could use the original
> enumerator behavior by default, but when the client requests a mutation I
> first notify all in-progress enumerators [on that connection], which will
> immediately read the rest of their rows into memory.
>

I think you said something earlier about a fear that the record set might
be too big to fit in memory (or wanting to avoid that possibility). You
could select the record set you want to a temp table then select *that*
while running updates on the original tables. Probably something you
already thought of (or maybe I subconsciously read it from someone else
already; sorry if adding noise), but thought I'd toss it out.

SDR
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Request: Per-connection equivalent to SQLITE_CONFIG_LOG

[Jens Alfke]

Our library is registering a SQLITE_CONFIG_LOG callback* to log error messages. 
This can provide useful information about errors returned by SQLite. However, 
we’ve found a side effect that, since this callback is global, it gets called 
due to activity from other clients using SQLite in the same process. (This is 
on iOS, where SQLite is a ubiquitous system-provided DLL that is used by a lot 
of OS APIs, as well as many 3rd party libraries used in apps.)

So for example, we sometimes end up logging
WARNING: SQLite error (code 1): no such table: cfurl_cache_response
which has nothing to do with our library, rather with the system CFNetwork 
framework.

It’s not a big deal, but some developers freak about warning messages and will 
either file support tickets or ask for help on our forums, which increases our 
workload.

My request is for some equivalent of this functionality which is scoped to a 
specific database connection, so that we only hear about SQLite warnings 
regarding our own activity.

(Yes, statically linking our own copy of SQLite would solve this problem. But 
it would increase our code size by a megabyte or so, which is undesirable for a 
mobile-focused library. Moreover, there are database corruption issues that can 
be caused by having multiple copies of SQLite code running in a single process.)

—Jens

* https://sqlite.org/c3ref/c_config_covering_index_scan.html
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bitten by lack of isolation between SELECT and UPDATE on the same connection

[Richard Hipp]

On 1/31/17, Jens Alfke  wrote:
>
> My immediate workaround (implemented last night) is to iterate over the
> statement at the moment the query is run, saving all the rows in memory. Our
> enumerator object then just reads and returns successive rows from that
> list.

That's how client/server SQL database engines do it.  They run the
query to completion before starting on any of the updates.
-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bitten by lack of isolation between SELECT and UPDATE on the same connection

[Jens Alfke]

> On Jan 31, 2017, at 9:39 AM, James K. Lowden  wrote:
> 
> According the SQL standard, every SQL statement is atomic.  SELECT has
> no beginning and no end: the results it returns reflect the state of
> the database as of the moment the statement was executed.  If you fetch
> the last row six days after the first, it still belongs to the database
> as it stood when you began.  

That is the behavior I was assuming and desiring, but it’s not what actually 
occurs. If there are concurrent mutations in the same connection, the rows 
returned by SELECT do _not_ reflect the prior state of the database, but suffer 
from “undefined” behavior. In other words, there is a lack of isolation between 
the SELECT and the concurrent UPDATEs.

It’s possible I’m misunderstanding your point, though!

My immediate workaround (implemented last night) is to iterate over the 
statement at the moment the query is run, saving all the rows in memory. Our 
enumerator object then just reads and returns successive rows from that list.

In the medium term I have ideas for optimizations that can let us avoid this 
memory hit in most circumstances (since most queries are not made at the same 
time as mutations.) For example, I could use the original enumerator behavior 
by default, but when the client requests a mutation I first notify all 
in-progress enumerators [on that connection], which will immediately read the 
rest of their rows into memory.

—Jens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Bitten by lack of isolation between SELECT and UPDATE on the same connection

[James K. Lowden]

On Mon, 30 Jan 2017 19:29:40 -0800
Jens Alfke  wrote:

> if I iterate over the the rows in a table using sqlite3_step, and
> update each row after it?s returned, Bad Stuff happens. Specifically,
> my query is just getting the first row over and over and over again,
> and the iteration runs forever.  

I think you've solved your immediate problem and come to grips with
SQLite's behavior.  I thought it might be helpful to explain why it
works that way, and that what you want is also valid, but goes by
another name: a cursor.  

According the SQL standard, every SQL statement is atomic.  SELECT has
no beginning and no end: the results it returns reflect the state of
the database as of the moment the statement was executed.  If you fetch
the last row six days after the first, it still belongs to the database
as it stood when you began.  

SQLite in WAL mode gives you that isolation.  You weren't bitten by a
*lack* of isolation; you were bitten by isolation you didn't expect.   

The idea of updating rows as they are read -- without completing the
transaction -- is supported in SQL with a cursor.  Standard SQL has
DECLARE CURSOR syntax; some cursors can be declared as FOR UPDATE and
have behavior much like what you expected.  That syntax, as you know, is
not supported by SQLite.  

Technically speaking the product of ORDER BY is also a cursor, and it's
not hard to find references to sqlite3_step as using "a cursor".  All
meaning stands in context, and those should not be confused with an SQL
cursor.  

--jkl


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] BUG: Illegal initialization in icu.c : sqlite3IcuInit

[Scott Robison]

On Tue, Jan 31, 2017 at 9:48 AM, James K. Lowden 
wrote:

> On Mon, 30 Jan 2017 13:32:46 -0700
> Scott Robison  wrote:
>
> > Basing source on "ANSI C" (as much as possible) just gives you the
> > biggest possible distribution / compatibility.
>
> Yes, but it also limits you to C as it stood 20 years ago.  And
> counting.  Is there no point at which a more recent standard should be
> adopted?  Among features of C11 I use:
>
> stdint.h
> stdbool.h
> VLA
> designated initialization
>

{snipped}


>
> Knowing what I do of the philosophy of the SQLite developers -- if we
> can speak of such -- I honestly think they (you) would find C11
> amenable.  It's a better language.  It's every bit as respectful of its
> environment, and is more standardized and more easily conformed to it.
>

I agree with all your points regarding the superiority of C99 to ANSI C.
The only downside to it is the "baked in" compatibility. I'm not aware of
any C99 conforming compiler that does not also claim to support C89, but
not vice versa.

It doesn't really matter much to me, as I'm not personally trying to target
such platforms. When I write my own code, I prefer C++11 or later,
personally. But I'm not trying to provide a robust library that works
practically everywhere, either.

-- 
Scott Robison
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] BUG: Illegal initialization in icu.c : sqlite3IcuInit

[James K. Lowden]

On Mon, 30 Jan 2017 13:32:46 -0700
Scott Robison  wrote:

> Basing source on "ANSI C" (as much as possible) just gives you the
> biggest possible distribution / compatibility. 

Yes, but it also limits you to C as it stood 20 years ago.  And
counting.  Is there no point at which a more recent standard should be
adopted?  Among features of C11 I use:

stdint.h
stdbool.h
VLA
designated initialization

SQLite would benefit from all of those, plus UTF-8 string constants.  

I full well understand the workarounds for integer sizes and Booleans
are baked into SQLite, and there's no real need to adopt the new (er,
new-ish) forms.  VLAs and designated initializers should not be
underestimated, though.  There are many places in SQLite where VLAs
could replace local malloc/free pairs, and SQLite code is rife with
structures that can be more succintly initialized with the modern
syntax.  

> [iBook] From a little searching online, 10.2 was the last release in
> 2002, and it seems that it may have included GCC 3.3. GCC 3.3

In 2002, that might have been the best available.  15 years later, I
suspect better compilers are available for that OS.  The state of
compilers then is not the question.  The state of compilers now is.  

> given that a fix for this issue has already been committed, and
> allows the code to work with both C89 & C99, the original report is
> no longer a consideration.

True.  I never thought the original report was problematic or
symptomatic.  

When I took my C out of its rusty scabbard, back around when that iBook
was new, I adhered to my K  None of that size_t stuff for me!  The
Internet visited its instruction, though, and over time I grew to
appreciate why lseek(2) returns off_t and so on.  

Nowawdays, once again making a living writing C code and having the
daily opportunity to experiment the the new language features, I've
come to like them.  C is a sharp-edged language, and has become a bit
fussy as the compilers writers have adopted, as we've discussed here, 
pedantic notions of UB.  OTOH, it's an improvement to relieve the
language of some of its ancient constraints, such as static array sizes
and the prohibition on type-punning in unions (which began as SOP, then
was UB, and is now OK again).  

Knowing what I do of the philosophy of the SQLite developers -- if we
can speak of such -- I honestly think they (you) would find C11
amenable.  It's a better language.  It's every bit as respectful of its
environment, and is more standardized and more easily conformed to it.  

--jkl
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Does SQLite use field definitions?

[Hick Gunter]

Yes. See http://sqlite.org/lang_createtable.html for details. I also suggest 
you look at http://sqlite.org/datatype3.html too

-Ursprüngliche Nachricht-
Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im 
Auftrag von Clyde Eisenbeis
Gesendet: Dienstag, 31. Jänner 2017 17:07
An: SQLite mailing list 
Betreff: [sqlite] Does SQLite use field definitions?

In the past, when using Access as a database, I have specified field 
definitions.  These field definitions have been used when creating a table.

public const string stFIELD_DEFINITIONS = " fstPriority TEXT, fstInfo MEMO, 
fstDateCreated TEXT, fstDateModified TEXT, fiKeyID INTEGER PRIMARY KEY ";

Does SQLite (System.Data.SQLite) use field definitions too?
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
 Gunter Hick
Software Engineer
Scientific Games International GmbH
FN 157284 a, HG Wien
Klitschgasse 2-4, A-1130 Vienna, Austria
Tel: +43 1 80100 0
E-Mail: h...@scigames.at

This communication (including any attachments) is intended for the use of the 
intended recipient(s) only and may contain information that is confidential, 
privileged or legally protected. Any unauthorized use or dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please immediately notify the sender by return e-mail message and 
delete all copies of the original communication. Thank you for your cooperation.


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Does SQLite use field definitions?

[Clyde Eisenbeis]

In the past, when using Access as a database, I have specified field
definitions.  These field definitions have been used when creating a
table.

public const string stFIELD_DEFINITIONS = " fstPriority TEXT, fstInfo
MEMO, fstDateCreated TEXT, fstDateModified TEXT, fiKeyID INTEGER
PRIMARY KEY ";

Does SQLite (System.Data.SQLite) use field definitions too?
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] can't open db when path length extends 512 characters (on linux)

[Rowan Worth]

On 31 January 2017 at 17:32, Jan Nijtmans  wrote:

> 2017-01-30 19:37 GMT+01:00 Dan Kennedy:
> > On 01/31/2017 12:48 AM, Nir Paz wrote:
> >> Linux doesn't have that limit, my thought is to change the define of
> >> MAX_PATHNAME, is there a better option?
> >
> > I don't think there is a better way to do that. Define MAX_PATHNAME to
> > something and see how it goes.
> >
> > There is one very subtle problem that we know of:
> >
> >http://www.sqlite.org/src/artifact/ff1232b3088a3?ln=2771-2775
>
> Well, starting with this commit:
> 
> the default page size is now 4096.
>

That only applies to new databases though. The millions of files created
before this change will still use their original page size (unless
explicitly changed via PRAGMA page_size + VACUUM).

-Rowan
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] can't open db when path length extends 512 characters (on linux)

[Jan Nijtmans]

2017-01-30 19:37 GMT+01:00 Dan Kennedy:
> On 01/31/2017 12:48 AM, Nir Paz wrote:
>> Linux doesn't have that limit, my thought is to change the define of
>> MAX_PATHNAME, is there a better option?
>
> I don't think there is a better way to do that. Define MAX_PATHNAME to
> something and see how it goes.
>
> There is one very subtle problem that we know of:
>
>http://www.sqlite.org/src/artifact/ff1232b3088a3?ln=2771-2775

Well, starting with this commit:

the default page size is now 4096. So I don't think the risk is
really big. Upgrading to 1024 (or even 4096) should be
no problem.

See also:


Regards,
   Jan Nijtmans
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users