Re: [sqlite] Can we create a table where the value of one particular column <> another column?

2017-12-19 Thread Keith Medcalf

And of course you will need an index on edges.parent and one on edges.child 
since you need indexes on foreign keys.  (unless you do not intend to enforce 
them and they are merely for "documentation" of intent to perhaps have a 
consistent database mayhaps perchance).

You will probably also want a unique constraint (index) on edges (parent, 
chaild) so you do not have multiple edges going from the same parent to the 
same child.

You might also need a trigger to make sure that you do not have "duplicate" 
edges if your edges are "bidirectional" rather than directional.  
This will probably require a unique (child, parent) index as well.

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.

>-Original Message-
>From: sqlite-users [mailto:sqlite-users-
>boun...@mailinglists.sqlite.org] On Behalf Of Simon Slavin
>Sent: Tuesday, 19 December, 2017 23:50
>To: SQLite mailing list
>Subject: Re: [sqlite] Can we create a table where the value of one
>particular column <> another column?
>
>
>
>On 20 Dec 2017, at 6:30am, Shane Dev  wrote:
>
>> Let's say we have nodes and edges tables -
>>
>> sqlite> .sch nodes
>> CREATE TABLE nodes(id integer primary key, description text);
>> sqlite> .sch edges
>> CREATE TABLE edges(parent references nodes, child references
>nodes);
>>
>> Can we restrict the edges table so that inserting or updating a row
>where
>> edges.parent = edges.child is not allowed and ideally would produce
>an
>> error message?
>>
>> sqlite> insert into nodes select 1, 'node1';
>> sqlite> insert into nodes select 2, 'node2';
>> sqlite> insert into edges select 1, 2;
>> sqlite> insert into edges select 1, 1;
>> -- should be an error here
>
>Yes !
>
>CREATE TABLE edges(
>parent INTEGER references nodes,
>child INTEGER references nodes,
>CONSTRAINT NotMyOwnGrandpa CHECK (child <> parent)
>);
>
>You’ll get a failure result code from the INSERT.
>
>Simon.
>___
>sqlite-users mailing list
>sqlite-users@mailinglists.sqlite.org
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Can we create a table where the value of one particular column <> another column?

2017-12-19 Thread Simon Slavin


On 20 Dec 2017, at 6:30am, Shane Dev  wrote:

> Let's say we have nodes and edges tables -
> 
> sqlite> .sch nodes
> CREATE TABLE nodes(id integer primary key, description text);
> sqlite> .sch edges
> CREATE TABLE edges(parent references nodes, child references nodes);
> 
> Can we restrict the edges table so that inserting or updating a row where
> edges.parent = edges.child is not allowed and ideally would produce an
> error message?
> 
> sqlite> insert into nodes select 1, 'node1';
> sqlite> insert into nodes select 2, 'node2';
> sqlite> insert into edges select 1, 2;
> sqlite> insert into edges select 1, 1;
> -- should be an error here

Yes !

CREATE TABLE edges(
parent INTEGER references nodes,
child INTEGER references nodes,
CONSTRAINT NotMyOwnGrandpa CHECK (child <> parent)
);

You’ll get a failure result code from the INSERT.  

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] [EXTERNAL] Can we create a table where the value of one particular column <> another column?

2017-12-19 Thread Hick Gunter
Use a trigger and make it raise an exception, something like (not tested, just 
an example of how it might work):

CREATE TRIGGER  BEFORE INSERT ON  WHEN NEW.parent == NEW.child 
BEGIN RAISE(FAIL, 'connecting to self not allowed') END;

You may also require a similar BEFORE UPDATE trigger to avoid changes that 
violate the condition.


-Ursprüngliche Nachricht-
Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im 
Auftrag von Shane Dev
Gesendet: Mittwoch, 20. Dezember 2017 07:30
An: SQLite mailing list 
Betreff: [EXTERNAL] [sqlite] Can we create a table where the value of one 
particular column <> another column?

Let's say we have nodes and edges tables -

sqlite> .sch nodes
CREATE TABLE nodes(id integer primary key, description text);
sqlite> .sch edges
CREATE TABLE edges(parent references nodes, child references nodes);

Can we restrict the edges table so that inserting or updating a row where 
edges.parent = edges.child is not allowed and ideally would produce an error 
message?

sqlite> insert into nodes select 1, 'node1'; insert into nodes select 2,
sqlite> 'node2'; insert into edges select 1, 2; insert into edges select
sqlite> 1, 1;
-- should be an error here
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
 Gunter Hick | Software Engineer | Scientific Games International GmbH | 
Klitschgasse 2-4, A-1130 Vienna | FN 157284 a, HG Wien, DVR: 0430013 | (O) +43 
1 80100 - 0

May be privileged. May be confidential. Please delete if not the addressee.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] [EXTERNAL] Re: printf() with UTF-8 and \n \t format

2017-12-19 Thread Hick Gunter
The most common "problem" with UTF-8 and string lengths is that multibyte UTF-8 
characters (most often characters with diacritical marks, e.g. german umlaut or 
special characters like the EUR sign) get truncated in between their 
constituent bytes. This leads to invalid byte sequences at the "end" of 
strings, which picky programs gag on.

The second most common "problem" I have come across is "double translation", 
i.e. an ISO input strings gets converted to UTF-8 and the result is subjected 
to another conversion. This will lead to invalid byte sequences "in the middle 
of" a string, to the same effect as above.

-Ursprüngliche Nachricht-
Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im 
Auftrag von Tony Papadimitriou
Gesendet: Dienstag, 19. Dezember 2017 21:24
An: SQLite mailing list 
Betreff: [EXTERNAL] Re: [sqlite] printf() with UTF-8 and \n \t format

-Original Message-
From: R Smith

On 2017/12/19 8:37 PM, Tony Papadimitriou wrote:
>> A couple of questions about printf
>>
>> 1. Does it work with UTF-8? If so, how?
>
>- Yes.
>- Very nicely.

I'm using SQL v3.21 and UTF-8 does not work correctly.  (Not from the command 
line.)

I tried with latest trunk and it works fine.  Hmmm.
Let's try with the precompiled Windows binary for v3.21.0 It works.  Hmmm!
(...many, many trials later...)

Let's try with different default options.

Ta da!!!

When using the -column option (my own binary has this as default) the problem 
shows up.  With the official default of -list option the problem is not there.
And, it happens with the latest trunk, also.

So, that looks like a bug.

>> 2. Does it understand \n and \t?  I put actual line breaks inside the
>> string >which is OK if run from script file but it won’t work with
>> one-liners on the >command-line.>
>
>The \n, \t, \r etc. are really dependent on some factors (OS etc.).

Yes, I know all that.  The question was if it understands them, not how they 
might behave depending on OS.

When I use \t or \n I get actual \t and \n strings displayed instead of tabbing 
and advancing line, respectively.

So, is there any way to advance to next line from a command line printf()?

>Ryan

Thanks

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
 Gunter Hick | Software Engineer | Scientific Games International GmbH | 
Klitschgasse 2-4, A-1130 Vienna | FN 157284 a, HG Wien, DVR: 0430013 | (O) +43 
1 80100 - 0

May be privileged. May be confidential. Please delete if not the addressee.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] Can we create a table where the value of one particular column <> another column?

2017-12-19 Thread Shane Dev
Let's say we have nodes and edges tables -

sqlite> .sch nodes
CREATE TABLE nodes(id integer primary key, description text);
sqlite> .sch edges
CREATE TABLE edges(parent references nodes, child references nodes);

Can we restrict the edges table so that inserting or updating a row where
edges.parent = edges.child is not allowed and ideally would produce an
error message?

sqlite> insert into nodes select 1, 'node1';
sqlite> insert into nodes select 2, 'node2';
sqlite> insert into edges select 1, 2;
sqlite> insert into edges select 1, 1;
-- should be an error here
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread Keith Medcalf
>>Until the underlying system memory allocator fails and then it go
>boom.  How
>much goes boom is OS dependent.  Some OSes will only allow the errant
>process go boom.  Others (such as those from Microsoft) the entire OS
>go
>boom if the out of memory condition encompases the entire V=V address
>space.
>
>
>Thanks Keith. So sqlite does look to increase the temp table's cache
>size if 
>it's not big enough? Looking at the results I posted earlier, why did
>that
>not happen until cache_size was set to -500? Why didn't sqlite just
>increased the cache_size from -400 to the required memory? Mind you,
>I'm
>assuming the test results for cache_size <= -400 were slower because
>sqlite
>resorted to HD but maybe that's not the case.

Actually I think it is dependant on exactly what you are doing.  If you are 
using a :memory: database, it will continue to grow until all memory is 
consumed.  I think for temp space, however, if you have set "memory only" then 
I think it will fail allocation when the temp size is used up (but I could be 
wrong here -- I have never actually run into that issue).  However, if you use 
disk-backed storage (ie, a db file or temp-files, then the memory cache will 
spill to disk when it is "full" which will generally (hopefully) happen long 
before you have run out of virtual memory.

I've actually ever only run out of system-wide virtual memory a few times in 
the last several years.  Memory is cheap and so I have huge page and temp sizes 
set by default and lots of memory, and third-level page redirection disabled 
(ie, no non-RAM backing store).  The most troubling thing is that one gets used 
to have NVMe drives that can I/O in excess of 2 GB/s which creates a real 
problem moving to machines that do I/O through a pinhole.  Those things can 
usually be fixed though without too much difficulty (usually by using more RAM 
so as to not do I/O in excess of the pinhole capabilities).




___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread Simon Slavin
On 19 Dec 2017, at 9:22pm, curmudgeon  wrote:

> Thanks Keith. So sqlite does look to increase the temp table's cache size if
> it's not big enough?

No.  Memory allocated to that cache is memory which can’t be used by anything 
else.  There’s probably a far better use for that memory than using it for one 
obscure SQLite table.

If the table gets so big it can’t all fit into the cache allocated to it, it’s 
written to disk instead.  Of course, it’s written to disk using OS calls, so 
there’s a chance that the OS will decide that that data should be cached.

Your ideas portray a world where a computer really has infinite RAM, if only it 
would just allocate it.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Clemens Ladisch
Karl Forner wrote:
> On Tue, Dec 19, 2017 at 3:12 PM, Clemens Ladisch  wrote:
>> Where does the hex number in "excluded_mice_temp2b5036f270" come from?
>> Is it random?
>
> Yes, thanks, I think I got it. In order to alter tables, we first move them
> to a new table with a temp name, copy them, then drop them.
> The temp name is random, so is different between the two computers.
>
> After doing a "vacuum" on both files, they are now identical !
>
> So it is our fault, we should do a vacuum after the drop table.

No.   If you care about repeatability, don't use random numbers.
Instead use a deterministic algorithm to generate the temp table name.


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] printf() with UTF-8 and \n \t format

2017-12-19 Thread R Smith


On 2017/12/19 11:13 PM, Tony Papadimitriou wrote:
Great! Didn't think of the char() function at all.  (Although I would 
prefer a platform independent \n)


"\n" is NOT platform independent.  Char(10) on the other hand IS 
platform independent. That's perhaps the root of the misunderstanding.


I mean there are other factors too... Like on Linux we like to use only 
Char(10) (which can sometimes be signaled by "\n" to your app if the 
user pleases) having the great advantage of being nice and lean as a 
line and record separator.


On Windows we use both a Char(13) and Char(10) to signal a line-break or 
record separator (or "\r\n" if you will) which is fatter but allows you 
to use a normal linefeed character within records without 
breaking/ending the record.


These things are all OS dependent and so are the escape sequences 
"\anything" - it has nothing to do with data storage. In sqlite (or any 
other database) we use the actual really real control characters, not 
some escaped representation of it.  If you need to produce database 
output that will be fed via a command-line or console to a next 
application and want to pop out those escapes character sequences in 
stead of the real McCoy, then you can easily do that too by making a UDF 
or even doing this:


SELECT  replace(replace(replace(replace( "SomeDataColumn", char(10), 
'\n'), char(13), '\r'), char(09), '\t')


But I have to tell you, that makes my spine twitch

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread curmudgeon
>Until the underlying system memory allocator fails and then it go boom.  How
much goes boom is OS dependent.  Some OSes will only allow the errant
process go boom.  Others (such as those from Microsoft) the entire OS go
boom if the out of memory condition encompases the entire V=V address space. 


Thanks Keith. So sqlite does look to increase the temp table's cache size if
it's not big enough? Looking at the results I posted earlier, why did that
not happen until cache_size was set to -500? Why didn't sqlite just
increased the cache_size from -400 to the required memory? Mind you, I'm
assuming the test results for cache_size <= -400 were slower because sqlite
resorted to HD but maybe that's not the case.
 



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] printf() with UTF-8 and \n \t format

2017-12-19 Thread Tony Papadimitriou
Great! Didn't think of the char() function at all.  (Although I would prefer 
a platform independent \n)


Thanks.

-Original Message- 
From: Keith Medcalf



So, is there any way to advance to next line from a command line
printf()?


sqlite> select printf('%s%s%s', 'line 1', char(10), 'line 2');
line 1
line 2

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] printf() with UTF-8 and \n \t format

2017-12-19 Thread R Smith



On 2017/12/19 10:24 PM, Tony Papadimitriou wrote:



2. Does it understand \n and \t?  I put actual line breaks inside 
the string >which is OK if run from script file but it won’t work 
with one-liners on the >command-line.>


The \n, \t, \r etc. are really dependent on some factors (OS etc.).


Yes, I know all that.  The question was if it understands them, not 
how they might behave depending on OS.


But that is the entire point.
SQLite doesn't handle or understand \n, \t, \r, \x, \Father 
Christmas  Those things are OS-dependent escapes to get 
non-printable characters into command line input. SQlite calls a 
Line-Feed a Line-Feed. It is agnostic to  how the Line-Feed gets passed 
to it. The OS (sometimes) facilitates a conversion of the character 
sequence "\" + "n" as a "Line-Feed"  when passing it to a command-line 
interface, such as sqlite.exe in our case, but by the time sqlite sees 
it, it is already a Linefeed character (i.e. ASCII/Unicode/UTF8 
Character no. 10) - sqlite doesn't know anything about "\n", it cannot, 
does not, and need not interpret it, it is completely irrelevant to 
sqlite.  "\n" is a function of the OS to allow the user to more easily 
send character(10) to a console application, the console application 
(thank goodness) need not know anything about "\n". i.e if your SQLite 
DOESN'T understand \n, it is a shortcoming/by design/feature of your OS, 
it has nothing to do with SQLite.


Is that more clear?

The same goes for UTF8.

SQLite isn't so much "ABLE" to read UTF8, as it is indeed the only way 
it communicates, it doesn't do non-UTF8. UTF8 is the only language it 
speaks. So if your UTF8 doesn't get passed correctly to SQLite it is 
typically a problem of the OS Code page or the sqlite-using application 
running in the console. (Btw. sqlite3.exe and its ilk are all command 
line applications and isn't the sqlite engine itself, your fixes wrt the 
UTF8 probably fixed the CLI in some way and it may well be a bug there, 
so if you can submit a test case it would be helpful - please give all 
the details).



So, is there any way to advance to next line from a command line 
printf()?


Extremely easily:
SELECT Char(10);
- or -
SELECT printf( '%s', Char(10) );

And before you ask
Char(07) = BELL
Char(08) = BACKSPACE
Char(09) = TAB = \t
Char(10) = LF = \n
Char(11) = VTAB (Vertical Tab)
Char(12) = FF (FormFeed) - in case you print to an old-style line 
printer (LPT1)

Char(13) = CR = \r  (Carriage Return)
Char(27) = ESC
Char(32) = SPACE

That more or less covers the important non-printables / invisible 
characters.


Hope that helps :)
Ryan








Ryan


Thanks
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] printf() with UTF-8 and \n \t format

2017-12-19 Thread Keith Medcalf

>So, is there any way to advance to next line from a command line
>printf()?

print a linefeed.  That is how you tell a computer output device to advance to 
the beginning of the next line.

sqlite> select printf('%s%s%s', 'line 1', char(10), 'line 2');
line 1
line 2
sqlite>

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.


>
>>Ryan
>
>Thanks
>
>___
>sqlite-users mailing list
>sqlite-users@mailinglists.sqlite.org
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] printf() with UTF-8 and \n \t format

2017-12-19 Thread Tony Papadimitriou
-Original Message- 
From: R Smith


On 2017/12/19 8:37 PM, Tony Papadimitriou wrote:

A couple of questions about printf

1. Does it work with UTF-8? If so, how?


- Yes.
- Very nicely.


I'm using SQL v3.21 and UTF-8 does not work correctly.  (Not from the 
command line.)


I tried with latest trunk and it works fine.  Hmmm.
Let's try with the precompiled Windows binary for v3.21.0
It works.  Hmmm!
(...many, many trials later...)

Let's try with different default options.

Ta da!!!

When using the -column option (my own binary has this as default) the 
problem shows up.  With the official default of -list option the problem is 
not there.

And, it happens with the latest trunk, also.

So, that looks like a bug.

2. Does it understand \n and \t?  I put actual line breaks inside the 
string >which is OK if run from script file but it won’t work with 
one-liners on the >command-line.>


The \n, \t, \r etc. are really dependent on some factors (OS etc.).


Yes, I know all that.  The question was if it understands them, not how they 
might behave depending on OS.


When I use \t or \n I get actual \t and \n strings displayed instead of 
tabbing and advancing line, respectively.


So, is there any way to advance to next line from a command line printf()?


Ryan


Thanks 


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] printf() with UTF-8 and \n \t format

2017-12-19 Thread Keith Medcalf

Which printf?  There are a lot of them.

Assuming that you mean the SQLite3 built-in function printf() (as in SELECT 
PRINTF(...);) that function does not interpret backslash escape sequences.  
Interpretation of such things are a user I/O feature, not a data 
storage/retrieval feature.  

As for the first question, every string in SQLite3 is UTF-8, unless you tell it 
explicitly that you want one of the UTF-16 formats or use blob (bag-o-bytes) 
which can be any old bag-o-bytes you happen to like and it is just that, a 
bag-o-bytes containing just, well, bytes.

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.


>-Original Message-
>From: sqlite-users [mailto:sqlite-users-
>boun...@mailinglists.sqlite.org] On Behalf Of Tony Papadimitriou
>Sent: Tuesday, 19 December, 2017 11:38
>To: General Discussion of SQLite Database
>Subject: [sqlite] printf() with UTF-8 and \n \t format
>
>A couple of questions about printf
>
>1. Does it work with UTF-8? If so, how?
>
>2. Does it understand \n and \t?  I put actual line breaks inside the
>string which is OK if run from script file but it won’t work with
>one-liners on the command-line.
>
>Thank you
>___
>sqlite-users mailing list
>sqlite-users@mailinglists.sqlite.org
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] printf() with UTF-8 and \n \t format

2017-12-19 Thread R Smith


On 2017/12/19 8:37 PM, Tony Papadimitriou wrote:

A couple of questions about printf

1. Does it work with UTF-8? If so, how?


- Yes.
- Very nicely.


2. Does it understand \n and \t?  I put actual line breaks inside the string 
which is OK if run from script file but it won’t work with one-liners on the 
command-line.


The \n, \t, \r etc. are really dependent on some factors (OS etc.). It 
gets interpreted outside of SQLite and differently via different 
IO/Console mechanisms - Which command line are you using? Do you mean 
the sqlite3.exe CLI? (Or one of its bretheren on other OSes?)


If the first question about UTF8 is also based on the Command-Line 
interface's IO handling of UTF8, then that might get tricky. In Windows 
(for instance) it depends on the selected Code-Page. Perhaps just try it 
and if you don't get it right, please ask on here stating your exact OS 
and tried method, then I promise you someone will show you the easy way 
to achieve it (the question comes along often and always gets answered 
well, but it always depends on specifics).


At this point we know very little about your use-case, so it's a little 
hard to answer.



Good luck!
Ryan

___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread Keith Medcalf

Until the underlying system memory allocator fails and then it go boom.  How 
much goes boom is OS dependent.  Some OSes will only allow the errant process 
go boom.  Others (such as those from Microsoft) the entire OS go boom if the 
out of memory condition encompases the entire V=V address space.

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.


>-Original Message-
>From: sqlite-users [mailto:sqlite-users-
>boun...@mailinglists.sqlite.org] On Behalf Of Simon Slavin
>Sent: Tuesday, 19 December, 2017 11:12
>To: SQLite mailing list
>Subject: Re: [sqlite] What happens if an in memory database runs out
>of memory
>
>
>
>On 19 Dec 2017, at 6:01pm, curmudgeon  wrote:
>
>> 1) I read in those links that each temp table is given (by default)
>its own
>> page cache of 500 pages. Is this a separately created page cache or
>is it
>> 500 pages from THEE page chache? If it's the latter that will
>explain the
>> slowdown for cache_size < -500. Or does the cache_size pragma
>dictate the
>> size of the separately created page cache?
>
>Each temporary table gets its own cache.
>
>It works like this: Each attached file gets its own page cache, with
>a size governed by SQLITE_DEFAULT_CACHE_SIZE and/or "PRAGMA
>cache_size" .  Each temporary table declared is considered to be a
>separate attached file.
>
>> 3) Similar to the original question, if you set temp_store = 2
>(memory) and
>> there isn't enough memory for the table what happens?
>
>By 'memory' that web page is referring to whatever your operating
>system thinks is memory.  So the same thing happens as would happen
>to any application which tries to use a lot of memory: the virtual
>memory mechanism kicks in and memory is swapped to and from disk.
>
>Simon.
>___
>sqlite-users mailing list
>sqlite-users@mailinglists.sqlite.org
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Atomic DELETE index optimisation?

2017-12-19 Thread R Smith


On 2017/12/19 6:15 PM, Dinu wrote:

2) Structure alterations; either drop table, or drop indexes : I am
reluctant to do this; my evangelical instinct tells me hacking the semantics
of life might lead to implosion of Earth :)


Oh the irony

Your instinct (like most such instincts) is simply wrong.

But don't take our word for it. Take a copy of the data somewhere else, 
construct some readers mimicking the in-use case (or whatever it takes 
to adequately convince yourself) and try the suggested fix.


Not only will it work, it will do so really fast and prove to be not 
only the best, but also the absolutely correct fix.
The good people here have even tried to build suggested scripts around 
your irrational fears, but in truth the simplest version would work the 
best, which is:


In a Transaction -
- Rename the Table,
- Recreate a new empty version of it,
- Copy the rows you want to keep.
- Drop the renamed table.
- Recreate any Indexes/Triggers.

A solution that might be slightly slower but much less complicated and 
still very fast is simply:

In a Transaction -
- Drop the Indexes/Triggers,
- Do the deletes,
- Recreate the Indexes/Triggers.

This will work 100% even while other readers are active (thanks to the 
WAL [1]) and without making them implode (perhaps pause a little bit[2], 
but not implode).


Why am I (and others here) so confident this will work? Because this is 
a fundamental design principle and indeed a requirement of a 
Transactional and Relational Database (Which SQLite is) - Plus we do it 
all the time very much relying on exactly those characteristics. There 
is no "hacking" going on in any of our suggestions, this is quite the 
mainstream way in which it works.


It's just like learning to ride a bicycle. Initially your fears feel 
justified and the physics seem impossible magic from where you watch the 
others... until that first day you find your balance and soar  Then 
pretty soon, you do it without hands on the steering wheel, just like 
the other cool kids.  :)


Cheers,
Ryan

[1] - It will work even with other Journal modes than WAL, it's just 
that the readers then will wait quite a bit more on the transaction to 
finish, whereas the WAL allows updates to not affect concurrent readers 
until a checkpoint/commit.


[2] - The file is several Gigabytes in size, it's never going to be 
instant, there WILL be some waiting, but it won't take very long - try 
do it at a quiet time though. You will get a feeling for the time-frame 
if you do the test-case thing.




___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread curmudgeon
>> 3) Similar to the original question, if you set temp_store = 2 (memory)
and 
>> there isn't enough memory for the table what happens? 

>By 'memory' that web page is referring to whatever your operating system
thinks is memory.  So the >same thing happens as would happen to any
application which tries to use a lot of memory: the virtual >memory
mechanism kicks in and memory is swapped to and from disk.

Thanks Simon but is that to say sqlite attempts to increase the temp table's
page cache size beyond the size it was originally allotted? If the answer to
that is yes then that also answers question 2).  




--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] printf() with UTF-8 and \n \t format

2017-12-19 Thread Tony Papadimitriou
A couple of questions about printf

1. Does it work with UTF-8? If so, how?

2. Does it understand \n and \t?  I put actual line breaks inside the string 
which is OK if run from script file but it won’t work with one-liners on the 
command-line.

Thank you
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread Simon Slavin


On 19 Dec 2017, at 6:01pm, curmudgeon  wrote:

> 1) I read in those links that each temp table is given (by default) its own
> page cache of 500 pages. Is this a separately created page cache or is it
> 500 pages from THEE page chache? If it's the latter that will explain the
> slowdown for cache_size < -500. Or does the cache_size pragma dictate the
> size of the separately created page cache?

Each temporary table gets its own cache.

It works like this: Each attached file gets its own page cache, with a size 
governed by SQLITE_DEFAULT_CACHE_SIZE and/or "PRAGMA cache_size" .  Each 
temporary table declared is considered to be a separate attached file.

> 3) Similar to the original question, if you set temp_store = 2 (memory) and
> there isn't enough memory for the table what happens?

By 'memory' that web page is referring to whatever your operating system thinks 
is memory.  So the same thing happens as would happen to any application which 
tries to use a lot of memory: the virtual memory mechanism kicks in and memory 
is swapped to and from disk.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread curmudgeon
Thanks Simon/Gunter. I thought those sections cleared things up until I tried
a few tests.
I opened the DB, set temp_store to STORE, cache_size to CACHE and then
calculated the average secs taken (over 2 runs) to run the following

[Tbl has integer primary key ID, contains 10,570 records & is cross joined
to itself to create a (pointless and not distinct) large insert of
111,724,900 recs]

create temp table Key (ID int);
INSERT = secs to run 
insert into Key select ID from Tbl cross join Tbl using (ID);
DELETE = secs to run
delete from Key;

The results for different values of temp_store and cache_size were as
follows  

STORE, CACHE,   INSERT, DELETE
0, 0,   23.00, 1.87
2, 0,   21.10, 1.27
2, -8, 20.36, 1.30
2, -80, 20.61, 1.30
2, -200, 20.70, 1.30
2, -300, 20.72, 1.30
2, -400, 20.70, 1.30
2, -500, 13.58, 1.29
2, -800, 13.48, 1.29
2, -8000, 13.52,1.28
2, -800, 13.64, 1.29

At cache_size = -500 the timings come more into line with the memory results
from my old tests. Some things I need cleared up

1) I read in those links that each temp table is given (by default) its own
page cache of 500 pages. Is this a separately created page cache or is it
500 pages from THEE page chache? If it's the latter that will explain the
slowdown for cache_size < -500. Or does the cache_size pragma dictate the
size of the separately created page cache?

2) My DB page size is 1024. If the temp table is allocated a separate page
cache of 500 * 1024 bytes this means that sqlite managed to store
111,724,900 records in 500 KB? That's an average of 0.0046 bytes per record? 

3) Similar to the original question, if you set temp_store = 2 (memory) and
there isn't enough memory for the table what happens? 



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Atomic DELETE index optimisation?

2017-12-19 Thread Paul Sanderson
Dinu

Option 2, dropping and recreating the index with the transaction, seems to
be the way forward - I would suggest that if the author of SQlite (Dr Hipp)
has put this forward as a solution, as he did earlier in this thread,  then
it is probably a safe option and will not lead to an implosion of anything.

Paul
www.sandersonforensics.com
skype: r3scue193
twitter: @sandersonforens
Tel +44 (0)1326 572786
http://sandersonforensics.com/forum/content.php?195-SQLite-Forensic-Toolkit
-Forensic Toolkit for SQLite
email from a work address for a fully functional demo licence

On 19 December 2017 at 16:15, Dinu  wrote:

> Hi sub sk79,
> I have so far from this thread the following suggestions:
> 1) Copy table -> TRUNCATE -> copy back; this doesn't work, what was
> supposed
> to be TRUNCATE semantics (DELETE FROM without WHERE) has the same
> performance as with WHERE.
> 2) Structure alterations; either drop table, or drop indexes : I am
> reluctant to do this; my evangelical instinct tells me hacking the
> semantics
> of life might lead to implosion of Earth :)
> 3) "Deleted" bit field - presumably the "soft delete" as you call it; I am
> analyzing this, but here the question is whether we include the bit in the
> indexes. If so, performing a heap of UPDATEs should be even more
> inefficient; if we don't include it in the index, the problem of the cost
> of
> filtering the row needs some analysis which I will probably do; the problem
> with this solution is that is has residual effects: we run some pretty
> complex queries against this table, with complicated joins and we already
> got some surprising execution plans that needed query rewriting. So with
> this "deleted" bit out of the index pool, we need to check various other
> queries to make sure they are still optimized to what we need.
>
> All this said and done, 3 hours to delete 15G of data seems atrocious even
> if you do it by standards resulted from generations of DOD and NSA
> inbreeding... so I'm still hopeful for some DB-related solution.
>
>
>
> --
> Sent from: http://sqlite.1065341.n5.nabble.com/
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Atomic DELETE index optimisation?

2017-12-19 Thread Simon Slavin


On 19 Dec 2017, at 4:15pm, Dinu  wrote:

> 3) "Deleted" bit field - presumably the "soft delete" as you call it;

If you do try this, the 'bit' column should be declared as INTEGER and the 
values stored should be 0 and 1.  SQLite is extremely efficient at 
storing/sorting these values.

Including this column in indexes should not slow things down much.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Atomic DELETE index optimisation?

2017-12-19 Thread Dinu
Hi sub sk79,
I have so far from this thread the following suggestions:
1) Copy table -> TRUNCATE -> copy back; this doesn't work, what was supposed
to be TRUNCATE semantics (DELETE FROM without WHERE) has the same
performance as with WHERE.
2) Structure alterations; either drop table, or drop indexes : I am
reluctant to do this; my evangelical instinct tells me hacking the semantics
of life might lead to implosion of Earth :)
3) "Deleted" bit field - presumably the "soft delete" as you call it; I am
analyzing this, but here the question is whether we include the bit in the
indexes. If so, performing a heap of UPDATEs should be even more
inefficient; if we don't include it in the index, the problem of the cost of
filtering the row needs some analysis which I will probably do; the problem
with this solution is that is has residual effects: we run some pretty
complex queries against this table, with complicated joins and we already
got some surprising execution plans that needed query rewriting. So with
this "deleted" bit out of the index pool, we need to check various other
queries to make sure they are still optimized to what we need.

All this said and done, 3 hours to delete 15G of data seems atrocious even
if you do it by standards resulted from generations of DOD and NSA
inbreeding... so I'm still hopeful for some DB-related solution.



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Atomic DELETE index optimisation?

2017-12-19 Thread Subsk79
On Dec 17, 2017, at 3:53 PM, Dinu Marina 
 Is there any way to fix/improve this in userland? 

I think the thread already has a viable solution but still if userland fix is 
an option: would a soft-delete based design work for you?

https://dba.stackexchange.com/questions/125431/hard-delete-vs-soft-delete-performance

-SK

___
*StepSqlite* enhanced-PL/SQL 
on SQLite and BerkeleyDB.
**Coming Soon**
 Be Done at the Speed of Lite! --->
 https://metatranz.com/stepsqlite/ShowSignUp?guestlogin=ON




___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Karl Forner
On Tue, Dec 19, 2017 at 3:12 PM, Clemens Ladisch  wrote:

> Karl Forner wrote:
> > Here's a screenshot of some of the diffs using vbindiff
> >
> > https://ibb.co/kNm0X6
>
> SQLite uses hash tables for schema objects, so different names might
> result in such differences.
>
> Where does the hex number in "excluded_mice_temp2b5036f270" come from?
> Is it random?
>

Yes, thanks, I think I got it. In order to alter tables, we first move them
to a new table with a temp name, copy them, then drop them.
The temp name is random, so is different between the two computers.

After doing a "vacuum" on both files, they are now identical !

So it is our fault, we should do a vacuum after the drop table.

Thanks a lot for your help !




>
>
> Regards,
> Clemens
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Karl Forner
>
> PRAGMA integrity_check;
>

all ok


>
>
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Odd question

2017-12-19 Thread Nelson, Erik - 2
Mark Sent: Tuesday, December 19, 2017 1:32 AM

On Mon Dec 18, 2017 at 04:48:27PM +, Nelson, Erik - 2 wrote:
> Nomad Sent: Sunday, December 17, 2017 4:11 PM
> >On Sat Dec 16, 2017 at 07:53:22PM +, Nelson, Erik - 2 wrote:
>  
> >> Select 1 as value from (insert into table1 values(a, b, c)) I've
> >> tried a number of options but haven't been able to get anything
> ...snip...
> >> to work.  Is it possible?
> 
>> >How about a user defined function that does the insert in the
>> >background?
>> 
>> >SELECT my_insert(table_name, val1, val2);
>> 
>> That's a good suggestion- the only input control I have is sql
>> statements that must return at least one row.  AFAIK there's no way
>> to make UDFs within that constraint?

>That is quite a constraint. Unfortunately then the UDF option is not
>available to you.

Alas, yes.  Thanks to all for the excellent ideas, we can consider my question 
closed.

--
This message, and any attachments, is for the intended recipient(s) only, may 
contain information that is privileged, confidential and/or proprietary and 
subject to important terms and conditions available at 
http://www.bankofamerica.com/emaildisclaimer.   If you are not the intended 
recipient, please delete this message.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Clemens Ladisch
Karl Forner wrote:
> Here's a screenshot of some of the diffs using vbindiff
>
> https://ibb.co/kNm0X6

SQLite uses hash tables for schema objects, so different names might
result in such differences.

Where does the hex number in "excluded_mice_temp2b5036f270" come from?
Is it random?


Regards,
Clemens
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Simon Slavin
On 19 Dec 2017, at 1:30pm, Karl Forner  wrote:

> and the dumps usign the .dump sqlite3.exe command are also identical.

Paul has a better chance of understanding the hex dump than I do, but what I 
think I’m seeing is trivial differences in file organisation.  In other words 
some pieces of data are stored in a different order in the two databases.

I’m not sure why this could happen in a situation where you start with two 
identical Docker images.  Might they run out of caching space at different 
times because background processes ran differently ?  Having caches written to 
the database in a different order could cause the sort of thing you’re showing 
us.

Your tests — sha3sum checksum and using .dump — both say that the databases 
have identical contents.  As a last possible check you might want to run this 
in the shell tool on both copies:

PRAGMA integrity_check;

I’m betting that you see no errors on either copy.  Whatever the differences 
between the two copies are, they're harmless as far as SQLite is concerned.  
But I have no idea why they might have happened.  I hope Mr Sanderson or one of 
the development team have some ideas.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Keith Medcalf

So the "used bytes" are the same, but the "unused bytes" are not.  That would 
seem to indicate the possibility that the "empty page" initialization is the 
difference.  Some OSes will return zero'd out "new" sectors while some are 
perfectly happy to return whatever data happened to reside in the sector prior 
to it being allocated.  If this is the case the "used contents" will be the 
same but the "unused bytes" will be different.

Is this what you are seeing?

---
The fact that there's a Highway to Hell but only a Stairway to Heaven says a 
lot about anticipated traffic volume.


>-Original Message-
>From: sqlite-users [mailto:sqlite-users-
>boun...@mailinglists.sqlite.org] On Behalf Of Karl Forner
>Sent: Tuesday, 19 December, 2017 06:30
>To: SQLite mailing list
>Subject: Re: [sqlite] same code produces a different database file on
>different computers
>
>and the dumps usign the .dump sqlite3.exe command are also identical.
>
>On Tue, Dec 19, 2017 at 2:19 PM, Karl Forner 
>wrote:
>
>> > Using the sqlite3 shell tool please give the command
>> > SELECT * FROM sqlite_master;
>>
>> the results are identical
>>
>> table|_meta|_meta|2|CREATE TABLE "_meta"("TABLENAME" TEXT NOT
>NULL  ,
>> "COLNAME" TEXT NOT NULL  , "VARNAMES" TEXT  , "LABELS" TEXT  ,
>"PREFS"
>> TEXT   , PRIMARY KEY(TABLENAME,COLNAME))
>> index|sqlite_autoindex__meta_1|_meta|3|
>> table|pk_model|pk_model|39020|CREATE TABLE "pk_model"("MODEL"
>TEXT
>> , PRIMARY KEY(MODEL))
>> index|sqlite_autoindex_pk_model_1|pk_model|39021|
>> table|tumor_data|tumor_data|39027|CREATE TABLE "tumor_data"(
>"CRO"
>> TEXT  , "MODEL" TEXT  , "SUBTYPE" TEXT  , "QB_SUBTYPE" TEXT  ,
>> "SUBTYPE_DEFINED_FOR_QUARTZBIO_INTERNAL_HISTO_SUBTYPING_CRO_
>> SUBTYPING_CONSIDERED_IN_CASE_IT_IS_MORE_SUBTYPE_SPECIFIC" TEXT  ,
>> "TREATMENT_ARM_NO" INTEGER  , "TREATMENT" TEXT  , "DAY" TEXT  ,
>"MEASURE"
>> TEXT  , "LBSTRESC" TEXT , FOREIGN KEY(MODEL) REFERENCES
>pk_model(MODEL) ON
>> UPDATE NO ACTION ON DELETE NO ACTION)
>> table|sequencing_metadata|sequencing_metadata|7|CREATE TABLE
>> "sequencing_metadata"("ANALYSIS_ANALYSIS_NAME" TEXT  ,
>> "ANALYSIS_DATASET_NAME" TEXT  , "ANALYSIS_PROJECT_NAME" TEXT  ,
>> "METADATA_NAME" TEXT  , "METADATA_NUM" TEXT  , "METADATA_STR" TEXT
>,
>> "SAMPLE_BATCH" TEXT  , "SAMPLE_NAME" TEXT  , "MODEL" TEXT , FOREIGN
>> KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE
>NO
>> ACTION)
>> table|excluded_mice|excluded_mice|1319|CREATE TABLE
>"excluded_mice"(
>> "MODEL" TEXT  , "TREATMENT" TEXT  , "COMMENT" TEXT  , "X" TEXT ,
>FOREIGN
>> KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE
>NO
>> ACTION)
>> table|excluded_visits|excluded_visits|1320|CREATE TABLE
>> "excluded_visits"("MODEL" TEXT  , "TREATMENT" TEXT  , "VISIT"
>TEXT  ,
>> "COMMENT" TEXT , FOREIGN KEY(MODEL) REFERENCES pk_model(MODEL) ON
>UPDATE NO
>> ACTION ON DELETE NO ACTION)
>> table|tumor_lungsubtypes|tumor_lungsubtypes|1321|CREATE TABLE
>> "tumor_lungsubtypes"("MODEL" TEXT  , "SUBTYPE_COMBINED" TEXT ,
>FOREIGN
>> KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE
>NO
>> ACTION)
>>
>>
>>
>> Here's a screenshot of some of the diffs using vbindiff
>>
>> https://ibb.co/kNm0X6
>>
>> On Tue, Dec 19, 2017 at 1:48 PM, Simon Slavin
>
>> wrote:
>>
>>>
>>>
>>> On 19 Dec 2017, at 12:43pm, Karl Forner 
>wrote:
>>>
>>> > All the software in the docker container. so it is exactly the
>same
>>> version
>>> > of all software and libraries, except the linux kernel. Moreover
>the
>>> > differences are not in the header, rather interspersed with what
>looks
>>> like
>>> > CREATE statements.
>>>
>>> Using the sqlite3 shell tool please give the command
>>>
>>> SELECT * FROM sqlite_master;
>>>
>>> on one copy of the database.  Are the CREATE commands in the
>result the
>>> same as the CREATE command you just told us about ?
>>>
>>> Issue the same SELECT command for the other copy of the database.
>Do you
>>> get identical output, in the same order, as you got from the first
>copy of
>>> the database ?
>>>
>>> Simon.
>>> ___
>>> sqlite-users mailing list
>>> sqlite-users@mailinglists.sqlite.org
>>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-
>users
>>>
>>
>>
>___
>sqlite-users mailing list
>sqlite-users@mailinglists.sqlite.org
>http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users



___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Karl Forner
and the dumps usign the .dump sqlite3.exe command are also identical.

On Tue, Dec 19, 2017 at 2:19 PM, Karl Forner  wrote:

> > Using the sqlite3 shell tool please give the command
> > SELECT * FROM sqlite_master;
>
> the results are identical
>
> table|_meta|_meta|2|CREATE TABLE "_meta"("TABLENAME" TEXT NOT NULL  ,
> "COLNAME" TEXT NOT NULL  , "VARNAMES" TEXT  , "LABELS" TEXT  , "PREFS"
> TEXT   , PRIMARY KEY(TABLENAME,COLNAME))
> index|sqlite_autoindex__meta_1|_meta|3|
> table|pk_model|pk_model|39020|CREATE TABLE "pk_model"("MODEL" TEXT
> , PRIMARY KEY(MODEL))
> index|sqlite_autoindex_pk_model_1|pk_model|39021|
> table|tumor_data|tumor_data|39027|CREATE TABLE "tumor_data"("CRO"
> TEXT  , "MODEL" TEXT  , "SUBTYPE" TEXT  , "QB_SUBTYPE" TEXT  ,
> "SUBTYPE_DEFINED_FOR_QUARTZBIO_INTERNAL_HISTO_SUBTYPING_CRO_
> SUBTYPING_CONSIDERED_IN_CASE_IT_IS_MORE_SUBTYPE_SPECIFIC" TEXT  ,
> "TREATMENT_ARM_NO" INTEGER  , "TREATMENT" TEXT  , "DAY" TEXT  , "MEASURE"
> TEXT  , "LBSTRESC" TEXT , FOREIGN KEY(MODEL) REFERENCES pk_model(MODEL) ON
> UPDATE NO ACTION ON DELETE NO ACTION)
> table|sequencing_metadata|sequencing_metadata|7|CREATE TABLE
> "sequencing_metadata"("ANALYSIS_ANALYSIS_NAME" TEXT  ,
> "ANALYSIS_DATASET_NAME" TEXT  , "ANALYSIS_PROJECT_NAME" TEXT  ,
> "METADATA_NAME" TEXT  , "METADATA_NUM" TEXT  , "METADATA_STR" TEXT  ,
> "SAMPLE_BATCH" TEXT  , "SAMPLE_NAME" TEXT  , "MODEL" TEXT , FOREIGN
> KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE NO
> ACTION)
> table|excluded_mice|excluded_mice|1319|CREATE TABLE "excluded_mice"(
> "MODEL" TEXT  , "TREATMENT" TEXT  , "COMMENT" TEXT  , "X" TEXT , FOREIGN
> KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE NO
> ACTION)
> table|excluded_visits|excluded_visits|1320|CREATE TABLE
> "excluded_visits"("MODEL" TEXT  , "TREATMENT" TEXT  , "VISIT" TEXT  ,
> "COMMENT" TEXT , FOREIGN KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO
> ACTION ON DELETE NO ACTION)
> table|tumor_lungsubtypes|tumor_lungsubtypes|1321|CREATE TABLE
> "tumor_lungsubtypes"("MODEL" TEXT  , "SUBTYPE_COMBINED" TEXT , FOREIGN
> KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE NO
> ACTION)
>
>
>
> Here's a screenshot of some of the diffs using vbindiff
>
> https://ibb.co/kNm0X6
>
> On Tue, Dec 19, 2017 at 1:48 PM, Simon Slavin 
> wrote:
>
>>
>>
>> On 19 Dec 2017, at 12:43pm, Karl Forner  wrote:
>>
>> > All the software in the docker container. so it is exactly the same
>> version
>> > of all software and libraries, except the linux kernel. Moreover the
>> > differences are not in the header, rather interspersed with what looks
>> like
>> > CREATE statements.
>>
>> Using the sqlite3 shell tool please give the command
>>
>> SELECT * FROM sqlite_master;
>>
>> on one copy of the database.  Are the CREATE commands in the result the
>> same as the CREATE command you just told us about ?
>>
>> Issue the same SELECT command for the other copy of the database.  Do you
>> get identical output, in the same order, as you got from the first copy of
>> the database ?
>>
>> Simon.
>> ___
>> sqlite-users mailing list
>> sqlite-users@mailinglists.sqlite.org
>> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>>
>
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Karl Forner
> Using the sqlite3 shell tool please give the command
> SELECT * FROM sqlite_master;

the results are identical

table|_meta|_meta|2|CREATE TABLE "_meta"("TABLENAME" TEXT NOT NULL  ,
"COLNAME" TEXT NOT NULL  , "VARNAMES" TEXT  , "LABELS" TEXT  , "PREFS"
TEXT   , PRIMARY KEY(TABLENAME,COLNAME))
index|sqlite_autoindex__meta_1|_meta|3|
table|pk_model|pk_model|39020|CREATE TABLE "pk_model"("MODEL" TEXT   ,
PRIMARY KEY(MODEL))
index|sqlite_autoindex_pk_model_1|pk_model|39021|
table|tumor_data|tumor_data|39027|CREATE TABLE "tumor_data"("CRO" TEXT
, "MODEL" TEXT  , "SUBTYPE" TEXT  , "QB_SUBTYPE" TEXT  ,
"SUBTYPE_DEFINED_FOR_QUARTZBIO_INTERNAL_HISTO_SUBTYPING_CRO_SUBTYPING_
CONSIDERED_IN_CASE_IT_IS_MORE_SUBTYPE_SPECIFIC" TEXT  , "TREATMENT_ARM_NO"
INTEGER  , "TREATMENT" TEXT  , "DAY" TEXT  , "MEASURE" TEXT  , "LBSTRESC"
TEXT , FOREIGN KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON
DELETE NO ACTION)
table|sequencing_metadata|sequencing_metadata|7|CREATE TABLE
"sequencing_metadata"("ANALYSIS_ANALYSIS_NAME" TEXT  ,
"ANALYSIS_DATASET_NAME" TEXT  , "ANALYSIS_PROJECT_NAME" TEXT  ,
"METADATA_NAME" TEXT  , "METADATA_NUM" TEXT  , "METADATA_STR" TEXT  ,
"SAMPLE_BATCH" TEXT  , "SAMPLE_NAME" TEXT  , "MODEL" TEXT , FOREIGN
KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE NO
ACTION)
table|excluded_mice|excluded_mice|1319|CREATE TABLE "excluded_mice"(
"MODEL" TEXT  , "TREATMENT" TEXT  , "COMMENT" TEXT  , "X" TEXT , FOREIGN
KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE NO
ACTION)
table|excluded_visits|excluded_visits|1320|CREATE TABLE
"excluded_visits"("MODEL" TEXT  , "TREATMENT" TEXT  , "VISIT" TEXT  ,
"COMMENT" TEXT , FOREIGN KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO
ACTION ON DELETE NO ACTION)
table|tumor_lungsubtypes|tumor_lungsubtypes|1321|CREATE TABLE
"tumor_lungsubtypes"("MODEL" TEXT  , "SUBTYPE_COMBINED" TEXT , FOREIGN
KEY(MODEL) REFERENCES pk_model(MODEL) ON UPDATE NO ACTION ON DELETE NO
ACTION)



Here's a screenshot of some of the diffs using vbindiff

https://ibb.co/kNm0X6

On Tue, Dec 19, 2017 at 1:48 PM, Simon Slavin  wrote:

>
>
> On 19 Dec 2017, at 12:43pm, Karl Forner  wrote:
>
> > All the software in the docker container. so it is exactly the same
> version
> > of all software and libraries, except the linux kernel. Moreover the
> > differences are not in the header, rather interspersed with what looks
> like
> > CREATE statements.
>
> Using the sqlite3 shell tool please give the command
>
> SELECT * FROM sqlite_master;
>
> on one copy of the database.  Are the CREATE commands in the result the
> same as the CREATE command you just told us about ?
>
> Issue the same SELECT command for the other copy of the database.  Do you
> get identical output, in the same order, as you got from the first copy of
> the database ?
>
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] [EXTERNAL] Re: same code produces a different database file on different computers

2017-12-19 Thread Hick Gunter
The internal table sqlite_master has a field named sql that contains the text 
of the create statements. This may be what you are looking at. Does the 
difference show up in the sqlite_master table contents?

-Ursprüngliche Nachricht-
Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im 
Auftrag von Karl Forner
Gesendet: Dienstag, 19. Dezember 2017 13:44
An: SQLite mailing list 
Betreff: [EXTERNAL] Re: [sqlite] same code produces a different database file 
on different computers

Thanks for your reply.
All the software in the docker container. so it is exactly the same version of 
all software and libraries, except the linux kernel. Moreover the differences 
are not in the header, rather interspersed with what looks like CREATE 
statements.



On Tue, Dec 19, 2017 at 1:40 PM, Paul Sanderson < sandersonforens...@gmail.com> 
wrote:

> SQLite stores the verison number of the librrary in the database header.
> Different SQlite libraries on different computers would cause this error.
>
> Paul
> www.sandersonforensics.com
> skype: r3scue193
> twitter: @sandersonforens
> Tel +44 (0)1326 572786
> http://sandersonforensics.com/forum/content.php?195-SQLite-
> Forensic-Toolkit
> -Forensic Toolkit for SQLite
> email from a work address for a fully functional demo licence
>
> On 19 December 2017 at 12:32, Karl Forner  wrote:
>
> > Hello,
> >
> > I encountered a weird behavior recently.
> > The exact same code (executed from a docker container using the same
> image)
> > produced a different database file on two computers, as verified by
> > a MD5 or sha256 hash of the two files.
> > But using the* .sha3sum* of sqlite3.exe I could check that the
> > contents
> of
> > the two databases are indeed identical.
> >
> > To sum up, same content, but the files are somewhat different.
> >
> > Is this something expected or known ?
> > If so, is there a way to prevent this ?
> >
> > Our use case  if that we track the sha256 hashes of the files, that
> > could have been produced on different computers to know of the
> > outputs are up
> to
> > date. Until now, since ~ 4 years it has always seem to work.
> >
> > Thanks.
> > ___
> > sqlite-users mailing list
> > sqlite-users@mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
 Gunter Hick | Software Engineer | Scientific Games International GmbH | 
Klitschgasse 2-4, A-1130 Vienna | FN 157284 a, HG Wien, DVR: 0430013 | (O) +43 
1 80100 - 0

May be privileged. May be confidential. Please delete if not the addressee.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Simon Slavin


On 19 Dec 2017, at 12:43pm, Karl Forner  wrote:

> All the software in the docker container. so it is exactly the same version
> of all software and libraries, except the linux kernel. Moreover the
> differences are not in the header, rather interspersed with what looks like
> CREATE statements.

Using the sqlite3 shell tool please give the command

SELECT * FROM sqlite_master;

on one copy of the database.  Are the CREATE commands in the result the same as 
the CREATE command you just told us about ?

Issue the same SELECT command for the other copy of the database.  Do you get 
identical output, in the same order, as you got from the first copy of the 
database ?

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Karl Forner
Thanks for your reply.
All the software in the docker container. so it is exactly the same version
of all software and libraries, except the linux kernel. Moreover the
differences are not in the header, rather interspersed with what looks like
CREATE statements.



On Tue, Dec 19, 2017 at 1:40 PM, Paul Sanderson <
sandersonforens...@gmail.com> wrote:

> SQLite stores the verison number of the librrary in the database header.
> Different SQlite libraries on different computers would cause this error.
>
> Paul
> www.sandersonforensics.com
> skype: r3scue193
> twitter: @sandersonforens
> Tel +44 (0)1326 572786
> http://sandersonforensics.com/forum/content.php?195-SQLite-
> Forensic-Toolkit
> -Forensic Toolkit for SQLite
> email from a work address for a fully functional demo licence
>
> On 19 December 2017 at 12:32, Karl Forner  wrote:
>
> > Hello,
> >
> > I encountered a weird behavior recently.
> > The exact same code (executed from a docker container using the same
> image)
> > produced a different database file on two computers, as verified by a MD5
> > or sha256 hash of the two files.
> > But using the* .sha3sum* of sqlite3.exe I could check that the contents
> of
> > the two databases are indeed identical.
> >
> > To sum up, same content, but the files are somewhat different.
> >
> > Is this something expected or known ?
> > If so, is there a way to prevent this ?
> >
> > Our use case  if that we track the sha256 hashes of the files, that could
> > have been produced on different computers to know of the outputs are up
> to
> > date. Until now, since ~ 4 years it has always seem to work.
> >
> > Thanks.
> > ___
> > sqlite-users mailing list
> > sqlite-users@mailinglists.sqlite.org
> > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> >
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] same code produces a different database file on different computers

2017-12-19 Thread Paul Sanderson
SQLite stores the verison number of the librrary in the database header.
Different SQlite libraries on different computers would cause this error.

Paul
www.sandersonforensics.com
skype: r3scue193
twitter: @sandersonforens
Tel +44 (0)1326 572786
http://sandersonforensics.com/forum/content.php?195-SQLite-Forensic-Toolkit
-Forensic Toolkit for SQLite
email from a work address for a fully functional demo licence

On 19 December 2017 at 12:32, Karl Forner  wrote:

> Hello,
>
> I encountered a weird behavior recently.
> The exact same code (executed from a docker container using the same image)
> produced a different database file on two computers, as verified by a MD5
> or sha256 hash of the two files.
> But using the* .sha3sum* of sqlite3.exe I could check that the contents of
> the two databases are indeed identical.
>
> To sum up, same content, but the files are somewhat different.
>
> Is this something expected or known ?
> If so, is there a way to prevent this ?
>
> Our use case  if that we track the sha256 hashes of the files, that could
> have been produced on different computers to know of the outputs are up to
> date. Until now, since ~ 4 years it has always seem to work.
>
> Thanks.
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


[sqlite] same code produces a different database file on different computers

2017-12-19 Thread Karl Forner
Hello,

I encountered a weird behavior recently.
The exact same code (executed from a docker container using the same image)
produced a different database file on two computers, as verified by a MD5
or sha256 hash of the two files.
But using the* .sha3sum* of sqlite3.exe I could check that the contents of
the two databases are indeed identical.

To sum up, same content, but the files are somewhat different.

Is this something expected or known ?
If so, is there a way to prevent this ?

Our use case  if that we track the sha256 hashes of the files, that could
have been produced on different computers to know of the outputs are up to
date. Until now, since ~ 4 years it has always seem to work.

Thanks.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Does sqlite have official development testing tool?

2017-12-19 Thread advancenOO
Thanks for all your suggestions. 
Is it correct to add my compile-options(such as
-DSQLITE_DEFAULT_MMAP_SIZE=0x7fff) just after CFLAGS in Makefile? As I
found some weird bugs if I do so. 
For example, sometimes wal2.test can pass but sometimes it reports "cannot
open savepoint - SQL statements in progress".
 
Sorry for all the questions...

Thanks.



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] [EXTERNAL] Re: What happens if an in memory database runs out of memory

2017-12-19 Thread Hick Gunter
Pragma temp_store and the preprocessor macro SQLITE_TEMP_STORE determine where 
temp tables and indices are stored.

See http://sqlite.org/pragma.html#pragma_temp_store


-Ursprüngliche Nachricht-
Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im 
Auftrag von curmudgeon
Gesendet: Dienstag, 19. Dezember 2017 11:35
An: sqlite-users@mailinglists.sqlite.org
Betreff: [EXTERNAL] Re: [sqlite] What happens if an in memory database runs out 
of memory

My apologies. The inserts in the above tests were made on a TEMP table which I 
assumed was being created in the existing database. I've since discovered that 
isn't the case.

*Tables created using the "CREATE TEMP TABLE" syntax are only visible to the 
database connection in which the "CREATE TEMP TABLE" statement is originally 
evaluated. These TEMP tables, together with any associated indices, triggers, 
and views, are collectively stored in a separate temporary database file that 
is created as soon as the first "CREATE TEMP TABLE" statement is seen.*

Running the same tests again using an actual table (TEMP keyword omitted) shows 
the in memory INSERT (and DELETE) to be twice as fast on the database loaded 
into memory compared to when it's accessed from the SSD.The timings in the 
original tests were similar to the in memory database suggesting the TEMP table 
for those tests were created in memory regardless of whether the actual 
database was loaded in memory or not. I can't see any documentation suggesting 
that though. Could someone clarify?





--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


___
 Gunter Hick | Software Engineer | Scientific Games International GmbH | 
Klitschgasse 2-4, A-1130 Vienna | FN 157284 a, HG Wien, DVR: 0430013 | (O) +43 
1 80100 - 0

May be privileged. May be confidential. Please delete if not the addressee.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread Simon Slavin


On 19 Dec 2017, at 11:40am, Simon Slavin  wrote:

> See section 3 of
> 
> 

Sorry.  See section 4 too.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread Simon Slavin


On 19 Dec 2017, at 9:33am, curmudgeon  wrote:

> Running the same tests again using
> an actual table (TEMP keyword omitted) shows the in memory INSERT (and
> DELETE) to be twice as fast on the database loaded into memory compared to
> when it's accessed from the SSD.The timings in the original tests were
> similar to the in memory database suggesting the TEMP table for those tests
> were created in memory regardless of whether the actual database was loaded
> in memory or not. I can't see any documentation suggesting that though.
> Could someone clarify?

See section 3 of



A working assumption is that temporary files are created in memory and only if 
they get big enough to trouble virtual memory do they require disk access.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread curmudgeon
My apologies. The inserts in the above tests were made on a TEMP table which
I assumed was being created in the existing database. I've since discovered
that isn't the case.

*Tables created using the "CREATE TEMP TABLE" syntax are only visible to the
database connection in which the "CREATE TEMP TABLE" statement is originally
evaluated. These TEMP tables, together with any 
associated indices, triggers, and views, are collectively stored in a
separate temporary database file that is created as soon as the first
"CREATE TEMP TABLE" statement is seen.*

Running the same tests again using an actual table (TEMP keyword omitted)
shows the in memory INSERT (and DELETE) to be twice as fast on the database
loaded into memory compared to when it's accessed from the SSD.The timings
in the original tests were similar to the in memory database suggesting the
TEMP table for those tests were created in memory regardless of whether the
actual database was loaded in memory or not. I can't see any documentation
suggesting that though. Could someone clarify? 





--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] Macro expansion of B0 on Linux PPC system

2017-12-19 Thread Rowan Worth
On 16 December 2017 at 11:21, Richard Hipp  wrote:

> On 12/15/17, Lee, Greg  wrote:
> > I never got a reply on this issue and someone else tripped up on it:
> >
> > https://github.com/spack/spack/issues/6698
> >
> > Any help or even acknowledgement of this message would be appreciated.
> >
>
> I believe the problem was fixed on the highlighted check-in here:
> https://www.sqlite.org/src/timeline?y=ci=b9a58daca=9
>
> That fix should have appeared in SQLite version 3.19.0.  Version
> 3.21.0 is the latest.
>
> Are you still having issues?


Looks like the #undef was lost when shell.c.in was introduced:

https://www.sqlite.org/src/info/17e0bb12d82b510b

-Rowan
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users


Re: [sqlite] What happens if an in memory database runs out of memory

2017-12-19 Thread curmudgeon
My apologies. The inserts in the above tests were made on a TEMP table which
I assumed was being created in the existing database. I've since discovered
that isn't the case.*Tables created using the "CREATE TEMP TABLE" syntax are
only visible to the database connection in which the "CREATE TEMP TABLE"
statement is originally evaluated. These TEMP tables, together with any
associated indices, triggers, and views, are collectively stored in a
separate temporary database file that is created as soon as the first
"CREATE TEMP TABLE" statement is seen. *Running the same tests again using
an actual table (TEMP keyword omitted) shows the in memory INSERT (and
DELETE) to be twice as fast on the database loaded into memory compared to
when it's accessed from the SSD.The timings in the original tests were
similar to the in memory database suggesting the TEMP table for those tests
were created in memory regardless of whether the actual database was loaded
in memory or not. I can't see any documentation suggesting that though.
Could someone clarify?



--
Sent from: http://sqlite.1065341.n5.nabble.com/
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users