[sqlite] Mailing list policy change

I think I received about four, which I removed in a couple of seconds. Obviously it is a problem, but I don't think it calls for a change that makes it impossible to see the sender of each message. I always open messages from the SqLite developers sort of by default, for instance, which I can

[sqlite] Simple Math Question

Sorry, I missed out my point: SQLite version 3.8.10.2 2015-05-20 18:17:19 Enter ".help" for usage hints. sqlite> CREATE TABLE t(r REAL PRIMARY KEY,t TEXT); sqlite> INSERT INTO t VALUES (21.0,'twenty one point zero'); sqlite> INSERT INTO t VALUES (9.2+7.9+0+1.0+1.3+1.6, 'calculation'); sqlite>

[sqlite] Simple Math Question

At 23:34 28/10/2015, you wrote: >--- > > Those binary representations can be converted back into precise decimal > > representations, but those decimal representations will not be the > original > > decimal values, because they were translated from decimal strings into > > binary floating-point

[sqlite] Simple Math Question

On 28 Oct 2015, at 11:23pm, SQLite mailing list wrote: > This can't possibly work. "Fuzzy equality" is not transitive (x is close > enough to y, y is close enough to z, but x is just far enough from z to be > non-equal), which would break any indexing scheme. Oh crumbs. You're right. I

[sqlite] Simple Math Question

On 29 October 2015 at 09:46, SQLite mailing list < sqlite-users at mailinglists.sqlite.org> wrote: > > which I understood to mean, "if you can represent it in decimal, you > can represent it in binary". I didn't think that was true, but there > seemed to be concensus that it was. > The consensus

[sqlite] Simple Math Question

On 29 Oct 2015, at 2:09am, SQLite mailing list wrote: > The consensus was the other way: "If you can represent it in binary, you > can represent it in decimal." Well that one is actually true. If you can represent any non-recurring fraction in binary, in decimal it's a non-recurring fraction

[sqlite] Simple Math Question

On Thu, 29 Oct 2015 10:09:28 +0800 SQLite mailing list wrote: > The consensus was the other way: "If you can represent it in binary, > you can represent it in decimal." Gah, I see now. Thank you for the clarification. --jkl

[sqlite] Mailing list policy change

On Wed, Oct 28, 2015 at 6:52 PM, General Discussion of SQLite Database < sqlite-users at mailinglists.sqlite.org> wrote: > Effective immediately, the sender email address for mailing list posts > will be elided. All replies must go back to the mailing list itself. > Please reconsider. Not

[sqlite] Casting ctype functions' arguments

On 10/28/15, SQLite mailing list wrote: > Hi, everyone. > > I've been auditing the OpenBSD codebase for calls to ctype functions > with potentially signed chars. This is undefined on some platforms. I > found a number of instances in Sqlite, so I cloned your repo and ran my > script on it.

[sqlite] Mailing list policy change

Assuming the A*spammer is a basic algo subscribed to the list, and sending to any sender in a reasonably short time after posting; Question: Is it possible for the admin to easily backup the list, bisect it, and test for spam? That technique should identify the offending address in

[sqlite] How would sqlite read this from disk?

I'm trying to figure out how SQLite3 would read the schema and data off the disk below. I read https://www.sqlite.org/fileformat2.html but didn't find what I was looking for. In college databases, we calculated the estimated number of blocks (512-byte device blocks) read given schema and

[sqlite] How would sqlite read this from disk?

On Thu, Oct 29, 2015 at 3:29 PM, Jason H wrote: > I'm trying to figure out how SQLite3 would read the schema and data off > the disk below. I read https://www.sqlite.org/fileformat2.html but didn't > find what I was looking for. > In college databases, we calculated the estimated number of

[sqlite] Backing up SQLite DB with unbuffered I/O

Is there a way to perform an online backup with unbuffered I/O? I am currently performing backups with the command-line tool and ".backup", but this appears to wreak havoc on the OS disk cache. I am able to mitigate this by disabling buffering on the destination drive, but that cannot be used

[sqlite] Mailing list policy change

On 29 Oct 2015, at 2:21pm, Adam Devita wrote: > Question: Is it possible for the admin to easily backup the list, > bisect it, and test for spam? > That technique should identify the offending address in > log2(N-Users-Subscribed) attempts. You don't even need to mess with the genuine list

[sqlite] Alexa has been banished. Was: Mailing list policy change

On 10/29/15, Simon Slavin wrote: > > You don't even need to mess with the genuine list server. Just grab all the > addresses it sends to and use another computer to send messages directly to > those addresses using the binary-chop thing. Dan wrote a scribe to do this, which identified the

[sqlite] Backing up SQLite DB with unbuffered I/O

Eric Grange wrote: > Is there a way to perform an online backup with unbuffered I/O? A tool like ionice changes the I/O priority, which is something different. You would have to modify SQLite, or write your own VFS. Regards, Clemens

[sqlite] How would sqlite read this from disk?

On 29 Oct 2015, at 2:29pm, Jason H wrote: > In college databases, we calculated the estimated number of blocks (512-byte > device blocks) read given schema and statistics. Now, I'm asking how would > SQLite actually do that? SQLite uses a block structure in its database files, but one file

[sqlite] Alexa has been banished. Was: Mailing list policy change

On 29 Oct 2015, at 3:28pm, Richard Hipp wrote: > Dan wrote a scribe to do this, which identified the offender. Woo hoo ! Thanks to Dan for this and everything else he does for the list. Simon.

[sqlite] How would sqlite read this from disk?

> Sent: Thursday, October 29, 2015 at 12:10 PM > From: "Simon Slavin" > To: "SQLite mailing list" > Subject: Re: [sqlite] How would sqlite read this from disk? > > > On 29 Oct 2015, at 2:29pm, Jason H wrote: > > > In college databases, we calculated the estimated number of blocks > >

[sqlite] How would sqlite read this from disk?

On Thu, Oct 29, 2015 at 10:20 AM, Jason H wrote: > > If I could ask a followup question. You made the statement "SQLite reads > that row of the table from storage, from the first column to the last > column needed by the SELECT, but perhaps not all the way to the end of the > columns in the

[sqlite] How would sqlite read this from disk?

On 29 Oct 2015, at 5:20pm, Jason H wrote: > Thanks, this is some really great information! You're welcome. > If I could ask a followup question. You made the statement "SQLite reads that > row of the table from storage, from the first column to the last column > needed by the SELECT, but

[sqlite] How would sqlite read this from disk?

> Sent: Thursday, October 29, 2015 at 1:34 PM > From: "Scott Hess" > To: "SQLite mailing list" > Subject: Re: [sqlite] How would sqlite read this from disk? > > On Thu, Oct 29, 2015 at 10:20 AM, Jason H wrote: > > > > If I could ask a followup question. You made the statement "SQLite reads >

[sqlite] Documentation change suggestion: formatchng.html

I find nothing in error on the page but it could perhaps be updated to mention that from version 3.9.0 of SQlite, file format changes can happen only when the middle number changes. Or something of the sort. Irrelevant to that, I somehow expected the

[sqlite] How would sqlite read this from disk?

On Thu, Oct 29, 2015 at 10:59 AM, Jason H wrote: > > The documentation does not go into the detail of the engine is able to > skip the reading of unneeded interior rows. In theory, it can because the > length is contained in the header. So instead of read() on every column in > the row, it can

[sqlite] How would sqlite read this from disk?

It reads a complete page at a time so there is no seeking other than to the start of each row - in the sense of a disk seek. Note that there may be multiple required rows on the same page if the row length is much less than the page length, or if rows are longer than a size determined by some

[sqlite] How would sqlite read this from disk?

> Sent: Thursday, October 29, 2015 at 1:53 PM > From: "Simon Slavin" > To: "SQLite mailing list" > Subject: Re: [sqlite] How would sqlite read this from disk? > > > On 29 Oct 2015, at 5:20pm, Jason H wrote: > > > Thanks, this is some really great information! > > You're welcome. > > > If

[sqlite] How would sqlite read this from disk?

On 29 Oct 2015, at 6:09pm, Jason H wrote: > This seems to be at odds with my understanding of "2.1 Record Format" of the > document. Given that it reads the row varint, which contains the length of > itself and the serial types of the columns, in order, it should be completely > able to skip

[sqlite] How would sqlite read this from disk?

On 10/29/15, Jason H wrote: > > This seems to be at odds with my understanding of "2.1 Record Format" of the > document. Given that it reads the row varint, which contains the length of > itself and the serial types of the columns, in order, it should be > completely able to skip the reading of

[sqlite] How would sqlite read this from disk?

> Sent: Thursday, October 29, 2015 at 2:04 PM > From: "Paul Sanderson" > To: "SQLite mailing list" > Subject: Re: [sqlite] How would sqlite read this from disk? > > It reads a complete page at a time so there is no seeking other than > to the start of each row - in the sense of a disk seek. > >

[sqlite] How would sqlite read this from disk?

On Thu, Oct 29, 2015 at 7:09 PM, Jason H wrote: > This seems to be at odds with my understanding of "2.1 Record Format" of > the document. Given that it reads the row varint, which contains the length > of itself and the serial types of the columns, in order, it should be > completely able to

[sqlite] How would sqlite read this from disk?

On Thu, Oct 29, 2015 at 7:22 PM, Jason H wrote: > > Or is that just crazy talk? > Mostly, yes. You seem to think that reading from disk at arbitrary offsets and in arbitrarily small increments will be efficient. It won't be. And as you pointed out, that ignores transactionality, WAL mode, etc...

[sqlite] How would sqlite read this from disk?

On 10/29/15, Jason H wrote: > > Ah, so this is what I seem to have missed. The pages... This is unfortunate > as the read-heavy application won't likely benefit from SQLite. Your filesystem and your disk hardware work the same way. Your application might only ask for 10 bytes, but the

[sqlite] How would sqlite read this from disk?

[sqlite] Documentation change suggestion: formatchng.html

On Thu, Oct 29, 2015 at 6:59 PM, Simon Slavin wrote: > I find nothing in error on the page > > > > but it could perhaps be updated to mention that from version 3.9.0 of > SQlite, file format changes can happen only when the middle number > changes. Or

[sqlite] How would sqlite read this from disk?

On Thu, Oct 29, 2015 at 7:32 PM, Wade, William wrote: > From https://www.sqlite.org/fileformat2.html, if a row does not fit into > a single block, its overflow is stored in a linked list. The implication > (to me) is that if a particular row requires twenty blocks of storage, and > I want to

[sqlite] How would sqlite read this from disk?

On 10/29/15, Wade, William wrote: > From https://www.sqlite.org/fileformat2.html, if a row does not fit into a > single block, its overflow is stored in a linked list. The implication (to > me) is that if a particular row requires twenty blocks of storage, and I > want to read a one-byte field

[sqlite] Documentation change suggestion: formatchng.html

On 10/29/15, Dominique Devienne wrote: > On Thu, Oct 29, 2015 at 6:59 PM, Simon Slavin wrote: > >> I find nothing in error on the page >> >> >> That page was really old and mostly obsolete - at least the header text was. I've rewritten it. Please press

[sqlite] Security vulnerability in the SQLite Encryption Extension

Part of the interface design to the SQLite Encryption Extension makes it possible for an application to accidentally supply weak keys. This only happens if the keying interface is misused. Though to be fair, the documentation could be clearer about how to avoid misusing the interface. If an

[sqlite] How would sqlite read this from disk?

On 10/29/15, Dominique Devienne wrote: > > I've discussed blobs a few times already :). Especially the fact you can't > have transactional and incremental updates of blobs (i.e. the transaction > "copies" only the blob pages actually modified). Large blobs are transactional when they are stored

[sqlite] How would sqlite read this from disk?

On Thu, Oct 29, 2015 at 8:24 PM, Richard Hipp wrote: > On 10/29/15, Dominique Devienne wrote: > > > > I've discussed blobs a few times already :). Especially the fact you > can't > > have transactional and incremental updates of blobs (i.e. the transaction > > "copies" only the blob pages

[sqlite] How would sqlite read this from disk?

> On 10/29/15, Jason H wrote: > > > > Ah, so this is what I seem to have missed. The pages... This is unfortunate > > as the read-heavy application won't likely benefit from SQLite. > > Your filesystem and your disk hardware work the same way. Your > application might only ask for 10 bytes, but

[sqlite] How would sqlite read this from disk?

On 10/29/15, Jason H wrote: >> > If I were to try to work around... Before we go any further, have you actually measured a performance problem? Or are you optimizing without prior knowledge of where your application is spending time? -- D. Richard Hipp drh at sqlite.org

[sqlite] Documentation change suggestion: formatchng.html

On 29 Oct 2015, at 6:58pm, Richard Hipp wrote: > That page was really old and mostly obsolete - at least the header > text was. I've rewritten it. Please press "Reload" Good. Will it be updated for everything up to and including partial indexes ? It's a lot of work so I don't expect it

[sqlite] Documentation change suggestion: formatchng.html

On 10/29/15, Simon Slavin wrote: > > On 29 Oct 2015, at 6:58pm, Richard Hipp wrote: > >> That page was really old and mostly obsolete - at least the header >> text was. I've rewritten it. Please press "Reload" > > Good. Will it be updated for everything up to and including partial indexes > ?

[sqlite] How would sqlite read this from disk?

> > On 10/29/15, Jason H wrote: > >> > > If I were to try to work around... > > Before we go any further, have you actually measured a performance > problem? Or are you optimizing without prior knowledge of where your > application is spending time? Currently, I have a SQLite database of

[sqlite] How would sqlite read this from disk?

On 10/29/15, Jason H wrote: > > I'm open to ideas, but I was going to use this as an excuse to invent > something of a general tool. Post your schema and query. Also run ANALYZE and post the content of the sqlite_stat1 table. -- D. Richard Hipp drh at sqlite.org

[sqlite] How would sqlite read this from disk?

On 29 Oct 2015, at 8:59pm, Jason H wrote: > Currently, I have a SQLite database of around 10gig that takes 25 minutes to > run a single query against no other activity (it's never queried in > read/write, just read). I've created indexes the best I can. You can do better. I have a 43

[sqlite] How would sqlite read this from disk?

On 29 Oct 2015, at 6:22pm, Jason H wrote: > Ah, so this is what I seem to have missed. The pages... This is unfortunate > as the read-heavy application won't likely benefit from SQLite. Only a small > fraction of the row data is used each query, and it seems like It'll have to > read the

[sqlite] How would sqlite read this from disk?

On 29 Oct 2015, at 7:49pm, Jason H wrote: > If I were to try to work around all this excess reading, how good or bad > would it be to take the following approach: > Define a set of smaller tables (column count and size) with a common key, > then create a view for the specific query, then

[sqlite] How would sqlite read this from disk?

> On 10/29/15, Jason H wrote: > > > > I'm open to ideas, but I was going to use this as an excuse to invent > > something of a general tool. > > Post your schema and query. Also run ANALYZE and post the content of > the sqlite_stat1 table. I really appreciate the offer, but unfortunately, I

[sqlite] Alexa has been banished. Was: Mailing list policy change

damn! I wanted a date! > On 10/29/15, Simon Slavin wrote: > > > > You don't even need to mess with the genuine list server. Just grab all the > > addresses it sends to and use another computer to send messages directly to > > those addresses using the binary-chop thing. > > Dan wrote a

[sqlite] Alexa has been banished. Was: Mailing list policy change

I save one. I can forward it to you, if you'd like. > On Oct 29, 2015, at 5:37 PM, Nicolas J?ger wrote: > > damn! I wanted a date! > >>> On 10/29/15, Simon Slavin wrote: >>> >>> You don't even need to mess with the genuine list server. Just grab all the >>> addresses it sends to and use

[sqlite] Alexa has been banished. Was: Mailing list policy change

On 29 Oct 2015, at 9:34pm, Nicolas J?ger wrote: > damn! I wanted a date! Erm ... those pictures are of Angie Varona, a teenager from California. Her name is not even like close to Alexa. They just stole her picture. Unless, of course, you wanted a date with the woman who wrote the text.

[sqlite] Alexa has been banished. Was: Mailing list policy change

> Erm ... those pictures are of Angie Varona, a teenager from California. ok, you even know the name... their parents should be punished to let their daughter posting such pictures... stupid parents who don't take care of their children... sry about writting that, but that's exactly how I

[sqlite] SQLite build dependency on tclsh

Hello everyone. > > Hi, > > with the recent replacement of awk build scripts with tclsh ones, a > > problem has risen. The tcl is not commonly available when building the > > environment/distribution from ground up (bootstrap), and as a > > consequence, SQLite can no longer be built in these