Re: [sqlite] Coding standard

2019-12-12 Thread Valentin Davydov
e point of view of program logic it looks similar (at least for me), but shifts security burden from you to authors of libc. And of course this should calm static analyzers anxious about strlen(), sprintf() etc. Valentin Davydov. ___ sqlite-users

Re: [sqlite] How to increase performance when inserting a lot of small data into table using indices

2019-09-12 Thread Valentin Davydov
er successfull commit. Thus you shift the burden from SQL to filesystem which is less limited by natural data structure and might perform better. Valentin Davydov. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqli

[sqlite] 2D query

2017-12-13 Thread Valentin Davydov
t;M"); How to construct a query which returns coalesced sex but individual names, such as "F: Alex, Jane. M: Alex, John."? Sincerely, Valentin Davydov. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] whish list for 2016

2015-12-28 Thread Valentin Davydov
On Fri, Dec 25, 2015 at 12:51:55PM +, Simon Slavin wrote: > > On 25 Dec 2015, at 12:39pm, Valentin Davydov > wrote: > > > It would be desirable to improve algorithm of INTEGRITY_CHECK pragma. > > Presently it is generally useless for indexed databases which don't

[sqlite] whish list for 2016

2015-12-25 Thread Valentin Davydov
Hi, All! It would be desirable to improve algorithm of INTEGRITY_CHECK pragma. Presently it is generally useless for indexed databases which don't fit entirely in RAM (and which usually need checking much strongly than smaller ones). Valentin Davydov.

[sqlite] Native sqlite4 on FreeBSD

2015-09-11 Thread Valentin Davydov
did in the past) produce some nice bugs. Attention to developers. > Note that there isn't a port, I know. Despite the code being frozen for almost a full year, nobody wants to take a trouble of maintaining FreeBSD port (perhaps me too). Valentin Davydov.

[sqlite] Native sqlite4 on FreeBSD

2015-09-10 Thread Valentin Davydov
Hi, all! How to build native sqlite4 (preferrably statically linked shell only, without any extensions) on a recent version of FreeBSD? It has clang instead of gcc as a default system compiler, so makefiles bundled with sqlite sources don't work. Valentin Davydov.

[sqlite] AUTOINC vs. UUIDs

2015-05-21 Thread Valentin Davydov
On Wed, May 20, 2015 at 11:52:08PM +, Peter Aronson wrote: > Now you're just getting silly. ?What if the application sets all rowids, > everywhere to 1? ?The fact is, the chance of collision on a UUID is pretty > astronomically low as long as a decent source of entropy is used >

Re: [sqlite] Using Sqlite3 as a Change Time Recording Data Store in Glusterfs

2014-11-24 Thread Valentin Davydov
o the RAM cache. Of course, cache itself should be populated in advance to give this benefit, and, given current RAM prices, it seems not very feasible to steal available memory from smart applications in favour of dumb cache. Hope, this considerations will help you in tuning your code. V

Re: [sqlite] 50% faster than 3.7.17

2014-09-22 Thread Valentin Davydov
> number of CPU cycles. Is there any similar benchmarks with regard to disk i/o operations rather than CPU? Especially random read of cache misses, I mean. Valentin Davydov. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:

Re: [sqlite] Slow select from database

2014-03-12 Thread Valentin Davydov
n abovementioned index, changing autovacuum to forced vacuum sceduled at specific time of day etc.) or give it more resources (say, by putting the database on an SSD drive). Valentin Davydov. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] SQLite version 3.8.2 running 2.5x faster for some queries.

2013-11-24 Thread Valentin Davydov
not been done yet. To my opinion, the most general solution is to let to the application programmer to decide whether to calcucale the function once (say, at the beginning of a transaction), store the result and then access the stored value, or to make the new call t

Re: [sqlite] SQLite scalability

2013-11-22 Thread Valentin Davydov
se. The only definive SQLite limits are documentet in the relevant manual page. Valentin Davydov. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Write performance question for 3.7.15

2012-12-29 Thread Valentin Davydov
On Fri, Dec 28, 2012 at 03:35:17PM -0600, Dan Frankowski wrote: > > 3. Would horizontal partitioning (i.e. creating multiple tables, each for a > different key range) help? This would seriously impair read performance (you'd have to access two indices instead of one). Valenti

Re: [sqlite] Fwd: Write performance question for 3.7.15

2012-12-29 Thread Valentin Davydov
haps related to table size) that would change > this write performance? CACHE_SIZE. It makes sense to enlarge it up to the all available memory. Valentin Davydov. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Any limitation of file size on sqlite?

2012-12-17 Thread Valentin Davydov
es of many terabytes in size, provided underlying filesystem supports them, of course. Be prepared to wait quite a long for non-sqlite operations (copy, delete etc.) to complete on such a big files. Valentin Davydov. ___ sqlite-users mailing list sqlite-

Re: [sqlite] IP address support (was: Consequences of lexicographic sorting of keys in SQLite4?)

2012-07-11 Thread Valentin Davydov
are nicely supported in the form of unsigned integers, and prefixes/ranges - as contiguous ranges of such integers. For example, to determine whether given IP address belongs to a particular subnet, one can calculate "IP between NETWORK_MIN and NETWORK_MAX"

Re: [sqlite] Can't create empty database

2012-06-25 Thread Valentin Davydov
"pragma page_size=512; create table t(c);drop table t;vacuum;" $ wc -c test.db 512 test.db $ sqlite3 test.db "pragma page_size;" 512 $ Valentin Davydov. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] index creation for single-use DB

2012-05-13 Thread Valentin Davydov
On Sun, May 13, 2012 at 11:02:54AM +0300, Baruch Burstein wrote: > I am using an in-memory DB to load data into it, do a few sort / find > duplicates / SELECTs, and then dispose of the DB. It can vary in size from > a few thousand rows to over a million. > Would the time used for creating an index

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-09 Thread Valentin Davydov
suring validity of input data such as right number of fields in each row etc.) and not rely on the sqlite shell csv-parsing capabilities. Valentin Davydov. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-05 Thread Valentin Davydov
e file size or read pointer position in the input data file. Both of them grow linearly with amount of data processed. > I've started several loads only to find out hours later that > nothing has been loaded. Anyway be prepared to spend some of your t

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-04 Thread Valentin Davydov
ble RAM would help a bit. Also, don't forget to turn off journaling and wrap all in a single transaction when creating database for the first time. Valentin Davydov. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] question about zipvfs & performance

2012-03-25 Thread Valentin Davydov
cy storage, such as SSD drive for small databases or RAID array with plenty ow write cache memory for huge ones. > is there an overhead with zipvfs ? You can easily measure this overhead yourself on the in-memory database. RAM is cheap now ;-) Valentin Davydov. ___

Re: [sqlite] Slow inserts in indexed table

2012-02-09 Thread Valentin Davydov
On Thu, Feb 09, 2012 at 02:05:53PM +, Simon Slavin wrote: > > On 9 Feb 2012, at 1:45pm, Valentin Davydov wrote: > > > CREATE TABLE T(X, Y, Z, ..., UNIQUE (X, Y, Z, ...) ON CONFLICT IGNORE); > > CREATE INDEX IX ON T(X); > > CREATE INDEX IY ON T(Y);

[sqlite] Slow inserts in indexed table

2012-02-09 Thread Valentin Davydov
, and all time is spent waiting for disk _reading_ (rarely interspersed with fast and happy write bursts on each COMMIT). What is sqlite reading there? Does it try to perfectly balance each index on each insert (million times per transaction) or something else? Sincerely, Valentin Davydov

[sqlite] Shell bug: incorrect handling of some BLOBs.

2011-11-10 Thread Valentin Davydov
Subsystem: Shell Title: Incorrect handling of some BLOBs. Type: Code_Defect Severity: Important Priority: Unknown Affected: All SQLite versions containing output_hex_blob() up to 3.7.9. Environment: Tested on various FreeBSD 8.x versions, both i386 and amd64. Probably, other operating