Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

On 2014/05/28 20:26, Warren Young wrote: On 5/28/2014 11:20, jose isaias cabrera wrote: I would rather have the speed then the size. Rather speed than size is an Engineering decision which is easy on a desktop/server system (such as most Windows/OSX/*nix implementations) but it may well

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

On May 28, 2014 12:36 PM, "Drago, William @ MWG - NARDAEAST" > Don't modern compilers consider what effects the speed optimizations will have on the pipeline and the cache and optimize accordingly? I think they might try to in a broad way, but we live in a world with multiple models of CPUs from

Re: [sqlite] Load time performance

> -Original Message- > From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- > boun...@sqlite.org] On Behalf Of Stephen Chrzanowski > Sent: woensdag 28 mei 2014 17:33 > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Load time performance > > Caching or not, I

Re: [sqlite] Reading compressed database files

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/05/14 02:26, Hadashi, Rinat wrote: > I have 13 tables, of which 2 are huge, 2 are medium and the rest are > very small. My huge tables have 3 columns: numeric, numeric and varchar > with millions of rows. I keep an index on the numeric columns.

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

"Richard Hipp" wrote... On Wed, May 28, 2014 at 1:20 PM, jose isaias cabrera wrote: 3. Is there a spot anywhere that has clear steps on creating the Sqlite3 DLL? http://www.sqlite.org/draft/howtocompile.html#dll The "draft" page above will be promoted to the

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

On Wed, May 28, 2014 at 1:20 PM, jose isaias cabrera wrote: > > 3. Is there a spot anywhere that has clear steps on creating the Sqlite3 > DLL? > http://www.sqlite.org/draft/howtocompile.html#dll The "draft" page above will be promoted to the official website at the next

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

On 5/28/2014 12:35, Drago, William @ MWG - NARDAEAST wrote: Bigger code gets kicked out of the processor cache faster, so the processor has to go back to main memory more often. Don't modern compilers consider what effects the speed optimizations will have on the pipeline and the cache and

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

On 5/28/2014 12:26, Warren Young wrote: On 5/28/2014 11:20, jose isaias cabrera wrote: I would rather have the speed then the size. in today's L1/L2/L3 world, size *is* speed. Also, there is a pretty hard limit on how much micro code optimizations can help a DBMS. It's a fundamentally

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

> -Original Message- > From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- > boun...@sqlite.org] On Behalf Of Warren Young > Sent: Wednesday, May 28, 2014 2:26 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 >

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

On Wed, May 28, 2014 at 12:26 PM, Warren Young wrote: > On 5/28/2014 11:20, jose isaias cabrera wrote: > >> >> I would rather have the speed >> then the size. >> > > Many years ago, I read an article written by a Microsoft employee where > they said they built Windows' own

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

On 5/28/2014 11:20, jose isaias cabrera wrote: I would rather have the speed then the size. Many years ago, I read an article written by a Microsoft employee where they said they built Windows' own binaries optimized for size rather than speed, since in today's L1/L2/L3 world, size *is*

Re: [sqlite] Questions regarding using test_onefile as VFS

Hi, no you have to compile your vfs with sqlite3 amalgamation and its shell in order to use it. After compiling and linking, you will be able to run your vfs. Remember to register your vfs in order to have it available. Hope this helps, Regards, Luca Il 28/mag/2014 14:41 "김병준"

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

"Richard Hipp" wrote... On Wed, May 28, 2014 at 12:56 PM, jose isaias cabrera

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

On Wed, May 28, 2014 at 12:56 PM, jose isaias cabrera wrote: > > Just noticed something... It may be nothing, but the MinGW built DLL has > a size of 645KB while the MSVC built one has a size of 962KB. Just under > 33% bigger. I hope there is nothing missing on the

Re: [sqlite] New DLLs and sources. Was: SQLite version 3.8.5 beta

"Richard Hipp" wrote... On Tue, May 27, 2014 at 10:50 AM, Richard Hipp wrote: On Tue, May 27, 2014 at 10:39 AM, jose isaias cabrera < cabr...@wrc.xerox.com> wrote: H... I am running the original DLL created for 3.8.4.3 on the WinXP and it works fine, so it was not a

Re: [sqlite] Reading compressed database files

On Wed, May 28, 2014 at 11:33 AM, Hadashi, Rinat wrote: > Fixed data. I write the databases once in a different flow, and then I > only read them. > How well does gzip compress the database? In other words, if you do: ls -l original.db gzip original.db

Re: [sqlite] Reading compressed database files

Fixed data. I write the databases once in a different flow, and then I only read them. -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Simon Slavin Sent: Wednesday, May 28, 2014 4:18 PM To: General Discussion of SQLite

Re: [sqlite] Load time performance

Caching or not, I don't experience the same thing with opening a read only text file in Notepad. For S'n'Gs, I had a random text file sitting on my desktop (A config file for a game, and it is pure text) and decided to use it as a subject of abuse. Its original attribute is set so read only is

Re: [sqlite] DELETE & INSERT vs. REPLACE

On 28 May 2014, at 3:55pm, Drago, William @ MWG - NARDAEAST wrote: > Can someone tell me what an appropriate use for REPLACE is? REPLACE in SQlite is just a way of writing INSERT OR REPLACE ... It's best use is when you are not sure whether a record already exists

Re: [sqlite] DELETE & INSERT vs. REPLACE

> -Original Message- > From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users- > boun...@sqlite.org] On Behalf Of Simon Slavin > Sent: Tuesday, May 27, 2014 5:21 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] DELETE & INSERT vs. REPLACE > > > On 27 May 2014, at

Re: [sqlite] Load time performance

On 28 May 2014, at 1:44pm, Stephen Chrzanowski wrote: > What I found was that even changing the files read-only attribute > (Old-school DOS 3.3 days file attribute kinda thing) the initial query lag > hit. The same thing happened when I unset the R/O attrib and set the

Re: [sqlite] Reading compressed database files

On 28 May 2014, at 8:25am, Hadashi, Rinat wrote: > My databases are very big (almost 100 GB). > I am looking for a compression solution. Are these databases with fixed data which can be opened read-only ? Or do you have to be able to make changes to them ? It makes

Re: [sqlite] SQLite3 database on windows network drive and unreliable connection

SQLite is a desktop application, not a network aware application. The file locking mechanisms lie to SQLite which makes it an EXTREMELY HIGH CHANCE that connectivity and any WRITE statements WILL cause data corruption. This isn't the fault of SQLite but the network file system locking. AFAIK,

Re: [sqlite] Load time performance

Read Only mode is going to cause initial lag. I've written an analysis to what I saw at least on a Windows machine several months ago. I didn't try a Linux machine I don't think, but I can re-do it if anyone is interested. That analysis was done on a local drive (RAID-0 SSD setup), not on a

[sqlite] Questions regarding using test_onefile as VFS

The documentation seems to state that in order to use test_onefile, instead of providing an option when compiling sqlite3, test_onefile must be set as vfs with the -vfs command option in the shell. Is my understanding correct? The documentation does not provide examples using vfs demo files

Re: [sqlite] Load time performance

Hello Rinat, I found that if I opened the DB and read it using normal file IO, throwing away the data I read, then closing it and opening it with Sqlite could reduce this initial delay. It depends on the size of the DB though. It's mostly useful for small DB's. C Wednesday, May 28,

Re: [sqlite] Reading compressed database files

Hello Rinat, I compress my data into blobs using Gzip before insert, and decompress when I read the blobs back out but, leave the other data in the DB un-compressed. In that way, I get compression but, normal operations remain fast. This works if the data to compress > 4Kish. My blobs can be 200K

Re: [sqlite] Reading compressed database files

On Wed, May 28, 2014 at 5:26 AM, Hadashi, Rinat wrote: > Hi Roger > > Where can I learn how to characterize my database? > How much does ZIP or gzip compress your database? The amount of compression obtained by CEROD is usually very close to the compression obtained

Re: [sqlite] Reading compressed database files

Hi Roger Where can I learn how to characterize my database? I work read-only, single connection. I have 13 tables, of which 2 are huge, 2 are medium and the rest are very small. My huge tables have 3 columns: numeric, numeric and varchar with millions of rows. I keep an index on the numeric

Re: [sqlite] Reading compressed database files

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/05/14 00:25, Hadashi, Rinat wrote: > My databases are very big (almost 100 GB). I am looking for a > compression solution. > > Did anyone have an experience with reading a compressed database? It would be helpful if you characterise your data

Re: [sqlite] Reading compressed database files

On 2014/05/28 09:25, Hadashi, Rinat wrote: Hi My databases are very big (almost 100 GB). I am looking for a compression solution. Did anyone have an experience with reading a compressed database? What was the degradation in performance of queries? Severe. Well, this depends - if the table

[sqlite] Reading compressed database files

Hi My databases are very big (almost 100 GB). I am looking for a compression solution. Did anyone have an experience with reading a compressed database? What was the degradation in performance of queries? Thanks Rinat Hadashi

Re: [sqlite] Load time performance

Hi, I work in READ ONLY mode. My application connects the DB only once, at the beginning. I can't really work with local files. (I log to any machine and get my files from the network.) Perhaps there are some intermediate files generated in the first load that I can prepare in advance? Rinat