[sqlite] Performance issue

2015-04-08 Thread Jeff Roux
Thanks everyone for your answers, I made some changes to the database according to the information you gave me. It improved the performance of the query by about 20% (the request now takes 4 seconds instead of 5). Here are some more information, regarding all the suggestions I received: - The

[sqlite] Performance issue

2015-04-08 Thread Dominique Devienne
On Wed, Apr 8, 2015 at 11:16 AM, Jeff Roux wrote: > # For mysql, I use: > /etc/init.d/mysql stop; /etc/init.d/mysql start; \ > time echo 'SELECT SUM(nbBytesDecompOut+nbBytesCompIn) as vol, portLan \ > as item FROM flows WHERE timestamp>=1383770600 AND \ > timestamp<=1384770600 AND

[sqlite] Performance issue

2015-04-08 Thread Simon Slavin
On 8 Apr 2015, at 10:16am, Jeff Roux wrote: > SELECT SUM(nbBytesDecompOut+nbBytesCompIn) as vol, portLan \ > as item FROM flows WHERE timestamp>=1383770600 AND \ > timestamp<=1384770600 AND protocol IN (17, 6) GROUP BY portLan \ > ORDER BY vol DESC LIMIT 6 If you want us to

[sqlite] Performance issue

2015-04-08 Thread R.Smith
On 2015-04-08 11:35 AM, Dominique Devienne wrote: > On Wed, Apr 8, 2015 at 11:16 AM, Jeff Roux wrote: > >> # For mysql, I use: >> /etc/init.d/mysql stop; /etc/init.d/mysql start; \ >> time echo 'SELECT SUM(nbBytesDecompOut+nbBytesCompIn) as vol, portLan \ >> as item FROM flows WHERE

[sqlite] Performance issue

2015-04-08 Thread Dominique Devienne
On Wed, Apr 8, 2015 at 1:24 PM, R.Smith wrote: > On 2015-04-08 11:35 AM, Dominique Devienne wrote: > >> On Wed, Apr 8, 2015 at 11:16 AM, Jeff Roux wrote: >> >>> time echo 'SELECT ... FROM flows WHERE timestamp>=1383770600 AND \ >>> timestamp<=1384770600 AND protocol IN (17, 6) GROUP BY

[sqlite] Performance issue

2015-04-08 Thread Richard Hipp
On 4/8/15, Dominique Devienne wrote: > With a LIMIT clause, in > such a GROUP BY ORDER BY returning a large result set, would SQLite: > 1) sort the whole result-set and then keep only the first top-N rows? > 2) or instead do a partial-sort of the first top-N rows only, SQLite must examine all

[sqlite] Performance issue

2015-04-08 Thread Dominique Devienne
On Wed, Apr 8, 2015 at 2:09 PM, Richard Hipp wrote: > On 4/8/15, Dominique Devienne wrote: > > With a LIMIT clause, in > > such a GROUP BY ORDER BY returning a large result set, would SQLite: > > 1) sort the whole result-set and then keep only the first top-N rows? > > 2) or instead do a

[sqlite] Performance issue

2015-04-08 Thread R.Smith
On 2015-04-08 01:57 PM, Dominique Devienne wrote: >> No Dominique, it's not that - >> > Perhaps. But that still doesn't get to my point. With a LIMIT clause, in > such a GROUP BY ORDER BY returning a large result set, would SQLite: > 1) sort the whole result-set and then keep only the first

[sqlite] Transpose selected rows into columns

2015-04-08 Thread Drago, William @ CSG - NARDA-MITEQ
Jim, This works quite well. Thank you. And thanks to all others who replied. -- Bill Drago Senior Engineer L3 Narda-MITEQ 435 Moreland Road Hauppauge, NY 11788 631-272-5947 / William.Drago at L-3COM.com > -Original Message- > From: sqlite-users-bounces at mailinglists.sqlite.org

[sqlite] Transpose selected rows into columns

2015-04-08 Thread Jim Callahan
You are welcome. The crosstab followed by a calculation is a common pattern in statistics, political science and accounting (if you added a percent change column after the diff you would have the classic "accounting variance" report). Using an intermediate TABLE or VIEW is an "information

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread Fabian Pröbstl
Hi there! Currently, we are using SQLite as our application file format for a Windows 7/C#/System.Data.SQLite based desktop application. We only allow one instance to open the file by running "set locking_mode=EXCLUSIVE;BEGIN EXCLUSIVE;COMMIT" when connecting to the database. This all works

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread Hick Gunter
Even if there were a simple way to protect an SQLite db file from being casually (or even maliciously) overwritten by a user (which there isn't), it is quite impossible to prevent a user with "root privileges" from accessing/altering/deleting/moving/renaming ANY file on any operating system

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread R.Smith
On 2015-04-08 04:18 PM, Fabian Pr?bstl wrote: > Hi there! > > Currently, we are using SQLite as our application file format for a Windows > 7/C#/System.Data.SQLite based desktop application. We only allow one instance > to open the file by running "set locking_mode=EXCLUSIVE;BEGIN >

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread Simon Slavin
On 8 Apr 2015, at 3:51pm, R.Smith wrote: > BEGIN EXCLUSIVE - Locks the database from other SQLite3 database connections > for the time being. > COMMIT; - Unlocks it again - so calling all this in one go is pointless. > > That said, database locking serves only to protect from other database

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread Dan Kennedy
On 04/08/2015 09:51 PM, R.Smith wrote: > > > On 2015-04-08 04:18 PM, Fabian Pr?bstl wrote: >> Hi there! >> >> Currently, we are using SQLite as our application file format for a >> Windows 7/C#/System.Data.SQLite based desktop application. We only >> allow one instance to open the file by

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread R.Smith
On 2015-04-08 05:38 PM, Dan Kennedy wrote: > On 04/08/2015 09:51 PM, R.Smith wrote: >> >> >> On 2015-04-08 04:18 PM, Fabian Pr?bstl wrote: >>> Hi there! >>> >>> Currently, we are using SQLite as our application file format for a >>> Windows 7/C#/System.Data.SQLite based desktop application. We

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread maarten.most...@stakepoint.com
I have the same problem but in order to prevent overriding I actually make a hidden copy of the file in the local app directory when opening it. This allows me to recover it if the application breaks and to implement save, save As etc. and if it is no longer there I can still save where it

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread R.Smith
On 2015-04-08 06:00 PM, Dan Kennedy wrote: > On 04/08/2015 10:52 PM, R.Smith wrote: >> >> >> On 2015-04-08 05:38 PM, Dan Kennedy wrote: >>> On 04/08/2015 09:51 PM, R.Smith wrote: On 2015-04-08 04:18 PM, Fabian Pr?bstl wrote: > Hi there! > > Currently, we are using

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 04/08/2015 07:18 AM, Fabian Pr?bstl wrote: > Is there an easy way of configuring / changing this so that > SHARED_WRITE is not acquired? Will SQLite even function? Is it just > easier to create a hidden copy and work on that? For something this

[sqlite] fts5

2015-04-08 Thread Scott Hess
On Wed, Apr 8, 2015 at 12:32 PM, Dan Kennedy wrote: > On 04/08/2015 04:49 AM, Scott Hess wrote: >> Something that bugged me a lot was that I had used deletion markers to >> cancel out hits, but did not provide a way for deletion markers to >> cancel out. The main problem with this was that a

[sqlite] Prevent database file from being overwritten by other processes

2015-04-08 Thread Keith Medcalf
In the Amalgamation Source search for the line (around 37836): dwShareMode = FILE_SHARE_READ | FILE_SHARE_WRITE; If you change this to: dwShareMode = 0 then use this version of sqlite3.c in your application. This will open the file for "exclusive" access and not shared access. This