Re: [sqlite] SQLite slow when lots of tables

2019-01-30 Thread Wout Mertens
Mertens wrote: > > >To: SQLite mailing list > >Subject: Re: [sqlite] SQLite slow when lots of tables > > > > I always have to explain to people that there's no magic sauce that > > "real databases" add versus SQLite. > > > SQLite us

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Igor Korot
Peter, On Tue, Jan 29, 2019 at 7:53 PM Peter da Silva wrote: > > On Tue, Jan 29, 2019, 7:46 PM Igor Korot > > You can install mySQL/MariaDB for free and use it for your needs. > > I believe that if you pay to Oracle/MariaDB Foundation, it will be > > just for support. (I may be wrong though). >

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Peter da Silva
On Tue, Jan 29, 2019, 7:46 PM Igor Korot You can install mySQL/MariaDB for free and use it for your needs. > I believe that if you pay to Oracle/MariaDB Foundation, it will be > just for support. (I may be wrong though). > Or better, PostgreSQL. We have a system that uses SQLite as an optional

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Igor Korot
Hi, On Tue, Jan 29, 2019 at 7:06 PM Keith Medcalf wrote: > > On Tuesday, 29 January, 2019 16:28, Wout Mertens wrote: > > >To: SQLite mailing list > >Subject: Re: [sqlite] SQLite slow when lots of tables > > > > I always have to explain to people that ther

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Keith Medcalf
On Tuesday, 29 January, 2019 16:28, Wout Mertens wrote: >To: SQLite mailing list >Subject: Re: [sqlite] SQLite slow when lots of tables > > I always have to explain to people that there's no magic sauce that > "real databases" add versus SQLite. > SQLite uses the s

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Wout Mertens
I always have to explain to people that there's no magic sauce that "real databases" add versus SQLite. SQLite uses the same techniques all databases use, and thanks to the absence of a network later, you avoid a lot of latency, so it can actually be faster. (I do believe that SQLite optimizes

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread James K. Lowden
On Tue, 29 Jan 2019 12:00:49 +0100 (CET) mzz...@libero.it wrote: > what happens if I put all data in a single table and this table > become very huge (for example millions of rows)? Big tables are your friend, actually. A binary search on 1 million rows requires at most 20 operations. A

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread mzzdvd
thanks a lot for fast answers. Now I'll try to modify my DB structure to use one table. Thanks to all. Bye. > > Il 29 gennaio 2019 alle 12.29 Simon Slavin ha > scritto: > > On 29 Jan 2019, at 11:00am, mzz...@libero.it wrote: > > > > > > what happens if I put all

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Rob Willett
Millions of rows is not even large, never mind huge or very huge :) We have tables with hundreds of millions of rows, we got to billions of rows in a single table before we changed the logic. From memory we had 67GB for a single database and I reckon 60GB was one table. Not many issues at all

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Andy Bennett
Hi, what happens if I put all data in a single table and this table become very huge (for example millions of rows)? Will I have same performace problems? The INSERTing profile has been covered by others but I'll just add something about SELECT as well. It depends on how you query it.

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Simon Slavin
On 29 Jan 2019, at 11:00am, mzz...@libero.it wrote: > what happens if I put all data in a single table and this table become very > huge (for example millions of rows)? > > Will I have same performace problems? I used to use a 43 Gigabyte SQLite database of which 42 Gigabytes was taken up

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Chris Locke
Ryan Smith has already covered this scenario. "And to add to the slew of "Few-tables-many-rows rather than Many-tables-few-rows" solutions offered, one thing to note: After a single table contains a few million rows, INSERTing will become slightly slower, but the difference will still be

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread mzzdvd
Dear all, what happens if I put all data in a single table and this table become very huge (for example millions of rows)? Will I have same performace problems? Thanks. Regards. > > Il 28 gennaio 2019 alle 17.28 Simon Slavin ha > scritto: > > On 28 Jan 2019, at 4:17pm,

Re: [sqlite] SQLite slow when lots of tables

2019-01-29 Thread Chris Locke
I don't know - that's why I asked. Thanks for the clarification. On Mon, Jan 28, 2019 at 10:10 PM Warren Young wrote: > On Jan 28, 2019, at 2:44 PM, Chris Locke wrote: > > > >> The table name should not be meaningful to your application; nothing in > >> your application should conjure up a

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread Simon Slavin
On 28 Jan 2019, at 9:44pm, Chris Locke wrote: >> The table name should not be meaningful to your application; nothing in your >> application should conjure up a table name. > > I can't get my head around this advice. Is this just for this occasion, or > for every application? What if I'm

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread Warren Young
On Jan 28, 2019, at 2:44 PM, Chris Locke wrote: > >> The table name should not be meaningful to your application; nothing in >> your application should conjure up a table name. > > I can't get my head around this advice. Is this just for this occasion, or > for every application? What if I'm

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread Chris Locke
> The table name should not be meaningful to your application; nothing in > your application should conjure up a table name. I can't get my head around this advice. Is this just for this occasion, or for every application? What if I'm writing a customer address book? Am I allowed a table

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread Warren Young
On Jan 28, 2019, at 9:17 AM, mzz...@libero.it wrote: > > I developed an application that need to create 1 table with thousand of rows > every time when a certain event occours. > > This works in a good way, but when the number of the tables become huge > (about 15000/2 tables) the first

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread James K. Lowden
On Mon, 28 Jan 2019 16:28:41 + Simon Slavin wrote: > SQL is not designed to have a variable number of tables in a > database. All the optimization is done assuming that you will have a > low number of tables, and rarely create or drop tables. This. The table name should not be meaningful

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread Igor Korot
Hi, On Mon, Jan 28, 2019 at 10:17 AM wrote: > > Dear, > > I developed an application that need to create 1 table with thousand of rows > every time when a certain event occours. Are you coming from the FoxBase/ForPro world? Thank you. > > This works in a good way, but when the number of the

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread R Smith
When the same solution is offered in rapid intervals, it must be the correct one then! :) To elaborate on what Chris mentioned regarding relational database methodology and related tables, here is why and a good description of how to do that correctly:

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread R Smith
On 2019/01/28 6:17 PM, mzz...@libero.it wrote: I developed an application that need to create 1 table with thousand of rows every time when a certain event occours. This works in a good way, but when the number of the tables become huge (about 15000/2 tables) the first DataBase reading

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread Chris Locke
Why do you need to create a new table each time? Its easier to create a relational database. This means create one main table that might (for example) have a 'tableID' field, which points to just one other table. This means you only need two tables. Not 20,000+ Just an idea. Depends on what

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread Simon Slavin
On 28 Jan 2019, at 4:17pm, mzz...@libero.it wrote: > when the number of the tables become huge (about 15000/2 tables) the > first DataBase reading query, after Database open, is very slow (about 4sec.) > while next reading operations are faster. > > How can I speed up? Put all the data in

Re: [sqlite] SQLite slow when lots of tables

2019-01-28 Thread Richard Hipp
On 1/28/19, mzz...@libero.it wrote: > Dear, > > I developed an application that need to create 1 table with thousand of rows > every time when a certain event occours. > > This works in a good way, but when the number of the tables become huge > (about 15000/2 tables) the first DataBase

[sqlite] SQLite slow when lots of tables

2019-01-28 Thread mzzdvd
Dear, I developed an application that need to create 1 table with thousand of rows every time when a certain event occours. This works in a good way, but when the number of the tables become huge (about 15000/2 tables) the first DataBase reading query, after Database open, is very slow