Re: [sqlite] Parallel access to read only in memory database

Daniel Meyer wrote: We are interested in using sqlite as a read only, in memory, parallel access database. We have database files that are on the order of 100GB that we are loading into memory. We have found great performance when reading from a single thread. We need to scale up to have many

Re: [sqlite] Parallel access to read only in memory database

Duarte) > 15. Re: Error trying to make sqlite3 documentation (Richard Hipp) > 16. Fwd: Lemon: Simple recursive rule causes assertion failed: > stateno <= YY_SHIFT_COUNT (Conor O) > > > -------------- > > Message: 1 > Date: Fri, 7 Oct 2016 13:37:47 -0700 > From: Daniel Meyer <me

Re: [sqlite] Parallel access to read only in memory database

Jens Alfke wrote: On Oct 9, 2016, at 10:41 AM, Howard Chu wrote: As for code freshness, I've seen no compelling new features from 3.8.x onward that would improve performance so there's been no reason to update further. Perhaps, but there’s important new functionality in

Re: [sqlite] Parallel access to read only in memory database

> On Oct 9, 2016, at 10:41 AM, Howard Chu wrote: > > As for code freshness, I've seen no compelling new features from 3.8.x onward > that would improve performance so there's been no reason to update further. Perhaps, but there’s important new functionality in newer versions,

Re: [sqlite] Parallel access to read only in memory database

Jens Alfke wrote: On Oct 9, 2016, at 8:15 AM, Howard Chu wrote: Use SQLightning, it's designed specifically for write once read many workloads. "SQLite3 ported to use LMDB instead of its original Btree code” — sounds great, and the performance figures quoted in the readme

Re: [sqlite] Parallel access to read only in memory database

> On Oct 9, 2016, at 8:15 AM, Howard Chu wrote: > > Use SQLightning, it's designed specifically for write once read many > workloads. "SQLite3 ported to use LMDB instead of its original Btree code” — sounds great, and the performance figures quoted in the readme are

Re: [sqlite] Parallel access to read only in memory database

Hello ! But SQLightning is a dead project, isn't it ? version 3.7.17 hyc committed on 12 Sep 2013 Cheers ! On 09/10/16 12:15, Howard Chu

Re: [sqlite] Parallel access to read only in memory database

Daniel Meyer wrote: We are interested in using sqlite as a read only, in memory, parallel access database. We have database files that are on the order of 100GB that we are loading into memory. We have found great performance when reading from a single thread. We need to scale up to have many

Re: [sqlite] Parallel access to read only in memory database

> On Oct 7, 2016, at 1:45 PM, Joe Mistachkin wrote: > > Have you tried using the URI "file::memory:?cache=shared” Shared cache will definitely help when using multiple read-only connections, but I have a feeling memory-mapping would help even more since it would

Re: [sqlite] Parallel access to read only in memory database

igurations off the boat from china) > -Original Message- > From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] > On Behalf Of Simon Slavin > Sent: Friday, 7 October, 2016 14:49 > To: SQLite mailing list > Subject: Re: [sqlite] Parallel access to read only

Re: [sqlite] Parallel access to read only in memory database

(My two cents) I just setup two brand new machines in our Colo for ESX. Both machines had 256gig of memory. Not unheard of in server situations. ;) On Fri, Oct 7, 2016 at 4:48 PM, Simon Slavin wrote: > > On 7 Oct 2016, at 9:37pm, Daniel Meyer wrote: >

Re: [sqlite] Parallel access to read only in memory database

On 7 Oct 2016, at 9:37pm, Daniel Meyer wrote: > We have database files that are on the order of 100GB [...] in memory You have 100GB memory ? Simon. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org

Re: [sqlite] Parallel access to read only in memory database

Daniel Meyer wrote: > > How can we allow many reader threads on an in memory, write once read many > times database and achieve multi-core performance? Is this possible with > sqlite? > Have you tried using the URI "file::memory:?cache=shared" with one of the sqlite3_open*() C APIs? Further

[sqlite] Parallel access to read only in memory database

We are interested in using sqlite as a read only, in memory, parallel access database. We have database files that are on the order of 100GB that we are loading into memory. We have found great performance when reading from a single thread. We need to scale up to have many parallel reader