Re: [sqlite] Stored Procedures

On 14 May 2018 at 01:08, Richard Damon wrote: > On 5/13/18 12:55 PM, Rowan Worth wrote: > > On 9 May 2018 at 08:56, Richard Hipp wrote: > > > >> But with > >> SQLite, there is no round-trip latency. A "round-trip" to and > >> database is just a

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

Techno Magos wrote: > So, memory sqlite is not really usable with multiple threads (readers). > While one might expect that multiple readers of *memory *content could > scale even better than with file content. Concurrent accesses to the same in-memory data structures must be serialized. In

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

I have a few ideas you could try with a file db. 1. Use VFS with no locks - named "win32-none" or "unix-none" depending on your system (4th argument to sqlite_open_v2). 2. Run "pragma locking_mode=exclusive;" on each connection or compile SQLite with -DSQLITE_DEFAULT_LOCKING_MODE=1 3. Compile

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

Clemens Ladisch wrote: Techno Magos wrote: So, memory sqlite is not really usable with multiple threads (readers). While one might expect that multiple readers of *memory *content could scale even better than with file content. Concurrent accesses to the same in-memory data structures must

Re: [sqlite] shell edit quoting

David Burgess wrote: >> Where do the quotes around the value come from? > > I typed them. Simgle set of double quotes I meant the quotes around the entire value returned by the SELECT. >> Are you using the standard command-line shell, and which output mode? > > yes and the default mode The

[sqlite] Multi threaded readers on memory sqlite cannot scale

Hello I do not have clear examples to post on this but would like to report findings around multi threaded read access (single process) in a large system that uses sqlite. This may be a known issue/restriction of memory sqlite behaviour, but wanted to check with the list first: 1. Running 2,

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

On Sun, May 13, 2018 at 9:01 AM, Dennis Clarke wrote: > On 05/13/2018 11:57 AM, Kevin O'Gorman wrote: > >> The arguments here are simplified >> > > > Will you stop top posting please? > > I am trying to follow along here about some x86 boxen stuff but > you are top posting

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

It's not clear to me why reads must be serialized at all. Maybe this could be re-thought? Maybe there should be a way to tell SQLite that a certain DB or table is to be read-only and unserialized? On Sun, May 13, 2018 at 7:15 AM, Keith Medcalf wrote: > > Say Hi to Gene! >

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

On 13 May 2018, at 11:50am, Techno Magos wrote: > So, memory sqlite is not really usable with multiple threads (readers). > While one might expect that multiple readers of *memory *content could > scale even better than with file content. > > Can this restriction be

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

The arguments here are simplified, and assume some things that may or may not be true. The server I keep in my garage has 16 real cores, 32 threads. More importantly, it uses DDR4 memory which I think means there are 4 channels to memory which can be used in parallel -- perhaps not on exactly

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

On 13 May 2018, at 17:01, Dennis Clarke wrote: > On 05/13/2018 11:57 AM, Kevin O'Gorman wrote: > Also is that a single socket machine with a single big memory bank or > is it NUMA and multiple sockets or is it just a single motherboard unit? And I'd be curious to know

Re: [sqlite] Stored Procedures

On 9 May 2018 at 08:56, Richard Hipp wrote: > But with > SQLite, there is no round-trip latency. A "round-trip" to and > database is just a function call, and is very very cheap. > I want to emphasise that Dr. Hipp's usage of "round-trip" only includes the latency of

Re: [sqlite] Stored Procedures

On 5/13/18 12:55 PM, Rowan Worth wrote: > On 9 May 2018 at 08:56, Richard Hipp wrote: > >> But with >> SQLite, there is no round-trip latency. A "round-trip" to and >> database is just a function call, and is very very cheap. >> > I want to emphasise that Dr. Hipp's usage of

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

Say Hi to Gene! https://en.wikipedia.org/wiki/Amdahl%27s_law So I believe what you are saying is something like this: If I take a child and have it count as fast as it can then it can count to X in an hour. However, I take the same child but have it count as fast as it can at five minute

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

On 13 May 2018, at 4:57pm, Kevin O'Gorman wrote: > More importantly, it uses DDR4 memory which I think means there > are 4 channels to memory which can be used in parallel -- perhaps not on > exactly the same address but the memory is spread among 16 DIMMs. Suppose your

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

Kevin O'Gorman wrote: > why is a read-only database being serialized at all? The database file is read only, the in-memory data structures are not. For example, when the cache size is smaller than the DB size, pages must be removed from and added to the internal list of cached pages. When using

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

On 05/13/2018 11:57 AM, Kevin O'Gorman wrote: The arguments here are simplified Will you stop top posting please? I am trying to follow along here about some x86 boxen stuff but you are top posting madly. Also is that a single socket machine with a single big memory bank or is it NUMA and

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

On 5/13/18 11:57 AM, Kevin O'Gorman wrote: > More importantly, it uses DDR4 memory which I think means there > are 4 channels to memory which can be used in parallel DDR4 does NOT have 4 independent memory channels. DDR4 is the forth generation of the Double Data Rate interface standard. DDR

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

>2. Running the same example on sqlite *file *(multi threaded mode; >WAL journal) scales almost linearly; so 6 threads provide nearly 6xN >throughput. Single threaded throughput is a bit slower (around 15- >20%) than single threaded in-memory access (expected). So, there is some "part" of the

Re: [sqlite] shell edit quoting

> And it works for me: I'm pleased for you. ___ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Multi threaded readers on memory sqlite cannot scale

Try to open N separate database connections (without shared cache) and load content using sqlite3_deserialize with SQLITE_DESERIALIZE_READONLY flag. http://www.sqlite.org/c3ref/deserialize.html SQLite won't copy data but use provided buffer so you won't have N copies of databse. 2018-05-13 12:50

[sqlite] Problem b building sqlite on macOS android.

Hi, I am having problems building sqlite with the android NDK for macos. Weirdly, it *does* build fine on windows ndk. The errors I am getting are shown below. Bye, Mark '-- errors building sqlite on macOS android ndk