Re: [sqlite] Thread safety of serialized mode

On 18 February 2017 at 01:16, James K. Lowden wrote: > It's why I like Go: it's the first language in 30 years to incorporate > concurrency in its design, and finally support a theoretically sound > model. > I like Go too, but this is giving it a bit too much credit.

Re: [sqlite] Thread safety of serialized mode

> On Feb 17, 2017, at 11:11 AM, Dominique Devienne wrote: > > No they are not. They may be scheduled on threads, but they are not threads. We're disagreeing on terminology, but I believe I’m correct. Threads don’t have to be implemented at the OS level. Threads

Re: [sqlite] Thread safety of serialized mode

On Fri, Feb 17, 2017 at 6:58 PM, Jens Alfke wrote: > [from another email] > > It's why I like Go: it's the first language in 30 years to incorporate > > concurrency in its design, and finally support a theoretically sound > > model. > > Goroutines are still threads, and Go

Re: [sqlite] Thread safety of serialized mode

> On Feb 17, 2017, at 9:18 AM, James K. Lowden wrote: > > It's the OS. A thread is an OS abstraction, not a machine feature. You don’t need an OS to create threads. All you need is setjmp/longjmp or the equivalent. “Green” threads are more awkward to use than

Re: [sqlite] Thread safety of serialized mode

> On Feb 17, 2017, at 3:48 AM, Simon Slavin wrote: > > It is insane that a CPU would allow two threads to interfere with each-other > in such a way as to 'break' an INC instruction. It may be insane to you, but it’s simply how multiprocessor computer architectures work,

Re: [sqlite] Thread safety of serialized mode

.com/2016/11/30/15-years-of-concurrency/ - Deon -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of James K. Lowden Sent: Friday, February 17, 2017 9:17 AM To: sqlite-users@mailinglists.sqlite.org Subject: Re: [sqlite] Thread sa

Re: [sqlite] Thread safety of serialized mode

On Fri, Feb 17, 2017 at 5:10 AM, Deon Brewis wrote: > If you look at the original underlying NT I/O architecture that Cutler > implemented - it is a thing of beauty that's based in async patterns, and > not threads. > Thanks. That Cutler reference led me to

Re: [sqlite] Thread safety of serialized mode

On Thu, 16 Feb 2017 19:35:51 -0800 Jens Alfke wrote: > > On Feb 16, 2017, at 6:26 PM, James K. Lowden > > wrote: > > > > It doesn't change the fact that the OS has subverted the > > guarantees your language would otherwise provide, such as the > >

Re: [sqlite] Thread safety of serialized mode

On Fri, 17 Feb 2017 04:10:09 + Deon Brewis wrote: > If you look at the original underlying NT I/O architecture that > Cutler implemented - it is a thing of beauty that's based in async > patterns, and not threads. ... > If instead NT initially only exposed the Nt API's and

Re: [sqlite] Thread safety of serialized mode

> On Feb 17, 2017, at 12:21 AM, Warren Young wrote: > > How can we expect people to write threaded programs when even a simple > integer increment is prone to race conditions and read-modify-write errors? "… we did not (and still do not) believe in the standard

Re: [sqlite] Thread safety of serialized mode

riday, February 17, 2017 6:48 AM To: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> Subject: Re: [sqlite] Thread safety of serialized mode On 17 Feb 2017, at 9:27am, Clemens Ladisch <clem...@ladisch.de> wrote: > X86(-64) has always had "INC [mem]" and "LOCK I

Re: [sqlite] Thread safety of serialized mode

On 17 Feb 2017, at 9:27am, Clemens Ladisch wrote: > X86(-64) has always had "INC [mem]" and "LOCK INC [mem]". It is insane that a CPU would allow two threads to interfere with each-other in such a way as to 'break' an INC instruction. But yes, we have drifted a long way

Re: [sqlite] Thread safety of serialized mode

Jens Alfke wrote: > A read-modify-write cycle on an address in main memory is intrinsically > _not_ atomic on a multiprocessor system, not unless the CPU goes through > some expensive efforts to make it so (cache invalidation, bus locking, > etc.) Most modern CPUs have caches, and any _normal_

Re: [sqlite] Thread safety of serialized mode

On 2017-02-14 5:05 PM, Darren Duncan wrote: On 2017-02-14 4:46 PM, Richard Hipp wrote: This is yet another reason why I say "threads are evil". For whatever reason, programmers today think that "goto" and pointers and assert() are the causes of all errors, but threads are cool and healthful.

Re: [sqlite] Thread safety of serialized mode

Jens Alfke wrote: > PS: I’m not aware of _any_ current CPUs that can increment main memory > in one instruction, atomically or not. X86(-64) has always had "INC [mem]" and "LOCK INC [mem]". And MSP430 calls itself RISC, but is so orthogonal that any operand can be in memory. Regards,

Re: [sqlite] Thread safety of serialized mode

On 17 Feb 2017, at 8:54am, Clemens Ladisch wrote: > Bob Friesenhahn wrote: > >> Does anyone have an idea about this specific problem that we encountered? >> >> It is not clear to me if this is a threading issue, or memory corruption >> issue > > It's probably memory

Re: [sqlite] Thread safety of serialized mode

Bob Friesenhahn wrote: > Does anyone have an idea about this specific problem that we encountered? > > It is not clear to me if this is a threading issue, or memory corruption issue It's probably memory corruption causes by a threading issue. The original mail said: > Any thread may acquire and

Re: [sqlite] Thread safety of serialized mode

ty with 16 bit API's, which long predated Cutler. - Deon -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of James K. Lowden Sent: Thursday, February 16, 2017 6:26 PM To: sqlite-users@mailinglists.sqlite.org Subject: Re: [sqlite] Thread

Re: [sqlite] Thread safety of serialized mode

> On Feb 16, 2017, at 6:26 PM, James K. Lowden wrote: > > It doesn't change the fact that the OS has subverted the > guarantees your language would otherwise provide, such as the atomicity > of ++i noted elsewhere in this thread. It’s not the OS, it’s the

Re: [sqlite] Thread safety of serialized mode

On Thu, 16 Feb 2017 21:49 + Tim Streater wrote: > > What's inherently wrong with threads in principle is that there is > > no logic that describes them, and consequently no compiler to > > control that logic. > > [snip remainder of long whinge about threads] > >

Re: [sqlite] Thread safety of serialized mode

On Thu, 16 Feb 2017 16:21:06 -0700 Warren Young wrote: > https://www2.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf > > Threads aren?t just distasteful from an implementation standpoint, > they?re *mathematically unsound*. Thank you for that. I think I encounted

Re: [sqlite] Thread safety of serialized mode

On Thu, 16 Feb 2017, Warren Young wrote: Taking it off-list, since there is zero remaining connection to SQLite now: Thank you for taking it off list. How can we expect people to write threaded programs when even a simple integer increment is prone to race conditions and read-modify-write

Re: [sqlite] Thread safety of serialized mode

Taking it off-list, since there is zero remaining connection to SQLite now: > On Feb 16, 2017, at 2:49 PM, Tim Streater wrote: > > On 16 Feb 2017 at 18:30, James K. Lowden wrote: > >> On Tue, 14 Feb 2017 17:05:30 -0800 >> Darren Duncan

Re: [sqlite] Thread safety of serialized mode

> On Feb 16, 2017, at 11:49 AM, Warren Young wrote: > > A software developer who refuses to learn about his processor’s assembly > language is like trying to become an electrical engineer without learning > anything about physics. In this case what you need to read is

Re: [sqlite] Thread safety of serialized mode

On 16 Feb 2017 at 18:30, James K. Lowden wrote: > On Tue, 14 Feb 2017 17:05:30 -0800 > Darren Duncan wrote: > >> There is nothing inherently wrong with threads in principle > > What's inherently wrong with threads in principle is that there is

Re: [sqlite] Thread safety of serialized mode

It seems like the discussion has turned into a general programming discussion unrelated to SQLite3. Does anyone have an idea about this specific problem that we encountered (see quoted message below)? It is not clear to me if this is a threading issue, or memory corruption issue, or if it is

Re: [sqlite] Thread safety of serialized mode

a separate supervisor process. > -Original Message- > From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] > On Behalf Of James K. Lowden > Sent: Thursday, 16 February, 2017 11:30 > To: sqlite-users@mailinglists.sqlite.org > Subject: Re: [sqlite] Thre

Re: [sqlite] Thread safety of serialized mode

On Feb 15, 2017, at 5:03 AM, a...@zator.com wrote: > > I suppose someday, programming languages can do an analogous translation in > our limited but safe, sequential programs. Not as long as we require side effects to achieve anything of practical value. Any form of I/O is a “side effect” by

Re: [sqlite] Thread safety of serialized mode

On Feb 15, 2017, at 4:40 AM, Darren Duncan wrote: > > On 2017-02-15 2:40 AM, Clemens Ladisch wrote: >> Cecil Westerhof wrote: >> >> And just like with assembly code, you also have to count the time spent >> writing it, and debugging the result. > > Also, its a long

Re: [sqlite] Thread safety of serialized mode

On Tue, 14 Feb 2017 17:05:30 -0800 Darren Duncan wrote: > There is nothing inherently wrong with threads in principle What's inherently wrong with threads in principle is that there is no logic that describes them, and consequently no compiler to control that logic.

Re: [sqlite] Thread safety of serialized mode

On Wed, 15 Feb 2017 09:40:13 -0800 Jens Alfke wrote: > https://en.wikipedia.org/wiki/Communicating_sequential_processes > Also search YouTube for Rob Pike's presentations on CSP in Go. It will help clarify

Re: [sqlite] Thread safety of serialized mode

On Wed, 15 Feb 2017 12:34:51 + Simon Slavin wrote: > Two disadvantages are that threads are indistinguishable to anything > but the owner and don?t know how to keep out of each-other?s way. By > the time you?ve devised some sort of mutex/locking/blocking mechanism >

Re: [sqlite] Thread safety of serialized mode

On Wed, 15 Feb 2017 07:55:16 -0700 "Keith Medcalf" wrote: > Note that for several modern OSes, the OS is nothing more than a > discontiguous saved segment (DCSS) which is mapped into *every* > process space and that process isolation is more of a myth than a > reality. Are

Re: [sqlite] Thread safety of serialized mode

It turns out that I have more data on the problem. The error message reported reads something like: SQLITE_CORRUPT: database disk image is malformed database corruption at line 70273 of [17efb4209f] We are using version 3.10.2. Looking at amalgamation code I see that the error is returned

Re: [sqlite] Thread safety of serialized mode

> On Feb 15, 2017, at 3:44 AM, Cecil Westerhof wrote: > > ​As I said before: I did not work much with threads. Mostly for GUI > performance. Do you (or anyone else) have any resources about those > concurrency models​? Theory: https://en.wikipedia.org/wiki/Actor_model

Re: [sqlite] Thread safety of serialized mode

> On Feb 14, 2017, at 11:58 PM, Clemens Ladisch wrote: > > But "go parallel" does not necessarily imply threads. There are many > ways to allow code running on different CPUs(/cores) to communicate > with each other (e.g., files, sockets, message queues, pipes, shared >

Re: [sqlite] Thread safety of serialized mode

On Wednesday, 15 February, 2017 03:16, Cecil Westerhof said: > 2017-02-15 8:58 GMT+01:00 Clemens Ladisch : > > Jens Alfke wrote: > > Threading is the most extreme method of achieving parallelism, and > > therefore should be used only as the last

Re: [sqlite] Thread safety of serialized mode

On Tue, 14 Feb 2017, Jens Alfke wrote: If we have two threads executing sqlite3_step() on the same connection and using their own prepared statement, is there any magic in sqlite3 which would keep sqlite3_step() and sqlite3_column_foo() from consuming (or disrupting) the results from the

Re: [sqlite] Thread safety of serialized mode

On Tue, 14 Feb 2017, Richard Hipp wrote: On 2/14/17, Bob Friesenhahn wrote: Due to memory constraints (at least 1MB is consumed per connection!), only one database connection is used. Any thread may acquire and use this one database connection at any time.

Re: [sqlite] Thread safety of serialized mode

On 15 Feb 2017, at 10:16am, Cecil Westerhof wrote: > 2017-02-15 8:58 GMT+01:00 Clemens Ladisch : > >> Jens Alfke wrote: >> Threading is the most extreme method of achieving parallelism, and >> therefore should be used only as the last resort. (I'd

Re: [sqlite] Thread safety of serialized mode

> Mensaje original > De: Richard Hipp <d...@sqlite.org> > Para: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> > Fecha: Tue, 14 Feb 2017 20:15:49 -0500 > Asunto: Re: [sqlite] Thread safety of serialized mode >> > > On 2/14/17, Darren

Re: [sqlite] Thread safety of serialized mode

2017-02-15 12:40 GMT+01:00 Darren Duncan : > Similarly with threads, for the vast majority of people, using other > concurrency models with supported languages are better; they will still get > the performance benefit of using multiple CPU cores but do it much more >

Re: [sqlite] Thread safety of serialized mode

On 2017-02-15 2:40 AM, Clemens Ladisch wrote: Cecil Westerhof wrote: 2017-02-15 8:58 GMT+01:00 Clemens Ladisch : Threading is the most extreme method of achieving parallelism, and therefore should be used only as the last resort. (I'd compare it to assembly code in this

Re: [sqlite] Thread safety of serialized mode

Cecil Westerhof wrote: > 2017-02-15 8:58 GMT+01:00 Clemens Ladisch : >> Threading is the most extreme method of achieving parallelism, and >> therefore should be used only as the last resort. (I'd compare it to >> assembly code in this regard.) > > ​At the moment I am not

Re: [sqlite] Thread safety of serialized mode

2017-02-15 8:58 GMT+01:00 Clemens Ladisch : > Jens Alfke wrote: > Threading is the most extreme method of achieving parallelism, and > therefore should be used only as the last resort. (I'd compare it to > assembly code in this regard.) > ​At the moment I am not using it

Re: [sqlite] Thread safety of serialized mode

On Wed, Feb 15, 2017 at 2:07 AM, Jens Alfke wrote: > > Entire programming languages are invited (I'm thinking of > > Java) to make goto and pointers impossible or to make assert() > > impossible (Go) and yet at the same time encourage people to use > > threads. > > [...]. And

Re: [sqlite] Thread safety of serialized mode

Jens Alfke wrote: > With clock speed having stalled, the only way to take advantage of > modern CPUs (and GPUs!) is to go parallel. But "go parallel" does not necessarily imply threads. There are many ways to allow code running on different CPUs(/cores) to communicate with each other (e.g.,

Re: [sqlite] Thread safety of serialized mode

> On Feb 14, 2017, at 5:15 PM, Richard Hipp wrote: > > Nor is there anything wrong with goto, pointers, and assert(), in > principle. And yet they are despised while threads are adored, in > spite of the fact that goto/pointer/assert() errors are orders of > magnitude easier

Re: [sqlite] Thread safety of serialized mode

On 2/14/17, Darren Duncan wrote: > > There is nothing inherently wrong with threads in principle, Nor is there anything wrong with goto, pointers, and assert(), in principle. And yet they are despised while threads are adored, in spite of the fact that

Re: [sqlite] Thread safety of serialized mode

On Tue, Feb 14, 2017 at 5:05 PM, Darren Duncan wrote: > On 2017-02-14 4:46 PM, Richard Hipp wrote: > >> This is yet another reason why I say "threads are evil". For >> whatever reason, programmers today think that "goto" and pointers and >> assert() are the causes of

Re: [sqlite] Thread safety of serialized mode

> On Feb 14, 2017, at 4:46 PM, Richard Hipp wrote: > > This is yet another reason why I say "threads are evil”. I agree, and it’s a pretty widely held opinion these days, going back at least to Edward Lee’s 2006 paper “The Problem With Threads”.[1] Actually the problem

Re: [sqlite] Thread safety of serialized mode

On 2017-02-14 4:46 PM, Richard Hipp wrote: This is yet another reason why I say "threads are evil". For whatever reason, programmers today think that "goto" and pointers and assert() are the causes of all errors, but threads are cool and healthful. Entire programming languages are invited

Re: [sqlite] Thread safety of serialized mode

> On Feb 14, 2017, at 3:51 PM, Bob Friesenhahn > wrote: > > Due to timing constraints, it performs all read queries in one thread and > creates a temporary POSIX thread for each update query (this is the > developer's reasoning). To me that seems kind of

Re: [sqlite] Thread safety of serialized mode

On 2/14/17, Bob Friesenhahn wrote: > Due to memory constraints > (at least 1MB is consumed per connection!), only one database > connection is used. Any thread may acquire and use this one database > connection at any time. This is yet another reason why I say

Re: [sqlite] Thread safety of serialized mode

On Wed, 15 Feb 2017, Simon Slavin wrote: On 14 Feb 2017, at 11:51pm, Bob Friesenhahn wrote: One of our Linux programs (not written by me) is reporting errors of the form "SQLITE_CORRUPT: database disk image is malformed database corruption". Is the database

Re: [sqlite] Thread safety of serialized mode

On 14 Feb 2017, at 11:51pm, Bob Friesenhahn wrote: > One of our Linux programs (not written by me) is reporting errors of the form > "SQLITE_CORRUPT: database disk image is malformed database corruption". Is the database actually corrupt ? Even if your other