Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-12-09 Thread Kevin O'Gorman
hecked memory, just an undetected memory >> fault which will cause random AHTBL. >> >> > -Original Message- >> > From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org >> ] >> > On Behalf Of Kevin O'Gorman >> > Sent: Su

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-12-07 Thread Kevin O'Gorman
mory, just an undetected memory fault > which will cause random AHTBL. > > > -Original Message- > > From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] > > On Behalf Of Kevin O'Gorman > > Sent: Sunday, 4 December, 2016 09:21 > > To: SQLite ma

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-12-04 Thread Keith Medcalf
4 December, 2016 09:21 > To: SQLite mailing list > Subject: Re: [sqlite] I keep getting seg faults building my database using > python sqlite3 > > Well, the i7 system failed again, but this time it was quite different. > And peculiar. > The system wasn't doing anything, but it

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-12-04 Thread Kevin O'Gorman
Well, the i7 system failed again, but this time it was quite different. And peculiar. The system wasn't doing anything, but it should have been. So I tried something. It didn't matter what, because I could not get the mouse or keyboard to work -- it was like they weren't plugged in. Really like i

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-21 Thread Kevin O'Gorman
On Mon, Nov 21, 2016 at 9:41 AM, Roger Binns wrote: > On 19/11/16 08:08, Kevin O'Gorman wrote: > > System with problems: Running Xubuntu Linux 16.04.1, Python 3.5.2. > [...] > > System without this problem: Running Ubuntu Linux 14.04.5, Python 3.4.3. > > You are good on Python versions then. My

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-21 Thread Roger Binns
On 19/11/16 08:08, Kevin O'Gorman wrote: > System with problems: Running Xubuntu Linux 16.04.1, Python 3.5.2. [...] > System without this problem: Running Ubuntu Linux 14.04.5, Python 3.4.3. You are good on Python versions then. My remaining recommendation is to make the process that does SQLite

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-19 Thread Simon Slavin
On 19 Nov 2016, at 10:22pm, Kevin O'Gorman wrote: > Ran Memtest86+ 5.01, two complete passes, with no errors. Well, thanks for doing it anyway. It removes one possible source of problems. Simon. ___ sqlite-users mailing list sqlite-users@mailinglist

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-19 Thread Kevin O'Gorman
On Fri, Nov 18, 2016 at 10:18 AM, James K. Lowden wrote: > On Fri, 18 Nov 2016 08:55:11 -0800 > "Kevin O'Gorman" wrote: > > > All of the python code is a single thread. The closest I come > > is a few times where I use subprocess.Popen to create what amounts to > > a pipeline, and one place whe

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-19 Thread Kevin O'Gorman
Ran Memtest86+ 5.01, two complete passes, with no errors. On Sat, Nov 19, 2016 at 8:19 AM, Kevin O'Gorman wrote: > > > On Sat, Nov 19, 2016 at 8:11 AM, Simon Slavin > wrote: > >> >> On 19 Nov 2016, at 4:08pm, Kevin O'Gorman >> wrote: >> >> > I have two different machines running this stuff.

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-19 Thread Roger Binns
On 18/11/16 15:19, James K. Lowden wrote: > Good catch, Roger. It's a liability, but I slightly disagree with your > characterization. > >> - Running any Python code (destructors can be called which then run in >> the parent and child) > > Yes, if those destructors affect shared resources. Th

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-19 Thread Kevin O'Gorman
On Sat, Nov 19, 2016 at 8:11 AM, Simon Slavin wrote: > > On 19 Nov 2016, at 4:08pm, Kevin O'Gorman wrote: > > > I have two different machines running this stuff. Only one is having the > > seg faults, but they are different enough that this does not convince me > > to blame hardware. > > Could

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-19 Thread Simon Slavin
On 19 Nov 2016, at 4:08pm, Kevin O'Gorman wrote: > I have two different machines running this stuff. Only one is having the > seg faults, but they are different enough that this does not convince me > to blame hardware. Could you, anyway, run a memory test on the computer which is failing ? I

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-19 Thread Kevin O'Gorman
On Fri, Nov 18, 2016 at 3:19 PM, James K. Lowden wrote: > On Fri, 18 Nov 2016 10:56:37 -0800 > Roger Binns wrote: > > > Popen calls fork (it seems like you are doing Unix/Mac, not Windows). > > fork() duplicates the process including all open file descriptors. > > One or more of those descriptor

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread James K. Lowden
On Fri, 18 Nov 2016 10:56:37 -0800 Roger Binns wrote: > Popen calls fork (it seems like you are doing Unix/Mac, not Windows). > fork() duplicates the process including all open file descriptors. > One or more of those descriptors belong to open SQLite databases and > ancillary files. Good catch,

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Roger Binns
On 18/11/16 08:55, Kevin O'Gorman wrote: >> I am not. All of the python code is a single thread. The closest I come > is a few times where I use subprocess.Popen to create what amounts to a > pipeline, and one place where I start a number of copies of a C program in > parallel, but each is a sepa

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Simon Slavin
On 18 Nov 2016, at 6:18pm, James K. Lowden wrote: > The most likely culprit in my mind is RAM. You're exercising new memory > pretty hard, running a bunch of processes at it at full tilt. Any > defect in the chips or DMA could explain what you're seeing. An easy > test, not necessarily cheap,

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread James K. Lowden
On Fri, 18 Nov 2016 08:55:11 -0800 "Kevin O'Gorman" wrote: > All of the python code is a single thread. The closest I come > is a few times where I use subprocess.Popen to create what amounts to > a pipeline, and one place where I start a number of copies of a C > program in parallel, but each i

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Kevin O'Gorman
On Fri, Nov 18, 2016 at 8:38 AM, Roger Binns wrote: > On 17/11/16 19:14, Kevin O'Gorman wrote: > > SO: I need help bifurcating this problem. For instance, how can I tell > if > > the fault lies in SQLite, or in python? Or even in the hardware, given > that > > the time to failure is so variable?

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Kevin O'Gorman
On Fri, Nov 18, 2016 at 3:11 AM, Simon Slavin wrote: > Forgot to say ... > > Most of these problems result from attempting to reuse memory you've > already released. Even if the error is happening inside a SQLite routine, > it will be because you passed it a pointer to an SQLite connection which

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Roger Binns
On 17/11/16 19:14, Kevin O'Gorman wrote: > SO: I need help bifurcating this problem. For instance, how can I tell if > the fault lies in SQLite, or in python? Or even in the hardware, given that > the time to failure is so variable? Are you using threads, threading related settings etc in any way

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Simon Slavin
Forgot to say ... Most of these problems result from attempting to reuse memory you've already released. Even if the error is happening inside a SQLite routine, it will be because you passed it a pointer to an SQLite connection which had already been _close()'d or a statement which had already

Re: [sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-18 Thread Simon Slavin
On 18 Nov 2016, at 3:14am, Kevin O'Gorman wrote: > SO: I need help bifurcating this problem. For instance, how can I tell if > the fault lies in SQLite, or in python? Or even in the hardware, given that > the time to failure is so variable? Normal recommendation is to use GDB. You can find se

[sqlite] I keep getting seg faults building my database using python sqlite3

2016-11-17 Thread Kevin O'Gorman
I ran this thing 3 times with identical inputs, it is deterministic, but it failed after 66, 128 and 96 minutes respectively. Each run started with no database at all, and gets a single input from which the rest is calculated. The calculations are cached (in flat files), so and it never got to th