Re: [sqlite] Large Database Windows vs Linux

2013-06-29 Thread James K. Lowden
On Fri, 28 Jun 2013 08:54:07 -0400 "Christopher W. Steenwyk" wrote: > I have a rather large database (11 GB) that has two tables (one with > approximately 500,000 rows and another with approximately 50,000,000 > rows). In this database I am performing a query that joins these two > tables to prod

Re: [sqlite] Large Database Windows vs Linux

2013-06-29 Thread Bradley Giesbrecht
On Jun 28, 2013, at 5:54 AM, Christopher W. Steenwyk wrote: > I have a rather large database (11 GB) that has two tables (one with > approximately 500,000 rows and another with approximately 50,000,000 rows). > In this database I am performing a query that joins these two tables to > produce appro

Re: [sqlite] Large Database Windows vs Linux

2013-06-29 Thread Olaf Schmidt
Am 28.06.2013 14:54, schrieb Christopher W. Steenwyk: I have a rather large database (11 GB) that has two tables (one with approximately 500,000 rows and another with approximately 50,000,000 rows). In this database I am performing a query that joins these two tables to produce approximately 4

Re: [sqlite] Large Database Windows vs Linux

2013-06-29 Thread Jeremy Nicoll - ml sqlite users
Roger Binns wrote: > Here is the list of monitored extensions: > > http://msdn.microsoft.com/en-us/library/windows/desktop/aa378870(v=vs.85).aspx That's a frighteningly long list - 574 separate extensions... And they're not all obscure ones either. For example it includes .DATA .SRC

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Simon Slavin
On 29 Jun 2013, at 1:22am, Howard Chu wrote: > David de Regt wrote: >> It's the kind of useful help like this that makes me love the FOSS movement. > > All based in facts, of course. http://blog.zorinaq.com/?e=74 Nevertheless, the remark was not helpful and is therefore best forgotten. If an

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Howard Chu
lf Of Walter Hurry Sent: Friday, June 28, 2013 5:09 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Large Database Windows vs Linux On Fri, 28 Jun 2013 15:22:57 -0600, Keith Medcalf wrote: That would explain why the best thing to be done with System Destroyer (System Restore) is the same as the

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread David de Regt
t: Re: [sqlite] Large Database Windows vs Linux On Fri, 28 Jun 2013 15:22:57 -0600, Keith Medcalf wrote: > That would explain why the best thing to be done with System Destroyer > (System Restore) is the same as the best way to handle the Hardware > Destroyer (Power Management) in Window

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Walter Hurry
On Fri, 28 Jun 2013 15:22:57 -0600, Keith Medcalf wrote: > That would explain why the best thing to be done with System Destroyer > (System Restore) is the same as the best way to handle the Hardware > Destroyer (Power Management) in Windows. Disable it completely. > The best thing to do with Wi

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread David de Regt
nal Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Keith Medcalf Sent: Friday, June 28, 2013 2:23 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Large Database Windows vs Linux That would explain why the best thing

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Keith Medcalf
-users- > boun...@sqlite.org] On Behalf Of Roger Binns > Sent: Friday, 28 June, 2013 15:07 > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Large Database Windows vs Linux > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 28/06/13 13:17, RSm

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 28/06/13 13:17, RSmith wrote: > Best guess is some other system is trying to also look into that file, > making the Windows file manager stutter quite possibly the Win7 > Preview pane, a 3rd party file indexer service, an anti-virus system or > som

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread RSmith
I think your assumption about the file system is correct - It is hard for the code to produce widely differing times under different systems as the basic algorithms do not change between systems, only dependencies on file-system or VFS specific api's etc. The NTFS file system in Windows (hoping

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Paul Corke
On 28 June 2013 18:08, Stephan Beal wrote: > i've seen sqlite3 tests of mine in a 32-bit VM running on > 64-bit hardware run twice as fast as that same code on the > 64-bit hardware (outside the VM) One of our customers uses our product on a VM, and it appears that the hypervisor lies about havin

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Stephan Beal
On Fri, Jun 28, 2013 at 7:18 PM, Simon Slavin wrote: > Probably that your entire VM is in cache memory on your computer, but the > program running on your hardware gets to write to a physical disk drive. > That's certainly the most likely hypothesis i've heard so far. i hadn't considered the eff

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Simon Slavin
On 28 Jun 2013, at 6:08pm, Stephan Beal wrote: > On Fri, Jun 28, 2013 at 6:55 PM, Christopher W. Steenwyk < > csteen...@gmail.com> wrote: > >> I also did try version 3.7.13 and that did run faster. So for whatever >> reason my shell 3.7.17 (32 or 64 bit) is significantly slower on windows >> th

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Stephan Beal
On Fri, Jun 28, 2013 at 6:55 PM, Christopher W. Steenwyk < csteen...@gmail.com> wrote: > I also did try version 3.7.13 and that did run faster. So for whatever > reason my shell 3.7.17 (32 or 64 bit) is significantly slower on windows > than my 3.7.13 32-bit. > Vaguely related to those observatio

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread Christopher W. Steenwyk
Thanks for the input. I did recompile in 64-bit mode with no difference. I also did try version 3.7.13 and that did run faster. So for whatever reason my shell 3.7.17 (32 or 64 bit) is significantly slower on windows than my 3.7.13 32-bit. On Fri, Jun 28, 2013 at 10:30 AM, og wrote: > I had a

Re: [sqlite] Large Database Windows vs Linux

2013-06-28 Thread og
I had a similar problem and it was the antivirus (win 7 prof)... My table has about 63 million rows and a description very similar to yours... but now I use the same data for FTS4, etc... on Debian wheeze (simple workstation... and currently all ok...) example: sqlite> select count(*) from parte;

[sqlite] Large Database Windows vs Linux

2013-06-28 Thread Christopher W. Steenwyk
Hi, I have been struggling with a problem and was hoping I could get some insight. I have a rather large database (11 GB) that has two tables (one with approximately 500,000 rows and another with approximately 50,000,000 rows). In this database I am performing a query that joins these two tables