Re: [sqlite] Vista frustrations

2008-09-18 Thread Mark Spiegel
EMAIL PROTECTED] Behalf Of Mark Spiegel > Sent: Thursday, September 18, 2008 11:25 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Vista frustrations > > > I'm sorry, I have to take issue with that statement. The design of the > file system/cache m

Re: [sqlite] Vista frustrations

2008-09-18 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jay A. Kreibich wrote: > SQLite is not "lying." After poking around a bit to refresh my > understanding of SQLite's file structure, I think it is safe to say > that SQLite will almost never do a sequential file read, even if > you're doing a

Re: [sqlite] Vista frustrations

2008-09-18 Thread Fred Williams
I never said a word aboout SQLite. You ass U Me too much I suspect. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Mark Spiegel Sent: Thursday, September 18, 2008 11:25 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations

Re: [sqlite] Vista frustrations

2008-09-18 Thread Mark Spiegel
mpson > Sent: Thursday, September 18, 2008 10:31 AM > To: 'General Discussion of SQLite Database' > Subject: Re: [sqlite] Vista frustrations > > > After watching Molly Brown's Channel9 videos on the cache manager, I'm > convinced the behavior for SQLite should be to not give the

Re: [sqlite] Vista frustrations

2008-09-18 Thread Robert Simpson
a workaround for an OS design flaw is not unique to Windows by any stretch. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Fred Williams Sent: Thursday, September 18, 2008 8:54 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations

Re: [sqlite] Vista frustrations

2008-09-18 Thread Fred Williams
' Subject: Re: [sqlite] Vista frustrations After watching Molly Brown's Channel9 videos on the cache manager, I'm convinced the behavior for SQLite should be to not give the filesystem any hints about caching and let the cache manager itself figure it out. The exception being Windows CE, where we

Re: [sqlite] Vista frustrations

2008-09-18 Thread Mark Spiegel
FILE_FLAG_RANDOM_ACCESS and FILE_FLAG_SEQUENTIAL_SCAN are hints to the cache manager (CC) in Windows and the underlying file system(s). With respect to the cache manager, it is going to affect whether or not there is read ahead, how much read ahead will be used, and how long data will remain

Re: [sqlite] Vista frustrations

2008-09-17 Thread Robert Simpson
: http://msdn.microsoft.com/en-us/library/ms923949.aspx Robert -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Roger Binns Sent: Wednesday, September 17, 2008 4:24 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations

Re: [sqlite] Vista frustrations

2008-09-17 Thread Jay A. Kreibich
On Wed, Sep 17, 2008 at 06:00:45PM -0700, Roger Binns scratched on the wall: > The second is that SQLite when opening a file under Windows explicitly > tells Windows that the file will be used for random access even though > that is not the case. Windows uses this hint to override its builtin >

Re: [sqlite] Vista frustrations

2008-09-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Brown, Daniel wrote: > Has anyone tried to replicate this bug on WindowsXP too? There are two separate bugs here. One is that when Vista is told a file is used for random access and then there is a lot of file access, it may use a little too much

Re: [sqlite] Vista frustrations

2008-09-17 Thread Brown, Daniel
Has anyone tried to replicate this bug on WindowsXP too? -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Roger Binns Sent: Wednesday, September 17, 2008 4:24 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations -BEGIN

Re: [sqlite] Vista frustrations

2008-09-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Simpson wrote: > Ideally, at least on non-CE platforms, I'd like see SQLite not give the OS > any hints about caching. However, I'm not sure what kind of performance hit > (if any) that would have on Windows. It's already been proven that >

Re: [sqlite] Vista frustrations

2008-09-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Fred Williams wrote: > Have you ever actually used a version of Windows? Windows 1.0 (once), Windows 2/286, Windows 3, 3.1, 3.11, 95, 98, ME, 2000, XP, 2003 and Vista. > ANY OS that attempts to read in a xGigibyte file into real memory to the >

Re: [sqlite] Vista frustrations

2008-09-17 Thread Jeffrey Becker
n Behalf Of Roger Binns > Sent: Wednesday, September 17, 2008 2:02 PM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Vista frustrations > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Robert Simpson wrote: >> The purpose of a cache is to improve

Re: [sqlite] Vista frustrations

2008-09-17 Thread Robert Simpson
: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Roger Binns Sent: Wednesday, September 17, 2008 2:02 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Simpson wrote: > The purpose of a ca

Re: [sqlite] Vista frustrations

2008-09-17 Thread Fred Williams
: [sqlite] Vista frustrations -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Simpson wrote: > The purpose of a cache is to improve performance and responsiveness. Any > cache that uses all physical memory, forces all other apps to the paging > file All current operating systems do th

Re: [sqlite] Vista frustrations

2008-09-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Simpson wrote: > The purpose of a cache is to improve performance and responsiveness. Any > cache that uses all physical memory, forces all other apps to the paging > file All current operating systems do this, using heuristics to determine

Re: [sqlite] Vista frustrations

2008-09-17 Thread Virgilio Alexandre Fornazin
Database Subject: Re: [sqlite] Vista frustrations SQLite in general and the .Net provider in particular are most often shipped as components of other applications. I dont think having developers tell their end users to disable superfetch is a viable solution. As much as I hate to propose this maybe

Re: [sqlite] Vista frustrations

2008-09-17 Thread Jeffrey Becker
y A. Kreibich > Sent: quarta-feira, 17 de setembro de 2008 13:01 > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Vista frustrations > > On Wed, Sep 17, 2008 at 01:17:51AM -0700, Roger Binns scratched on the wall: >> Robert Simpson wrote: >> > To me this

Re: [sqlite] Vista frustrations

2008-09-17 Thread Virgilio Alexandre Fornazin
, 17 de setembro de 2008 13:30 To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Vista frustrations I've run the tests with superfetch and prefetch disabled and enabled. Results are consistent with or without these running. The only thing that has any affect

Re: [sqlite] Vista frustrations

2008-09-17 Thread Robert Simpson
] [mailto:[EMAIL PROTECTED] On Behalf Of Jay A. Kreibich Sent: quarta-feira, 17 de setembro de 2008 13:01 To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations On Wed, Sep 17, 2008 at 01:17:51AM -0700, Roger Binns scratched on the wall: > Robert Simpson wrote: > &

Re: [sqlite] Vista frustrations

2008-09-17 Thread Virgilio Alexandre Fornazin
PROTECTED] On Behalf Of Jay A. Kreibich Sent: quarta-feira, 17 de setembro de 2008 13:01 To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations On Wed, Sep 17, 2008 at 01:17:51AM -0700, Roger Binns scratched on the wall: > Robert Simpson wrote: > > To me this s

Re: [sqlite] Vista frustrations

2008-09-17 Thread Jay A. Kreibich
On Wed, Sep 17, 2008 at 01:17:51AM -0700, Roger Binns scratched on the wall: > Robert Simpson wrote: > > To me this seems like an obvious bug in Vista, > > Actually I'd argue that it is behaving as designed. You could argue it is behaving as designed, but I'd still argue it is behaving

Re: [sqlite] Vista frustrations

2008-09-17 Thread Fred Williams
[mailto:[EMAIL PROTECTED] Behalf Of Robert Simpson Sent: Wednesday, September 17, 2008 9:54 AM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Vista frustrations The real frustration is that this seems to be a rather obvious bug in Vista, and definitely not SQLite's responsibilit

Re: [sqlite] Vista frustrations

2008-09-17 Thread Robert Simpson
Discussion of SQLite Database' Subject: Re: [sqlite] Vista frustrations > I ran some tests using the command-line sqlite3.exe, and observed that > Windows Vista (SP1) is actually trying to cache the entire 5gb file > into > memory during the table scan! The system slows to a co

Re: [sqlite] Vista frustrations

2008-09-17 Thread Robert Simpson
poor Vista people need SQLite to work well with their OS. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Roger Binns Sent: Wednesday, September 17, 2008 1:18 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations -BEGIN

Re: [sqlite] Vista frustrations

2008-09-17 Thread Robert Simpson
:[EMAIL PROTECTED] On Behalf Of Roger Binns Sent: Wednesday, September 17, 2008 1:18 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Vista frustrations -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Simpson wrote: > To me this seems like an obvious bug in Vista, Actually

Re: [sqlite] Vista frustrations

2008-09-17 Thread Filip Navara
"(*) Sun's ZFS can even detect strided sequential access - ie reading X amount of data every Y kilobytes." ... and so can the NT cache manager since the very first Windows NT release ;-) It's good to see that people are finally adapting these features 15 years later. F. On Wed, Sep 17, 2008 at

Re: [sqlite] Vista frustrations

2008-09-17 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Robert Simpson wrote: > To me this seems like an obvious bug in Vista, Actually I'd argue that it is behaving as designed. Generally filesystem code will try to detect what is going on under the hood. In particular if it looks like you are doing

Re: [sqlite] Vista frustrations

2008-09-17 Thread Sherief N. Farouk
> I ran some tests using the command-line sqlite3.exe, and observed that > Windows Vista (SP1) is actually trying to cache the entire 5gb file > into > memory during the table scan! The system slows to a complete crawl and > becomes unresponsive. The sqlite3.exe's memory remains very minimal, >

[sqlite] Vista frustrations

2008-09-16 Thread Robert Simpson
I recently had a user of the ADO.NET provider report a problem on Vista. His database is 5gb, and he's doing a full table scan of 16 million rows. Yea. I ran some tests using the command-line sqlite3.exe, and observed that Windows Vista (SP1) is actually trying to cache the entire 5gb file