Re: [sqlite] Very Slow DB Access After Reboot on Windows

[Samet YASLAN]

I have tried changing the extension and also "copy my.db nul:". The 
result does not change a bit. Still 40sec.
I also tried reading the file with fread before openning db connection like;

FILE* fp = fopen(fileName.c_str(), "rb");
 if ( fp ) {
 char pBuffer[1024*32];
 while ( fread(pBuffer, 1024, 32, fp) > 0 )
 {
 }
 fclose(fp);
 }

This is very helpful and really decreases the total read performance 
including this operation.
But if the file size is too big than this may lead to some problems again.

Anyway there must be some users reaching gigabytes of DB size and I am 
not sure they are waiting one hour after restarting their PCs.
There must be a way to solve this problem. As I said before Linux is 
just fine. I wish to see Linux on every PC one day.

*Samet YASLAN*

On 22.07.2010 22:51, Kees Nuyt wrote:
> On Thu, 22 Jul 2010 10:44:50 -0500, "Black, Michael (IS)"
>   wrote:
>
>
>> Wrongread the docs...if copy didn't do binary by default there would so 
>> many screwed up computers in the world
>>
>>
>> http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/copy.mspx?mfr=true
>>
>> Using /b
>>  
> I stand corrected, thanks for the research.
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Very Slow DB Access After Reboot on Windows

[Kees Nuyt]

On Thu, 22 Jul 2010 10:44:50 -0500, "Black, Michael (IS)"
 wrote:

>Wrongread the docs...if copy didn't do binary by default there would so 
>many screwed up computers in the world
> 
> 
>http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/copy.mspx?mfr=true
>
>Using /b 

I stand corrected, thanks for the research.
-- 
  (  Kees Nuyt
  )
c[_]
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Very Slow DB Access After Reboot on Windows

[Griggs, Donald]

Regarding win/dos  COPY command and /b option:

Thanks for correcting me, Michael.  I somehow thought that NUL being the target 
would introduce some sort of "text affinity" but it's good to know the truth 
instead.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Very Slow DB Access After Reboot on Windows

[Black, Michael (IS)]

Wrongread the docs...if copy didn't do binary by default there would so 
many screwed up computers in the world
 
 
http://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/copy.mspx?mfr=true

Using /b 

/b directs the command interpreter to read the number of bytes specified by the 
file size in the directory. /b is the default value for copy, unless copy 
combines files. 

When /b precedes a list of files on the command line, it applies to all listed 
files until copy encounters /a. In this case, /a applies to the file preceding 
/a.

When /b follows a list of files on the command line, it applies to all listed 
files until copy encounters /a. In this case, /a applies to the file preceding 
/a. 

The effect of /b depends on its position in the commandline string. When /b 
follows Source, copy copies the entire file, including any end-of-file 
character.

When /b follows Destination, copy does not add an end-of-file character. 

Michael D. Black
Senior Scientist
Advanced Analytics Directorate
Northrop Grumman Information Systems
 



From: sqlite-users-boun...@sqlite.org on behalf of Kees Nuyt
Sent: Thu 7/22/2010 10:18 AM
To: sqlite-users@sqlite.org
Subject: EXTERNAL:Re: [sqlite] Very Slow DB Access After Reboot on Windows



On Thu, 22 Jul 2010 08:56:31 -0400, "Griggs, Donald"
<donald.gri...@allscripts.com> wrote:

>
>
>Regarding:
>Also...try doing a "copy my.db nul:" to get it cached once before you use 
> it.
>
>
>Am I right in thinking he may want to include the "/b" (binary) option so that 
>the copy doesn't stop at the first nul byte?
>
>  copy /b my.db nul

You are right.
--
  (  Kees Nuyt
  )
c[_]
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Very Slow DB Access After Reboot on Windows

[Kees Nuyt]

On Thu, 22 Jul 2010 08:56:31 -0400, "Griggs, Donald"
 wrote:

> 
>
>Regarding:
>Also...try doing a "copy my.db nul:" to get it cached once before you use 
> it.
> 
> 
>Am I right in thinking he may want to include the "/b" (binary) option so that 
>the copy doesn't stop at the first nul byte?
>
>  copy /b my.db nul

You are right.
-- 
  (  Kees Nuyt
  )
c[_]
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Very Slow DB Access After Reboot on Windows

[Griggs, Donald]

 

Regarding:
Also...try doing a "copy my.db nul:" to get it cached once before you use 
it.
 
 
Am I right in thinking he may want to include the "/b" (binary) option so that 
the copy doesn't stop at the first nul byte?

  copy /b my.db nul
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Very Slow DB Access After Reboot on Windows

[Black, Michael (IS)]

Also...try doing a "copy my.db nul:" to get it cached once before you use it.
 
You're probably running into disk head seeking the first time (due to random 
placement of your data relative to your query) which will slow you down a lot.
 
Michael D. Black
Senior Scientist
Advanced Analytics Directorate
Northrop Grumman Information Systems
 



From: sqlite-users-boun...@sqlite.org on behalf of Simon Slavin
Sent: Wed 7/21/2010 9:28 PM
To: General Discussion of SQLite Database
Subject: EXTERNAL:Re: [sqlite] Very Slow DB Access After Reboot on Windows




On 21 Jul 2010, at 4:56pm, Samet YASLAN wrote:

> I have a 30MB DB file with 4 tables.
> Execution time for a query is 1 sec normally but it is like 40 secs
> after restarting window.
> This seems to be related with file caching of Windows. The same source
> code does not cause this problem on Linux.

What is the name of the database file ?  I understand that there is a Windows 
bug where it tries to cache all of files with certain extensions like '.db'.  
Try changing the filename extention to '.sqlite' or something else unusual.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users


___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] Very Slow DB Access After Reboot on Windows

[Simon Slavin]

On 21 Jul 2010, at 4:56pm, Samet YASLAN wrote:

> I have a 30MB DB file with 4 tables.
> Execution time for a query is 1 sec normally but it is like 40 secs 
> after restarting window.
> This seems to be related with file caching of Windows. The same source 
> code does not cause this problem on Linux.

What is the name of the database file ?  I understand that there is a Windows 
bug where it tries to cache all of files with certain extensions like '.db'.  
Try changing the filename extention to '.sqlite' or something else unusual.

Simon.
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Very Slow DB Access After Reboot on Windows

[Samet YASLAN]

I have a 30MB DB file with 4 tables.
Execution time for a query is 1 sec normally but it is like 40 secs 
after restarting window.
This seems to be related with file caching of Windows. The same source 
code does not cause this problem on Linux.

Any ideas?
-- 
*Samet YASLAN*
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users