Hi, Bert I understand what WAL mode is not suitable for svn. But my patch uses PERSIST mode which give very good results, e.g. for NTFS on Windows as I shown in tests.
One problem, as says Philip Martin, is reduced performance on NFS disks. But I don't understand why. The number of file operations in PERSIST mode greatly reduced. Unfortunaly I don't have NFS disk, if it means Network File System developed by Sun Microsystems. Windows is using SMB protocol to access files via network. I will test SBM and put result here later. Best regards, Vasily Tunegov. On Wed, Sep 19, 2012 at 7:30 PM, Bert Huijben <b...@vmoo.com> wrote: > systems like NTFS on Windows > MIME-Version: 1.0 > Content-Type: multipart/alternative; boundary=20cf3074b2a0fd54e504ca0ed837 > > --20cf3074b2a0fd54e504ca0ed837 > Content-Type: text/plain; charset="utf-8" > Content-Transfer-Encoding: 7bit > > For your own install you can enable the WAL on the file and other users > of the db will use it. I don't think we can enable it as default (like > you said) > > Bert Huijben (Cell phone) > From: Vasily Tunegov > Sent: 19-9-2012 7:30 > To: Philip Martin > Cc: dev@subversion.apache.org > Subject: Re: Subject: [PATCH] WC rep cache optimization for some file > systems like NTFS on Windows > Hi, Philip > > I'm using SQLite 3.7.14. > I tried to test checkout for our project code base, hosted on svn 1.7.6 > server (93 984 Files, 3 204 Folders, 1.97GB). There are results: > > svn co snv://server/project/branches/branche src -q > > Standard disk: > svn.r1387070 - 256.8 sec > svn.patch - 152.2 sec > > My research shows that PERSIST journal mode greatly reducing the number of > I/O operations, such as creating, opening and closing SQLite journal files > on the disk. The best option here is WAL journal mode. But it works only if > SQLite database is used from one computer (it use memory mapped files API). > I don't know is this restriction is important to svn client. > > Best regards, > Vasily Tunegov. > > On Wed, Sep 19, 2012 at 2:30 PM, Philip Martin > <philip.mar...@wandisco.com>wrote: > > > Vasily Tunegov <vasily.tune...@gmail.com> writes: > > > > > Please review my patch (see attached file patch.txt). It improve svn > > client > > > performance on file systems like NTFS on Windows. Maybe on some other > > file > > > systems, e.g. NFS, but I didn't test it. This patch switch rep cache db > > > journal mode from DELETE (by default) to PERSISTENT. For more info > about > > > journal modes in SQLite see > > > http://www.sqlite.org/pragma.html#pragma_journal_mode. > > > > > > I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7 > x64 > > > Enterprise) for two disk drives: standard and ssd. For tests I used > > > Subversion Benchmark Tool, see > > > > > > https://ctf.open.collab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool > > . > > > For tested application I used svn 1.8.0 from trunk, revision 1387070. > > > > > > Subversion Benchmark Tool tests, total results: > > > > > > Standard disk: > > > svn.r1387070 - 3:05.895 > > > svn.patch - 2:06.548 > > > > > > SSD disk: > > > svn.r1387070 - 2:11.504 > > > svn.patch - 1:38.397 > > > > Which version of SQLite? > > > > On my Linux systems this gives a small improvement in checkout > > performance for working copies on local disks, unfortunately it also > > reduces checkout performance for working copies on NFS disks. > > > > I'm using SQLite 3.7.12.1 and 3.7.13. > > > > -- > > Certified & Supported Apache Subversion Downloads: > > http://www.wandisco.com/subversion/download > > > > --20cf3074b2a0fd54e504ca0ed837 > Content-Type: text/html; charset="utf-8" > Content-Transfer-Encoding: quoted-printable > > <html><head><meta content=3D"text/html; charset=3Dutf-8" > http-equiv=3D"Cont= > ent-Type"></head><body><div><div style=3D"font-family: Calibri,sans-serif; > = > font-size: 11pt;">For your own install you can enable the WAL on the file > a= > nd other users of the db will use it. I don't think we can enable it as > def= > ault (like you said)<br><br>Bert Huijben (Cell > phone)<br></div></div><hr><s= > pan style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; font-weight: > = > bold;">From: </span><span style=3D"font-family: Tahoma,sans-serif; > font-siz= > e: 10pt;">Vasily Tunegov</span><br><span style=3D"font-family: > Tahoma,sans-= > serif; font-size: 10pt; font-weight: bold;">Sent: </span><span > style=3D"fon= > t-family: Tahoma,sans-serif; font-size: 10pt;">19-9-2012 > 7:30</span><br><sp= > an style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; font-weight: > b= > old;">To: </span><span style=3D"font-family: Tahoma,sans-serif; font-size: > = > 10pt;">Philip Martin</span><br><span style=3D"font-family: > Tahoma,sans-seri= > f; font-size: 10pt; font-weight: bold;">Cc: </span><span > style=3D"font-fami= > ly: Tahoma,sans-serif; font-size: 10pt;">dev@subversion.apache.org > </span><b= > r><span style=3D"font-family: Tahoma,sans-serif; font-size: 10pt; > font-weig= > ht: bold;">Subject: </span><span style=3D"font-family: Tahoma,sans-serif; > f= > ont-size: 10pt;">Re: Subject: [PATCH] WC rep cache optimization for some > fi= > le systems like NTFS on Windows</span><br><br></body></html><div > style=3D"c= > > olor:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;background-c= > olor:rgb(255,255,255)">Hi,=C2=A0Philip</div><div > style=3D"color:rgb(34,34,3= > > 4);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255,255= > ,255)"> > <br></div><span > style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;f= > ont-size:13px;background-color:rgb(255,255,255)">I'm using SQLite > 3.7.1= > 4.=C2=A0</span><div > style=3D"color:rgb(34,34,34);font-family:arial,sans-ser= > if;font-size:13px;background-color:rgb(255,255,255)"> > I tried to test checkout for our project code base, hosted on svn 1.7.6 > ser= > ver (93 984 Files, 3 204 Folders, 1.97GB). There are results:</div><div > sty= > > le=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13px;backg= > round-color:rgb(255,255,255)"> > <br></div><div > style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;fo= > nt-size:13px;background-color:rgb(255,255,255)">svn co > snv://server/project= > /branches/branche src -q</div><div > style=3D"color:rgb(34,34,34);font-family= > :arial,sans-serif;font-size:13px;background-color:rgb(255,255,255)"> > <br></div><div > style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;fo= > nt-size:13px;background-color:rgb(255,255,255)">Standard > disk:<br>svn.r1387= > 070 - 256.8 sec<br>svn.patch - 152.2 sec<br><br>My research shows that > PERS= > IST journal mode greatly reducing the number of I/O operations, such as > cre= > ating, opening and closing SQLite journal files on the disk. The best > optio= > n here is WAL journal mode. But it works only if SQLite database is used > fr= > om one computer (it use memory mapped files API). I don't know is this > = > restriction is important to svn client.</div> > <div > style=3D"color:rgb(34,34,34);font-family:arial,sans-serif;font-size:13= > px;background-color:rgb(255,255,255)"><br></div><div > style=3D"color:rgb(34,= > > 34,34);font-family:arial,sans-serif;font-size:13px;background-color:rgb(255= > ,255,255)"> > Best regards,</div><div > style=3D"color:rgb(34,34,34);font-family:arial,sans= > -serif;font-size:13px;background-color:rgb(255,255,255)">Vasily > Tunegov.</d= > iv><br><div class=3D"gmail_quote">On Wed, Sep 19, 2012 at 2:30 PM, Philip > M= > artin <span dir=3D"ltr"><<a href=3D"mailto:philip.mar...@wandisco.com"; > t= > arget=3D"_blank">philip.mar...@wandisco.com</a>></span> wrote:<br> > <blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 > .8ex;border-left:1p= > x #ccc solid;padding-left:1ex"><div class=3D"im">Vasily Tunegov <<a > href= > =3D"mailto:vasily.tune...@gmail.com";>vasily.tune...@gmail.com</a>> > write= > s:<br> > > <br> > > Please review my patch (see attached file patch.txt). It improve svn > c= > lient<br> > > performance on file systems like NTFS on Windows. Maybe on some other > = > file<br> > > systems, e.g. NFS, but I didn't test it. This patch switch rep > cac= > he db<br> > > journal mode from DELETE (by default) to PERSISTENT. For more info > abo= > ut<br> > > journal modes in SQLite see<br> > > <a href=3D"http://www.sqlite.org/pragma.html#pragma_journal_mode"; > targ= > et=3D"_blank">http://www.sqlite.org/pragma.html#pragma_journal_mode > </a>.<br= > > > ><br> > > I have tested this patch on my desktop (Core i7-2600, 16Gb, Windows 7 > = > x64<br> > > Enterprise) for two disk drives: standard and ssd. For tests I > used<br= > > > > Subversion Benchmark Tool, see<br> > > <a href=3D" > https://ctf.open.collab.net/sf/frs/do/listReleases/projects= > .csvn/frs.subversion_benchmark_tool" target=3D"_blank"> > https://ctf.open.col= > lab.net/sf/frs/do/listReleases/projects.csvn/frs.subversion_benchmark_tool > <= > /a>.<br> > > > For tested application I used svn 1.8.0 from trunk, revision > 1387070.<= > br> > ><br> > > Subversion Benchmark Tool tests, total results:<br> > ><br> > > Standard disk:<br> > > svn.r1387070 - 3:05.895<br> > > svn.patch - 2:06.548<br> > ><br> > > SSD disk:<br> > > svn.r1387070 - 2:11.504<br> > > svn.patch - 1:38.397<br> > <br> > </div>Which version of SQLite?<br> > <br> > On my Linux systems this gives a small improvement in checkout<br> > performance for working copies on local disks, unfortunately it also<br> > reduces checkout performance for working copies on NFS disks.<br> > <br> > I'm using SQLite 3.7.12.1 and 3.7.13.<br> > <span class=3D"HOEnZb"><font color=3D"#888888"><br> > --<br> > Certified & Supported Apache Subversion Downloads:<br> > <a href=3D"http://www.wandisco.com/subversion/download"; > target=3D"_blank">h= > ttp://www.wandisco.com/subversion/download</a><br> > </font></span></blockquote></div><br> > > --20cf3074b2a0fd54e504ca0ed837-- >
