Re: [sqlite] Idea for improving page cache

2009-10-30 Thread John Crenshaw
> Just for the sake of discussion I've attached a performance > graph for various C++ data structures plus the Unrolled LL. > The tests where run on a dell vostro 1400 laptop. As you can > see the graphs show the ULL to be quite efficient for > insert/delete from the front/back of the list. I

Re: [sqlite] Idea for improving page cache

2009-10-29 Thread Ken
into the list. So this would be very problematic for page movement. Ken --- On Tue, 10/27/09, John Crenshaw <johncrens...@priacta.com> wrote: > From: John Crenshaw <johncrens...@priacta.com> > Subject: Re: [sqlite] Idea for improving page cache > To: "General Dis

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread John Crenshaw
cpy are looked at). John -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Nicolas Williams Sent: Tuesday, October 27, 2009 6:29 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Idea for improving page cache On Tue, Oct

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread Nicolas Williams
On Tue, Oct 27, 2009 at 04:28:11PM -0400, John Crenshaw wrote: > "advantage" kind of depends. ULL is more specialized. You gain some > benefit, but also lose some as well. For example, consider what is > involved in doing a sorted insert into an ULL. On the other hand, you > can get all of the

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread John Crenshaw
users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Kristoffer Danielsson Sent: Tuesday, October 27, 2009 3:50 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] Idea for improving page cache I really like the concept of ULL. Check this one out: http://blogs.msdn.com/devd

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread Pavel Ivanov
> wrote: > >> From: Pavel Ivanov <paiva...@gmail.com> >> Subject: Re: [sqlite] Idea for improving page cache >> To: kennethinbox-sql...@yahoo.com, "General Discussion of SQLite Database" >> <sqlite-users@sqlite.org> >> Date: Tuesday, October 27

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread Ken
--- On Tue, 10/27/09, Pavel Ivanov <paiva...@gmail.com> wrote: > From: Pavel Ivanov <paiva...@gmail.com> > Subject: Re: [sqlite] Idea for improving page cache > To: kennethinbox-sql...@yahoo.com, "General Discussion of SQLite Database" > <sqlite-users@sq

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread Kristoffer Danielsson
; From: johncrens...@priacta.com > To: sqlite-users@sqlite.org > Subject: Re: [sqlite] Idea for improving page cache > > Supposing that the reduced cache misses are worth it, I think it would be > better to simply allocate the nodes from a pool. Allocating from a pool > maximiz

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread John Crenshaw
-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Pavel Ivanov Sent: Tuesday, October 27, 2009 1:38 PM To: kennethinbox-sql...@yahoo.com; General Discussion of SQLite Database Subject: Re: [sqlite] Idea for improving page cache Are you sure that there will be improvement

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread Pavel Ivanov
@live.se> > wrote: > >> From: Kristoffer Danielsson <kristoffer.daniels...@live.se> >> Subject: Re: [sqlite] Idea for improving page cache >> To: sqlite-users@sqlite.org >> Date: Tuesday, October 27, 2009, 1:03 PM >> >> In game development you se

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread Ken
--- On Tue, 10/27/09, Kristoffer Danielsson <kristoffer.daniels...@live.se> wrote: > From: Kristoffer Danielsson <kristoffer.daniels...@live.se> > Subject: Re: [sqlite] Idea for improving page cache > To: sqlite-users@sqlite.org > Date: Tuesday, October 27, 20

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread Kristoffer Danielsson
qlite.org > Subject: Re: [sqlite] Idea for improving page cache > > Are you sure that there will be improvement with ULL? > If you're talking about improving due to CPU internal cache then first > of all you have to store in the list pointers to pages, not pages > themselves (you don't

Re: [sqlite] Idea for improving page cache

2009-10-27 Thread Pavel Ivanov
Are you sure that there will be improvement with ULL? If you're talking about improving due to CPU internal cache then first of all you have to store in the list pointers to pages, not pages themselves (you don't want to store several pages in one chunk of memory, do you?). So you're getting one

[sqlite] Idea for improving page cache

2009-10-27 Thread Ken
Hi All, I have an idea that could improve the page cache performance. Instead of using a regular linked list to connect pages that are on the cache use an "unrolled linked list". On some architectures due to the CPU caching the ULL is about 40 times faster. Still this is mostly