Re: [HACKERS] Buffer Requests Trace

2014-10-16 Thread Lucas Lersch
Answering your first question: running tpcc for 1 minute, in a database with 64 warehouses (6~7GB), with a buffer pool of 128MB (around 1.8% of database size) and a hit ratio of ~91%, I get a throughput of 45~50 transactions per second. I did some experiments and I got the following information ab

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Jeff Janes
On Wed, Oct 15, 2014 at 6:22 AM, Lucas Lersch wrote: > So is it a possible normal behavior that running tpcc for 10min only > access 50% of the database? Furthermore, is there a guideline of parameters > for tpcc (# of warehouses, execution time, operations weight)? > > I'm not familiar with your

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Lucas Lersch
I got the following numbers from my tpcc database: Data size: ~6059MB Index size: ~1390MB Total size: ~7400MB Even considering index-only scans, the ratio of around 50% of the database pages being accessed seems unrealistic to me. On Wed, Oct 15, 2014 at 3:50 PM, Stephen Frost wrote: > * Luc

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > So is it a possible normal behavior that running tpcc for 10min only access > 50% of the database? Furthermore, is there a guideline of parameters for > tpcc (# of warehouses, execution time, operations weight)? Depends- you may be aware that we supp

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Lucas Lersch
So is it a possible normal behavior that running tpcc for 10min only access 50% of the database? Furthermore, is there a guideline of parameters for tpcc (# of warehouses, execution time, operations weight)? On Wed, Oct 15, 2014 at 3:09 PM, Simon Riggs wrote: > On 15 October 2014 13:44, Lucas Le

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Simon Riggs
On 15 October 2014 13:44, Lucas Lersch wrote: > I am recording the BufferDesc.tag.blockNum for the buffer along with the > spcNode, dbNode, relNode, also present in the tag. The TPC-C I/O is random, so if you run it for longer you should see a wider set. Cacheing isn't possible as a way to impr

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Lucas Lersch
I am recording the BufferDesc.tag.blockNum for the buffer along with the spcNode, dbNode, relNode, also present in the tag. On Wed, Oct 15, 2014 at 2:27 PM, Simon Riggs wrote: > On 15 October 2014 12:49, Lucas Lersch wrote: > > Sorry for taking so long to answer. I am sending attached the patch

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Simon Riggs
On 15 October 2014 12:49, Lucas Lersch wrote: > Sorry for taking so long to answer. I am sending attached the patch with the > changes I did to pgsql code. I followed the steps for compiling and > installing pgsql from: > http://www.postgresql.org/docs/current/static/install-short.html Are you re

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Lucas Lersch
Sorry for taking so long to answer. I am sending attached the patch with the changes I did to pgsql code. I followed the steps for compiling and installing pgsql from: http://www.postgresql.org/docs/current/static/install-short.html In summary, the page_id of the page being released in ReleaseBuff

Re: [HACKERS] Buffer Requests Trace

2014-10-15 Thread Simon Riggs
On 14 October 2014 17:08, Lucas Lersch wrote: > Unfortunately, in the generated trace with over 2 million buffer requests, > only ~14k different pages are being accessed, out of the 800k of the whole > database. Am I missing something here? We can't tell what you're doing just by knowing the num

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > I see this... but ReleaseBuffer() simply decrements the reference count of > page the buffer currently holds. Assuming that a ReadBuffer() - > ReleaseBuffer() pattern is used for interacting with the shared_buffers, > there will be a ReleaseBuffer() c

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
I see this... but ReleaseBuffer() simply decrements the reference count of page the buffer currently holds. Assuming that a ReadBuffer() - ReleaseBuffer() pattern is used for interacting with the shared_buffers, there will be a ReleaseBuffer() call for any page (heap or index) "loaded" into the sha

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > Aren't heap and index requests supposed to go through the shared buffers > anyway? Sure they do, but a given page in shared_buffers can be used over and over again for different heap and index pages.. Thanks, Stephen signa

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
Aren't heap and index requests supposed to go through the shared buffers anyway? On Tue, Oct 14, 2014 at 7:02 PM, Stephen Frost wrote: > * Lucas Lersch (lucasler...@gmail.com) wrote: > > shared_buffers is 128MB and the version of pgsql is 9.3.5 > > I suspect you're not tracking what you think yo

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > shared_buffers is 128MB and the version of pgsql is 9.3.5 I suspect you're not tracking what you think you're tracking, which is why I brought up shared_buffers. ~14k * 8192 (page size) = ~110MB What it sounds like you're actually tracking are shar

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
shared_buffers is 128MB and the version of pgsql is 9.3.5 On Tue, Oct 14, 2014 at 6:31 PM, Lucas Lersch wrote: > Sorry, I do not understand the question. > > But I forgot to give an additional information: I am printing the page id > for the trace file in ReleaseBuffer() only if it is a shared b

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
Sorry, I do not understand the question. But I forgot to give an additional information: I am printing the page id for the trace file in ReleaseBuffer() only if it is a shared buffer, I am not considering local buffers. I assumed that local buffers were used only for temporary tables. On Tue, Oct

Re: [HACKERS] Buffer Requests Trace

2014-10-14 Thread Stephen Frost
* Lucas Lersch (lucasler...@gmail.com) wrote: > Unfortunately, in the generated trace with over 2 million buffer requests, > only ~14k different pages are being accessed, out of the 800k of the whole > database. Am I missing something here? What do you have shared_buffers set to..? Thanks

[HACKERS] Buffer Requests Trace

2014-10-14 Thread Lucas Lersch
Hello, I changed the buffer manager code in order to generate a trace of page requests from the buffer manager perspective. In summary, whenever ReleaseBuffer() or ReleaseAndReadBuffer() are called, I print the page currently being released which is identified by the tuple (tableSpace, dbNode, rel