Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-07-22 Thread Kyotaro HORIGUCHI
Hello, I'm very sory to take your time on this mess. ul 2013 16:06:11 +0530, Amit Kapila amit.kap...@huawei.com wrote in 014201ce7bc6$f71eb950$e55c2bf0$@kap...@huawei.com I understood your patch's algorithm, but still I have doubt in my mind that if the next analyze can correct the estimates,

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-07-08 Thread Amit Kapila
On Wednesday, July 03, 2013 1:21 PM Kyotaro HORIGUCHI wrote: Hello, I could see the same output with your latest script, also I could reproduce the test if I run the test with individual sql statements. One of the main point for reproducing individual test was to keep autovacuum =

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-07-08 Thread Josh Berkus
On 07/03/2013 12:51 AM, Kyotaro HORIGUCHI wrote: It is tough to decide how to modify there. Currently I decided to preserve vac_estimate_reltuples as possible as it is. For that objective, I picked up old_rel_tuples as intermediate variable for the aid to 'deceive' the function. This can be

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-07-03 Thread Kyotaro HORIGUCHI
Hello, I could see the same output with your latest script, also I could reproduce the test if I run the test with individual sql statements. One of the main point for reproducing individual test was to keep autovacuum = off. I see. Autovacuum's nap time is 60 sconds for the default

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-07-02 Thread Amit Kapila
On Thursday, June 27, 2013 4:58 PM Amit Kapila wrote: On Wednesday, June 26, 2013 7:40 AM Kyotaro HORIGUCHI wrote: I've recovered from messing up. snip Please let me have a bit of time to diagnose this. I was completely messed up and walking on the wrong way. I looked into the

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-27 Thread Amit Kapila
On Wednesday, June 26, 2013 7:40 AM Kyotaro HORIGUCHI wrote: I've recovered from messing up. snip Please let me have a bit of time to diagnose this. I was completely messed up and walking on the wrong way. I looked into the vacuum for UPDATEs, not DELETE's so it's quite resonable to have

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-27 Thread Robert Haas
On Thu, Jun 27, 2013 at 7:27 AM, Amit Kapila amit.kap...@huawei.com wrote: Now I can look into it further, I have still not gone through in detail about your new approach to calculate the reltuples, but I am wondering whether there can be anyway with which estimates can be improved with

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-25 Thread Kyotaro HORIGUCHI
Hello, I have tried to reproduce the problem in different m/c's, but couldn't reproduce it. I have ran tests with default configuration. I think you had reproduced it. Output on Windows: --- postgres=# create table t (a int, b int); (snip) postgres=# select n_live_tup,

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-25 Thread Kyotaro HORIGUCHI
Sorry for lots of mistakes I've made, I noticed that I was dancing on the 9.3dev at sometime, so I reran on 9.4devel up to date, having the same result from the view of inaccuracy of pg_class.reltuples after vacuuming. Although, what differs from my old testset, vacuum reported that it sanned

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-25 Thread Kyotaro HORIGUCHI
I've recovered from messing up. snip Please let me have a bit of time to diagnose this. I was completely messed up and walking on the wrong way. I looked into the vacuum for UPDATEs, not DELETE's so it's quite resonable to have such results. The renewed test script attached shows the verbose

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-25 Thread Amit Kapila
On Tuesday, June 25, 2013 2:06 PM Kyotaro HORIGUCHI wrote: Hello, I have tried to reproduce the problem in different m/c's, but couldn't reproduce it. I have ran tests with default configuration. I think you had reproduced it. Output on Windows: --- postgres=#

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-24 Thread Amit Kapila
On Friday, June 14, 2013 2:05 PM Kyotaro HORIGUCHI wrote: Hello, Postgresql estimates the number of live tuples after the vacuum has left some buffers unscanned. This estimation does well for most cases, but makes completely different result with a strong imbalance of tuple density. For

Re: [HACKERS] Reduce maximum error in tuples estimation after vacuum.

2013-06-14 Thread Kyotaro HORIGUCHI
Sorry, I made an mistake. Kyotaro HORIGUCHI horiguchi.kyot...@lab.ntt.co.jp: Overall algorithm could be illistrated as below, - summing up used bytes, max offnum(PageGetMaxOffsetNumber), Not max offnum, the number of linp's used after page vacuum. maximum free bytes for tuple data ,