Re: [HACKERS] RFC: replace pg_stat_activity.waiting with something more descriptive

2015-11-13 Thread Amit Kapila
On Wed, Sep 16, 2015 at 11:22 PM, Robert Haas wrote: > On Wed, Sep 16, 2015 at 12:29 PM, Alexander Korotkov > wrote: > > >> I think it's reasonable to consider reporting this data in the PGPROC > >> using a 4-byte integer rather than reporting it through a singe byte > >> in the backend status st

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Amit Kapila
On Fri, Nov 13, 2015 at 9:16 PM, Thom Brown wrote: > > On 13 November 2015 at 15:22, Amit Kapila wrote: > > > > There will be hardly any difference in nodes for each worker and it could > > be very long plan for large number of workers. What kind of additional > > information you want which can'

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Tom Lane
Dean Rasheed writes: > On 14 November 2015 at 00:16, Dean Rasheed wrote: >> I can't see a way to make that work reliably. It would need to be >> 10^estimate_ln_weight(base) and the problem is that both exp and >> ln_weight could be too big to fit in double variables, and become >> HUGE_VAL, losin

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Dean Rasheed
On 14 November 2015 at 00:16, Dean Rasheed wrote: > I can't see a way to make that work reliably. It would need to be > 10^estimate_ln_weight(base) and the problem is that both exp and > ln_weight could be too big to fit in double variables, and become > HUGE_VAL, losing all precision. Of course

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Dean Rasheed
On 13 November 2015 at 23:10, Tom Lane wrote: > One more thing: the approach you used in power_var() of doing a whole > separate exp * ln(base) calculation to approximate the result weight > seems mighty expensive, even if it is done at minimal precision. > Couldn't we get a good-enough approximat

Re: [HACKERS] check for interrupts in set_rtable_names

2015-11-13 Thread Jeff Janes
On Fri, Nov 13, 2015 at 3:13 PM, Tom Lane wrote: > Jeff Janes writes: >> Someone sent my server a deranged query, it tripped my >> auto_explain.log_min_duration setting, that hit some kind of >> pathological case while assigning aliases, and now it sits >> uninterruptibly in set_rtable_names for

Re: [HACKERS] check for interrupts in set_rtable_names

2015-11-13 Thread Tom Lane
Jeff Janes writes: > Someone sent my server a deranged query, it tripped my > auto_explain.log_min_duration setting, that hit some kind of > pathological case while assigning aliases, and now it sits > uninterruptibly in set_rtable_names for hours. > Is there any reason we can't check for interru

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Tom Lane
Dean Rasheed writes: > On 13 November 2015 at 18:36, Tom Lane wrote: >> Seems like we arguably should do this whenever the weight isn't zero, >> so as to minimize the number of sqrt() steps. > It's a bit arbitrary. There is a tradeoff here -- computing ln(10) is > more expensive than doing a sqr

[HACKERS] check for interrupts in set_rtable_names

2015-11-13 Thread Jeff Janes
Someone sent my server a deranged query, it tripped my auto_explain.log_min_duration setting, that hit some kind of pathological case while assigning aliases, and now it sits uninterruptibly in set_rtable_names for hours. Is there any reason we can't check for interrupts in set_rtable_names, like

Re: [HACKERS] [DESIGN] ParallelAppend

2015-11-13 Thread Robert Haas
On Thu, Nov 12, 2015 at 12:09 AM, Kouhei Kaigai wrote: > I'm now designing the parallel feature of Append... > > Here is one challenge. How do we determine whether each sub-plan > allows execution in the background worker context? I've been thinking about these questions for a bit now, and I thin

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Dean Rasheed
On 13 November 2015 at 21:00, Tom Lane wrote: > BTW, something I find confusing and error-prone is that this patch keeps > on using the term "weight" to refer to numbers expressed in decimal digits > (ie, the approximate log10 of something). Basically everywhere in the > existing code, "weights"

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Dean Rasheed
On 13 November 2015 at 18:36, Tom Lane wrote: > Next question: in the additional range-reduction step you added to ln_var, > why stop there, ie, what's the rationale for this magic number: > > if (Abs((x.weight + 1) * DEC_DIGITS) > 10) > > Seems like we arguably should do this whenever the

Re: [HACKERS] proposal: multiple psql option -c

2015-11-13 Thread David G. Johnston
On Fri, Nov 13, 2015 at 1:54 PM, Catalin Iacob wrote: > So I promised I'd try to document this. I had a look at the proposed > semantics of -C and I think in the patch they're too complicated which > makes explaining them hard. > > My assumptions about behaviour without this patch, from reading t

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Tom Lane
BTW, something I find confusing and error-prone is that this patch keeps on using the term "weight" to refer to numbers expressed in decimal digits (ie, the approximate log10 of something). Basically everywhere in the existing code, "weights" are measured in base-NBASE digits, while "scales" are m

Re: [HACKERS] proposal: multiple psql option -c

2015-11-13 Thread Catalin Iacob
So I promised I'd try to document this. I had a look at the proposed semantics of -C and I think in the patch they're too complicated which makes explaining them hard. My assumptions about behaviour without this patch, from reading the docs and some experimenting, correct me if I'm wrong: 1. psql

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-13 Thread Andres Freund
On 2015-10-31 11:02:12 +0530, Amit Kapila wrote: > On Thu, Oct 8, 2015 at 11:05 PM, Simon Riggs wrote: > > > > On 1 October 2015 at 23:30, Josh Berkus wrote: > >> > >> On 10/01/2015 07:43 AM, Robert Haas wrote: > >> > On Thu, Oct 1, 2015 at 9:44 AM, Fujii Masao > wrote: > >> >> I wonder how much

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-13 Thread Bruce Momjian
On Tue, Nov 3, 2015 at 09:03:49AM +0530, Amit Kapila wrote: > I think in that case we can call it as page info map or page state map, but > I find retaining visibility map name in this case or for future (if we want to > add another bit) as confusing.  In-fact if you find "visibility and freeze >

Re: [HACKERS] Foreign Data Wrapper

2015-11-13 Thread Corey Huinker
On Fri, Nov 13, 2015 at 1:46 PM, Big Mike wrote: > Writing a Foreign Data Wrapper and interested in isolating the WHERE > clause to speed up the access of an indexed file on my filesystem. I'm > attempting to understand the inner workings of how the data is retrieved so > I'm writing code to just

[HACKERS] Foreign Data Wrapper

2015-11-13 Thread Big Mike
Writing a Foreign Data Wrapper and interested in isolating the WHERE clause to speed up the access of an indexed file on my filesystem. I'm attempting to understand the inner workings of how the data is retrieved so I'm writing code to just handle one case at the moment: WHERE clause on a single co

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Tom Lane
Dean Rasheed writes: > On 12 November 2015 at 21:01, Tom Lane wrote: >> I started to look at this patch, and was immediately bemused by the >> comment in estimate_ln_weight: > That's nonsense. The comment is perfectly correct. It's not saying the > logarithm is negative, it's saying that the *we

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Jeff Janes
On Wed, Nov 11, 2015 at 6:53 AM, Robert Haas wrote: > > I've committed most of this, except for some planner bits that I > didn't like, and after a bunch of cleanup. Instead, I committed the > consider-parallel-v2.patch with some additional planner bits to make > up for the ones I removed from yo

Re: [HACKERS] [PATCH] Refactoring of LWLock tranches

2015-11-13 Thread Ildus Kurbangaliev
On Thu, 12 Nov 2015 14:59:59 -0500 Robert Haas wrote: > On Wed, Nov 11, 2015 at 6:50 AM, Ildus Kurbangaliev > wrote: > > Attached a new version of the patch that moves SLRU tranches and LWLocks to > > SLRU control structs. > > > > `buffer_locks` field now contains LWLocks itself, so we have some

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Thom Brown
On 13 November 2015 at 15:22, Amit Kapila wrote: > On Fri, Nov 13, 2015 at 7:59 PM, Thom Brown wrote: >> >> On 13 November 2015 at 13:38, Amit Kapila wrote: >> > On Wed, Nov 11, 2015 at 11:40 PM, Pavel Stehule >> > >> > wrote: >> >> >> >> >> >> yes - the another little bit unclean in EXPLAIN is

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Amit Kapila
On Fri, Nov 13, 2015 at 7:59 PM, Thom Brown wrote: > > On 13 November 2015 at 13:38, Amit Kapila wrote: > > On Wed, Nov 11, 2015 at 11:40 PM, Pavel Stehule > > wrote: > >> > >> > >> yes - the another little bit unclean in EXPLAIN is number of workers. If I > >> understand to the behave, the quer

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Thom Brown
On 13 November 2015 at 13:38, Amit Kapila wrote: > On Wed, Nov 11, 2015 at 11:40 PM, Pavel Stehule > wrote: >> >> >> yes - the another little bit unclean in EXPLAIN is number of workers. If I >> understand to the behave, the query is processed by two processes if workers >> in the explain is one.

Re: [HACKERS] pg_rewind and replication slots

2015-11-13 Thread Michael Paquier
On Fri, Nov 13, 2015 at 7:41 PM, Vladimir Borodin wrote: > Should pg_rewind ignore pg_replslot dir at all? As it does pg_basebackup, > for example. To keep the code of pg_rewind simple, the decision has been made to make pg_rewind copy everything, the necessary post-cleanup or pre-process saving,

Re: [HACKERS] eXtensible Transaction Manager API

2015-11-13 Thread Michael Paquier
On Thu, Nov 12, 2015 at 11:23 PM, Robert Haas wrote: > On Mon, Nov 9, 2015 at 2:47 PM, Simon Riggs wrote: >> On 9 November 2015 at 18:46, Robert Haas wrote: >>> One point I'd like to mention is that it's absolutely critical to >>> design this in a way that minimizes network roundtrips without >>

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Amit Kapila
On Wed, Nov 11, 2015 at 11:40 PM, Pavel Stehule wrote: > > yes - the another little bit unclean in EXPLAIN is number of workers. If I > understand to the behave, the query is processed by two processes if > workers in the explain is one. > > You are right and I think that is current working model

Re: [HACKERS] eXtensible Transaction Manager API

2015-11-13 Thread Michael Paquier
On Tue, Nov 10, 2015 at 3:46 AM, Robert Haas wrote: > On Sun, Nov 8, 2015 at 6:35 PM, Michael Paquier > wrote: >> Sure. Now imagine that the pg_twophase entry is corrupted for this >> transaction on one node. This would trigger a PANIC on it, and >> transaction would not be committed everywhere.

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Amit Kapila
On Fri, Nov 13, 2015 at 6:17 PM, Thom Brown wrote: > > > > > The number of shared buffers hit could be different across different runs > > because the read sequence of parallel workers can't be guaranteed, also > > I don't think same is even guaranteed for Seq Scan node, the other > > operations >

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Amit Kapila
On Fri, Nov 13, 2015 at 10:56 AM, Robert Haas wrote: > > On Thu, Nov 12, 2015 at 10:39 PM, Amit Kapila wrote: > > The number of shared buffers hit could be different across different runs > > because the read sequence of parallel workers can't be guaranteed, also > > I don't think same is even gu

Re: [HACKERS] Parallel Seq Scan

2015-11-13 Thread Thom Brown
On 13 November 2015 at 03:39, Amit Kapila wrote: > On Thu, Nov 12, 2015 at 9:05 PM, Thom Brown wrote: >> >> On 12 November 2015 at 15:23, Amit Kapila wrote: >> > On Wed, Nov 11, 2015 at 11:29 PM, Pavel Stehule >> > >> > wrote: >> >> >> >> Hi >> >> >> >> I have a first query >> >> >> >> I looked

[HACKERS] pg_rewind and replication slots

2015-11-13 Thread Vladimir Borodin
Hi. Should pg_rewind ignore pg_replslot dir at all? As it does pg_basebackup, for example. -- May the force be with you… https://simply.name

Re: [HACKERS] Inaccurate results from numeric ln(), log(), exp() and pow()

2015-11-13 Thread Dean Rasheed
On 12 November 2015 at 21:01, Tom Lane wrote: > Dean Rasheed writes: >> These results are based on the attached, updated patch which includes >> a few minor improvements. > > I started to look at this patch, and was immediately bemused by the > comment in estimate_ln_weight: > > /* >