Re: [PERFORM] Query planner gaining the ability to replanning after start of query execution.

2017-11-13 Thread Arne Roland
Hello, that method is bound to introduce errors if the physical location of a row correlates strongly with a column, imagine "and my_timestamp> now() - INTERVAL '1 year'" as part of a where clause of a query without limit on an insert only table which is one and a half years old with a seqscan.

Re: [PERFORM] Query planner gaining the ability to replanning after start of query execution.

2017-11-13 Thread Arne Roland
Hello, I'd love to have some sort of dynamic query feedback, yet it's very complicated to do it right. I am not convinced that changing the plan during a single execution is the right way to do it, not only because it sounds intrusive to do crazy things in the executor, but also because don't u

Re: [PERFORM] Query planner gaining the ability to replanning after start of query execution.

2017-11-13 Thread Oliver Mattos
> You can't just restart from scratch, because we may already have shipped > rows to the client For v1, replanning wouldn't be an option if rows have already been shipped, or for DML statements. > parallel plans and most importantly cursors? Parallel plans look do-able with the same approach, b

Re: [PERFORM] Query planner gaining the ability to replanning after start of query execution.

2017-11-13 Thread Tom Lane
Oliver Mattos writes: >> Can you be more elaborate how you'd want to go about it? > ... If another candidate plan is now lower cost, the current plan would be > terminated[1] by setting a flag instructing each execnode to return as > if it had reached the end of the input, although still caching

Re: [PERFORM] Query planner gaining the ability to replanning after start of query execution.

2017-11-13 Thread Oliver Mattos
> Can you be more elaborate how you'd want to go about it? My initial approach would be to try to identify places in the plan where selectivity is seriously over or underestimated. I would reuse the instrumentation infrastructure's counts of filtered and returned tuples for each execnode, and pe

[PERFORM] Query planner gaining the ability to replanning after start of query execution.

2017-11-13 Thread Oliver Mattos
I am interested in giving the query planner the ability to replan (or re-rank plans) after query execution has begun, based on the progression of the query so far. Example use case: * A LIMIT 1 query is planned using an expensive scan which the planner expects to return a large number of results