Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-21 Thread Robert Haas
2010/4/20 Pavel : > For now I know it is not commitable in actual state, but for my thesis it is > enough and I know it will not be commitable with this design at all. In case > of GSoC it will depends on the time I will be able to spend on it, if I will > consider some other design. I am not sure

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-20 Thread Pavel
Greg Smith wrote: pavelbaros wrote: I am also waiting for approval for my repository named "materialized_view" on git.postgresql.org, so I could publish completed parts. Presuming that you're going to wander there and get assigned what looks like an official repo name for this project is a

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-13 Thread Tom Lane
Josh Berkus writes: > There are basically 2 major parts for materialized views: > A) Planner: Getting the query planner to swap in the MatView for part of > a query automatically for query plan portions which the MatView supports; > B) Maintenance: maintaining the MatView data according to the p

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Greg Smith
Josh Berkus wrote: I just worry about any feature which doesn't get as far as a user-visible implementation. If someone doesn't do the rest of the parts soon, such features tend to atrophy because nobody is using them. While they're limited, there are complexly viable prototype quality imp

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Josh Berkus
> I don't want to see Materialized Views wander down the same path as > partitioning, where lots of people produce "fun parts" patches, while > ignoring the grunt work of things like production quality catalog > support for the feature. I think Pavel's proposal got that part right > by starting w

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Greg Smith
Josh Berkus wrote: What would be the use case for (1) by itself? There isn't any use case for just working on the infrastructure, just like there's no use case for "Syntax for partitioning" on its own. That why people rarely work on that part of these problems--it's boring and produces n

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Josh Berkus
Greg, > I'm not saying someone can't jump right into (3), using the current > implementations for (1) and (2) that are floating around out there. I > just think it would end up wasting a fair amount of work on prototypes > that don't work quite the same way as the eventual fully integrated > vers

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Robert Haas
On Mon, Apr 12, 2010 at 3:43 PM, Greg Smith wrote: > Josh Berkus wrote: >> >> There are basically 2 major parts for materialized views: >> A) Planner: Getting the query planner to swap in the MatView for part of >> a query automatically for query plan portions which the MatView supports; >> B) Mai

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Greg Smith
Josh Berkus wrote: There are basically 2 major parts for materialized views: A) Planner: Getting the query planner to swap in the MatView for part of a query automatically for query plan portions which the MatView supports; B) Maintenance: maintaining the MatView data according to the programmed

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Robert Haas
On Mon, Apr 12, 2010 at 1:50 PM, Josh Berkus wrote: > On 4/9/10 1:36 PM, pavelbaros wrote: >> 2) change rewriter >> - usually, view is relation with defined rule and when rewriting, rule >> is fired and relation (view) is replaced by definition of view. If >> relation do not have rule, planner and

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Josh Berkus
On 4/9/10 1:36 PM, pavelbaros wrote: > 2) change rewriter > - usually, view is relation with defined rule and when rewriting, rule > is fired and relation (view) is replaced by definition of view. If > relation do not have rule, planner and executor behave to it as physical > table (relation). In c

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-12 Thread Robert Haas
On Mon, Apr 12, 2010 at 2:16 AM, Pavel Stehule wrote: > I am sure so > dynamical materialised views is bad task for GSoC - it is too large, > too complex. Manually refreshed views is adequate to two months work > and it has sense. That is my feeling also - though I fear that even the simplest pos

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Pavel Stehule
2010/4/12 Robert Haas : > On Sun, Apr 11, 2010 at 5:24 AM, Greg Smith wrote: >> From the rest of your comments, I'm comfortable that you're in sync with the >> not necessarily obvious risky spots here I wanted to raise awareness of. >>  It's unreasonable to expect we'll have exactly the same prior

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Heikki Linnakangas
Robert Haas wrote: > On Sun, Apr 11, 2010 at 10:26 AM, Heikki Linnakangas > wrote: >> Robert Haas wrote: >>> 2010/4/10 Andrew Dunstan : Heikki Linnakangas wrote: > 1. Keep the materialized view up-to-date when the base tables change. > This can be further divided into many steps, you

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Robert Haas
On Sun, Apr 11, 2010 at 5:24 AM, Greg Smith wrote: > From the rest of your comments, I'm comfortable that you're in sync with the > not necessarily obvious risky spots here I wanted to raise awareness of. >  It's unreasonable to expect we'll have exactly the same priorities  here, > and I doubt it

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Robert Haas
On Sun, Apr 11, 2010 at 10:13 PM, Florian G. Pflug wrote: > If continuous updates prove to be too hard initially, you could instead > update the view on select if it's outdated. Such a materialized view > would be a kind of inter-session cache for subselects. > > The hard part would probably be to

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Florian G. Pflug
On 11.04.10 20:47 , Robert Haas wrote: On Sun, Apr 11, 2010 at 10:26 AM, Heikki Linnakangas wrote: Robert Haas wrote: 2010/4/10 Andrew Dunstan: Heikki Linnakangas wrote: 1. Keep the materialized view up-to-date when the base tables change. This can be further divided into many steps, you ca

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Robert Haas
On Sun, Apr 11, 2010 at 10:26 AM, Heikki Linnakangas wrote: > Robert Haas wrote: >> 2010/4/10 Andrew Dunstan : >>> Heikki Linnakangas wrote: 1. Keep the materialized view up-to-date when the base tables change. This can be further divided into many steps, you can begin by supporting

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Heikki Linnakangas
Robert Haas wrote: > 2010/4/10 Andrew Dunstan : >> Heikki Linnakangas wrote: >>> 1. Keep the materialized view up-to-date when the base tables change. >>> This can be further divided into many steps, you can begin by supporting >>> automatic updates only on very simple views with e.g a single table

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-11 Thread Greg Smith
Robert Haas wrote: I also think that you're underestimating the number of problems that will have to be solved to get this done. It's going to take some significant work - both design work and coding work - to figure out how this should integrate into the rest of the system. (What should be the

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-10 Thread Robert Haas
On Sat, Apr 10, 2010 at 11:40 PM, Greg Smith wrote: > To be frank, that makes for a materalized view implementation of little > value over what you can currently do as far as I'm concerned.  It might be > interesting as a prototype, but that's not necessarily going to look like > what's needed to

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-10 Thread Greg Smith
Robert Haas wrote: It's not obvious to me that a brief full-table lock wouldn't be acceptable for an initial implementation. Obviously it wouldn't be suitable for every use case but since we're talking about manually refreshed views that was bound to be true anyway. There already is an init

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-10 Thread Greg Smith
Heikki Linnakangas wrote: Your proposal basically describes doing 1, in a limited fashion where the view is not updated automatically, but only when the DBA runs a command to refresh it. I'm not sure if that's useful enough on its own, writing "CREATE MATERIALIZED VIEW ... SELECT ..." doesn't see

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-10 Thread Kevin Grittner
Greg Smith wrote: > And work on MERGE support is itself blocked behind the fact that > PostgreSQL doesn't have a good way to lock access to a key value > that doesn't exist yet--what other databases call key range > locking. The bulk of the serializable implementation WIP is work to implement

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-10 Thread Robert Haas
2010/4/10 Andrew Dunstan : > Heikki Linnakangas wrote: >> >> 1. Keep the materialized view up-to-date when the base tables change. >> This can be further divided into many steps, you can begin by supporting >> automatic updates only on very simple views with e.g a single table and >> a where clause

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-10 Thread Andrew Dunstan
Heikki Linnakangas wrote: 1. Keep the materialized view up-to-date when the base tables change. This can be further divided into many steps, you can begin by supporting automatic updates only on very simple views with e.g a single table and a where clause. Then extend that to support joins, ag

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-10 Thread Heikki Linnakangas
Greg Smith wrote: > The main hidden complexity in this particular project relates to > handling view refreshes. The non-obvious problem is that when the view > updates, you need something like a SQL MERGE to really handle that in a > robust way that doesn't conflict with concurrent access to queri

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-09 Thread Robert Haas
2010/4/9 Greg Smith : > The main hidden complexity in this particular project relates to handling > view refreshes.  The non-obvious problem is that when the view updates, you > need something like a SQL MERGE to really handle that in a robust way that > doesn't conflict with concurrent access to q

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-09 Thread Greg Smith
pavelbaros wrote: I am also waiting for approval for my repository named "materialized_view" on git.postgresql.org, so I could publish completed parts. Presuming that you're going to wander there and get assigned what looks like an official repo name for this project is a bit...optimistic.

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-09 Thread Robert Haas
2010/4/9 pavelbaros : > Implementation:  could be divided to few steps: > > 1) create materialized view > - modify grammar (parser): CREATE MATERIALIZED VIEW mv_name AS SELECT ... > - change executor, so that it will create physical table defined by select > statement This basically needs to work

Re: [HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-09 Thread Kevin Grittner
pavelbaros wrote: > I am also waiting for approval for my repository named > "materialized_view" on git.postgresql.org They seem to prefer that you get a repository under your name and use materialized_view as a branch name. See my account on git.postgresql.org and its serializable branch for

[HACKERS] GSoC - proposal - Materialized Views in PostgreSQL

2010-04-09 Thread pavelbaros
Hello, I am sending my proposal on GSoC. Details are listed below. Please, if you have ideas, tips, or if you only want to say you opinion about my project, go ahead. thanks, Pavel Baros Abstract: It is effort to implement snapshot materialized view (are only updated when refreshed) in Pos