Re: [HACKERS] Postgresql Materialized views

2008-01-17 Thread Florian G. Pflug
Tom Lane wrote: Well, my point is that taking automatic rewriting as a required feature has at least two negative impacts: * it rules out any form of lazy update, even though for many applications an out-of-date summary view would be acceptable for some purposes; * requiring MVCC consistency

Re: [HACKERS] Postgresql Materialized views

2008-01-16 Thread Heikki Linnakangas
Merlin Moncure wrote: On Jan 12, 2008 4:19 PM, Simon Riggs [EMAIL PROTECTED] wrote: On Sat, 2008-01-12 at 13:27 +0100, Jean-Michel Pouré wrote: Please pick-up this important issue for developpers. There is no need to concentrate on complex issues, when handling materialized views could boost

Re: [HACKERS] Postgresql Materialized views

2008-01-16 Thread Simon Riggs
On Wed, 2008-01-16 at 11:16 +, Heikki Linnakangas wrote: Merlin Moncure wrote: On Jan 12, 2008 4:19 PM, Simon Riggs [EMAIL PROTECTED] wrote: On Sat, 2008-01-12 at 13:27 +0100, Jean-Michel Pouré wrote: Please pick-up this important issue for developpers. There is no need to

Re: [HACKERS] Postgresql Materialized views

2008-01-16 Thread Andreas Pflug
Simon Riggs wrote: My thinking was if you load a 1000 rows and they all have the same key in your summary table then you'll be doing 1000 updates on a single row. This is true because the statement level triggers are still rudimentary, with no OLD and NEW support. A single AFTER statement

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Simon Riggs
On Sun, 2008-01-13 at 17:44 -0800, Sean Utt wrote: It is not my contention that the core developers need to be different in any way. It is also not my contention that the users need to be different in any way. First, this is an open forum, so thank you for expressing your views openly in

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Csaba Nagy
On Mon, 2008-01-14 at 09:22 +, Simon Riggs wrote: So I support Mark Mielke's views on writing code. Anybody who wants to code, can. There's probably a project of a size and complexity that's right for your first project. The main problem is that usually that initial thing is not what you

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Jean-Michel Pouré
Dear Friends, I hope that this flame war can stop, as it is useless. The logic of free software is that developers pick-up issues, based on their skills and interest. The power of the cummunity is to gather very talented developers from all over the planet. Freedom is the logic and there is no

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Mark Mielke
Jean-Michel Pouré wrote: When posting this thread, I hope that a talented developer would some day pick-up the materialized view issue and work on it, during a process of discussion. In case it was lost in the noise - there are several capable people that have been personally on this

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Alvaro Herrera
Mark Mielke wrote: FYI, my triggers are perhaps 10 lines each, and I believe I have three triggers in the 1500 ms - 1 ms example. I have a view and a summary table. I update the summary table from the view. In my opinion, this solution is very manageable given the 1500:1 performance

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Roberts, Jon
-Original Message- From: Tom Lane [mailto:[EMAIL PROTECTED] Sent: Sunday, January 13, 2008 8:18 PM To: Sean Utt Cc: Andrew Dunstan; Joshua D. Drake; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Postgresql Materialized views Sean Utt [EMAIL PROTECTED] writes: My point

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Martijn van Oosterhout
On Mon, Jan 14, 2008 at 08:30:53AM -0600, Roberts, Jon wrote: My point is that you should be able to query _table and the system should automatically use the view, without you saying so (except by initially creating them). I agree! From a BI perspective, a materialized view is worthless

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Zeugswetter Andreas ADI SD
But you had to modify your queries. I would think that a materialized views implementation worth its salt would put the view to work on the original, unmodified queries. I might be slow today (everyday? :-) ) - but what do you mean by this? The only difference between *_table and

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Andrew Dunstan
Roberts, Jon wrote: What gets implemented is whatever individual contributors choose to work on, either because they find it interesting or (in some cases) because someone pays them to do something specific. Certainly, some contributors pay attention to what's being requested, but I see no

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Greg Sabino Mullane
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 My point is simply this: The lack of a clear formal process for feature requests leads to this degradation in the conversation. Without a formalized structure, the conversation devolves rapidly into an argument over semantics and word

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Tom Lane
Zeugswetter Andreas ADI SD [EMAIL PROTECTED] writes: Traditionally materialized views exist, so that you do not need to code differently. Your queries still run on the detail table, but are silently answered by a suitable MV. The MV might have count + other aggregated columns grouped by some

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Joshua D. Drake
Roberts, Jon wrote: 2) In the end, this is an open source *community*; no amount of formal feature requesting will have any material impact on what actually gets implemented, because there isn't any central control. Wow. Being new to Open Source, this amazes me. Well it depends on what

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Zeugswetter Andreas ADI SD
Traditionally materialized views exist, so that you do not need to code differently. Your queries still run on the detail table, but are silently answered by a suitable MV. The MV might have count + other aggregated columns grouped by some columns, and thus be able e.g. shortcircuit a

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Ron Mayer
Mark Mielke wrote: Mark Mielke wrote: Counts, because as we all know, PostgreSQL count(*) is slow, and in any case, my count(*) is not on the whole table, but on a subset. Doing this in a general way seems complex to me as it would need to be able to evaluate whether a given INSERT or UPDATE

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Bruce Momjian
Added to TODO: * Add the ability to automatically create materialized views Right now materialized views require the user to create triggers on the main table to keep the summary table current. SQL syntax should be able to manager the triggers

Re: [HACKERS] Postgresql Materialized views

2008-01-14 Thread Tom Lane
Zeugswetter Andreas ADI SD [EMAIL PROTECTED] writes: Note that you just raised the minimum bar for implementation of the feature by a couple orders of magnitude. Yes, unfortunately. But don't you also think that this is what makes it a worthwhile feature ? Well, my point is that taking

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Jean-Michel Pouré
In fairness to Jean-Michel, he has spent hundreds of hours in the past doing just that and far more for the pgAdmin users in the community - I'm sure we can excuse him for asking for what many do think would be a useful feature in the hopes that someone listening might just decide to pick it

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread James Mansion
Mark Mielke wrote: Joshua D. Drake wrote: Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would be to open up the source code, and give it a try yourself? Because users possibly want to do that - use it? Some of us have better things to do

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Mark Mielke
FYI: I don't like being attacked for stating the truth, nor distracting the mailing list with these emotional discussions. However, there are things that need to be clarified. Feel free to kill the thread in your mail browser. James Mansion wrote: Mark Mielke wrote: Joshua D. Drake wrote:

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Alvaro Herrera
James Mansion wrote: If your view of the community is that it should be insular and closed to those who can't or won't be developers, then fine. But taking that attitude will pretty much guarantee that your system will never amount to more than a hill of beans. Keep in mind that Mark

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Merlin Moncure
On Jan 12, 2008 4:19 PM, Simon Riggs [EMAIL PROTECTED] wrote: On Sat, 2008-01-12 at 13:27 +0100, Jean-Michel Pouré wrote: Please pick-up this important issue for developpers. There is no need to concentrate on complex issues, when handling materialized views could boost somme web apps. by

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Sean Utt
sarcasmGood to see/sarcasm things haven't changed, and requests for features and improvements on the pgsql-hackers list can still degenerate rapidly into a discussion about how to request features and improvements. As Joshua Drake has pointed out before, most of the core people working on

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Joshua D. Drake
Sean Utt wrote: sarcasmGood to see/sarcasm things haven't changed, and requests for features and improvements on the pgsql-hackers list can still degenerate rapidly into a discussion about how to request features and improvements. As Joshua Drake has pointed out before, most of the core

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Webb Sprague
Just my two cents on this (rapidly degenerating) thread. On 1/13/08, Sean Utt [EMAIL PROTECTED] wrote: sarcasmGood to see/sarcasm things haven't changed, and requests for features and improvements on the pgsql-hackers list can still degenerate rapidly into a discussion about how to request

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Tom Lane
Webb Sprague [EMAIL PROTECTED] writes: May I propose the following: (1) can we put materialized views on the TODO, perhaps as a library or as core, still subject to a lot of design work and with no particular deadline? Actually, I had thought they *were* on the TODO list, because certainly

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Andrew Dunstan
Sean Utt wrote: As Joshua Drake has pointed out before, most of the core people working on PostgreSQL don't actually use it for anything themselves. I will expand a little on that and say that this means that while they are extremely good at what they do, they really don't have a clue what

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Joshua D. Drake
Andrew Dunstan wrote: Sean Utt wrote: As Joshua Drake has pointed out before, most of the core people working on PostgreSQL don't actually use it for anything themselves. I will expand a little on that and say that this means that while they are extremely good at what they do, they really

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Andrew Dunstan
Joshua D. Drake wrote: Andrew Dunstan wrote: Sean Utt wrote: As Joshua Drake has pointed out before, most of the core people working on PostgreSQL don't actually use it for anything themselves. I will expand a little on that and say that this means that while they are extremely good at

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Joshua D. Drake
Andrew Dunstan wrote: I'm not going to bother trying, because you just moved the goalposts (managing in a production environment vs using). And why should the number of DBAs matter one whit? Why should they matter more than, say application developers, when it comes to language level

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Andrew Dunstan
Joshua D. Drake wrote: Andrew Dunstan wrote: I'm not going to bother trying, because you just moved the goalposts (managing in a production environment vs using). And why should the number of DBAs matter one whit? Why should they matter more than, say application developers, when it

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Sean Utt
My point is simply this: The lack of a clear formal process for feature requests leads to this degradation in the conversation. Without a formalized structure, the conversation devolves rapidly into an argument over semantics and word choice. It is not my contention that the core developers

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Andrew Dunstan
Sean Utt wrote: My point is simply this: The lack of a clear formal process for feature requests leads to this degradation in the conversation. Without a formalized structure, the conversation devolves rapidly into an argument over semantics and word choice. It is not my contention that the

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Tom Lane
Sean Utt [EMAIL PROTECTED] writes: My point is simply this: The lack of a clear formal process for feature requests leads to this degradation in the conversation. Two comments: 1) The existing informal process has served us very well for more than ten years now. I'm disinclined to consider

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Mark Mielke
Sean Utt wrote: My point is simply this: The lack of a clear formal process for feature requests leads to this degradation in the conversation. Without a formalized structure, the conversation devolves rapidly into an argument over semantics and word choice. It is not my contention that the

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Andrew Chernow
I think, though, that telling them that they must work on a certain feature, because that's what the users are asking for, is the wrong approach. Not to say that is exactly what you are requesting, but I suggest that is where you are leading. Cheers, mark The more communication between

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Webb Sprague
I realize that some very important navel-gazing (^H^H^H group process) is happening, but let us remember where bona-fide feature requests should go: http://www.postgresql.org/docs/faqs.TODO.html So far, I don't see any mention of materialized views on this page, and I did refresh ... :)

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Joshua D. Drake
Webb Sprague wrote: I realize that some very important navel-gazing (^H^H^H group process) is happening, but let us remember where bona-fide feature requests should go: http://www.postgresql.org/docs/faqs.TODO.html So far, I don't see any mention of materialized views on this page, and I did

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Mark Mielke
Andrew Chernow wrote: I think, though, that telling them that they must work on a certain feature, because that's what the users are asking for, is the wrong approach. Not to say that is exactly what you are requesting, but I suggest that is where you are leading. The more communication

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Joshua D. Drake
Mark Mielke wrote: Nobody is ignoring users or needs or wants. It is a question of priority. My priorities may not match yours may not match Tom's or one of the other core contributors. Valuable features are being added to FYI, the terminology core contributors is confusing. There is not

Re: [HACKERS] Postgresql Materialized views

2008-01-13 Thread Webb Sprague
But did you clear your cache? :P Freud might say it takes a lifetime to clear one's cache Luckily, in therapy you don't have to wait for those darn Postgres developers ;) Joshua D. Drake ---(end of broadcast)--- TIP 1: if

[HACKERS] Postgresql Materialized views

2008-01-12 Thread Jean-Michel Pouré
Dear Friends, In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs. You are well aware of Jonathan Gardner preliminary work: http://www.varlena.com/varlena/GeneralBits/Tidbits/matviews.html When do you plan to add MATERIALIZED VIEWS to

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would be to open up the source code, and give it a try

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Dave Page
On 12/01/2008, Mark Mielke [EMAIL PROTECTED] wrote: Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Joshua D. Drake
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Sat, 12 Jan 2008 10:26:49 -0500 Mark Mielke [EMAIL PROTECTED] wrote: Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay*

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Mark Mielke wrote: Counts, because as we all know, PostgreSQL count(*) is slow, and in any case, my count(*) is not on the whole table, but on a subset. Doing this in a general way seems complex to me as it would need to be able to evaluate whether a given INSERT or UPDATE or one of the

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Dave Page wrote: On 12/01/2008, Mark Mielke [EMAIL PROTECTED] wrote: Jean-Michel Pouré wrote: In my past development projects, I always used the concept of Materialized VIEW to speed-up SELECTs over INSERTs Unless you are going to *pay* for it - you do realize that the best way

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Simon Riggs
On Sat, 2008-01-12 at 13:27 +0100, Jean-Michel Pouré wrote: Please pick-up this important issue for developpers. There is no need to concentrate on complex issues, when handling materialized views could boost somme web apps. by a factor of 10 or more. It's more complex than you think, but the

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Mark Mielke
Joshua D. Drake wrote: Unless you are going to *pay* for it - you do realize that the best way to get it implemented, would be to open up the source code, and give it a try yourself? If it was so easy, and such a clear win, I think one of the very competent people using PostgreSQL today would

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Jonah H. Harris
On Jan 12, 2008 5:31 PM, Mark Mielke [EMAIL PROTECTED] wrote: Joshua D. Drake wrote: I think his email was very well written and a simple request of discussion of alternatives as well as future plans. Agreed, JD. Offensive is relative. I find it offensive when people demand things on one of

Re: [HACKERS] Postgresql Materialized views

2008-01-12 Thread Greg Smith
On Sat, 12 Jan 2008, Jean-Michel Pour? wrote: You are well aware of Jonathan Gardner preliminary work: http://www.varlena.com/varlena/GeneralBits/Tidbits/matviews.html There's also PostgreSQL::Snapshots ; intro at http://cunha17.cristianoduarte.pro.br/postgresql/snapshots.en_us.php and main