Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-19 Thread Marko Kreen
On 10/10/07, Marko Kreen [EMAIL PROTECTED] wrote:
 On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
  * Why is txid_current_snapshot() excluding subtransaction XIDs?  That
  might be all right for the current uses in Slony/Skytools, but it seems
  darn close to a bug for any other use.

 In queue usage the transaction that stores snapshots does not do
 any other work on its own, so it does not matter, main requirement
 is that txid_current()/txid_current_snapshot() be symmetric -
 whatever the txid_current() outputs, the txid_current_snapshot() measures.

 But I agree, supporting subtransactions makes the API more
 universal.  And it wouldn't break Slony/PgQ current usage.

I thought about it with a clear head, and am now on optinion
that the subtransactions should be left out from current API.

I really fail to imagine a scenario where it would be useful.

The module main use comes from the scenario where txid_current(),
txid_current_snapshot() and reader of them are all different
transactions.  Main guarantee that the APi makes is that as
soon you can see a inserted snapshot in table, you can also
see all inserted events in different table.

There does not seem to be any use of them intra-transaction.

Adding them currently in would be just premature bloat.

We can do it always later, when someone presents a case for
them.  Then we can also decide whether it should be added
to current API or have parallel API besides.  It should not
break Slony/Skytools either way.

-- 
marko

---(end of broadcast)---
TIP 6: explain analyze is your friend

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-11 Thread Marko Kreen
On 10/11/07, Tom Lane [EMAIL PROTECTED] wrote:
 Marko Kreen [EMAIL PROTECTED] writes:
  Could you describe bit more?  The is_visible_txid() works
  on data returned by txid_current_snapshot()?  How can there
  be any subtrans id's if txid_current_snapshot() wont return
  them?

 Ah, I see: txid_current() never reports a subxact ID so there's no need to
 consider them elsewhere in txids either.  OK, but this desperately needs
 to be documented.

Will do.

 BTW, I notice that use of txid_current will force assignment of an XID
 in a transaction that might not otherwise have one.  Does this matter,
 or is the expectation that it's only going to be used in transactions
 that are making DB modifications anyway?

Yes, the behaviour is fine - it is meant to be used in transactions
that do modifications.  Even if not, the lazy xid assignment should
stay internal optimization detail of backend and should not be
exposed to users that clearly.

-- 
marko

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-11 Thread Marko Kreen
On 10/10/07, Marko Kreen [EMAIL PROTECTED] wrote:
 On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
  * Why is txid_current_snapshot() excluding subtransaction XIDs?  That
  might be all right for the current uses in Slony/Skytools, but it seems
  darn close to a bug for any other use.

 In queue usage the transaction that stores snapshots does not do
 any other work on its own, so it does not matter, main requirement
 is that txid_current()/txid_current_snapshot() be symmetric -
 whatever the txid_current() outputs, the txid_current_snapshot() measures.

 But I agree, supporting subtransactions makes the API more
 universal.  And it wouldn't break Slony/PgQ current usage.

I thought about it with a clear head, and am now on optinion
that the subtransactions should be left out from current API.

I really fail to imagine a scenario where it would be useful.

The module main use comes from the scenario where txid_current(),
txid_current_snapshot() and reader of them are all different
transactions.  Main guarantee that the APi makes is that as
soon you can see a inserted snapshot in table, you can also
see all inserted events in different table.

There does not seem to be any use of them intra-transaction.

Adding them currently in would be just premature bloat.

We can do it always later, when someone presents a case for
them.  Then we can also decide whether it should be added
to current API or have parallel API besides.  It should not
break Slony/Skytools either way.

-- 
marko

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-11 Thread Bruce Momjian
Magnus Hagander wrote:
 
   The results have nothing to do with whether the process was followed.
   We do not ignore process violations just because the outcome was OK.
 
  Agreed. But reversing something that came out OK for no other reason
  than that the process was violated? I know you don't, but some people
  are asking for exactly that.
 
 So as long as something is committed, and only breaks certain (for now
 unnamed) platforms (until fixed that is), then procedure doesn't apply.

No, if someone does this again they are going to be raked over the coals
like Jan was.  That is probably enough to keep people following procedure.

--
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Magnus Hagander
   We allow /contrib to be more lax about beta changes.
  
  the postgresql ecosystem is growing and there is a lot of people like
  packagers that will be a quite irritated if we keep randomly adding
  completely new code and modules during BETA.
 
 Should packagers be concerned with /contrib at all ?

Our users want it. Because we have important features that live there.
 
 As noted before /contrib is a technical way of ensuring that something
 gets updated together with core, not a recommendation to include it in a
 package.

Then why did it get added there with the motivation that a lot of users will 
want it?

/Magnus 

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Hannu Krosing
Ühel kenal päeval, K, 2007-10-10 kell 07:44, kirjutas Magnus Hagander:
We allow /contrib to be more lax about beta changes.
   
   the postgresql ecosystem is growing and there is a lot of people like
   packagers that will be a quite irritated if we keep randomly adding
   completely new code and modules during BETA.
  
  Should packagers be concerned with /contrib at all ?
 
 Our users want it. Because we have important features that live there.

Sure, but some features are also moved from /contrib to pgfoundry, and
users still may want these too.

And sure there are features in contrib that most users don't want.

  
  As noted before /contrib is a technical way of ensuring that something
  gets updated together with core, not a recommendation to include it in a
  package.
 
 Then why did it get added there with the motivation that a lot of users will 
 want it?

I think that the main motivation was to ensure that a feature that a lot
of users will want will be stable, that is, maintained together with
core postgres.

exposing stable xid and snapshot to userspace is something needed by
more than one postgres add-on (Slony1 replication, Skytools universal
queueing and replication) makes it much easier to develop these packages
without need to have an extra package to maintain separately from these,
yet in sync with core postgres.


Hannu


---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Simon Riggs
On Tue, 2007-10-09 at 18:35 -0400, Andrew Dunstan wrote:

 I think this project has got too big for us to make things up as we go 
 along. We need to follow processes that are well understood and 
 transparent.

Well said, I very much agree.

Mostly we do, but since we've just spent more than 6 months between
Feature Freeze and Beta. There were no well understood or transparent
processes during that period, so nobody is on solid ground trying to
enforce a small number of rules with a single individual. Especially
when discussing what might be argued was a major item in 8.3

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Devrim GÜNDÜZ
Hi,

On Tue, 2007-10-09 at 17:10 -0700, Joshua D. Drake wrote:
  (Will all respect to pginstaller team, I'm *think* it won't take
 much
  time to add txid to installer, at least compared to the time that we
  spent discussing this issue.)
 
 With respect, you don't know. My understanding of the pginstaller
 project is that it is a fairly heft undertaking. It isn't as simple on
 Linux as just calling to the OS level dependencies because library
 versions etc.. 

To avoid any problem/misunderstanding: I do never ever underestimate any
packaging work. I fully respect what Dave, Magnus, Hiroshi and others do
for pginstaller -- and it is the work that I personally cannot do. I
just tried to compare the amount of work from RPM perspective.

Regards,
-- 
Devrim GÜNDÜZ
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/




signature.asc
Description: This is a digitally signed message part


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Magnus Hagander
 We allow /contrib to be more lax about beta changes.

the postgresql ecosystem is growing and there is a lot of people like
packagers that will be a quite irritated if we keep randomly adding
completely new code and modules during BETA.
   
   Should packagers be concerned with /contrib at all ?
  
  Our users want it. Because we have important features that live there.
 
 Sure, but some features are also moved from /contrib to pgfoundry, and
 users still may want these too.

Sure. This is why Dave created stackbuilder.

The point is users expect us to ship whatever isincluded in core postgresql, 
which includes contrib.
 
 And sure there are features in contrib that most users don't want.

Sure. There are features in the backend that most users don't even know about, 
much less use/want.
 
   
   As noted before /contrib is a technical way of ensuring that something
   gets updated together with core, not a recommendation to include it in a
   package.
  
  Then why did it get added there with the motivation that a lot of users 
  will want it?
 
 I think that the main motivation was to ensure that a feature that a lot
 of users will want will be stable, that is, maintained together with
 core postgres.

IMSHO, this is far from enough motivation to put something in post beta. I 
could accept it *with* proper discussion, during feature freze. Where the 
argument of not putting it in contrib, but backend-actual instead, could've 
been properly made.

  exposing stable xid and snapshot to userspace is something needed by
 more than one postgres add-on (Slony1 replication, Skytools universal
 queueing and replication) makes it much easier to develop these packages
 without need to have an extra package to maintain separately from these,
 yet in sync with core postgres.

So, it should be in the backend, not contrib. And since we're past beta, that 
means 8.4.

If discussion had happened just a day or two before the comit, we could've 
delayed beta a day to get it in

/Magnus 

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Mark Kirkwood

Simon Riggs wrote:


Personally, I want to see Jan contribute more, not less. The link with
Slony and related replication technology is critically important to
Postgres, which is why Jan has spent so long on it. 


Generally we should be encouraging everybody to contribute; the project
must have an orientation towards action and growth if we are to survive.
If Jan had not done this, would there have been a storm of protest?


  


+1

Having said that, the fact that the discussion happened in a forum other 
than -hackers is clearly something to be avoided in future (ISTR 
something similar happening with some changes coming from Bizgres a 
while ago). I don't know how we can avoid this sort of thing happening 
every now and again, except to encourage folk to include -hackers in 
their discussions before sending the final patch in (Bruce and others 
have stated this previously - maybe we need to re-iterate it few weeks 
*before* every feature freeze...)


Cheers

Mark


---(end of broadcast)---
TIP 4: Have you searched our list archives?

  http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Dave Page
Devrim GÜNDÜZ wrote:
 (Will all respect to pginstaller team, I'm *think* it won't take much
 time to add txid to installer, at least compared to the time that we
 spent discussing this issue.)

Time is not the issue.

/D

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Simon Riggs
On Wed, 2007-10-10 at 07:08 +0100, Simon Riggs wrote:
 On Tue, 2007-10-09 at 18:35 -0400, Andrew Dunstan wrote:
 
  I think this project has got too big for us to make things up as we go 
  along. We need to follow processes that are well understood and 
  transparent.
 
 Well said, I very much agree.
 
 Mostly we do, but since we've just spent more than 6 months between
 Feature Freeze and Beta. There were no well understood or transparent
 processes during that period, so nobody is on solid ground trying to
 enforce a small number of rules with a single individual. Especially
 when discussing what might be argued was a major item in 8.3

I should add that I'm not unhappy about how things have happened and I
have no complaints to lodge anywhere with anybody. Just wanted to give
Jan a bit of moral support, and I've done that now, so time for me to
stop.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Marko Kreen
On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote:
 On Tue, 9 Oct 2007 18:35:52 -0500
 Michael Glaesemann [EMAIL PROTECTED] wrote:
  On Oct 9, 2007, at 0:06 , Bruce Momjian wrote:
   I am surprised we are not backing
   out the patch and requiring that the patch go through the formal
   review
   process.
 
  I have no opinion as to the patch itself (other than the fact that
  it's a not bug fix), but I think this patch should be reverted
  because it's (a) after feature freeze, (b) had no discussion on
  hackers (or patches), (c) is not a bug fix. IMO rules can be bent
  but there should always at least be discussion before a new feature
  is committed after feature freeze and definitely after beta.
  Otherwise, the rule appears to be if you can get it in somehow, it's
  in.

 I think this almost says it all. My particular gripe about this whole
 thing is that there are other features that are not too intrusive (or
 appear so anyway) that are easily more useful that are not being
 considered at all. Namely,
 http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It
 makes the whole process seem tilted and subjective.

 IMO, the patch is reverted, and submitted for 8.4 or pgfoundry.

Yes, reverting is an option, but please, do that at least with
an understanding what actually happened.  Current discussion
seems to give picture that Jan committed some private piece of
code without consulting anybody which was not the case.

It was actually my patch that was reviewed by 2 senior PostgreSQL
developers: Jan and Tom, then later committed by Jan.  I don't
think the fact that Jan was an interested party by being Slony
developer invalidates his status as PostgreSQL developer.

Obviously that does not make skipping -hackers less mistake,
but there was no evil from anybody and the process for such
exceptional case was _mostly_ followed.

Now the skipping -hackers part - that was also my mistake,
I should have Cc-d the design and code review discussion here
also.  I just saw the contrib-acceptance as minor question,
the main issue was whether Slony was prepared to such a major
rewrite of its core parts on such short notice, so I wanted
to sync with them first.

Also I think several people are annoyed by the Jan asked permission
from -core part of the process.  But I think if you replace the
-core with release manager it will become more understandable.
The fact is there are only few people responsible for releases and
non-technical decisions need to be made by them.  And yes, it should
have been accompanied by technical review in -hackers.

-- 
marko

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Jan Wieck

On 10/10/2007 12:05 AM, Shane Ambler wrote:

Devrim GÜNDÜZ wrote:

Hi,

On Tue, 2007-10-09 at 16:50 -0700, Joshua D. Drake wrote:

IMO, the patch is reverted, and submitted for 8.4 or pgfoundry.


You know, txid was discussed in Slony-I + Skytools lists for a
reasonably long time, and Tom also commented in that thread. I agree
that we broke the policy this time, but this does not mean the end of
the world.


If it has been discussed and planned for so long then it should have 
been considered for inclusion earlier, not just slipped under the radar. 
Even if at feature freeze it wasn't ready it could have been discussed 
whether it could be added after feature freeze if it reached an 
acceptable standard.


If Slony or Skytools need this for a new feature in their x.y release 
then it can be a patch that is included with their release or be a 
prerequisite for their version x.y and detailed in their install steps.


There was no intend to slip it in under the radar. Discussion had 
happened and I failed to realize at the time the code actually looked 
good that the discussion had happened somewhere other than -hackers.


I can certainly take a good amount of flak, especially considering that 
there was a fault on my side. But what is going on right now here is 
getting annoying.


Also Slony doesn't need this module. We can certainly wait until we are 
forced by other reasons to bump Slony to version 3.0, declare 3.0 
incompatible with 8.3 and switch to using txid then. Slony can continue 
using its own xxid data type until then. No problem here.



Jan

--
#==#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.  #
#== [EMAIL PROTECTED] #

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Simon Riggs
On Wed, 2007-10-10 at 11:50 +0300, Marko Kreen wrote:
 On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote:

  IMO, the patch is reverted, and submitted for 8.4 or pgfoundry.
 
 Yes, reverting is an option

Reverting is only an option if we need to solve a technical problem. If
there is no problem then why would we revert it? The only argument I've
seen for reverting the patch is that it broke a process. 

It's hard enough for Developers to get code in without a team of
Anti-Developers trying to take it back out again. Perhaps we should have
an anti-credits section in the release notes to allow all those who've
managed to get work removed get full credit for their anti-efforts. :-)

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Magnus Hagander
On Wed, Oct 10, 2007 at 11:50:12AM +0300, Marko Kreen wrote:
 On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote:
  On Tue, 9 Oct 2007 18:35:52 -0500
  Michael Glaesemann [EMAIL PROTECTED] wrote:
   On Oct 9, 2007, at 0:06 , Bruce Momjian wrote:
I am surprised we are not backing
out the patch and requiring that the patch go through the formal
review
process.
  
   I have no opinion as to the patch itself (other than the fact that
   it's a not bug fix), but I think this patch should be reverted
   because it's (a) after feature freeze, (b) had no discussion on
   hackers (or patches), (c) is not a bug fix. IMO rules can be bent
   but there should always at least be discussion before a new feature
   is committed after feature freeze and definitely after beta.
   Otherwise, the rule appears to be if you can get it in somehow, it's
   in.
 
  I think this almost says it all. My particular gripe about this whole
  thing is that there are other features that are not too intrusive (or
  appear so anyway) that are easily more useful that are not being
  considered at all. Namely,
  http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It
  makes the whole process seem tilted and subjective.
 
  IMO, the patch is reverted, and submitted for 8.4 or pgfoundry.
 
 Yes, reverting is an option, but please, do that at least with
 an understanding what actually happened.  Current discussion
 seems to give picture that Jan committed some private piece of
 code without consulting anybody which was not the case.

At least I am fully aware that it's not a private piece of code. And in
general, I trust Jan (and of course Tom as well) to take a patch from
elsewhere and put it in.

My objections are twofold:

1) We don't add things after beta. I can live with adding it during feature
freeze since it's contrib, and reviewed by these people, but I think it's
horrible to do it after we've shipped beta1.

2) I get the strong feeling that it's going into contrib only because it
missed feature freeze. If it hadn't missed feature freeze, it wuold be in
the backend and not contrib. If the plan is that it lives in contrib
forever, that argument falls. But if the plan is to migrate it into the
backend for 8.4, then I strongly object to using contrib just as a way to
get it in even though we're feature-frozen.


 It was actually my patch that was reviewed by 2 senior PostgreSQL
 developers: Jan and Tom, then later committed by Jan.  I don't
 think the fact that Jan was an interested party by being Slony
 developer invalidates his status as PostgreSQL developer.

Certainly not. 

 Obviously that does not make skipping -hackers less mistake,
 but there was no evil from anybody and the process for such
 exceptional case was _mostly_ followed.

I don't think anybody thinks there were evil intentions behind it. I
certainly don't think Jan would've committed it if he expected people to
dislike it technically. 

All objections have been procedural, AFICS.


 Now the skipping -hackers part - that was also my mistake,
 I should have Cc-d the design and code review discussion here
 also.  I just saw the contrib-acceptance as minor question,
 the main issue was whether Slony was prepared to such a major
 rewrite of its core parts on such short notice, so I wanted
 to sync with them first.
 
 Also I think several people are annoyed by the Jan asked permission
 from -core part of the process.  But I think if you replace the
 -core with release manager it will become more understandable.
 The fact is there are only few people responsible for releases and
 non-technical decisions need to be made by them.  And yes, it should
 have been accompanied by technical review in -hackers.

AFAIK, our release manager is -hackers concensus. We don't *have* a
release manager as such. The closest thing you'd get in that case is the
-packagers list which is for those building the binary packages, and they
were not consulted...

But again, I don't see any issues with the technical side of things. It's
procedural, and placement (is contrib really the right place for it).

//Magnus

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Stefan Kaltenbrunner

Magnus Hagander wrote:

On Wed, Oct 10, 2007 at 11:50:12AM +0300, Marko Kreen wrote:

On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote:

On Tue, 9 Oct 2007 18:35:52 -0500
Michael Glaesemann [EMAIL PROTECTED] wrote:

On Oct 9, 2007, at 0:06 , Bruce Momjian wrote:

I am surprised we are not backing
out the patch and requiring that the patch go through the formal
review
process.

I have no opinion as to the patch itself (other than the fact that
it's a not bug fix), but I think this patch should be reverted
because it's (a) after feature freeze, (b) had no discussion on
hackers (or patches), (c) is not a bug fix. IMO rules can be bent
but there should always at least be discussion before a new feature
is committed after feature freeze and definitely after beta.
Otherwise, the rule appears to be if you can get it in somehow, it's
in.

I think this almost says it all. My particular gripe about this whole
thing is that there are other features that are not too intrusive (or
appear so anyway) that are easily more useful that are not being
considered at all. Namely,
http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It
makes the whole process seem tilted and subjective.

IMO, the patch is reverted, and submitted for 8.4 or pgfoundry.

Yes, reverting is an option, but please, do that at least with
an understanding what actually happened.  Current discussion
seems to give picture that Jan committed some private piece of
code without consulting anybody which was not the case.


At least I am fully aware that it's not a private piece of code. And in
general, I trust Jan (and of course Tom as well) to take a patch from
elsewhere and put it in.

My objections are twofold:

1) We don't add things after beta. I can live with adding it during feature
freeze since it's contrib, and reviewed by these people, but I think it's
horrible to do it after we've shipped beta1.


yeah that is exactly the point - if we do have a feature freeze we 
should hold to it. if we are in BETA we should not add any new code.




2) I get the strong feeling that it's going into contrib only because it
missed feature freeze. If it hadn't missed feature freeze, it wuold be in
the backend and not contrib. If the plan is that it lives in contrib
forever, that argument falls. But if the plan is to migrate it into the
backend for 8.4, then I strongly object to using contrib just as a way to
get it in even though we're feature-frozen.


yeah I agree that code like this should be either in core or somewhere 
else (either pgfoundry or even shipped as part of the replication 
solutions mentioned which is basically something slony did for ages with 
the xxid stuff). Just pushing it now into contrib results in people 
wanting to use one of those solution having to deal with 3 kinds of 
packages:


1. postgresql
2. postgresql-contrib
3. skytools/slony/...

instead of just two which does not strike me as much of an improvement.


Stefan

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Marko Kreen
On 10/10/07, Magnus Hagander [EMAIL PROTECTED] wrote:
 All objections have been procedural, AFICS.

Lets not talk about mistakes we made for a moment.

And I agree with the rest of the objections in general. But I'd
like to summarise why I still hope the exception can be made
even this late.

This is directly related to attitude to the first submission to 8.2:
unless Slony uses it we are not interested.  Now is the only
moment which won't come again in several years that it's possible
to unify txid handling in Slony and Skytools and also make the
functionality available to broader public.

This due to the fact that Slony 2.0 which will be released with 8.3
will not support PostgreSQL version lower then 8.3.

Yes, we realized the opportunity too late and now it's question
if PostgreSQL is flexible enough to react to this.

Note that rejection now does not cause any big problems to either
Slony and Skytools, we will keep our internal versions of the module,
invisible to anybody else.

But the potential use of the module is huge - it's killer feature is
that it allows to implement high-performance multi-reader / multi-writer
queues inside database.  Well, I know this sounds unimpressive, queues
do not belong to standard toolbox when doing database developement.
And those who have tried to implement them, carry a avoid at any cost tag,
because thus far there has not been a way to implement robust and
well-perfoming queue inside general-purpose database.

Now txid can change that.  E.g. in Skype, it has become irreplaceable
tool for coordinating work between several databases.  Here we are
probably going overboard with usage of queues...

-- 
marko

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Magnus Hagander
On Wed, Oct 10, 2007 at 03:27:17PM +0300, Marko Kreen wrote:
 On 10/10/07, Magnus Hagander [EMAIL PROTECTED] wrote:
  All objections have been procedural, AFICS.
 
 Lets not talk about mistakes we made for a moment.
 
 And I agree with the rest of the objections in general. But I'd
 like to summarise why I still hope the exception can be made
 even this late.
 
 This is directly related to attitude to the first submission to 8.2:
 unless Slony uses it we are not interested.  Now is the only
 moment which won't come again in several years that it's possible
 to unify txid handling in Slony and Skytools and also make the
 functionality available to broader public.
 
 This due to the fact that Slony 2.0 which will be released with 8.3
 will not support PostgreSQL version lower then 8.3.
 
 Yes, we realized the opportunity too late and now it's question
 if PostgreSQL is flexible enough to react to this.
 
 Note that rejection now does not cause any big problems to either
 Slony and Skytools, we will keep our internal versions of the module,
 invisible to anybody else.
 
 But the potential use of the module is huge - it's killer feature is
 that it allows to implement high-performance multi-reader / multi-writer
 queues inside database.  Well, I know this sounds unimpressive, queues
 do not belong to standard toolbox when doing database developement.
 And those who have tried to implement them, carry a avoid at any cost tag,
 because thus far there has not been a way to implement robust and
 well-perfoming queue inside general-purpose database.
 
 Now txid can change that.  E.g. in Skype, it has become irreplaceable
 tool for coordinating work between several databases.  Here we are
 probably going overboard with usage of queues...

If it is this irreplacable killer feature, it should *not* be in contrib.
It should be in the core backend, and we should be discussing if we can
bend the rules for that. This is the proper forum for discussing that, so
let's bring that question to the table.

Our beta-1 is already fairly broken (the locale stuff on our most
downloaded platform), so perhaps we should pull that one back, put this
stuff in the backend, and try to get a beta2 out ASAP?


Putting it in contrib just because we were too late to put it in the
backend, but it is reallyi really important for our users just doesn't
make sense.

//Magnus

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Bruce Momjian
Marko Kreen wrote:
 On 10/10/07, Joshua D. Drake [EMAIL PROTECTED] wrote:
  On Tue, 9 Oct 2007 18:35:52 -0500
  Michael Glaesemann [EMAIL PROTECTED] wrote:
   On Oct 9, 2007, at 0:06 , Bruce Momjian wrote:
I am surprised we are not backing
out the patch and requiring that the patch go through the formal
review
process.
  
   I have no opinion as to the patch itself (other than the fact that
   it's a not bug fix), but I think this patch should be reverted
   because it's (a) after feature freeze, (b) had no discussion on
   hackers (or patches), (c) is not a bug fix. IMO rules can be bent
   but there should always at least be discussion before a new feature
   is committed after feature freeze and definitely after beta.
   Otherwise, the rule appears to be if you can get it in somehow, it's
   in.
 
  I think this almost says it all. My particular gripe about this whole
  thing is that there are other features that are not too intrusive (or
  appear so anyway) that are easily more useful that are not being
  considered at all. Namely,
  http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It
  makes the whole process seem tilted and subjective.
 
  IMO, the patch is reverted, and submitted for 8.4 or pgfoundry.
 
 Yes, reverting is an option, but please, do that at least with
 an understanding what actually happened.  Current discussion
 seems to give picture that Jan committed some private piece of
 code without consulting anybody which was not the case.
 
 It was actually my patch that was reviewed by 2 senior PostgreSQL
 developers: Jan and Tom, then later committed by Jan.  I don't
 think the fact that Jan was an interested party by being Slony
 developer invalidates his status as PostgreSQL developer.
 
 Obviously that does not make skipping -hackers less mistake,
 but there was no evil from anybody and the process for such
 exceptional case was _mostly_ followed.
 
 Now the skipping -hackers part - that was also my mistake,
 I should have Cc-d the design and code review discussion here
 also.  I just saw the contrib-acceptance as minor question,
 the main issue was whether Slony was prepared to such a major
 rewrite of its core parts on such short notice, so I wanted
 to sync with them first.
 
 Also I think several people are annoyed by the Jan asked permission
 from -core part of the process.  But I think if you replace the
 -core with release manager it will become more understandable.
 The fact is there are only few people responsible for releases and
 non-technical decisions need to be made by them.  And yes, it should
 have been accompanied by technical review in -hackers.

I don't think this is accurate.  Jan talked to Tom, not all of core, and
Tom just gave general approval.  Tom still expected this to go through
the hackers review process.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Bruce Momjian

Agreed.  I think if we had followed procedure the code would have been
accepted post-beta1.

---

Marko Kreen wrote:
 On 10/10/07, Magnus Hagander [EMAIL PROTECTED] wrote:
  All objections have been procedural, AFICS.
 
 Lets not talk about mistakes we made for a moment.
 
 And I agree with the rest of the objections in general. But I'd
 like to summarise why I still hope the exception can be made
 even this late.
 
 This is directly related to attitude to the first submission to 8.2:
 unless Slony uses it we are not interested.  Now is the only
 moment which won't come again in several years that it's possible
 to unify txid handling in Slony and Skytools and also make the
 functionality available to broader public.
 
 This due to the fact that Slony 2.0 which will be released with 8.3
 will not support PostgreSQL version lower then 8.3.
 
 Yes, we realized the opportunity too late and now it's question
 if PostgreSQL is flexible enough to react to this.
 
 Note that rejection now does not cause any big problems to either
 Slony and Skytools, we will keep our internal versions of the module,
 invisible to anybody else.
 
 But the potential use of the module is huge - it's killer feature is
 that it allows to implement high-performance multi-reader / multi-writer
 queues inside database.  Well, I know this sounds unimpressive, queues
 do not belong to standard toolbox when doing database developement.
 And those who have tried to implement them, carry a avoid at any cost tag,
 because thus far there has not been a way to implement robust and
 well-perfoming queue inside general-purpose database.
 
 Now txid can change that.  E.g. in Skype, it has become irreplaceable
 tool for coordinating work between several databases.  Here we are
 probably going overboard with usage of queues...
 
 -- 
 marko

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Bruce Momjian
Magnus Hagander wrote:
  Now txid can change that.  E.g. in Skype, it has become irreplaceable
  tool for coordinating work between several databases.  Here we are
  probably going overboard with usage of queues...
 
 If it is this irreplacable killer feature, it should *not* be in contrib.
 It should be in the core backend, and we should be discussing if we can
 bend the rules for that. This is the proper forum for discussing that, so
 let's bring that question to the table.
 
 Our beta-1 is already fairly broken (the locale stuff on our most
 downloaded platform), so perhaps we should pull that one back, put this
 stuff in the backend, and try to get a beta2 out ASAP?
 
 
 Putting it in contrib just because we were too late to put it in the
 backend, but it is reallyi really important for our users just doesn't
 make sense.

I also agree with this.  We have to pretend it isn't in /contrib now,
figure out where want it, then put it there (contrib, pgfoundry, core).

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Andrew Dunstan



Magnus Hagander wrote:

If it is this irreplacable killer feature, it should *not* be in contrib.
It should be in the core backend, and we should be discussing if we can
bend the rules for that. This is the proper forum for discussing that, so
let's bring that question to the table.

Our beta-1 is already fairly broken (the locale stuff on our most
downloaded platform), so perhaps we should pull that one back, put this
stuff in the backend, and try to get a beta2 out ASAP?


Putting it in contrib just because we were too late to put it in the
backend, but it is reallyi really important for our users just doesn't
make sense.


  


I agree with most of this. If it's really that important we should 
abandon beta1 in effect and open the tree for just this and as much of 
the extra tsearch samples as we care to take. But that's a decision that 
should be taken openly, after discussion on -hackers. I don't think I'm 
being arrogant when I say that if I was completely unaware of this 
proposal then a great many other people probably were too.


cheers

andrew

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Shane Ambler

Magnus Hagander wrote:


If it is this irreplacable killer feature, it should *not* be in contrib.
It should be in the core backend, and we should be discussing if we can
bend the rules for that. This is the proper forum for discussing that, so
let's bring that question to the table.


+1 there, I don't think it should go into contrib just cause it was a 
late entry. It really seems to be a matter of whether it gets into 8.3 
or 8.4



Our beta-1 is already fairly broken (the locale stuff on our most
downloaded platform), so perhaps we should pull that one back, put this
stuff in the backend, and try to get a beta2 out ASAP?



The question there is how long will it take to reach a decision of where 
the patch belongs? (8.3 8.4 or contrib)



Putting it in contrib just because we were too late to put it in the
backend, but it is reallyi really important for our users just doesn't
make sense.


+1



--

Shane Ambler
[EMAIL PROTECTED]

Get Sheeky @ http://Sheeky.Biz

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Bruce Momjian
Simon Riggs wrote:
 On Tue, 2007-10-09 at 18:35 -0400, Andrew Dunstan wrote:
 
  I think this project has got too big for us to make things up as we go 
  along. We need to follow processes that are well understood and 
  transparent.
 
 Well said, I very much agree.
 
 Mostly we do, but since we've just spent more than 6 months between
 Feature Freeze and Beta. There were no well understood or transparent
 processes during that period, so nobody is on solid ground trying to
 enforce a small number of rules with a single individual. Especially
 when discussing what might be argued was a major item in 8.3

What?  Even Jan agrees he didn't follow the procedures.  I have no idea
how you are coming to the conclusion there are no well understood
processes.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes:
 On Wed, Oct 10, 2007 at 03:27:17PM +0300, Marko Kreen wrote:
 Now txid can change that.  E.g. in Skype, it has become irreplaceable
 tool for coordinating work between several databases.  Here we are
 probably going overboard with usage of queues...

 If it is this irreplacable killer feature, it should *not* be in contrib.
 It should be in the core backend, and we should be discussing if we can
 bend the rules for that.

It may be a killer feature for Skytools and Slony, but even so that's a
small part of our userbase.  I see nothing wrong with having it in
contrib now with an eye to migrating to the core later, when and if we
see there's enough demand for that.  Another reason for that approach
is that once it's in core it will be very much harder to make any tweaks
to the API; and with the prospective uses being largely unwritten as
yet, it hardly seems unlikely we might not want some changes.

I think our two realistic options today are (1) leave the code where
it is, or (2) remove it.  While Jan clearly failed to follow the agreed
procedures, I'm not convinced the transgression was severe enough to
justify (2).

regards, tom lane

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 Marko Kreen wrote:
 Also I think several people are annoyed by the Jan asked permission
 from -core part of the process.

 I don't think this is accurate.  Jan talked to Tom, not all of core, and
 Tom just gave general approval.  Tom still expected this to go through
 the hackers review process.

Well, my view of the discussion was that Jan asked core if any of us
would veto a late contrib addition.  I trust you'll agree that if anyone
on core were to vote against, it would not have gone in; and so it
seemed reasonable to me for Jan to check this before expending any
further effort on creating a patch.

I asked a couple questions and then said it sounded okay to me; I don't
recall now if anyone else commented, but this was definitely a
discussion on -core not personal email.

In any case, since that was in advance of seeing the code, I certainly
was expecting a -patches submission before commit ...

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Bruce Momjian [EMAIL PROTECTED] writes:
 I also agree with this.  We have to pretend it isn't in /contrib now,
 figure out where want it, then put it there (contrib, pgfoundry, core).

Putting it in core now would mean forcing a post-beta1 initdb, which
I don't think adequate cause has been shown for.

Possibly we should sit on the decision for awhile and see if any
initdb-forcing bugs are reported.  But for the moment I think only the
contrib or pgfoundry options are acceptable.

regards, tom lane

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Marko Kreen
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
 Magnus Hagander [EMAIL PROTECTED] writes:
  On Wed, Oct 10, 2007 at 03:27:17PM +0300, Marko Kreen wrote:
  Now txid can change that.  E.g. in Skype, it has become irreplaceable
  tool for coordinating work between several databases.  Here we are
  probably going overboard with usage of queues...

  If it is this irreplacable killer feature, it should *not* be in contrib.
  It should be in the core backend, and we should be discussing if we can
  bend the rules for that.

 It may be a killer feature for Skytools and Slony, but even so that's a
 small part of our userbase.  I see nothing wrong with having it in
 contrib now with an eye to migrating to the core later, when and if we
 see there's enough demand for that.  Another reason for that approach
 is that once it's in core it will be very much harder to make any tweaks
 to the API; and with the prospective uses being largely unwritten as
 yet, it hardly seems unlikely we might not want some changes.

Considering the core operations are now being in active use
some 6-7 years, I really fail to see how there can be anything
to tweak, unless you are speaking changing naming style.

IMHO the core operations are already as stable as PostgreSQL use
of MVCC, as the module just exports backend internal state...
Current set of functions is the minimal necessary to implement
queue operations, there is nothing more to remove.

We might want to add some helper functions for query generation,
but that does not affect core operation.

Another thing can can be done is more compact representation for
txid_snapshot type, but that also won't affect core operation.

I am very happy for txid being in contrib, I'm not arguing against
that, but the hint that txid module is something that just freshly
popped out of thin air is irking me.

 I think our two realistic options today are (1) leave the code where
 it is, or (2) remove it.  While Jan clearly failed to follow the agreed
 procedures, I'm not convinced the transgression was severe enough to
 justify (2).

Thanks for being understanding.

-- 
marko

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Magnus Hagander
On Wed, Oct 10, 2007 at 11:30:47AM -0400, Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  I also agree with this.  We have to pretend it isn't in /contrib now,
  figure out where want it, then put it there (contrib, pgfoundry, core).
 
 Putting it in core now would mean forcing a post-beta1 initdb, which
 I don't think adequate cause has been shown for.

Ok. In that case, my vote is pgfoundry (heh, I'm sure that's clear by now).
I don't think an adequate cause to break all our procedures to stick it in
core has been shown either.


 Possibly we should sit on the decision for awhile and see if any
 initdb-forcing bugs are reported.  But for the moment I think only the
 contrib or pgfoundry options are acceptable.

This sounds like a good fallback - if the option opens up, I really think
it should be put in the backend. (Assuming it's technically sound - I still
haven't checked the actual code, but I'm assuming it's Ok since Jan
approved it)

//Magnus

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Devrim GÜNDÜZ
Hi,

On Wed, 2007-10-10 at 09:19 +0100, Simon Riggs wrote:
 I should add that I'm not unhappy about how things have happened and I
 have no complaints to lodge anywhere with anybody. Just wanted to give
 Jan a bit of moral support 

I have the same feelings, so +1.

Regards,
-- 
Devrim GÜNDÜZ
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/




signature.asc
Description: This is a digitally signed message part


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Marko Kreen [EMAIL PROTECTED] writes:
 IMHO the core operations are already as stable as PostgreSQL use
 of MVCC, as the module just exports backend internal state...

Well, it exports backend internal state that did not exist before 8.2
(ie, XID epoch).  So it doesn't seem all that set in stone to me.

 Another thing can can be done is more compact representation for
 txid_snapshot type, but that also won't affect core operation.

That's another thing that's likely to become very much harder to change
once it's in core.  People keep threatening to produce a working
in-place-upgrade process, and once that's reality the on-disk
representation of core types is going to be hard to change.

regards, tom lane

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Simon Riggs
On Wed, 2007-10-10 at 10:29 -0400, Bruce Momjian wrote:
 Simon Riggs wrote:

  Mostly we do, but since we've just spent more than 6 months between
  Feature Freeze and Beta. There were no well understood or transparent
  processes during that period, so nobody is on solid ground trying to
  enforce a small number of rules with a single individual. Especially
  when discussing what might be argued was a major item in 8.3
 
 What?  Even Jan agrees he didn't follow the procedures.  I have no idea
 how you are coming to the conclusion there are no well understood
 processes.

Results look good to me, so I'm not lodging a complaint, just explaining
my position.

The main rule is discuss-it-first-on-Hackers and if Jan didn't follow
that then he is wrong. But since he nearly got it right by discussing it
with some people, nothing bad happened and the outcome is worth having I
don't see why we would want to pick Jan up for anything, or back-out
those changes.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Marko Kreen
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
 Marko Kreen [EMAIL PROTECTED] writes:
  IMHO the core operations are already as stable as PostgreSQL use
  of MVCC, as the module just exports backend internal state...

 Well, it exports backend internal state that did not exist before 8.2
 (ie, XID epoch).  So it doesn't seem all that set in stone to me.

Ok.  Lets say the API concepts are 6-7 years old.  The epoch was
a only thing missing in API ca. 2 years ago (the 8byte txid was
written on 8.0), so now the API is complete.

  Another thing can can be done is more compact representation for
  txid_snapshot type, but that also won't affect core operation.

 That's another thing that's likely to become very much harder to change
 once it's in core.  People keep threatening to produce a working
 in-place-upgrade process, and once that's reality the on-disk
 representation of core types is going to be hard to change.

Good point.  But txid_snapshot happens to have couple of free
bits that can be used to create backwards compatibility, so I
don't think that could be a big problem.

Anyway, the in-place upgrade seems a month-two away couple of
years now :)

-- 
marko

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Bruce Momjian
Simon Riggs wrote:
 On Wed, 2007-10-10 at 10:29 -0400, Bruce Momjian wrote:
  Simon Riggs wrote:
 
   Mostly we do, but since we've just spent more than 6 months between
   Feature Freeze and Beta. There were no well understood or transparent
   processes during that period, so nobody is on solid ground trying to
   enforce a small number of rules with a single individual. Especially
   when discussing what might be argued was a major item in 8.3
  
  What?  Even Jan agrees he didn't follow the procedures.  I have no idea
  how you are coming to the conclusion there are no well understood
  processes.
 
 Results look good to me, so I'm not lodging a complaint, just explaining
 my position.
 
 The main rule is discuss-it-first-on-Hackers and if Jan didn't follow
 that then he is wrong. But since he nearly got it right by discussing it
 with some people, nothing bad happened and the outcome is worth having I
 don't see why we would want to pick Jan up for anything, or back-out
 those changes.

The results have nothing to do with whether the process was followed. 
We do not ignore process violations just because the outcome was OK.

And Jan did not come even close to following procedure.  He just asked
core if they would object to such an addition post-feature freeze.  He
did not show code to core so there is no sense the core approved the
patch.  And Jan did not show the patch to hackers or discuss the patch
application.

I do not want to keep bashing Jan but Simon's statements require me to
set the record straight.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Magnus Hagander [EMAIL PROTECTED] writes:
 (Assuming it's technically sound - I still haven't checked the actual
 code, but I'm assuming it's Ok since Jan approved it)

I hadn't looked at it either, but here are a few things that need
review:

* Why no binary I/O support for the new datatype?  We tend to expect
that for all core types.

* Why is txid_current_snapshot() excluding subtransaction XIDs?  That
might be all right for the current uses in Slony/Skytools, but it seems
darn close to a bug for any other use.

* Why is txid_current_snapshot() reading SerializableSnapshot rather
than an actually current snap as its name suggests?  This isn't just
misleading, this will fail completely when SerializableSnapshot
goes away, as seems likely to happen in 8.4 (and no, we won't keep it
just because txid might want it).

regards, tom lane

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Magnus Hagander
On Wed, Oct 10, 2007 at 11:47:15AM -0400, Tom Lane wrote:
 Marko Kreen [EMAIL PROTECTED] writes:
  IMHO the core operations are already as stable as PostgreSQL use
  of MVCC, as the module just exports backend internal state...
 
 Well, it exports backend internal state that did not exist before 8.2
 (ie, XID epoch).  So it doesn't seem all that set in stone to me.
 
  Another thing can can be done is more compact representation for
  txid_snapshot type, but that also won't affect core operation.
 
 That's another thing that's likely to become very much harder to change
 once it's in core.  People keep threatening to produce a working
 in-place-upgrade process, and once that's reality the on-disk
 representation of core types is going to be hard to change.

Well, if that is a concern, than it's an equally big concern to have it in
contrib. If people start using it, they're not going to care about us
saying hey, it was in contrib, why did you use it, when we earlier said
in order do use our whiz-bang stuff, you must install from contrib. We'll
have complaints that it's too hard to install, but we won't manage to
escape from the responsibility to keep it working.

//Magnus

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Gregory Stark
Magnus Hagander [EMAIL PROTECTED] writes:

 On Wed, Oct 10, 2007 at 11:30:47AM -0400, Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  I also agree with this.  We have to pretend it isn't in /contrib now,
  figure out where want it, then put it there (contrib, pgfoundry, core).
 
 Putting it in core now would mean forcing a post-beta1 initdb, which
 I don't think adequate cause has been shown for.

 Ok. In that case, my vote is pgfoundry (heh, I'm sure that's clear by now).
 I don't think an adequate cause to break all our procedures to stick it in
 core has been shown either.

I just don't see the point in putting it in pgfoundry. It's already in
pgfoundry as part of Skytools. The whole point of having such a datatype is to
build common interface to abstract away the internals of the database. That
makes the pgfoundry modules (Skytools and Slony) easier to maintain
separately. Putting txid on pgfoundry just means splitting one pgfoundry
package into two. Moving code around doesn't make it any easier to maintain,
the portion that depends on the database internals will still depend on
database internals.

Putting it in core or contrib means that when we change the snapshot mechanics
in 8.4 the same developer will be able to fix the module at the same time (and
find out if his changes break it at the same time). Also different versions of
Postgres would include appropriate txid modules without having to maintain a
single version with a million ifdefs to cover multiple versions of Postgres.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Joshua D. Drake
On Wed, 10 Oct 2007 11:04:53 -0400
Tom Lane [EMAIL PROTECTED] wrote:

 Magnus Hagander [EMAIL PROTECTED] writes:
  On Wed, Oct 10, 2007 at 03:27:17PM +0300, Marko Kreen wrote:
  Now txid can change that.  E.g. in Skype, it has become
  irreplaceable tool for coordinating work between several
  databases.  Here we are probably going overboard with usage of
  queues...
 
  If it is this irreplacable killer feature, it should *not* be in
  contrib. It should be in the core backend, and we should be
  discussing if we can bend the rules for that.
 
 It may be a killer feature for Skytools and Slony, but even so that's
 a small part of our userbase. 

This is a valid point. Skytools is cool. Slony is cool, but their user
bases compared to PostgreSQL proper are miniscule. We need to be
considering the potential issues (if there are any) that this can cause
for the larger community.

I can think of a couple of immediate ones, I believe Tom has touched on
some of these elsewhere in the thread.

1) Maintainability
2) The eventual (lord it is about 5 years late) upgrade in place feature
3) This possibly should be in core not contrib.

*If* it should be in core, it needs to push to 8.4. It is that simple.
Else we need to open the tree and start reviewing all those patches
that got left behind before at feature freeze because of possible open
issues.

If it doesn't need to be in core, in certainly has zero need to be in
contrib and can push to pgFoundry.

 
 I think our two realistic options today are (1) leave the code where
 it is, or (2) remove it.  While Jan clearly failed to follow the
 agreed procedures, I'm not convinced the transgression was severe
 enough to justify (2).

Well I would like to step back and remove the Jan element. Jan has
really been taking a beating here. The reality is, A committer made a
mistake. It doesn't matter who it was.

The code needs to be removed. I just don't see a way around it,
unless we are willing to go back and review other items. This patch has
not been peer reviewed, it's benefit has not been discussed on
-hackers, and in general it hasn't been vetted.

Sincerely,

Joshua D. Drake
 


-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Joshua D. Drake
On Wed, 10 Oct 2007 18:33:03 +0300
Marko Kreen [EMAIL PROTECTED] wrote:

 Considering the core operations are now being in active use
 some 6-7 years, I really fail to see how there can be anything
 to tweak, unless you are speaking changing naming style.

Well that is the problem right there isn't it? As someone who has
financed, shipped, developed and tested an integrated Replication
solution for *years*, this statement is obvious naivety.

Your code, may be the most blessed, pristine and bug free code *in your
environment* but your environment is hardly the only environment out
there and things *always* come up. 

 
 IMHO the core operations are already as stable as PostgreSQL use
 of MVCC, as the module just exports backend internal state...
 Current set of functions is the minimal necessary to implement
 queue operations, there is nothing more to remove.

Having a hard time buying that. MVCC has the pleasure of being tested
everyday by hundreds of thousands of installations.


 We might want to add some helper functions for query generation,
 but that does not affect core operation.
 

But it does affect the inclusion argument.

 Another thing can can be done is more compact representation for
 txid_snapshot type, but that also won't affect core operation.


You are starting to bring up things in your own post that may need to
change before inclusion. This is *exactly* why the code should be
removed. It wasn't vetted on -hackers, and if it had been we may have
had a more complete piece of software.

 
 I am very happy for txid being in contrib, I'm not arguing against
 that, but the hint that txid module is something that just freshly
 popped out of thin air is irking me.

Certainly, I can understand this as you have had a long time to work
with, develop and mature the code. However it is just out of thin air.
It doesn't exist except for a very small part of the PostgreSQL world.

It may not be new to you, but it is certainly 100% new to many of the
long time contributors of this project. 


  I think our two realistic options today are (1) leave the code where
  it is, or (2) remove it.  While Jan clearly failed to follow the
  agreed procedures, I'm not convinced the transgression was severe
  enough to justify (2).
 
 Thanks for being understanding.
 

We all try to be :) but I do feel it needs to be removed, pgFoundry is
the perfect place for this.

Sincerely,

Joshua D. Drake

-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Joshua D. Drake
On Wed, 10 Oct 2007 13:01:54 +0200
Stefan Kaltenbrunner [EMAIL PROTECTED] wrote:

 yeah I agree that code like this should be either in core or
 somewhere else (either pgfoundry or even shipped as part of the
 replication solutions mentioned which is basically something slony
 did for ages with the xxid stuff). Just pushing it now into contrib
 results in people wanting to use one of those solution having to deal
 with 3 kinds of packages:
 
 1. postgresql
 2. postgresql-contrib
 3. skytools/slony/...
 
 instead of just two which does not strike me as much of an
 improvement.

I am not sure I am buying this argument. Skytools/slony is not
PostgreSQL. Should we also include Greg's recent replication software?
Or perhaps pgCluster?

I can think of at least a dozen modules that would be cool to have in
contrib... plpgsm or orafce?

Yes this is just a small bit but Marko has already made suggestions
in his own thread of items that should possibly change. 

This needs to be pulled out, put on pgFoundry or submitted for 8.4.

Sincerely,

Joshua D. Drake




-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Joshua D. Drake [EMAIL PROTECTED] writes:
 If it doesn't need to be in core, in certainly has zero need to be in
 contrib and can push to pgFoundry.

One advantage of having it in contrib is buildfarm testing, as indeed we
already found out ... although it's true that *keeping* it there now
that it passes probably won't teach us too much more.

But I think the argument that was being made was mostly that the Slony
and Skytools projects would find it easier to depend on a contrib module
than on something that has to be fetched separately from pgfoundry.
Now they could work around that by including copies of the pgfoundry
project in their own distributions, but then they have a collision
problem if someone tries to install both together.  (I have no idea how
likely that is, though; it might not be a big problem in practice?)

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Simon Riggs
On Wed, 2007-10-10 at 12:08 -0400, Bruce Momjian wrote:

 The results have nothing to do with whether the process was followed. 
 We do not ignore process violations just because the outcome was OK.
 
 And Jan did not come even close to following procedure.  He just asked
 core if they would object to such an addition post-feature freeze.  He
 did not show code to core so there is no sense the core approved the
 patch.  And Jan did not show the patch to hackers or discuss the patch
 application.
 
 I do not want to keep bashing Jan but Simon's statements require me to
 set the record straight.

Record straight; please consider my comments withdrawn.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Marko Kreen
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
 Magnus Hagander [EMAIL PROTECTED] writes:
  (Assuming it's technically sound - I still haven't checked the actual
  code, but I'm assuming it's Ok since Jan approved it)

 I hadn't looked at it either, but here are a few things that need
 review:

 * Why no binary I/O support for the new datatype?  We tend to expect
 that for all core types.

As I said, the current module is minimal, my goal was to
have code where there is nothing to remove.

But for a data type that targets core, yes binary i/o support
should be added.

 * Why is txid_current_snapshot() excluding subtransaction XIDs?  That
 might be all right for the current uses in Slony/Skytools, but it seems
 darn close to a bug for any other use.

In queue usage the transaction that stores snapshots does not do
any other work on its own, so it does not matter, main requirement
is that txid_current()/txid_current_snapshot() be symmetric -
whatever the txid_current() outputs, the txid_current_snapshot() measures.

But I agree, supporting subtransactions makes the API more
universal.  And it wouldn't break Slony/PgQ current usage.

 * Why is txid_current_snapshot() reading SerializableSnapshot rather
 than an actually current snap as its name suggests?  This isn't just
 misleading, this will fail completely when SerializableSnapshot
 goes away, as seems likely to happen in 8.4 (and no, we won't keep it
 just because txid might want it).

If you say so.  This code is from original xxid module and
has worked thus far. :)  If the requirement mentioned above
is not broken, the code needs to be brought in line with
current backend coding standards.

Will look into the problems and send patch tomorrow,
today has been too tiring already...

-- 
marko

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Hannu Krosing
Ühel kenal päeval, K, 2007-10-10 kell 12:18, kirjutas Tom Lane:
 Magnus Hagander [EMAIL PROTECTED] writes:
  (Assuming it's technically sound - I still haven't checked the actual
  code, but I'm assuming it's Ok since Jan approved it)
 
 I hadn't looked at it either, but here are a few things that need
 review:
 
 * Why no binary I/O support for the new datatype?  We tend to expect
 that for all core types.

should be easy to add, likely a copy-past from any other varlena type.

 * Why is txid_current_snapshot() excluding subtransaction XIDs?  That
 might be all right for the current uses in Slony/Skytools, but it seems
 darn close to a bug for any other use.

Just thinking aloud here :

I think that the reason has something to do with it being for stored
snapshots used by different transactions for determining info about
other committed transactions , and the stored snapshots and transaction
ids become visible from SQL level only after both are committed.

There may be cases where you want to use it from other places, say C
code for user-defined function dealing with visibility of other
transactions, but before adding in subtransactions and thus possibly
bloating the storage, we should first identify such case.

Most likely it is better to just use in-backend snapshots straight from
backend internals if you dont need to store them.

 * Why is txid_current_snapshot() reading SerializableSnapshot rather
 than an actually current snap as its name suggests? This isn't just
 misleading, this will fail completely when SerializableSnapshot
 goes away, as seems likely to happen in 8.4 (and no, we won't keep it
 just because txid might want it).

Why is SerializableSnapshot going away ? 

How will we do serialized isolation level in 8.4 then?


Hannu






---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Hannu Krosing
Ühel kenal päeval, K, 2007-10-10 kell 11:06, kirjutas Joshua D. Drake:
 On Wed, 10 Oct 2007 18:01:34 +0100
 Gregory Stark [EMAIL PROTECTED] wrote:
 
  Magnus Hagander [EMAIL PROTECTED] writes:
  
   On Wed, Oct 10, 2007 at 11:30:47AM -0400, Tom Lane wrote:
   Bruce Momjian [EMAIL PROTECTED] writes:
I also agree with this.  We have to pretend it isn't in /contrib
now, figure out where want it, then put it there (contrib,
pgfoundry, core).
  
  I just don't see the point in putting it in pgfoundry. It's already in
  pgfoundry as part of Skytools.
   The whole point of having such a
  datatype is to build common interface to abstract away the internals
  of the database. That makes the pgfoundry modules (Skytools and
  Slony) easier to maintain separately.
 
 I missed the part that it is part of Skytools already but as counter
 point, what makes sense at that point is for Skytools to remove it and
 make it it's own module.

Is'nt  this just what happened when it was moved to contrib ?

  That way Slony (which is not a pgfoundry
 project) or anyone else that wants to make use of it can.
 
  
  Putting it in core or contrib means that when we change the snapshot
  mechanics in 8.4 the same developer will be able to fix the module at
  the same time (and find out if his changes break it at the same
  time).
 
 Which is very cool, for *8.4* :)
 
 Joshua D. Drake
 
 


---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Joshua D. Drake
On Wed, 10 Oct 2007 18:01:34 +0100
Gregory Stark [EMAIL PROTECTED] wrote:

 Magnus Hagander [EMAIL PROTECTED] writes:
 
  On Wed, Oct 10, 2007 at 11:30:47AM -0400, Tom Lane wrote:
  Bruce Momjian [EMAIL PROTECTED] writes:
   I also agree with this.  We have to pretend it isn't in /contrib
   now, figure out where want it, then put it there (contrib,
   pgfoundry, core).
 
 I just don't see the point in putting it in pgfoundry. It's already in
 pgfoundry as part of Skytools.
  The whole point of having such a
 datatype is to build common interface to abstract away the internals
 of the database. That makes the pgfoundry modules (Skytools and
 Slony) easier to maintain separately.

I missed the part that it is part of Skytools already but as counter
point, what makes sense at that point is for Skytools to remove it and
make it it's own module. That way Slony (which is not a pgfoundry
project) or anyone else that wants to make use of it can.

 
 Putting it in core or contrib means that when we change the snapshot
 mechanics in 8.4 the same developer will be able to fix the module at
 the same time (and find out if his changes break it at the same
 time).

Which is very cool, for *8.4* :)

Joshua D. Drake


-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Marko Kreen
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
 Joshua D. Drake [EMAIL PROTECTED] writes:
  If it doesn't need to be in core, in certainly has zero need to be in
  contrib and can push to pgFoundry.

 One advantage of having it in contrib is buildfarm testing, as indeed we
 already found out ... although it's true that *keeping* it there now
 that it passes probably won't teach us too much more.

 But I think the argument that was being made was mostly that the Slony
 and Skytools projects would find it easier to depend on a contrib module
 than on something that has to be fetched separately from pgfoundry.
 Now they could work around that by including copies of the pgfoundry
 project in their own distributions, but then they have a collision
 problem if someone tries to install both together.  (I have no idea how
 likely that is, though; it might not be a big problem in practice?)

Well, if it is kicked from /contrib now, one way we could handle
it is by shipping the same module inside both skytools/slony.

That has obvious conflict problems.

Unfortunately the problem has very easy fix - each one keeps
using it's current module.  Very easy, no work required.

But that also scratches the common API possibility.

-- 
marko

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Hannu Krosing
Ühel kenal päeval, K, 2007-10-10 kell 18:23, kirjutas Magnus Hagander:
 On Wed, Oct 10, 2007 at 11:47:15AM -0400, Tom Lane wrote:
  Marko Kreen [EMAIL PROTECTED] writes:
   IMHO the core operations are already as stable as PostgreSQL use
   of MVCC, as the module just exports backend internal state...
  
  Well, it exports backend internal state that did not exist before 8.2
  (ie, XID epoch).  So it doesn't seem all that set in stone to me.
  
   Another thing can can be done is more compact representation for
   txid_snapshot type, but that also won't affect core operation.
  
  That's another thing that's likely to become very much harder to change
  once it's in core.  People keep threatening to produce a working
  in-place-upgrade process, and once that's reality the on-disk
  representation of core types is going to be hard to change.
 
 Well, if that is a concern, than it's an equally big concern to have it in
 contrib. If people start using it, they're not going to care about us
 saying hey, it was in contrib, why did you use it, when we earlier said
 in order do use our whiz-bang stuff, you must install from contrib. We'll
 have complaints that it's too hard to install, but we won't manage to
 escape from the responsibility to keep it working.

I guess Marko will be able to take that responsibility, as he has been
doing for pg_crypto for years, an recently also pl/python to some
degree.

tsearch lived in contrib for quite long, evolving a lot and going
through a big morphing when it moved to core. I can't see anything
nearly as big happening to txid/snapshot types.


Hannu



---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Joshua D. Drake [EMAIL PROTECTED] writes:
 Gregory Stark [EMAIL PROTECTED] wrote:
 Putting it in core or contrib means that when we change the snapshot
 mechanics in 8.4 the same developer will be able to fix the module at
 the same time (and find out if his changes break it at the same
 time).

 Which is very cool, for *8.4* :)

I think you missed one point: waiting for 8.4 is too late because of the
mechanics of the slony/skytools projects.  The reason this came up at
all is those projects' recognition that they had a narrow window to
standardize on a common bit of code.  Slony is breaking backward
compatibility for 8.3 in order to make use of the new backend
ENABLE/DISABLE REPLICA TRIGGER functionality.  They can fold in
dependence on an externally-supplied txid at the same time, but if they
miss doing so, they're hardly likely to break compatibility again
anytime in the near future.  So if there's no solution available for 8.3
then there's no point in doing anything at all.

This is not an argument why they cannot depend on a pgfoundry project
for 8.3 instead of a contrib module, but it is the reason why simply
waiting to propose the feature for 8.4 wasn't a viable alternative.

I had been thinking that the choice between pgfoundry and contrib
was technically neutral and only a matter of policy, but Greg's
argument does raise a valid technical point: if the code is in contrib
then it *will* track any redesign of the snapshot maintenance code,
whereas if it's in pgfoundry then it won't.  That could perhaps be
addressed by merging it into 8.4 before anyone does any snapshot fixing,
but our track record on causing such things to happen in a particular
sequence isn't great ...

regards, tom lane

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Hannu Krosing [EMAIL PROTECTED] writes:
 Ühel kenal päeval, K, 2007-10-10 kell 12:18, kirjutas Tom Lane:
 * Why is txid_current_snapshot() reading SerializableSnapshot rather
 than an actually current snap as its name suggests?

 Why is SerializableSnapshot going away ? 
 How will we do serialized isolation level in 8.4 then?

If we are in a serializable transaction, we'll keep that snap around
(though probably not stored exactly where it is now).  In a Read
Committed transaction we should discard snaps that are no longer going
to be used by any subsequent query; this will allow intratransaction
advancement of xmin with ensuing benefits for VACUUM etc.  (This has
been discussed repeatedly, though I'm too lazy to go searching the
archives at the moment.)

The proposed behavior of txid_current_snapshot would defeat any
possibility of such an optimization, because we'd have to keep around
the xact's oldest snapshot on the off chance that txid_current_snapshot
would be called later in the xact.

I think txid_current_snapshot should read ActiveSnapshot.  If the user
wants to get a beginning-of-xact rather than beginning-of-statement
snapshot from it, he should be required to call it in a serializable
transaction.

regards, tom lane

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Florian Pflug

Tom Lane wrote:

The proposed behavior of txid_current_snapshot would defeat any possibility
of such an optimization, because we'd have to keep around the xact's oldest
snapshot on the off chance that txid_current_snapshot would be called later
in the xact.

I think txid_current_snapshot should read ActiveSnapshot.  If the user wants
to get a beginning-of-xact rather than beginning-of-statement snapshot from
it, he should be required to call it in a serializable transaction.


Hm... does txid require that the snapshot it uses a valid in the sense that
its xmin follows OldestXmin? If not, we could keep the snapshot around for txid,
but still update our published xmin - which seems to be the main reason we care
about getting rid of old snapshots at all.

greetings, Florian Pflug


---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

   http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Joshua D. Drake
Hello,

Well this certainly turned into something bigger than I thought it ever
would. The questions that come into play with this whole thread are
larger than just, the process wasn't followed, what do we do?

We obviously don't want to make life difficult for our sibling projects
such as Slony and Skytools. On the other hand, we certainly want
processes followed because it leads to better overall quality control.
It also leads to fairness within community as a whole.

There are quite a few contributors that are upset that this whole
process went down the way that it did. I would say they are likely in
the majority versus the people that just want to leave it alone and
move on.

As I see it, we really only have a couple of choices:

Allow it as contrib which isn't really a good solution because, 
Tom and others have piped up that this may need to be in core. If it is
to go into core we have two choices, back off beta and
review some of those last patches that were kicked or... push it to 8.4.

  Why? Because people are already talking about changes to this
patch such as supporting binary I/O and helper functions that don't
exist in the current piece of code.

  That means it is not complete. Which means we might as well look at
Concurrent psql, Table function support, bitmap scan changes, and GIT
as well.

  Those patches were submitted long ago and tried to go through the
correct process and have been pushed to 8.4. Now, some of them may need
more work than is warranted (I can't actually speak to that) but
certainly a couple of them are worth a second look.

Another option, is to push the contrib module to pgfoundry. There is
zero loss here to PostgreSQL that I can see, in the current state of the
patch. It just means that the two projects supporting this patch
(Skytools and Slony) will have to cooperate. They have already proven
they can do this.

Of course this doesn't solve the should be in core issue but the should
be in core came about *after*.

Either way, I think everyone has had there say. Can we just make a
decision. Personally I think we should just push to pgfoundry and call
it good.

Sincerely,

Joshua D. Drake



-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Michael Glaesemann


On Oct 10, 2007, at 13:30 , Tom Lane wrote:


That could perhaps be
addressed by merging it into 8.4 before anyone does any snapshot  
fixing,

but our track record on causing such things to happen in a particular
sequence isn't great ...


Granted, everyone's focused on the 8.3 branch right now, but with the  
enthusiasm of those who want txid, I can't help but think there'd be  
a patch ready and waiting the day 8_3_STABLE is tagged. And there's  
no reason not to have something submitted to -patches right now  
(unless it's not ready)—there are patches in the patch queue that  
didn't make it in before feature freeze.


Michael Glaesemann
grzm seespotcode net



---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Marko Kreen [EMAIL PROTECTED] writes:
 On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
 * Why is txid_current_snapshot() excluding subtransaction XIDs?  That
 might be all right for the current uses in Slony/Skytools, but it seems
 darn close to a bug for any other use.
 ...
 But I agree, supporting subtransactions makes the API more
 universal.  And it wouldn't break Slony/PgQ current usage.

After looking at this more closely, I think txid_current_snapshot is
okay as is, but is_visible_txid is probably buggy: the latter should be
folding subtransaction IDs to top-transaction IDs, no?  If not, why not?
I hope the answer is no because otherwise the code will be at huge risk
from truncation of pg_subtrans, but it's not apparent why this behavior
is okay.

regards, tom lane

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Marko Kreen
On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
 Marko Kreen [EMAIL PROTECTED] writes:
  On 10/10/07, Tom Lane [EMAIL PROTECTED] wrote:
  * Why is txid_current_snapshot() excluding subtransaction XIDs?  That
  might be all right for the current uses in Slony/Skytools, but it seems
  darn close to a bug for any other use.
  ...
  But I agree, supporting subtransactions makes the API more
  universal.  And it wouldn't break Slony/PgQ current usage.

 After looking at this more closely, I think txid_current_snapshot is
 okay as is, but is_visible_txid is probably buggy: the latter should be
 folding subtransaction IDs to top-transaction IDs, no?  If not, why not?
 I hope the answer is no because otherwise the code will be at huge risk
 from truncation of pg_subtrans, but it's not apparent why this behavior
 is okay.

Could you describe bit more?  The is_visible_txid() works
on data returned by txid_current_snapshot()?  How can there
be any subtrans id's if txid_current_snapshot() wont return
them?

The basic idea is - only txid_current() and txid_current_snapshot()
communicate with backend, rest of functions work on data
returned by them.

-- 
marko

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Gregory Stark

Joshua D. Drake [EMAIL PROTECTED] writes:

 There are quite a few contributors that are upset that this whole
 process went down the way that it did. I would say they are likely in
 the majority versus the people that just want to leave it alone and
 move on.

   That means it is not complete. Which means we might as well look at
 Concurrent psql, Table function support, bitmap scan changes, and GIT
 as well.

That's just nonsense. We need to fix our other problems too and that means
getting substantive feedback to the authors of those patches so they can
complete the work. But that has no bearing whatsoever on the current
situation.

 Another option, is to push the contrib module to pgfoundry. There is
 zero loss here to PostgreSQL that I can see, in the current state of the
 patch. 

You keep saying this, do you have any justification for it?

I've explained why I think this code belongs in Postgres and not pgfoundry,
did you have any counter-argument?

And the complaints Tom brought up are mostly precisely the kind of interface
issues that actually argue well for it being in contrib. It serves its current
purpose well but future users might need binary i/o or subxid support and so
on. Until the interface is very stable being in contrib makes perfect sense.

-- 
  Gregory Stark
  EnterpriseDB  http://www.enterprisedb.com

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Joshua D. Drake
On Wed, 10 Oct 2007 21:02:30 +0100
Gregory Stark [EMAIL PROTECTED] wrote:

 
 Joshua D. Drake [EMAIL PROTECTED] writes:
 
  There are quite a few contributors that are upset that this whole
  process went down the way that it did. I would say they are likely
  in the majority versus the people that just want to leave it alone
  and move on.
 
That means it is not complete. Which means we might as well look
  at Concurrent psql, Table function support, bitmap scan changes,
  and GIT as well.
 
 That's just nonsense. We need to fix our other problems too and that
 means getting substantive feedback to the authors of those patches so
 they can complete the work. But that has no bearing whatsoever on the
 current situation.

You seem to be diverting my point. We can not provide preferential
treatment. Those patches are out there and have been out there for some
time. They followed the rules. Frankly, they deserve to be fully
reviewed and have the opportunity to be put in core *before* this
contrib patch.

Especially since this patch has already been marked as *not complete*.
There is already discussion happening on the patch and the changes that
need to be made.

 
  Another option, is to push the contrib module to pgfoundry. There is
  zero loss here to PostgreSQL that I can see, in the current state
  of the patch. 
 
 You keep saying this, do you have any justification for it?

I believe if you read my posts I have made plenty of justification. The
simplest of course being, process wasn't followed.

 
 I've explained why I think this code belongs in Postgres and not
 pgfoundry, did you have any counter-argument?

I believe I have mentioned that there is an argument for it to be in
PostgreSQL. 

 
 And the complaints Tom brought up are mostly precisely the kind of
 interface issues that actually argue well for it being in contrib.

Nothing that is in contrib can not also be maintained just as well with
pgFoundry. It just may take more proactiveness in the process.

 It
 serves its current purpose well but future users might need binary
 i/o or subxid support and so on. Until the interface is very stable
 being in contrib makes perfect sense.
 

I would state that until the interface is very stable pgfoundry also
makes perfect sense.

I am getting the impression that you think that I don't *want* this
module. I do, but I do not want it at the sacrifice of other modules
and code authors who did the job the right way.

I understand Tom's point about if we push to 8.4 that could cause
problems for Slony and Skytools. I certainly don't want to cause
problems for some very cool projects. I do however don't see those
problems existing if it was in pgFoundry.

Or are we saying that the only way to provide quality sofware to
PostgreSQL is if it is either in core or contrib? I do not believe you
are saying that.

Sincerely,

Joshua D. Drake






-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Bruce Momjian

Looking at the discussion, I think we should just keep it in /contrib. 
The code is tightly tied to our backend transaction system so there is
logic to have it in /contrib rather than pgfoundry.  I do think we
should just move it into core for 8.4 though.

---

Joshua D. Drake wrote:
-- Start of PGP signed section.
 On Wed, 10 Oct 2007 21:02:30 +0100
 Gregory Stark [EMAIL PROTECTED] wrote:
 
  
  Joshua D. Drake [EMAIL PROTECTED] writes:
  
   There are quite a few contributors that are upset that this whole
   process went down the way that it did. I would say they are likely
   in the majority versus the people that just want to leave it alone
   and move on.
  
 That means it is not complete. Which means we might as well look
   at Concurrent psql, Table function support, bitmap scan changes,
   and GIT as well.
  
  That's just nonsense. We need to fix our other problems too and that
  means getting substantive feedback to the authors of those patches so
  they can complete the work. But that has no bearing whatsoever on the
  current situation.
 
 You seem to be diverting my point. We can not provide preferential
 treatment. Those patches are out there and have been out there for some
 time. They followed the rules. Frankly, they deserve to be fully
 reviewed and have the opportunity to be put in core *before* this
 contrib patch.
 
 Especially since this patch has already been marked as *not complete*.
 There is already discussion happening on the patch and the changes that
 need to be made.
 
  
   Another option, is to push the contrib module to pgfoundry. There is
   zero loss here to PostgreSQL that I can see, in the current state
   of the patch. 
  
  You keep saying this, do you have any justification for it?
 
 I believe if you read my posts I have made plenty of justification. The
 simplest of course being, process wasn't followed.
 
  
  I've explained why I think this code belongs in Postgres and not
  pgfoundry, did you have any counter-argument?
 
 I believe I have mentioned that there is an argument for it to be in
 PostgreSQL. 
 
  
  And the complaints Tom brought up are mostly precisely the kind of
  interface issues that actually argue well for it being in contrib.
 
 Nothing that is in contrib can not also be maintained just as well with
 pgFoundry. It just may take more proactiveness in the process.
 
  It
  serves its current purpose well but future users might need binary
  i/o or subxid support and so on. Until the interface is very stable
  being in contrib makes perfect sense.
  
 
 I would state that until the interface is very stable pgfoundry also
 makes perfect sense.
 
 I am getting the impression that you think that I don't *want* this
 module. I do, but I do not want it at the sacrifice of other modules
 and code authors who did the job the right way.
 
 I understand Tom's point about if we push to 8.4 that could cause
 problems for Slony and Skytools. I certainly don't want to cause
 problems for some very cool projects. I do however don't see those
 problems existing if it was in pgFoundry.
 
 Or are we saying that the only way to provide quality sofware to
 PostgreSQL is if it is either in core or contrib? I do not believe you
 are saying that.
 
 Sincerely,
 
 Joshua D. Drake
 
 
 
 
 
 
 -- 
 
   === The PostgreSQL Company: Command Prompt, Inc. ===
 Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
 PostgreSQL solutions since 1997  http://www.commandprompt.com/
   UNIQUE NOT NULL
 Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
 PostgreSQL Replication: http://www.commandprompt.com/products/
 
-- End of PGP section, PGP failed!

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Joshua D. Drake
On Wed, 10 Oct 2007 17:10:17 -0400 (EDT)
Bruce Momjian [EMAIL PROTECTED] wrote:

 
 Looking at the discussion, I think we should just keep it
 in /contrib. The code is tightly tied to our backend transaction
 system so there is logic to have it in /contrib rather than
 pgfoundry.  I do think we should just move it into core for 8.4
 though.

In the interest of killing the problem. I agree. Everyone has had ample
opportunity for their opinions and frankly there isn't a good solution,
so let's take the one presented.

It's committed. It is in 8.3.

Sincerely,

Joshua D. Drake
-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Marko Kreen [EMAIL PROTECTED] writes:
 Could you describe bit more?  The is_visible_txid() works
 on data returned by txid_current_snapshot()?  How can there
 be any subtrans id's if txid_current_snapshot() wont return
 them?

Ah, I see: txid_current() never reports a subxact ID so there's no need to
consider them elsewhere in txids either.  OK, but this desperately needs
to be documented.

BTW, I notice that use of txid_current will force assignment of an XID
in a transaction that might not otherwise have one.  Does this matter,
or is the expectation that it's only going to be used in transactions
that are making DB modifications anyway?

regards, tom lane

---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tom Lane
Florian Pflug [EMAIL PROTECTED] writes:
 Tom Lane wrote:
 I think txid_current_snapshot should read ActiveSnapshot.  If the user wants
 to get a beginning-of-xact rather than beginning-of-statement snapshot from
 it, he should be required to call it in a serializable transaction.

 Hm... does txid require that the snapshot it uses a valid in the sense that
 its xmin follows OldestXmin? If not, we could keep the snapshot around for 
 txid,
 but still update our published xmin - which seems to be the main reason we 
 care
 about getting rid of old snapshots at all.

Why should we complicate the main code like that for txid?  I have not
heard any argument why the function should be examining
SerializableSnapshot instead of the current transaction snapshot.

regards, tom lane

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Hannu Krosing
Ühel kenal päeval, K, 2007-10-10 kell 10:10, kirjutas Joshua D. Drake:
 On Wed, 10 Oct 2007 18:33:03 +0300
 Marko Kreen [EMAIL PROTECTED] wrote:
 
  Considering the core operations are now being in active use
  some 6-7 years, I really fail to see how there can be anything
  to tweak, unless you are speaking changing naming style.
 
 Well that is the problem right there isn't it? As someone who has
 financed, shipped, developed and tested an integrated Replication
 solution for *years*, 

AH, now I see it , and I think I understand your concerns better ;)

 this statement is obvious naivety.

Then you should not feel threatened by including this in contrib 

 Your code, may be the most blessed, pristine and bug free code *in your
 environment* but your environment is hardly the only environment out
 there and things *always* come up. 

Meaning there will still be market for tried and tested commercially
supported wal-log based replication solutions. 

btw, can you publicly discuss how CommandPrompts  WAL-based replication
works ? 

Does it also also need something similar to snapshot type to do
replication effectively (or else you have to quess, which portion of
WAL is safe to commit and in what order) or does it somehow come out of
WAL log naturally ? 

And how do you map WAL records back to tables/indexes/sequences in slave
databases in an effective manner ?

  IMHO the core operations are already as stable as PostgreSQL use
  of MVCC, as the module just exports backend internal state...
  Current set of functions is the minimal necessary to implement
  queue operations, there is nothing more to remove.
 
 Having a hard time buying that. MVCC has the pleasure of being tested
 everyday by hundreds of thousands of installations.

that's what he is telling you - this code just exposes to userspace,
what has been tested everyday by hundreds of thousands of installations.

  We might want to add some helper functions for query generation,
  but that does not affect core operation.
  
 
 But it does affect the inclusion argument.

Probably makes it stronger, as we dont want different users to add
slightly different versions of these. And hopefully these will be
discussed on -hackers before final design is committed :)

  Another thing can can be done is more compact representation for
  txid_snapshot type, but that also won't affect core operation.
 
 
 You are starting to bring up things in your own post that may need to
 change before inclusion. This is *exactly* why the code should be
 removed. It wasn't vetted on -hackers, and if it had been we may have
 had a more complete piece of software.

None of these needs to change before inclusion, as they don't change
interfaces. they are either plans for future enchancemants in
implementation (you cant argue that nothing can get into contrib unless
all conceivable future enchancements are already done), or possible new
interfaces to be added in future if needed.

  I am very happy for txid being in contrib, I'm not arguing against
  that, but the hint that txid module is something that just freshly
  popped out of thin air is irking me.
 
 Certainly, I can understand this as you have had a long time to work
 with, develop and mature the code. However it is just out of thin air.
 It doesn't exist except for a very small part of the PostgreSQL world.
 
 It may not be new to you, but it is certainly 100% new to many of the
 long time contributors of this project. 
 
   I think our two realistic options today are (1) leave the code where
   it is, or (2) remove it.  While Jan clearly failed to follow the
   agreed procedures, I'm not convinced the transgression was severe
   enough to justify (2).
  
  Thanks for being understanding.
  
 
 We all try to be :) but I do feel it needs to be removed, pgFoundry is
 the perfect place for this.

We feel that it is may be a generally enabling technology, and hope that
if it is being included in postgreSQL baseline distribution it will
sprout new uses beyound high-performanse replication and queueing, as it
lowers the barriers for making new creative uses of postgreSQL's MVCC
mechanisms by people not very familiar with minute details of
postgreSQL's inner workings.


Hannu



---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Hannu Krosing
Ühel kenal päeval, K, 2007-10-10 kell 17:17, kirjutas Tom Lane:
 Marko Kreen [EMAIL PROTECTED] writes:
  Could you describe bit more?  The is_visible_txid() works
  on data returned by txid_current_snapshot()?  How can there
  be any subtrans id's if txid_current_snapshot() wont return
  them?
 
 Ah, I see: txid_current() never reports a subxact ID so there's no need to
 consider them elsewhere in txids either.  OK, but this desperately needs
 to be documented.
 
 BTW, I notice that use of txid_current will force assignment of an XID
 in a transaction that might not otherwise have one.  Does this matter,
 or is the expectation that it's only going to be used in transactions
 that are making DB modifications anyway?

yes, the common use in Skytools (and now in Slony) is in
insert/update/delete triggers to add current txid to log records.

   regards, tom lane
 
 ---(end of broadcast)---
 TIP 7: You can help support the PostgreSQL project by donating at
 
 http://www.postgresql.org/about/donate


---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Joshua D. Drake
On Thu, 11 Oct 2007 01:43:16 +0300
Hannu Krosing [EMAIL PROTECTED] wrote:

 AH, now I see it , and I think I understand your concerns better ;)
 
  this statement is obvious naivety.
 
 Then you should not feel threatened by including this in contrib 
 

Please do not mistake my concerns for having anything to do with
replicator. They don't. My concerns were purely about following correct
process within the community.

 
 btw, can you publicly discuss how CommandPrompts  WAL-based
 replication works ? 

It's my company, if course I am ;)... but not in this thread. If you
are interested feel free to email me directly or start a new thread.

Joshua D. Drake

 


-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Jan Wieck

On 10/10/2007 12:08 PM, Bruce Momjian wrote:
The results have nothing to do with whether the process was followed. 
We do not ignore process violations just because the outcome was OK.


Agreed. But reversing something that came out OK for no other reason 
than that the process was violated? I know you don't, but some people 
are asking for exactly that.



Jan

--
#==#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.  #
#== [EMAIL PROTECTED] #

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Florian Pflug

Tom Lane wrote:

Florian Pflug [EMAIL PROTECTED] writes:

Tom Lane wrote:
I think txid_current_snapshot should read ActiveSnapshot.  If the user 
wants to get a beginning-of-xact rather than beginning-of-statement 
snapshot from it, he should be required to call it in a serializable 
transaction.



Hm... does txid require that the snapshot it uses a valid in the sense that
 its xmin follows OldestXmin? If not, we could keep the snapshot around for
 txid, but still update our published xmin - which seems to be the main 
reason we care about getting rid of old snapshots at all.


Why should we complicate the main code like that for txid?  I have not heard 
any argument why the function should be examining SerializableSnapshot 
instead of the current transaction snapshot.


I wouldn't know. I just wanted to say that even if it needs to examine
SerializableSnapshot, that won't clash with the xmin optimizations planned for
8.4, as long as the snapshot won't be used for actual queries.

greetings, Florian Pflug


---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
  choose an index scan if your joining column's datatypes do not
  match


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Bruce Momjian
Tatsuo Ishii wrote:
 I don't undrestand why the txid stuff is in 8.3beta(this is an unsual
 case right?), but if we decide to keep it, please consider updating
 release.sgml. Bruce explained me that release.sgml will not be updated
 until the official release, but this is the unusual case and we need to
 break the rule, I think.

release.sgml is updated before every beta.

---


 --
 Tatsuo Ishii
 SRA OSS, Inc. Japan
 
  Looking at the discussion, I think we should just keep it in /contrib. 
  The code is tightly tied to our backend transaction system so there is
  logic to have it in /contrib rather than pgfoundry.  I do think we
  should just move it into core for 8.4 though.
  
  ---
  
  Joshua D. Drake wrote:
  -- Start of PGP signed section.
   On Wed, 10 Oct 2007 21:02:30 +0100
   Gregory Stark [EMAIL PROTECTED] wrote:
   

Joshua D. Drake [EMAIL PROTECTED] writes:

 There are quite a few contributors that are upset that this whole
 process went down the way that it did. I would say they are likely
 in the majority versus the people that just want to leave it alone
 and move on.

   That means it is not complete. Which means we might as well look
 at Concurrent psql, Table function support, bitmap scan changes,
 and GIT as well.

That's just nonsense. We need to fix our other problems too and that
means getting substantive feedback to the authors of those patches so
they can complete the work. But that has no bearing whatsoever on the
current situation.
   
   You seem to be diverting my point. We can not provide preferential
   treatment. Those patches are out there and have been out there for some
   time. They followed the rules. Frankly, they deserve to be fully
   reviewed and have the opportunity to be put in core *before* this
   contrib patch.
   
   Especially since this patch has already been marked as *not complete*.
   There is already discussion happening on the patch and the changes that
   need to be made.
   

 Another option, is to push the contrib module to pgfoundry. There is
 zero loss here to PostgreSQL that I can see, in the current state
 of the patch. 

You keep saying this, do you have any justification for it?
   
   I believe if you read my posts I have made plenty of justification. The
   simplest of course being, process wasn't followed.
   

I've explained why I think this code belongs in Postgres and not
pgfoundry, did you have any counter-argument?
   
   I believe I have mentioned that there is an argument for it to be in
   PostgreSQL. 
   

And the complaints Tom brought up are mostly precisely the kind of
interface issues that actually argue well for it being in contrib.
   
   Nothing that is in contrib can not also be maintained just as well with
   pgFoundry. It just may take more proactiveness in the process.
   
It
serves its current purpose well but future users might need binary
i/o or subxid support and so on. Until the interface is very stable
being in contrib makes perfect sense.

   
   I would state that until the interface is very stable pgfoundry also
   makes perfect sense.
   
   I am getting the impression that you think that I don't *want* this
   module. I do, but I do not want it at the sacrifice of other modules
   and code authors who did the job the right way.
   
   I understand Tom's point about if we push to 8.4 that could cause
   problems for Slony and Skytools. I certainly don't want to cause
   problems for some very cool projects. I do however don't see those
   problems existing if it was in pgFoundry.
   
   Or are we saying that the only way to provide quality sofware to
   PostgreSQL is if it is either in core or contrib? I do not believe you
   are saying that.
   
   Sincerely,
   
   Joshua D. Drake
   
   
   
   
   
   
   -- 
   
 === The PostgreSQL Company: Command Prompt, Inc. ===
   Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
   PostgreSQL solutions since 1997  http://www.commandprompt.com/
 UNIQUE NOT NULL
   Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
   PostgreSQL Replication: http://www.commandprompt.com/products/
   
  -- End of PGP section, PGP failed!
  
  -- 
Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
EnterpriseDB http://postgres.enterprisedb.com
  
+ If your life is a hard drive, Christ can be your backup. +
  
  ---(end of broadcast)---
  TIP 5: don't forget to increase your free space map settings

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a 

Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Tatsuo Ishii
I don't undrestand why the txid stuff is in 8.3beta(this is an unsual
case right?), but if we decide to keep it, please consider updating
release.sgml. Bruce explained me that release.sgml will not be updated
until the official release, but this is the unusual case and we need to
break the rule, I think.
--
Tatsuo Ishii
SRA OSS, Inc. Japan

 Looking at the discussion, I think we should just keep it in /contrib. 
 The code is tightly tied to our backend transaction system so there is
 logic to have it in /contrib rather than pgfoundry.  I do think we
 should just move it into core for 8.4 though.
 
 ---
 
 Joshua D. Drake wrote:
 -- Start of PGP signed section.
  On Wed, 10 Oct 2007 21:02:30 +0100
  Gregory Stark [EMAIL PROTECTED] wrote:
  
   
   Joshua D. Drake [EMAIL PROTECTED] writes:
   
There are quite a few contributors that are upset that this whole
process went down the way that it did. I would say they are likely
in the majority versus the people that just want to leave it alone
and move on.
   
  That means it is not complete. Which means we might as well look
at Concurrent psql, Table function support, bitmap scan changes,
and GIT as well.
   
   That's just nonsense. We need to fix our other problems too and that
   means getting substantive feedback to the authors of those patches so
   they can complete the work. But that has no bearing whatsoever on the
   current situation.
  
  You seem to be diverting my point. We can not provide preferential
  treatment. Those patches are out there and have been out there for some
  time. They followed the rules. Frankly, they deserve to be fully
  reviewed and have the opportunity to be put in core *before* this
  contrib patch.
  
  Especially since this patch has already been marked as *not complete*.
  There is already discussion happening on the patch and the changes that
  need to be made.
  
   
Another option, is to push the contrib module to pgfoundry. There is
zero loss here to PostgreSQL that I can see, in the current state
of the patch. 
   
   You keep saying this, do you have any justification for it?
  
  I believe if you read my posts I have made plenty of justification. The
  simplest of course being, process wasn't followed.
  
   
   I've explained why I think this code belongs in Postgres and not
   pgfoundry, did you have any counter-argument?
  
  I believe I have mentioned that there is an argument for it to be in
  PostgreSQL. 
  
   
   And the complaints Tom brought up are mostly precisely the kind of
   interface issues that actually argue well for it being in contrib.
  
  Nothing that is in contrib can not also be maintained just as well with
  pgFoundry. It just may take more proactiveness in the process.
  
   It
   serves its current purpose well but future users might need binary
   i/o or subxid support and so on. Until the interface is very stable
   being in contrib makes perfect sense.
   
  
  I would state that until the interface is very stable pgfoundry also
  makes perfect sense.
  
  I am getting the impression that you think that I don't *want* this
  module. I do, but I do not want it at the sacrifice of other modules
  and code authors who did the job the right way.
  
  I understand Tom's point about if we push to 8.4 that could cause
  problems for Slony and Skytools. I certainly don't want to cause
  problems for some very cool projects. I do however don't see those
  problems existing if it was in pgFoundry.
  
  Or are we saying that the only way to provide quality sofware to
  PostgreSQL is if it is either in core or contrib? I do not believe you
  are saying that.
  
  Sincerely,
  
  Joshua D. Drake
  
  
  
  
  
  
  -- 
  
=== The PostgreSQL Company: Command Prompt, Inc. ===
  Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
  PostgreSQL solutions since 1997  http://www.commandprompt.com/
  UNIQUE NOT NULL
  Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
  PostgreSQL Replication: http://www.commandprompt.com/products/
  
 -- End of PGP section, PGP failed!
 
 -- 
   Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
   EnterpriseDB http://postgres.enterprisedb.com
 
   + If your life is a hard drive, Christ can be your backup. +
 
 ---(end of broadcast)---
 TIP 5: don't forget to increase your free space map settings

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-10 Thread Magnus Hagander

  The results have nothing to do with whether the process was followed. 
  We do not ignore process violations just because the outcome was OK.
 
 Agreed. But reversing something that came out OK for no other reason 
 than that the process was violated? I know you don't, but some people 
 are asking for exactly that.

So as long as something is committed, and only breaks certain (for now unnamed) 
platforms (until fixed that is), then procedure doesn't apply.

And it helps if certain external projects ask for it. I'm not entirely clear on 
the criteria for those projects.

I don't particularly like that fact but I think it's good to have discussed it 
and spelled it out. And I will certainly accept it since it's been discussed in 
public and seems to have fair agreement between core members. 

The important thing is that we have documented a way around the rules so next 
time it's done noone needs to bother complaining.

/Magnus

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Bruce Momjian
Tom Lane wrote:
 Bruce Momjian [EMAIL PROTECTED] writes:
  I don't see how timing has anything to do with this.  You could have
  added it between beta1 and beta2 after sufficient hackers discussion. 
 
 Uh, it *was* after beta1.

Oh, so it didn't hold up beta1 --- that's good.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Andrew Dunstan



Bruce Momjian wrote:

Tom Lane wrote:
  

Bruce Momjian [EMAIL PROTECTED] writes:


I don't see how timing has anything to do with this.  You could have
added it between beta1 and beta2 after sufficient hackers discussion. 
  

Uh, it *was* after beta1.



Oh, so it didn't hold up beta1 --- that's good.

  


No it's not.

Can somebody please explain to me what beta means if you can commit new 
stuff after it has been declared?


cheers

andrew

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Magnus Hagander
Andrew Dunstan wrote:
 
 
 Bruce Momjian wrote:
 Tom Lane wrote:
  
 Bruce Momjian [EMAIL PROTECTED] writes:

 I don't see how timing has anything to do with this.  You could have
 added it between beta1 and beta2 after sufficient hackers
 discussion.   
 Uh, it *was* after beta1.
 

 Oh, so it didn't hold up beta1 --- that's good.

   
 
 No it's not.
 
 Can somebody please explain to me what beta means if you can commit new
 stuff after it has been declared?

Yeah, I'd like to know that as well. And specifically what kind of stuff
 it is that's ok...

If nothing else, that'll be good to know for packagers. Due to the
addition of this module we'll have to make code changes that aren't just
bugfixes to the win32 installer for example, which is also in feature
freeze...

//Magnus

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Bruce Momjian
Andrew Dunstan wrote:
 
 
 Bruce Momjian wrote:
  Tom Lane wrote:

  Bruce Momjian [EMAIL PROTECTED] writes:
  
  I don't see how timing has anything to do with this.  You could have
  added it between beta1 and beta2 after sufficient hackers discussion. 

  Uh, it *was* after beta1.
  
 
  Oh, so it didn't hold up beta1 --- that's good.
 

 
 No it's not.
 
 Can somebody please explain to me what beta means if you can commit new 
 stuff after it has been declared?

We allow /contrib to be more lax about beta changes.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Bruce Momjian
Jan Wieck wrote:
  I don't see how timing has anything to do with this.  You could have
  added it between beta1 and beta2 after sufficient hackers discussion. 
  Doing it the way you did with no warning, right before beta, and then
  leaving is the worse of all times.  I am surprised we are not backing
  out the patch and requiring that the patch go through the formal review
  process.
  
  This is not the first time you have had trouble with patches.  There was
  an issue with your patch of February, 2007:
  
  http://archives.postgresql.org/pgsql-hackers/2007-02/msg00385.php
 
 That email might contain the keyword COMMIT, but it doesn't have to do 
 with anything I committed to CVS. The trigger changes you are referring 
 to have been discussed and a patch for discussion was presented here:
 
  http://archives.postgresql.org/pgsql-hackers/2007-02/msg00146.php

Right, but at the time you didn't want to give a good explaination and I
had to ask for it.  That should not have been necessary.

  (In summary, you had to be coaxed to explain your patch to the
  community.)  Basically, I am not sure you understand the process that
  has to be followed, or feel you are somehow immune from following it.
 
 I don't see how you leap from the above example to that conclusion.

You have had only a few commits in 2007, and there have been two
problems.  That ratio seems too high to me, hence my questions above.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Andrew Dunstan



Bruce Momjian wrote:

Andrew Dunstan wrote:
  


Can somebody please explain to me what beta means if you can commit new 
stuff after it has been declared?



We allow /contrib to be more lax about beta changes.

  


I think we should be looking long and hard at that. I can't see any 
justification for it at all.



cheers

andrew

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Stefan Kaltenbrunner
Bruce Momjian wrote:
 Andrew Dunstan wrote:

 Bruce Momjian wrote:
 Tom Lane wrote:
   
 Bruce Momjian [EMAIL PROTECTED] writes:
 
 I don't see how timing has anything to do with this.  You could have
 added it between beta1 and beta2 after sufficient hackers discussion. 
   
 Uh, it *was* after beta1.
 
 Oh, so it didn't hold up beta1 --- that's good.

   
 No it's not.

 Can somebody please explain to me what beta means if you can commit new 
 stuff after it has been declared?
 
 We allow /contrib to be more lax about beta changes.

the postgresql ecosystem is growing and there is a lot of people like
packagers that will be a quite irritated if we keep randomly adding
completely new code and modules during BETA.


Stefan

---(end of broadcast)---
TIP 2: Don't 'kill -9' the postmaster


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Dave Page
Bruce Momjian wrote:
 Can somebody please explain to me what beta means if you can commit new 
 stuff after it has been declared?
 
 We allow /contrib to be more lax about beta changes.

Why? When people were complaining about not being able to use TSearch
because their ISPs wouldn't install contrib modules we couldn't
understand why they would think that way. If we are going to be less
stringent about /contrib, maybe they were right to cautious.

/D




---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Bruce Momjian
Dave Page wrote:
 Bruce Momjian wrote:
  Can somebody please explain to me what beta means if you can commit new 
  stuff after it has been declared?
  
  We allow /contrib to be more lax about beta changes.
 
 Why? When people were complaining about not being able to use TSearch
 because their ISPs wouldn't install contrib modules we couldn't
 understand why they would think that way. If we are going to be less
 stringent about /contrib, maybe they were right to cautious.

The idea is /contrib isn't installed by default and it isn't tied into
the core code, and can be tested easier because it is stand-alone.  We
can rethink that logic but that has been the guide in the past.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Stefan Kaltenbrunner
Magnus Hagander wrote:
 Andrew Dunstan wrote:

 Bruce Momjian wrote:
 Tom Lane wrote:
  
 Bruce Momjian [EMAIL PROTECTED] writes:

 I don't see how timing has anything to do with this.  You could have
 added it between beta1 and beta2 after sufficient hackers
 discussion.   
 Uh, it *was* after beta1.
 
 Oh, so it didn't hold up beta1 --- that's good.

   
 No it's not.

 Can somebody please explain to me what beta means if you can commit new
 stuff after it has been declared?
 
 Yeah, I'd like to know that as well. And specifically what kind of stuff
  it is that's ok...

I hate to say this - but this adding completely new steff after or
immediatly before beta business really scares the hell out of me and
somewhat starts to resemble the mysql way of adding new features at will
and even during stable release trains ...
There is no point in having any kind of feature freeze or even a
PatchStatus Page if we keep adding stuff (as useful as it might be) that
late in the cycle ...


Stefan

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Jan Wieck

On 10/9/2007 1:06 AM, Bruce Momjian wrote:

Jan Wieck wrote:

On 10/8/2007 1:34 PM, Tom Lane wrote:
 Magnus Hagander [EMAIL PROTECTED] writes:
 Marko Kreen wrote:
 Because of the bad timing it would have been -core call anyway
 whether it gets in or not so Jan asked -core directly.  That's
 my explanation about what happened, obviously Jan and Tom have
 their own opinion.
 
 Right. I can see your point, but it's my understanding that -hackers is

 really the ones supposed to decide on this.
 
 It would ultimately have been core's decision, but the discussion should

 have happened on -hackers.  There was no reason for it to be private.

That blame certainly belongs to me and I apologize for jumping that and 
adding it to contrib without any -hackers discussion.


It is definitely a timing issue since I write this very email from JFK, 
boarding a flight to Hong Kong in less than an hour and will be mostly 
offline for the rest of the week.


I don't see how timing has anything to do with this.  You could have
added it between beta1 and beta2 after sufficient hackers discussion. 
Doing it the way you did with no warning, right before beta, and then

leaving is the worse of all times.  I am surprised we are not backing
out the patch and requiring that the patch go through the formal review
process.

This is not the first time you have had trouble with patches.  There was
an issue with your patch of February, 2007:

http://archives.postgresql.org/pgsql-hackers/2007-02/msg00385.php


That email might contain the keyword COMMIT, but it doesn't have to do 
with anything I committed to CVS. The trigger changes you are referring 
to have been discussed and a patch for discussion was presented here:


http://archives.postgresql.org/pgsql-hackers/2007-02/msg00146.php



(In summary, you had to be coaxed to explain your patch to the
community.)  Basically, I am not sure you understand the process that
has to be followed, or feel you are somehow immune from following it.


I don't see how you leap from the above example to that conclusion.


Jan

--
#==#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.  #
#== [EMAIL PROTECTED] #

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Magnus Hagander
Bruce Momjian wrote:
 Dave Page wrote:
 Bruce Momjian wrote:
 Can somebody please explain to me what beta means if you can commit new 
 stuff after it has been declared?
 We allow /contrib to be more lax about beta changes.
 Why? When people were complaining about not being able to use TSearch
 because their ISPs wouldn't install contrib modules we couldn't
 understand why they would think that way. If we are going to be less
 stringent about /contrib, maybe they were right to cautious.
 
 The idea is /contrib isn't installed by default and it isn't tied into
 the core code, and can be tested easier because it is stand-alone.  We
 can rethink that logic but that has been the guide in the past.

I think you just outlined a whole lot of arguments for pgfoundry, and
not for contrib.

//Magnus


---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Dave Page
Bruce Momjian wrote:
 Dave Page wrote:
 Bruce Momjian wrote:
 Can somebody please explain to me what beta means if you can commit new 
 stuff after it has been declared?
 We allow /contrib to be more lax about beta changes.
 Why? When people were complaining about not being able to use TSearch
 because their ISPs wouldn't install contrib modules we couldn't
 understand why they would think that way. If we are going to be less
 stringent about /contrib, maybe they were right to cautious.
 
 The idea is /contrib isn't installed by default and it isn't tied into
 the core code, and can be tested easier because it is stand-alone.  We
 can rethink that logic but that has been the guide in the past.

Yes, I think we should if this is the result. It's one thing keeping
modules seperate for ease of testing, but it's another to become less
rigourous about how that code is maintained, developed and tested
compared to the core code.

/D


---(end of broadcast)---
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Jan Wieck

On 10/9/2007 4:22 PM, Bruce Momjian wrote:

Jan Wieck wrote:

 I don't see how timing has anything to do with this.  You could have
 added it between beta1 and beta2 after sufficient hackers discussion. 
 Doing it the way you did with no warning, right before beta, and then

 leaving is the worse of all times.  I am surprised we are not backing
 out the patch and requiring that the patch go through the formal review
 process.
 
 This is not the first time you have had trouble with patches.  There was

 an issue with your patch of February, 2007:
 
 	http://archives.postgresql.org/pgsql-hackers/2007-02/msg00385.php


That email might contain the keyword COMMIT, but it doesn't have to do 
with anything I committed to CVS. The trigger changes you are referring 
to have been discussed and a patch for discussion was presented here:


 http://archives.postgresql.org/pgsql-hackers/2007-02/msg00146.php


Right, but at the time you didn't want to give a good explaination and I
had to ask for it.  That should not have been necessary.


 (In summary, you had to be coaxed to explain your patch to the
 community.)  Basically, I am not sure you understand the process that
 has to be followed, or feel you are somehow immune from following it.

I don't see how you leap from the above example to that conclusion.


You have had only a few commits in 2007, and there have been two
problems.  That ratio seems too high to me, hence my questions above.


You are misrepresenting the situation. The discussion about the commit 
timestamp, where you asked for a complete functional specification of a 
multimaster replication system based on it before anything should be 
done feature wise at all, was not about any CVS activity that happened.



Jan

--
#==#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.  #
#== [EMAIL PROTECTED] #

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

  http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Bruce Momjian
Jan Wieck wrote:
 On 10/9/2007 4:22 PM, Bruce Momjian wrote:
  Jan Wieck wrote:
   I don't see how timing has anything to do with this.  You could have
   added it between beta1 and beta2 after sufficient hackers discussion. 
   Doing it the way you did with no warning, right before beta, and then
   leaving is the worse of all times.  I am surprised we are not backing
   out the patch and requiring that the patch go through the formal review
   process.
   
   This is not the first time you have had trouble with patches.  There was
   an issue with your patch of February, 2007:
   
http://archives.postgresql.org/pgsql-hackers/2007-02/msg00385.php
  You have had only a few commits in 2007, and there have been two
  problems.  That ratio seems too high to me, hence my questions above.
 
 You are misrepresenting the situation. The discussion about the commit 
 timestamp, where you asked for a complete functional specification of a 
 multimaster replication system based on it before anything should be 
 done feature wise at all, was not about any CVS activity that happened.

Here is a quote of exactly what I had to ask for, which I shouldn't have
had to ask for:

What I did want to hear is a layout of how the system would work,
and an exchange of ideas until almost everyone was happy.

Also, I saw the trigger patch with no explaination of why it was
important or who would use it --- that also isn't going to fly
well.

So, to add something, the community needs to hear how it is going to
help users, because every code addition has cost, and we don't want to
add things unless it has general utility.  If someone can't explain the
utility of an addition, I question whether the person has fully thought
through were they are going.

Not sure where you got the complete functional specification of a
multimaster replication system.

I go back to my original question, do you understand the process that
has to be followed for patch submission/application, and that it applies
to all of us, including you?   A simple yes is all I need to hear.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Jan Wieck

On 10/9/2007 5:13 PM, Bruce Momjian wrote:

Jan Wieck wrote:

On 10/9/2007 4:22 PM, Bruce Momjian wrote:
 Jan Wieck wrote:
  I don't see how timing has anything to do with this.  You could have
  added it between beta1 and beta2 after sufficient hackers discussion. 
  Doing it the way you did with no warning, right before beta, and then

  leaving is the worse of all times.  I am surprised we are not backing
  out the patch and requiring that the patch go through the formal review
  process.
  
  This is not the first time you have had trouble with patches.  There was

  an issue with your patch of February, 2007:
  
  	http://archives.postgresql.org/pgsql-hackers/2007-02/msg00385.php

 You have had only a few commits in 2007, and there have been two
 problems.  That ratio seems too high to me, hence my questions above.

You are misrepresenting the situation. The discussion about the commit 
timestamp, where you asked for a complete functional specification of a 
multimaster replication system based on it before anything should be 
done feature wise at all, was not about any CVS activity that happened.


Here is a quote of exactly what I had to ask for, which I shouldn't have
had to ask for:

What I did want to hear is a layout of how the system would work,
and an exchange of ideas until almost everyone was happy.

Also, I saw the trigger patch with no explaination of why it was
important or who would use it --- that also isn't going to fly
well.

So, to add something, the community needs to hear how it is going to
help users, because every code addition has cost, and we don't want to
add things unless it has general utility.  If someone can't explain the
utility of an addition, I question whether the person has fully thought
through were they are going.

Not sure where you got the complete functional specification of a
multimaster replication system.

I go back to my original question, do you understand the process that
has to be followed for patch submission/application, and that it applies
to all of us, including you?   A simple yes is all I need to hear.



Yes, Sir.


Jan

--
#==#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.  #
#== [EMAIL PROTECTED] #

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

  http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Simon Riggs
On Tue, 2007-10-09 at 17:13 -0400, Bruce Momjian wrote:

 I go back to my original question, do you understand the process that
 has to be followed for patch submission/application, and that it
 applies to all of us, including you?

I think you're braking a little hard here. Nothing bad has happened has
it? 

My understanding of committing stuff was about taking responsibility for
what's in there. Jan definitely has the knowledge and track record to be
trusted, plus we know he has time to fix any mistakes. That close to
release, only Core members should be doing that and Jan is Core.

Personally, I want to see Jan contribute more, not less. The link with
Slony and related replication technology is critically important to
Postgres, which is why Jan has spent so long on it. 

Generally we should be encouraging everybody to contribute; the project
must have an orientation towards action and growth if we are to survive.
If Jan had not done this, would there have been a storm of protest?

Anyway, its a good release and its out now.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
   subscribe-nomail command to [EMAIL PROTECTED] so that your
   message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Andrew Dunstan



Simon Riggs wrote:

That close to
release, only Core members should be doing that and Jan is Core.


  


My understanding (not being a member :-) ) is that Core is an 
administrative group, not a group of committers. Some members of Core 
are committers, some not, some committers are in Core, some not.


cheers

andrew

---(end of broadcast)---
TIP 1: if posting/reading through Usenet, please send an appropriate
  subscribe-nomail command to [EMAIL PROTECTED] so that your
  message can get through to the mailing list cleanly


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Simon Riggs
On Tue, 2007-10-09 at 17:55 -0400, Andrew Dunstan wrote:

 Simon Riggs wrote:
  That close to
  release, only Core members should be doing that and Jan is Core.
 

 My understanding (not being a member :-) ) is that Core is an 
 administrative group, not a group of committers. Some members of Core 
 are committers, some not, some committers are in Core, some not.

By observation, all committers are not equal. If they are equal then we
must censure others also, as well as Jan, but I see no need personally.
If they are not equal, then Jan deserves extra leeway, IMHO. Either way,
we should not focus just upon Jan, especially when so many will thank
him for his actions.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Joshua D. Drake
On Tue, 09 Oct 2007 17:55:48 -0400
Andrew Dunstan [EMAIL PROTECTED] wrote:

 
 
 Simon Riggs wrote:
  That close to
  release, only Core members should be doing that and Jan is Core.
 
 

 
 My understanding (not being a member :-) ) is that Core is an 
 administrative group, not a group of committers. Some members of Core 
 are committers, some not, some committers are in Core, some not.

That is my understanding as well and has been substantiated buy other
members of core.

Sincerely,

Joshua D. Drake


 
 cheers
 
 andrew
 
 ---(end of
 broadcast)--- TIP 1: if posting/reading
 through Usenet, please send an appropriate subscribe-nomail command
 to [EMAIL PROTECTED] so that your message can get through to
 the mailing list cleanly
 


-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Bruce Momjian
Andrew Dunstan wrote:
 
 
 Simon Riggs wrote:
  That close to
  release, only Core members should be doing that and Jan is Core.
 
 

 
 My understanding (not being a member :-) ) is that Core is an 
 administrative group, not a group of committers. Some members of Core 
 are committers, some not, some committers are in Core, some not.

Correct.

-- 
  Bruce Momjian  [EMAIL PROTECTED]http://momjian.us
  EnterpriseDB http://postgres.enterprisedb.com

  + If your life is a hard drive, Christ can be your backup. +

---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Magnus Hagander
Simon Riggs wrote:
 On Tue, 2007-10-09 at 17:55 -0400, Andrew Dunstan wrote:
 
 Simon Riggs wrote:
 That close to
 release, only Core members should be doing that and Jan is Core.

 
 My understanding (not being a member :-) ) is that Core is an 
 administrative group, not a group of committers. Some members of Core 
 are committers, some not, some committers are in Core, some not.
 
 By observation, all committers are not equal. If they are equal then we
 must censure others also, as well as Jan, but I see no need personally.
 If they are not equal, then Jan deserves extra leeway, IMHO. Either way,
 we should not focus just upon Jan, especially when so many will thank
 him for his actions.

Correct, we certainly shouldn't focus on any one person. We should focus
upon our own policies - were they broken or not. And if they weren't, it
seems that the majority of people want those policies changed (for next
time). If they were, what do we do about it?

//Magnus

---(end of broadcast)---
TIP 3: Have you checked our extensive FAQ?

   http://www.postgresql.org/docs/faq


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Simon Riggs
On Wed, 2007-10-10 at 00:19 +0200, Magnus Hagander wrote:
 Simon Riggs wrote:
  On Tue, 2007-10-09 at 17:55 -0400, Andrew Dunstan wrote:
  
  Simon Riggs wrote:
  That close to
  release, only Core members should be doing that and Jan is Core.
 
  
  My understanding (not being a member :-) ) is that Core is an 
  administrative group, not a group of committers. Some members of Core 
  are committers, some not, some committers are in Core, some not.
  
  By observation, all committers are not equal. If they are equal then we
  must censure others also, as well as Jan, but I see no need personally.
  If they are not equal, then Jan deserves extra leeway, IMHO. Either way,
  we should not focus just upon Jan, especially when so many will thank
  him for his actions.
 
 Correct, we certainly shouldn't focus on any one person. We should focus
 upon our own policies - were they broken or not. And if they weren't, it
 seems that the majority of people want those policies changed (for next
 time). If they were, what do we do about it?

Well for me: Nothing broke, so we needn't discuss policy at all.

-- 
  Simon Riggs
  2ndQuadrant  http://www.2ndQuadrant.com


---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Magnus Hagander
Simon Riggs wrote:
 On Wed, 2007-10-10 at 00:19 +0200, Magnus Hagander wrote:
 Simon Riggs wrote:
 On Tue, 2007-10-09 at 17:55 -0400, Andrew Dunstan wrote:

 Simon Riggs wrote:
 That close to
 release, only Core members should be doing that and Jan is Core.

 My understanding (not being a member :-) ) is that Core is an 
 administrative group, not a group of committers. Some members of Core 
 are committers, some not, some committers are in Core, some not.
 By observation, all committers are not equal. If they are equal then we
 must censure others also, as well as Jan, but I see no need personally.
 If they are not equal, then Jan deserves extra leeway, IMHO. Either way,
 we should not focus just upon Jan, especially when so many will thank
 him for his actions.
 Correct, we certainly shouldn't focus on any one person. We should focus
 upon our own policies - were they broken or not. And if they weren't, it
 seems that the majority of people want those policies changed (for next
 time). If they were, what do we do about it?
 
 Well for me: Nothing broke, so we needn't discuss policy at all.

Well, buildfarm broke. But it's been fixed now.

Also, does that mean you volunteer to fix pginstaller, and any other
packages that needs it? ;-)

//Magnus

---(end of broadcast)---
TIP 9: In versions below 8.0, the planner will ignore your desire to
   choose an index scan if your joining column's datatypes do not
   match


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Andrew Dunstan



Simon Riggs wrote:

On Tue, 2007-10-09 at 17:55 -0400, Andrew Dunstan wrote:

  

Simon Riggs wrote:


That close to
release, only Core members should be doing that and Jan is Core.

  


  
My understanding (not being a member :-) ) is that Core is an 
administrative group, not a group of committers. Some members of Core 
are committers, some not, some committers are in Core, some not.



By observation, all committers are not equal. If they are equal then we
must censure others also, as well as Jan, but I see no need personally.
If they are not equal, then Jan deserves extra leeway, IMHO. Either way,
we should not focus just upon Jan, especially when so many will thank
him for his actions.

  


I think this project has got too big for us to make things up as we go 
along. We need to follow processes that are well understood and 
transparent.


I have tried not to concentrate on Jan in this discussion. It could be 
argued, however, that contrary to being the recipients of extra slack, 
core members have a special responsibility to set a good example.


cheers

andrew

---(end of broadcast)---
TIP 6: explain analyze is your friend


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread David Fetter
On Tue, Oct 09, 2007 at 01:06:11AM -0400, Bruce Momjian wrote:
 Jan Wieck wrote:
  On 10/8/2007 1:34 PM, Tom Lane wrote:
   Magnus Hagander [EMAIL PROTECTED] writes:
   Marko Kreen wrote:
   Because of the bad timing it would have been -core call anyway
   whether it gets in or not so Jan asked -core directly.  That's
   my explanation about what happened, obviously Jan and Tom have
   their own opinion.
   
   Right. I can see your point, but it's my understanding that
   -hackers is really the ones supposed to decide on this.
   
   It would ultimately have been core's decision, but the
   discussion should have happened on -hackers.  There was no
   reason for it to be private.
  
  That blame certainly belongs to me and I apologize for jumping
  that and adding it to contrib without any -hackers discussion.
  
  It is definitely a timing issue since I write this very email from
  JFK, boarding a flight to Hong Kong in less than an hour and will
  be mostly offline for the rest of the week.
 
 I don't see how timing has anything to do with this.  You could have
 added it between beta1 and beta2 after sufficient hackers
 discussion.  Doing it the way you did with no warning, right before
 beta, and then leaving is the worse of all times.  I am surprised we
 are not backing out the patch and requiring that the patch go
 through the formal review process.

The feature is great, and it should stay on pgfoundry or other
suitable place until the 8.4 development cycle.  I believe it would
make a great addition to the core product, but only in 8.4.

Cheers,
David.
-- 
David Fetter [EMAIL PROTECTED] http://fetter.org/
phone: +1 415 235 3778AIM: dfetter666
  Skype: davidfetter

Remember to vote!
Consider donating to PostgreSQL: http://www.postgresql.org/about/donate

---(end of broadcast)---
TIP 4: Have you searched our list archives?

   http://archives.postgresql.org


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Michael Glaesemann


On Oct 9, 2007, at 0:06 , Bruce Momjian wrote:


I am surprised we are not backing
out the patch and requiring that the patch go through the formal  
review

process.


I have no opinion as to the patch itself (other than the fact that  
it's a not bug fix), but I think this patch should be reverted  
because it's (a) after feature freeze, (b) had no discussion on  
hackers (or patches), (c) is not a bug fix. IMO rules can be bent but  
there should always at least be discussion before a new feature is  
committed after feature freeze and definitely after beta. Otherwise,  
the rule appears to be if you can get it in somehow, it's in.


Again, I have no opinion regarding the patch itself, and these issues  
are regardless of who commits or submits. Personally, I regard Jan as  
a helpful guy and a solid coder who has contributed a lot to  
PostgreSQL in the past and I'm sure will contribute even more in the  
future.


Michael Glaesemann
grzm seespotcode net



---(end of broadcast)---
TIP 5: don't forget to increase your free space map settings


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Joshua D. Drake
On Tue, 9 Oct 2007 18:35:52 -0500
Michael Glaesemann [EMAIL PROTECTED] wrote:

 
 On Oct 9, 2007, at 0:06 , Bruce Momjian wrote:
 
  I am surprised we are not backing
  out the patch and requiring that the patch go through the formal  
  review
  process.
 
 I have no opinion as to the patch itself (other than the fact that  
 it's a not bug fix), but I think this patch should be reverted  
 because it's (a) after feature freeze, (b) had no discussion on  
 hackers (or patches), (c) is not a bug fix. IMO rules can be bent
 but there should always at least be discussion before a new feature
 is committed after feature freeze and definitely after beta.
 Otherwise, the rule appears to be if you can get it in somehow, it's
 in.

I think this almost says it all. My particular gripe about this whole
thing is that there are other features that are not too intrusive (or
appear so anyway) that are easily more useful that are not being 
considered at all. Namely,
http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php . It
makes the whole process seem tilted and subjective.

IMO, the patch is reverted, and submitted for 8.4 or pgfoundry.

Sincerely,

Joshua D. Drake


-- 

  === The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564   24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997  http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/



signature.asc
Description: PGP signature


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Devrim GÜNDÜZ
Hi,

On Tue, 2007-10-09 at 16:50 -0700, Joshua D. Drake wrote:
 IMO, the patch is reverted, and submitted for 8.4 or pgfoundry.

That means another delay for improving PostgreSQL replication.

I think we are all pretty sure that Jan knows what he is doing -- he has
involved in replication issues for years, and as a long-time core
member, I'm sure that he knows the rules -- but I think this patch will
speed up works on replication.

(Will all respect to pginstaller team, I'm *think* it won't take much
time to add txid to installer, at least compared to the time that we
spent discussing this issue.)

You know, txid was discussed in Slony-I + Skytools lists for a
reasonably long time, and Tom also commented in that thread. I agree
that we broke the policy this time, but this does not mean the end of
the world.

What we should to do is to prevent such things happening in the future,
rather than reverting this patch and delaying replication issues.

Regards,
-- 
Devrim GÜNDÜZ
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/




signature.asc
Description: This is a digitally signed message part


Re: [HACKERS] Skytools committed without hackers discussion/review

2007-10-09 Thread Neil Conway
On Tue, 2007-10-09 at 16:50 -0700, Joshua D. Drake wrote:
 I think this almost says it all. My particular gripe about this whole
 thing is that there are other features that are not too intrusive (or
 appear so anyway) that are easily more useful that are not being 
 considered at all. Namely,
 http://archives.postgresql.org/pgsql-patches/2007-10/msg00087.php .

That is NOT a good example. That patch is a first-cut of a non-trivial
optimizer feature that was submitted just before beta1 shipped, by
someone who hasn't modified the optimizer before. Jan's patch was a
contrib module that has been already developed by the Skype folks, and
it goes without saying that Jan has contributed to Postgres extensively.

 It makes the whole process seem tilted and subjective.

There are some fairly obvious, objective reasons why txid differs from
the inline-SQL-SRF patch.

That said, I agree that the process should have been followed in this
case.

-Neil



---(end of broadcast)---
TIP 6: explain analyze is your friend


  1   2   >