Re: [HACKERS] Index-only scans

2010-01-09 Thread Gokulakannan Somasundaram
Hi Heikki,
   I was recollecting our conversation that, if the index-only quals
were implemented through indexes by storing snapshots, broken data-types may
not be supported. I feel this problem might exist, if we go on design a
IOT(Index Organized Tables) / Clustered Indexes. IOT is again a index which
stores snapshot info together with it.
   So are we not going to have this feature ever in PG? What is our
stand in that case? I am asking this because i have an IOT implementation on
PG 8.2.

Thanks,
Gokul

On Tue, Jul 14, 2009 at 2:08 PM, Heikki Linnakangas 
heikki.linnakan...@enterprisedb.com wrote:

 Simon Riggs wrote:
  On Mon, 2009-07-13 at 10:16 +0300, Heikki Linnakangas wrote:
 
  Implementing index-only scans requires a few changes:
 
  I would like to see a clear exposition of the use cases and an an
  analysis of the costs and benefits of doing this. It sounds cool, but I
  want to know it is cool before we spend time solving all of the juicy
  problems.

 BTW, there's another trick that I'm *not* going to implement yet, which
 is to allow joins using data from indexes only, and fetching the rest of
 the columns after the join. For example:

 CREATE TABLE a (aid integer PRIMARY KEY, adata text);
 CREATE TABLE b (bid integer PRIMARY KEY, bdata text);

 SELECT aid, adata, bid, bdata FROM a, b WHERE aid = bid;

 If the join is very selective, IOW there's only a few matching rows, it
 is a lot more efficient to perform the join first using just the
 indexes, and fetch adata and bdata columns and check visibility for the
 matching rows only.

 You can get pretty close by clustering the tables, but the wider the
 tables the bigger the difference.

 --
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

 --
 Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
 To make changes to your subscription:
 http://www.postgresql.org/mailpref/pgsql-hackers



Re: [HACKERS] Serializable Isolation without blocking

2010-01-09 Thread Markus Wanner
Hi,

Kevin Grittner wrote:
 Nicolas Barbier nicolas.barb...@gmail.com wrote:
 AFAICS, detecting a rw dependency where the read executes after
 the write is rather easy: the writer has created a row version
 that is not visible to the reader's snapshot. Therefore, any time
 a reader reads a non-last version of a row, there is a rw
 dependency between it and the creators of any newer versions.

Sure.

As you have written below, we still need the SIREAD lock on the tuple
read to be able to detect rw dependencies (where the read executes
before the write).

I was trying to point out the difference between rw dependencies on
existing tuples (where the write is an update/delete) vs those on newly
created tuples (inserts) that *would* have been read. The later clearly
needs predicate locking. For the former, basing on table- as well as
row-level locking have been discussed so far.

What I'm thinking about is if it would make sense to use the very same
locking infrastructure for both, which needs to be some kind of
predicate locking. Maybe that was intended by Kevin and already clear to
others. It wasn't for me.

As we don't yet have predicate locking or tagging, let's check what SSI
needs from that building block. AFAICT we currently have the following
(long term) requirements:

A) reference rows that don't exist, yet
B) keep the lock (tag) until after the session that created it has
expired (yes, session, not only transaction).
C) fit in (shared) memory, no matter how many rows get tagged (thus
maybe use dynamical adjustment of granularity)

AFAICT using the existing locking structure will get complicated mainly
because of B), because we sometimes lack a PGPROC for the transaction
holding the lock. It's specific to SSI (not generally usable for
predicate locking).

A) and C) might be usable for a general purpose predicate locking
mechanism as well, but those requirements make it hard to map the object
to be locked to a LOCKTAG.

Unlike a general purpose predicate locking implementation, we don't need
to block on SIREAD locks, so we don't need waiting queues nor deadlock
detection for SSI.

 Basically, I have to confirm that
 the read will see *all* new versions of a row without jumping out
 early on any code path.

Well, a heap scan immediately returns the currently visible version of a
row to the caller, for example. That one may or may not continue the
scan. So I fear yes, current code paths jump out early very often (as
that's an optimization for the LIMIT case as well).

 I assume here that PG's non-SI-compatible behavior of not always
 rollbacking any transaction that writes to a non-last version will
 be disabled in serializable mode.
  
 Can that currently happen in PostgreSQL's snapshot isolation?!?  I
 thought that was a READ COMMITTED issue.  If I've missed something
 there, I need to understand what.  Anybody?

A write to a non-last (or non-head) version would lead to having
multiple last (or rather multiple head) versions, which is not
something that's ever supposed to happen in Postgres, IIRC.

Regards

Markus Wanner


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function

2010-01-09 Thread Dean Rasheed
2009/12/4 Dean Rasheed dean.a.rash...@googlemail.com:
 With CVS HEAD...

 create table foo (a int);

 create or replace function foo_trig_fn() returns trigger as $$
 begin
  raise notice 'In trigger: added %', new.ctid;
  return new;
 end
 $$ language plpgsql;

 create trigger foo_trig after insert on foo
  for each row execute procedure foo_trig_fn();

 insert into foo values(1);

 ERROR:  attribute number -1 exceeds number of columns 1


I started thinking about this again, and it does indeed seem to be the commit
http://archives.postgresql.org/pgsql-committers/2009-11/msg00035.php which
causes this. Specifically, the change

  * Avoid unnecessary scanner-driven lookups of plpgsql variables in
  places where it's not needed, which is actually most of the time;
  we do not need it in DECLARE sections nor in text that is a SQL
  query or expression.

So read_sql_construct() now disables plpgsql variable lookups in
plpgsql_parse_dblword(), and old.foo/new.foo are compiled into FieldSelect
nodes, where they used to be record field param nodes, which is a problem for
ExecEvalFieldSelect() if foo is a system attribute.

How much do you really save by avoiding the plpgsql variable lookups in this
case? Is this just trading compilation time for execution time? In theory the
new code will be slower to execute because ExecEvalFieldSelect() goes through
ExecEvalParam() to get (a copy of) the whole record in order to extract the
required field, whereas the old code just calls ExecEvalParam() with dtype of
PLPGSQL_DTYPE_RECFIELD to retrieve the field directly. So perhaps
plpgsql_parse_dblword() should always just do the variable lookups.

Thoughts?

Regards,
Dean

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Serializable Isolation without blocking

2010-01-09 Thread Nicolas Barbier
2010/1/8 Kevin Grittner kevin.gritt...@wicourts.gov:

 Nicolas Barbier nicolas.barb...@gmail.com wrote:

 I assume here that PG's non-SI-compatible behavior of not always
 rollbacking any transaction that writes to a non-last version will
 be disabled in serializable mode.

 Can that currently happen in PostgreSQL's snapshot isolation?!?  I
 thought that was a READ COMMITTED issue.  If I've missed something
 there, I need to understand what.  Anybody?

Ah woops, you are right. The manual says in [1]:

 8 
13.2.2. Serializable Isolation Level

[..]

UPDATE, DELETE, SELECT FOR UPDATE, and SELECT FOR SHARE commands
behave the same as SELECT in terms of searching for target rows: they
will only find target rows that were committed as of the transaction
start time. However, such a target row might have already been updated
(or deleted or locked) by another concurrent transaction by the time
it is found. In this case, the serializable transaction will wait for
the first updating transaction to commit or roll back (if it is still
in progress). If the first updater rolls back, then its effects are
negated and the serializable transaction can proceed with updating the
originally found row. But if the first updater commits (and actually
updated or deleted the row, not just locked it) then the serializable
transaction will be rolled back with the message

ERROR:  could not serialize access due to concurrent update
because a serializable transaction cannot modify or lock rows changed
by other transactions after the serializable transaction began.
 8 

Sorry for the noise,

Nicolas

[1] 
URL:http://www.postgresql.org/docs/8.4/static/transaction-iso.html#XACT-SERIALIZABLE

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Magnus Hagander
On Sat, Jan 9, 2010 at 05:54, Alvaro Herrera alvhe...@commandprompt.com wrote:
 Tom Lane escribió:
 David Fetter da...@fetter.org writes:
  On Fri, Jan 08, 2010 at 10:35:24PM -0500, Tom Lane wrote:
  Probably eventually we'll be on git and this will be moot, but that
  doesn't seem to be ready to happen.

  What still needs to happen on this?  Clearly this would be a post-8.5
  (or whatever the new release number is) thing, but apart from that?

 AFAIR, we still weren't convinced that we had a 100% conversion method
 (ie something that would preserve all the history) and there were still
 questions about how to work with multi-branch patches most effectively.
 I don't recall where the previous discussion died off exactly, but
 it definitely wasn't at the we're ready to do it stage.

 Somebody did a pull of all the tags, and some of them were missing files
 and failed to build.

That was from the current git mirror.

To re-itarate yet again, what I believe has been said many times before:

There are two ways to get from cvs to git.

The first one is reliable (at least from what I've heard). But it only
supports one-off migrations. It doesn't support incremental changes.
It was confused by some things that were plain broken in our cvs
repository way back (this happens with cvs, as we all know), but AFAIK
they have been fixed.

The second one supports incremental changes. And has issues with
back-branches. This is the one we are using.


If/when we are moving the main repository, we should use the first
one. Yes, this will invalidate all current git clones out there, but
that's a one-time cost. Will there be issues? Possibly. But we're
*never* going to get something that's *guaranteed* 100% safe, not when
going from something like CVS...

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes:
  I have always
 felt that the purpose of a CommitFest was to give everyone a fair
 shake at getting their patch reviewed, provided that they followed
 certain ground rules.  

Yes, like for example submitting the patch before the commit fest
begins.

 And I thought we had agreement that one of
 those ground rules was don't submit new, large patches to the final
 CommitFest in a particular release cycle.  No?

I don't remember this having been agreed upon. What I think have been
said before is that doing so would not help stabilizing the tree before
release.

You seem to be wanting to put a lot of energy into being successful at
following the current release schedule, which others seem to be seeing
as an hint or a wish more than anything else (it's the expected one, not
the one we're committed to, I'd venture).

Is it more important to follow the calendar or to be unable to know with
a month precision when we're going to release the best possible 8.5?
Again, it's a compromise to find. You're pushing towards the calendar,
we're advocating staying fair (opened?) with contributors even when it
means we're taking risks on the schedule.

Regards,
-- 
dim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Robert Haas
On Sat, Jan 9, 2010 at 8:07 AM, Dimitri Fontaine dfonta...@hi-media.com wrote:
 Robert Haas robertmh...@gmail.com writes:
  I have always
 felt that the purpose of a CommitFest was to give everyone a fair
 shake at getting their patch reviewed, provided that they followed
 certain ground rules.

 Yes, like for example submitting the patch before the commit fest
 begins.

Right.

 And I thought we had agreement that one of
 those ground rules was don't submit new, large patches to the final
 CommitFest in a particular release cycle.  No?

 I don't remember this having been agreed upon.

As far as I know we have always had this rule.  Here is Tom talking
about having it for 8.4 and trying to formalize it for 8.5.

http://archives.postgresql.org/pgsql-hackers/2009-06/msg01520.php

Here is Josh discussing the details of how the rule should be
implemented, while agreeing that it exists:

http://archives.postgresql.org/pgsql-hackers/2009-09/msg00141.php

There are also various messages in the archives where various patch
authors discuss development plans they have made to avoid running
afoul of this rule.

Basically, here's my feeling.  Either we have a rule that we can
bounce large, previously-unseen patches from the final CommitFest of
the release cycle, or we don't.  If we do, then we should go ahead and
do it, and we should do it early when it will have more effect rather
than putting a lot of time into those patches and doing it only once
the release is already late.  On the other hand, if we don't, then we
should have to core team publish a clear statement that all
CommitFests are equal and we're just going to slip the schedule if
there are too many patches for the last one.

Right now, what we have is some patch authors (like Jeff Davis)
holding off from submitting big new features to the last CommitFest,
essentially voluntarily, and others (like KaiGai Kohei, and I think
perhaps also Simon Riggs) making Herculean attempts to meet certain
deadlines even when they seemed somewhat artificial.  Then on the flip
side we have others like Teodor and Oleg who submitted large patches
at the end of the development cycle.  Of course, Teodor and Oleg are
free to do their development whenever they like, but why should we
review their work and not Jeff's?  It seems to me that having a rule
and not enforcing it is the worst of all possible worlds: it
essentially punishes people who have voluntarily followed it.

From a practical standpoint, it seems much better to me to have the
rule than not, because committing large patches earlier in the cycle
seems better from the point of view of having them well-tested and
stabilized before the release comes out.  But if the consensus is that
we have no such rule, then let's be honest about it.

 What I think have been
 said before is that doing so would not help stabilizing the tree before
 release.

Sorry, I'm not following this sentence.

 You seem to be wanting to put a lot of energy into being successful at
 following the current release schedule, which others seem to be seeing
 as an hint or a wish more than anything else (it's the expected one, not
 the one we're committed to, I'd venture).

 Is it more important to follow the calendar or to be unable to know with
 a month precision when we're going to release the best possible 8.5?
 Again, it's a compromise to find. You're pushing towards the calendar,
 we're advocating staying fair (opened?) with contributors even when it
 means we're taking risks on the schedule.

I am definitely pushing for the schedule.  It's a maxim of software
development that you can have time-based releases or feature-based
releases, but not both.  In this community, we have time-based
releases early in the cycle and then they change to feature-based
releases late in the cycle.  As we found out with 8.4, trying to be
both on time and feature-complete can result in failing at both.  I
feel that we've been eminently fair to contributors in the 8.5 cycle -
it's something that I have personally worked very hard on - and I
actually also feel that what I am proposing now is also fair.  It may
not be very popular, though.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Tidy up and refactor plperl.c.

2010-01-09 Thread Andrew Dunstan


This has broken MSVC builds - I am working on it.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Peter Eisentraut
On fre, 2010-01-08 at 21:01 -0500, Robert Haas wrote:
  The commitfest is a tool for people to track what is going on, not a
  tool to tell people what to do.
 
 I don't understand what you mean by this.  Can you please elaborate?

The proposal was apparently that a small, vocal group gets to decide
what features are more important than others, and then change the commit
fest listing to make everyone work on those features instead of some
other ones.  My opinion is that the commit fest should list everything
that was submitted and that participants can decide on their own what is
more important to them.

 And I thought we had agreement that one of
 those ground rules was don't submit new, large patches to the final
 CommitFest in a particular release cycle.  No?

I don't agree with that or the way it was assessed in this case.

The reason why I make these points is that if you make the commit fest
too selective, then, since you are not the employer of anyone, people
who don't agree with your choices will just ignore them and do something
else.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Peter Eisentraut
On fre, 2010-01-08 at 12:04 -0300, Alvaro Herrera wrote:
 Do .gitignore files have the same format as .cvsignore?  If that's the
 case then it's simply a matter of a find /source -name .cvsignore
 -exec
 cp {} .gitignore \; or similar, isn't it?  Doesn't sound like
 something
 anybody should sweat over. 

The format is the same, but while cvsignore files currently list a few
dozen files, the proposed gitignore would list all files that are ever
build anywhere.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Peter Eisentraut
On fre, 2010-01-08 at 20:03 -0800, David Fetter wrote:
 On Fri, Jan 08, 2010 at 10:35:24PM -0500, Tom Lane wrote:
  Probably eventually we'll be on git and this will be moot, but that
  doesn't seem to be ready to happen.
 
 What still needs to happen on this?  Clearly this would be a post-8.5
 (or whatever the new release number is) thing, but apart from that?

Probably someone to actually track the open items that are mentioned
every time this discussion happens.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Also update ChangerLog file.

2010-01-09 Thread Michael Meskes
On Fri, Jan 08, 2010 at 03:41:10PM +0100, Stefan Kaltenbrunner wrote:
 Hmm not sure I find that commit message really helpful - but is it
 actually of any use to maintain a Changelog file specifically for
 ECPG?

Makes it easier to see what's new in ecpg when doing a new release. But I guess
we could get all this from the cvs log too. The file has been there for a long 
time.

Michael
-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: mes...@jabber.org
VfL Borussia! Forca Barca! Go SF 49ers! Use: Debian GNU/Linux, PostgreSQL

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Dimitri Fontaine
Robert Haas robertmh...@gmail.com writes:
 Basically, here's my feeling.  Either we have a rule that we can
 bounce large, previously-unseen patches from the final CommitFest of
 the release cycle, or we don't.  If we do, then we should go ahead and
 do it, and we should do it early when it will have more effect rather
 than putting a lot of time into those patches and doing it only once
 the release is already late.  On the other hand, if we don't, then we
 should have to core team publish a clear statement that all
 CommitFests are equal and we're just going to slip the schedule if
 there are too many patches for the last one.

Yeah, problem being we're trying to solve at least two different
problems here: make contributor happier to contribute to PostgreSQL by
giving them early feedback and releasing their code sooner rather than
later, and having a time-based release.

The two points contradicts exactly at the end of the cycle, when we have
to decide we give the priority to the schedule or the feature
set. Knowing that being late now means being even more late on next
cycle, so contributions missing the boat are even more impacted.

All of this is stating the obvious one more time, but I think that's the
reason why the rule is not written on any wall, and why nobody tried to
enforce it in any way yet.

 What I think have been
 said before is that doing so would not help stabilizing the tree before
 release.

 Sorry, I'm not following this sentence.

Trying to state some more obvious, so as to be sure we're talking about
the same things.

 I am definitely pushing for the schedule.  It's a maxim of software
 development that you can have time-based releases or feature-based
 releases, but not both.  In this community, we have time-based
 releases early in the cycle and then they change to feature-based
 releases late in the cycle.  As we found out with 8.4, trying to be
 both on time and feature-complete can result in failing at both.  I
 feel that we've been eminently fair to contributors in the 8.5 cycle -
 it's something that I have personally worked very hard on - and I
 actually also feel that what I am proposing now is also fair.  It may
 not be very popular, though.

This has already said before, but let's try it again.

One way to solve the problem could be to have a dedicated release
management team, taking responsibilities after last alpha of the cycle
until release: open items, getting to beta, then fixing any bug testers
find, some advocacy people for having more testers, etc, then release
candidates management and then .0 release.

While this team would work on this, we could maybe have the next cycle
open for development, with its first CommitFest happening while 8.5.0 is
not yet out the door.

Of course it has been said more than once that some resources will have
to be there on both the teams, and that we want people to dedicate to
beta testing rather than new features (which is always more fun).

My guess is that current state of affairs is not working that well,
forcing people to concentrate on stabilizing current beta will push them
to procrastinate if that's not what they want to do. If instead they are
working some more on their next patch, what do we lose?

Now running the release management parallel to the first one or two
commit fest of the next cycle would mean less resources to review and
commit that ones, but maybe a better overall average.

The advantages of doing so, if not clear, are that developments never
stops and it's even easier to return a patch in the last commitfest,
we know when next one begins.

Frankly, forcing people into release management and quality assurance
when they do not want to do it does not sound the best way to offer the
best stable code possible at .0 time. And opening a commitfest were it's
possible nothing will get committed (but only reviewed) because resources
are not available sounds only fair. If you really want your stuff
committed, go help stabilizing the beta.

Regards,
-- 
dim

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread David Fetter
On Sat, Jan 09, 2010 at 12:47:00PM +0100, Magnus Hagander wrote:
 But we're *never* going to get something that's *guaranteed* 100%
 safe,

You can end the sentence right there.  *Everything* has a strictly
positive probability of catastrophic failure.

 not when going from something like CVS...

Cheers,
David.
-- 
David Fetter da...@fetter.org http://fetter.org/
Phone: +1 415 235 3778  AIM: dfetter666  Yahoo!: dfetter
Skype: davidfetter  XMPP: david.fet...@gmail.com
iCal: webcal://www.tripit.com/feed/ical/people/david74/tripit.ics

Remember to vote!
Consider donating to Postgres: http://www.postgresql.org/about/donate

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] paging Gevik Babakhani

2010-01-09 Thread Andrew Dunstan


Does anyone know how to contact Gevik Babakhani? He has not been 
responding to email, and his buildfarm member has been failing for over 
6 months due to an out of date version of flex. If I don't hear from him 
soon I'm going to stop it from reporting.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes:
 If/when we are moving the main repository, we should use the first
 one. Yes, this will invalidate all current git clones out there, but
 that's a one-time cost. Will there be issues? Possibly. But we're
 *never* going to get something that's *guaranteed* 100% safe, not when
 going from something like CVS...

Alvaro already mentioned the success criterion that we agreed to:
be able to pull all of the past release tags from the repository
and get something that matches the actual release tarballs (perhaps
with an exception for $PostgreSQL$ tags and such).  Surely the process
can be tested in advance.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Also update ChangerLog file.

2010-01-09 Thread Tom Lane
Michael Meskes mes...@postgresql.org writes:
 On Fri, Jan 08, 2010 at 03:41:10PM +0100, Stefan Kaltenbrunner wrote:
 Hmm not sure I find that commit message really helpful - but is it
 actually of any use to maintain a Changelog file specifically for
 ECPG?

 Makes it easier to see what's new in ecpg when doing a new
 release. But I guess we could get all this from the cvs log too. The
 file has been there for a long time.

Well, I'm pretty sure that release notes are prepared off of the cvs log
entries.  I've never looked at ecpg's Changelog when working on notes,
and I'd be prepared to bet that Bruce hasn't either.  My advice is to
drop the Changelog file and be sure you describe changes appropriately
in the commit messages.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Tom Lane
Peter Eisentraut pete...@gmx.net writes:
 On fre, 2010-01-08 at 12:04 -0300, Alvaro Herrera wrote:
 Do .gitignore files have the same format as .cvsignore?

 The format is the same, but while cvsignore files currently list a few
 dozen files, the proposed gitignore would list all files that are ever
 build anywhere.

The charter of the .cvsignore files is to ignore files that are not in
the repository but are nonetheless left behind after make distclean.
Any git-oriented replacement should behave the same IMO.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [COMMITTERS] pgsql: Also update ChangerLog file.

2010-01-09 Thread Magnus Hagander
On Sat, Jan 9, 2010 at 17:55, Tom Lane t...@sss.pgh.pa.us wrote:
 Michael Meskes mes...@postgresql.org writes:
 On Fri, Jan 08, 2010 at 03:41:10PM +0100, Stefan Kaltenbrunner wrote:
 Hmm not sure I find that commit message really helpful - but is it
 actually of any use to maintain a Changelog file specifically for
 ECPG?

 Makes it easier to see what's new in ecpg when doing a new
 release. But I guess we could get all this from the cvs log too. The
 file has been there for a long time.

 Well, I'm pretty sure that release notes are prepared off of the cvs log
 entries.  I've never looked at ecpg's Changelog when working on notes,
 and I'd be prepared to bet that Bruce hasn't either.  My advice is to
 drop the Changelog file and be sure you describe changes appropriately
 in the commit messages.

+1.
Seems really weird (and unnecessary) to have a changelog for just one
small subsystem, when we don't have it for the others.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Magnus Hagander
On Sat, Jan 9, 2010 at 17:32, Tom Lane t...@sss.pgh.pa.us wrote:
 Magnus Hagander mag...@hagander.net writes:
 If/when we are moving the main repository, we should use the first
 one. Yes, this will invalidate all current git clones out there, but
 that's a one-time cost. Will there be issues? Possibly. But we're
 *never* going to get something that's *guaranteed* 100% safe, not when
 going from something like CVS...

 Alvaro already mentioned the success criterion that we agreed to:
 be able to pull all of the past release tags from the repository
 and get something that matches the actual release tarballs (perhaps
 with an exception for $PostgreSQL$ tags and such).  Surely the process
 can be tested in advance.

If that's the only remaining obstacle, I'm willing to work up some
test scripts around that. But I'm not going to do that if it's going
to fall over on something else as well, because it'll be a nontrivial
amount of work to test ir properly :-)


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Peter Eisentraut
On lör, 2010-01-09 at 12:00 -0500, Tom Lane wrote:
 Peter Eisentraut pete...@gmx.net writes:
  On fre, 2010-01-08 at 12:04 -0300, Alvaro Herrera wrote:
  Do .gitignore files have the same format as .cvsignore?
 
  The format is the same, but while cvsignore files currently list a few
  dozen files, the proposed gitignore would list all files that are ever
  build anywhere.
 
 The charter of the .cvsignore files is to ignore files that are not in
 the repository but are nonetheless left behind after make distclean.
 Any git-oriented replacement should behave the same IMO.

Then it would be trivial, but that is not what is being proposed.




-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Aidan Van Dyk
* Magnus Hagander mag...@hagander.net [100109 12:03]:

 If that's the only remaining obstacle, I'm willing to work up some
 test scripts around that. But I'm not going to do that if it's going
 to fall over on something else as well, because it'll be a nontrivial
 amount of work to test ir properly :-)

It's already been done.  It was not a lot of work (just processor time).
It's even already been posted to -hackers, including what the
differences were (i.e. which $Tags$ differed, and where, and where the
CVS history had been hacked by hand).

a.

-- 
Aidan Van Dyk Create like a god,
ai...@highrise.ca   command like a king,
http://www.highrise.ca/   work like a slave.


signature.asc
Description: Digital signature


Re: [HACKERS] [COMMITTERS] pgsql: Also update ChangerLog file.

2010-01-09 Thread Michael Meskes
On Sat, Jan 09, 2010 at 11:55:00AM -0500, Tom Lane wrote:
 Well, I'm pretty sure that release notes are prepared off of the cvs log
 entries.  I've never looked at ecpg's Changelog when working on notes,
 and I'd be prepared to bet that Bruce hasn't either.  My advice is to
 drop the Changelog file and be sure you describe changes appropriately
 in the commit messages.

You definitely get a +1 from me as this file only means additional work that
tends to be forgotten. And yes, it does not have any information that the CVS
log doesn't have.

Michael
-- 
Michael Meskes
Michael at Fam-Meskes dot De, Michael at Meskes dot (De|Com|Net|Org)
Michael at BorussiaFan dot De, Meskes at (Debian|Postgresql) dot Org
ICQ: 179140304, AIM/Yahoo/Skype: michaelmeskes, Jabber: mes...@jabber.org
VfL Borussia! Forca Barca! Go SF 49ers! Use: Debian GNU/Linux, PostgreSQL

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Magnus Hagander
On Sat, Jan 9, 2010 at 18:33, Aidan Van Dyk ai...@highrise.ca wrote:
 * Magnus Hagander mag...@hagander.net [100109 12:03]:

 If that's the only remaining obstacle, I'm willing to work up some
 test scripts around that. But I'm not going to do that if it's going
 to fall over on something else as well, because it'll be a nontrivial
 amount of work to test ir properly :-)

 It's already been done.  It was not a lot of work (just processor time).
 It's even already been posted to -hackers, including what the
 differences were (i.e. which $Tags$ differed, and where, and where the
 CVS history had been hacked by hand).

Do you still have the scripts?

IIRC we fixed at least some of the brokenness in the CVS repo?

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes:
 On Sat, Jan 9, 2010 at 18:33, Aidan Van Dyk ai...@highrise.ca wrote:
 It's already been done.  It was not a lot of work (just processor time).
 It's even already been posted to -hackers, including what the
 differences were (i.e. which $Tags$ differed, and where, and where the
 CVS history had been hacked by hand).

 Do you still have the scripts?

 IIRC we fixed at least some of the brokenness in the CVS repo?

I think we had *not*, but clearly that would be an appropriate next step
to take (and then retry the import experiment).

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function

2010-01-09 Thread Tom Lane
Dean Rasheed dean.a.rash...@googlemail.com writes:
 ERROR:  attribute number -1 exceeds number of columns 1

I guess your previous message slipped through the cracks --- sorry about
that.  It looks like the best fix is to teach ExecEvalFieldSelect that
references to system columns are OK.  Working on it now.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Robert Haas
On Sat, Jan 9, 2010 at 9:33 AM, Peter Eisentraut pete...@gmx.net wrote:
 On fre, 2010-01-08 at 21:01 -0500, Robert Haas wrote:
  The commitfest is a tool for people to track what is going on, not a
  tool to tell people what to do.

 I don't understand what you mean by this.  Can you please elaborate?

 The proposal was apparently that a small, vocal group gets to decide
 what features are more important than others, and then change the commit
 fest listing to make everyone work on those features instead of some
 other ones.

I'm sorry it came across that way.  That wasn't my intention.  I am
afraid that I may have taken Tom's suggestion more seriously than it
was meant, or more seriously than I should have.  I have been spending
a very large amount of time on PostgreSQL lately for reasons that are
OT for this list, and most of that work has been directed toward
trying to make it possible for us to release on time.  I'm afraid that
I may have let myself get a little too wrapped up in this.  If there
is not a community consensus toward making an on-time release
possible, then it is not a good idea for me to devote as much time as
I have been to helping us get there, because (1) it won't work and (2)
I'll be frustrated when it doesn't.

 And I thought we had agreement that one of
 those ground rules was don't submit new, large patches to the final
 CommitFest in a particular release cycle.  No?

 I don't agree with that

If we accept large patches at the very end of the development cycle,
that's when people will submit them.  You've previously criticized the
high proportion of the release cycle that is set aside for CommitFest
and beta, so I'm surprised to see you advocating for a policy that
seems likely to lengthen the time for which the tree is closed.

 or the way it was assessed in this case.

Specifics?

 The reason why I make these points is that if you make the commit fest
 too selective, then, since you are not the employer of anyone, people
 who don't agree with your choices will just ignore them and do something
 else.

Individual contributors can do whatever they like, and they do.  We
have a number of committers, such as yourself, who could be very
helpful in getting us to release sooner, with more features, or both.
However, so far, Tom Lane is the only committer who has indicated any
willingness or time to work on any of these large patches, and so when
we say we don't want to bump any patches, are we really saying we just
want Tom to fit them into his schedule?  Some people, such as Dimitri,
have opined that we should go ahead and do first-level review of all
of them, and I'm fine with that.  But as far as committer review for
those large patches, we don't seem to have a better plan than hoping
Tom can work it all in.  And he may be able to do that, but he's
clearly said he doesn't think HS is ready for beta, so then beta will
be later.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] maintenance announcement for dekeni.postgresql.org and minshara.postgresql.org

2010-01-09 Thread Stefan Kaltenbrunner

Hi all!

In order to install some OS security upgrades we are going to execute 
planned maintainance one the following hosts affecting the mentioned 
services tomorrow (10th January 13:00-14:00 GMT). Actual expected 
downtime will be a few minutes at most so this is just a heads up.



minshara.postgresql.org affecting the following services:

* gothos.postgresql.org(git.postgresql.org)
* fornax.postgresql.org(ftp.postgresql.org)
* wysanti.postgresql.org(static web mirror)

dekeni.postgresql.org affecting the following services:

* coridan.postgresql.org(commitfest.postgresql.org)
* mintaka.postgresql.org(media.postgresql.org)


Stefan

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function

2010-01-09 Thread Dean Rasheed
2010/1/9 Tom Lane t...@sss.pgh.pa.us:
 Dean Rasheed dean.a.rash...@googlemail.com writes:
 ERROR:  attribute number -1 exceeds number of columns 1

 I guess your previous message slipped through the cracks --- sorry about
 that.  It looks like the best fix is to teach ExecEvalFieldSelect that
 references to system columns are OK.  Working on it now.


I wonder if it might be better to have plpgsql_parse_dblword() ignore
plpgsql_LookupIdentifiers, and always do the lookups. In addition to
fixing my original gripe, the resulting parse tree is simpler and slightly
faster to execute. Admittedly you have to work quite hard to contrive a
test case where the performance difference is noticeable, but with the
test code below patching plpgsql_parse_dblword() gives around a 4%
performance boost to the INSERT.

create table foo (val text, len int);

create or replace function foo_trig_fn() returns trigger as $$
begin
  new.len := length(new.val);
  return new;
end
$$ language plpgsql;

create trigger foo_trig before insert on foo
  for each row execute procedure foo_trig_fn();

insert into foo(val)
  select repeat('X', 1) from generate_series(1,10);

Regards,
Dean

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] synchronized snapshots

2010-01-09 Thread Markus Wanner
Hi

Joachim Wieland wrote:
 Since nobody objected to the idea in general, I have implemented it.

Great! I hope to get some spare cycles within the next few days to
review it.

Regards

Markus Wanner


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function

2010-01-09 Thread Tom Lane
Dean Rasheed dean.a.rash...@googlemail.com writes:
 I wonder if it might be better to have plpgsql_parse_dblword() ignore
 plpgsql_LookupIdentifiers, and always do the lookups.

Not if you'd like things to still work.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function

2010-01-09 Thread Tom Lane
Dean Rasheed dean.a.rash...@googlemail.com writes:
 2010/1/9 Tom Lane t...@sss.pgh.pa.us:
 Dean Rasheed dean.a.rash...@googlemail.com writes:
 I wonder if it might be better to have plpgsql_parse_dblword() ignore
 plpgsql_LookupIdentifiers, and always do the lookups.

 Not if you'd like things to still work.

 OK, I admit that I'm totally new that area of code, so I'm not seeing
 it - what does it break?

The main problem is it will throw errors in some cases where that's
premature.  For instance we might have a.x where a is a plpgsql
row variable that doesn't contain x ... but if the reference is
in a query where a is a table that contains x, and we are using
prefer-the-column rules, this is not an error case.  Also we do
not want any lookups while looking at DECLARE constructs ---
it doesn't matter whether there's an outer-scope variable of the
same name.

However, it turns out my solution isn't working too well either :-(.
I can make it work for some of the system columns, but not for xmin,
xmax, or cmin/cmax because those fields of a regular tuple are overlaid
with datum-tuple header fields.  So by the time ExecEvalFieldSelect gets
the tuple those values are irretrievably trashed.

I think that a variant of your idea could be made to work: change
plpgsql_LookupIdentifiers to a three-state variable (which'd basically
mean in DECLARE, in a SQL expression, anywhere else), do no lookups in
DECLARE, and in SQL expressions do lookups but never throw any errors.
I'll have a go at that.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] synchronized snapshots

2010-01-09 Thread Marcin Mańk



Dnia 2010-01-09 o godz. 20:37 Markus Wanner mar...@bluegap.ch napisał 
(a):



Hi

Joachim Wieland wrote:

Since nobody objected to the idea in general, I have implemented it.


How cool it would be if we could synchronize snapshots between the  
master and the (sr) standby?


The connection poolers could use that to send read-only queries to the  
standby, and when the first dml/ddl statement in a transaction comes  
up, they could switch to the master.


If it is hard to tell from the statement if it writes anything, the  
pooler could catch the error, and retry on the master


Regards
Marcin Mańk 
--

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function

2010-01-09 Thread Dean Rasheed
2010/1/9 Tom Lane t...@sss.pgh.pa.us:
 OK, I admit that I'm totally new that area of code, so I'm not seeing
 it - what does it break?

 The main problem is...

Ah OK. Thanks for the explanation.

 I think that a variant of your idea could be made to work: change
 plpgsql_LookupIdentifiers to a three-state variable (which'd basically
 mean in DECLARE, in a SQL expression, anywhere else), do no lookups in
 DECLARE, and in SQL expressions do lookups but never throw any errors.
 I'll have a go at that.


Sounds plausible.

Cheers,
Dean

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Roger Leigh
On Sat, Jan 09, 2010 at 12:47:00PM +0100, Magnus Hagander wrote:
 On Sat, Jan 9, 2010 at 05:54, Alvaro Herrera alvhe...@commandprompt.com 
 wrote:
  Tom Lane escribió:
  David Fetter da...@fetter.org writes:
   On Fri, Jan 08, 2010 at 10:35:24PM -0500, Tom Lane wrote:
   Probably eventually we'll be on git and this will be moot, but that
   doesn't seem to be ready to happen.
 
   What still needs to happen on this?  Clearly this would be a post-8.5
   (or whatever the new release number is) thing, but apart from that?
 
  AFAIR, we still weren't convinced that we had a 100% conversion method
  (ie something that would preserve all the history) and there were still
  questions about how to work with multi-branch patches most effectively.
  I don't recall where the previous discussion died off exactly, but
  it definitely wasn't at the we're ready to do it stage.
 
  Somebody did a pull of all the tags, and some of them were missing files
  and failed to build.
 
 That was from the current git mirror.
 
 To re-itarate yet again, what I believe has been said many times before:
 
 There are two ways to get from cvs to git.
 
 The first one is reliable (at least from what I've heard). But it only
 supports one-off migrations. It doesn't support incremental changes.
 It was confused by some things that were plain broken in our cvs
 repository way back (this happens with cvs, as we all know), but AFAIK
 they have been fixed.

As a git user who has now done a number of CVS-git migrations over the
past few years, I also found that the various tools to do the
conversion do have their own issues (such as cvsps) which can lead to
incorrect history in some corner cases.  Unfortunately, in any big
repo with a lot of history, you do tend to find you trip up on them
in a few places.

My experience was that the CVS-SVN conversion tended to be rather
more reliable and accurate.  As a result, going GIT-SVN-git can
give a much better history.  At least in my experience.


Regards,
Roger

-- 
  .''`.  Roger Leigh
 : :' :  Debian GNU/Linux http://people.debian.org/~rleigh/
 `. `'   Printing on GNU/Linux?   http://gutenprint.sourceforge.net/
   `-GPG Public Key: 0x25BFB848   Please GPG sign your mail.


signature.asc
Description: Digital signature


Re: [HACKERS] damage control mode

2010-01-09 Thread Peter Eisentraut
On lör, 2010-01-09 at 14:12 -0500, Robert Haas wrote:
 If we accept large patches at the very end of the development cycle,
 that's when people will submit them.  You've previously criticized the
 high proportion of the release cycle that is set aside for CommitFest
 and beta, so I'm surprised to see you advocating for a policy that
 seems likely to lengthen the time for which the tree is closed.

Just to clarify: I am for sticking to the agreed dates.  If some things
are not ready by the necessary date plus/minus one, they won't make the
release.  If it's obvious earlier that something won't make the date, it
shouldn't be committed, and maybe put on the backburner right now.  But
AFAICT, that's independent of when it was submitted.  Some things that
were submitted just the other day might be almost ready, some things
that were first submitted many months ago are still not ready.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Setting oom_adj on linux?

2010-01-09 Thread Peter Eisentraut
On fre, 2010-01-08 at 11:37 -0500, Tom Lane wrote:
 Alex Hunsaker bada...@gmail.com writes:
  On Fri, Jan 8, 2010 at 07:27, Tom Lane t...@sss.pgh.pa.us wrote:
  Then, somebody who wants the feature would build with, say,
 -DLINUX_OOM_ADJ=0
  or another value if they want that.
 
  Here is a stab at that.
 
 Anybody have an objection to this basic approach?  I'm in a bit of a
 hurry to get something like this into the Fedora RPMs, so barring
 objections I'm going to review this, commit it into HEAD, and then
 make a back-ported patch I can use with 8.4 in Fedora.

I find this whole approach a bit evil.  If word of this gets out, every
server process on Linux will excuse itself from the OOM killer.  And
then the kernel guys will add another setting to override the process
preference.  It's an arms race, but maybe that's what's needed.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Feature patch 1 for plperl [PATCH]

2010-01-09 Thread Peter Eisentraut
On fre, 2010-01-08 at 15:01 +, Tim Bunce wrote:
 I didn't get any significant feedback from the earlier draft so here's
 the finished 'feature patch 1' for plperl.

I think it would help if you could split this up into about 6 to 10
single-feature patches.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initial refactoring of plperl.c - updated

2010-01-09 Thread Peter Eisentraut
On fre, 2010-01-08 at 12:46 +, Tim Bunce wrote:
 *** 45,50 
 --- 45,55 
   
   include $(top_srcdir)/src/Makefile.shlib
   
 + plperl.o: perlchunks.h
 + 
 + perlchunks.h: plc_*.pl
 +   $(PERL) text2macro.pl --strip='^(\#.*|\s*)$$' plc_*.pl 
 perlchunks.htmp
 +   mv perlchunks.htmp perlchunks.h
   
   all: all-lib
   

What's the reason for the temp file here?


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initial refactoring of plperl.c - updated

2010-01-09 Thread Tim Bunce
On Fri, Jan 08, 2010 at 09:47:01PM -0500, Andrew Dunstan wrote:
 Tim Bunce wrote:
 
 I see you've not commited it yet, so to help out I've attached
 a new diff, over the current CVS head, with two minor changes:
 
 - Removed the test, as noted above.
 - Optimized pg_verifymbstr calls to avoid unneeded strlen()s.
 
 I have committed this with minor edits. That should give us a clean
 base for the features patch(es).

Wonderful. Many thanks Andrew.

Tim.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Robert Haas
On Sat, Jan 9, 2010 at 4:01 PM, Peter Eisentraut pete...@gmx.net wrote:
 On lör, 2010-01-09 at 14:12 -0500, Robert Haas wrote:
 If we accept large patches at the very end of the development cycle,
 that's when people will submit them.  You've previously criticized the
 high proportion of the release cycle that is set aside for CommitFest
 and beta, so I'm surprised to see you advocating for a policy that
 seems likely to lengthen the time for which the tree is closed.

 Just to clarify: I am for sticking to the agreed dates.  If some things
 are not ready by the necessary date plus/minus one, they won't make the
 release.  If it's obvious earlier that something won't make the date, it
 shouldn't be committed, and maybe put on the backburner right now.  But
 AFAICT, that's independent of when it was submitted.  Some things that
 were submitted just the other day might be almost ready, some things
 that were first submitted many months ago are still not ready.

The portion of the schedule I'm worried about is the one where we go
to beta by March 7th.

http://archives.postgresql.org/pgsql-hackers/2009-09/msg01251.php

I think we can get all the remaining large patches committed by
February 15th if Tom doesn't start working on the remaining open items
until February 15th - but then I do not think that we will have a beta
on March 7th.

http://archives.postgresql.org/pgsql-hackers/2010-01/msg00663.php

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Feature patch 1 for plperl [PATCH]

2010-01-09 Thread Tim Bunce
On Fri, Jan 08, 2010 at 10:36:43PM -0500, Robert Haas wrote:
 On Fri, Jan 8, 2010 at 10:01 AM, Tim Bunce tim.bu...@pobox.com wrote:
  I didn't get any significant feedback from the earlier draft so here's
  the finished 'feature patch 1' for plperl.  (This builds on my earlier
  plperl refactoring patch, and the follow-on ppport.h patch.)
 
  Significant changes from the first draft:
  - New GUC plperl.on_perl_init='...perl...' for admin use.
  - New GUC plperl.on_trusted_init='...perl...' for plperl user use.
  - New GUC plperl.on_untrusted_init='...perl...' for plperlu user use.
 
 I kind of thought Tom said these were a bad idea, and I think I kind
 of agree.

Tom had some concerns which I believe I've addressed.
I'd be happy to hear from Tom if he has any remaining concerns.

 We're not going to support multi-line values for GUCs
 AFAIK, so this is going to be pretty kludgy.

I'm not sure what you mean by this. Typical use-cases would be:
plperl.on_perl_init='use MyStuff;'
plperl.on_trusted_init='$some_global=42';

 What about making the value a comma-separated list of module names to
 use, or something?

That would force people who just want to set some global variable
to write a module. That seems overly painful for no significant gain.
The fact that multi-line values for GUCs aren't supported will naturally
enourage anyone wanting to execute many statements to write a module for
them. That sems like a perfectly reasonable balance.

  [...]
 
 The rest of this all seems pretty nice, though I haven't read the patch yet.

Thanks Robert. I look forward to your feedback if you do get a chance to read 
it.

Tim.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] win32 socket definition

2010-01-09 Thread Magnus Hagander
On Wed, Jan 6, 2010 at 22:42, Tom Lane t...@sss.pgh.pa.us wrote:
 Magnus Hagander mag...@hagander.net writes:
 Is there a good trick to find out if you've touched every place you
 need to, because I'm very unsure I have. I don't even get a warning in
 more than those two places, but there ought to be some way to trick
 the system to tell me?

 Can't think of one, but you could try grepping for the socket-related
 syscalls to see what variables are referenced there.

Found two more by going over it again that way.

Unless there are objections, I will apply this version tomorrow.


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/


socket.patch
Description: Binary data

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Setting oom_adj on linux?

2010-01-09 Thread Alex Hunsaker
On Sat, Jan 9, 2010 at 14:06, Peter Eisentraut pete...@gmx.net wrote:
 I find this whole approach a bit evil.

I would tend to agree but this type of thing has been known since about 2004...

See http://thoughts.j-davis.com/2009/11/29/linux-oom-killer/,
particularly the comment from Greg Smith.

 If word of this gets out, every
 server process on Linux will excuse itself from the OOM killer.  And
 then the kernel guys will add another setting to override the process
 preference.

Yes, and note debian is already doing that with things like ssh.  Who
knows what else. (Id be curious to know)

Plus maybe it will convince them its time to fix the damn thing.
Although postgres really is kind of special in this regard.  All the
other daemons on my system include X had way lower oom scores.
Alsamixer was 3 times more likely to get killed than the first daemon
with the highest score (hald) while postgres was 55 times more likely.
 Yes its the kernel being stupid, but its been known for more than 6
years...

(oom scores: alsamxier: 1497, hald: 487, postgres: 26558)

 It's an arms race, but maybe that's what's needed.

Well *shrug* regardless of what core does... Ill certainly be doing it
on my postgres linux builds :)  Maybe it would convince them more if
we could get distros to accept patches that fix the kernel to do
correct/better shared mem accounting?  May I add good luck? :)

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Setting oom_adj on linux?

2010-01-09 Thread Andrew Dunstan



Peter Eisentraut wrote:

On fre, 2010-01-08 at 11:37 -0500, Tom Lane wrote:
  

Alex Hunsaker bada...@gmail.com writes:


On Fri, Jan 8, 2010 at 07:27, Tom Lane t...@sss.pgh.pa.us wrote:
  

Then, somebody who wants the feature would build with, say,
   -DLINUX_OOM_ADJ=0
or another value if they want that.


Here is a stab at that.
  

Anybody have an objection to this basic approach?  I'm in a bit of a
hurry to get something like this into the Fedora RPMs, so barring
objections I'm going to review this, commit it into HEAD, and then
make a back-ported patch I can use with 8.4 in Fedora.



I find this whole approach a bit evil.  If word of this gets out, every
server process on Linux will excuse itself from the OOM killer.  And
then the kernel guys will add another setting to override the process
preference.  It's an arms race, but maybe that's what's needed.
  


The trouble is that the OOM heuristics are pretty bad, and many Linux 
hackers aren't interested in improving them. One of the most prominent 
told me some years ago Just turn it off.


And the point of this patch is to allow the postmaster to *remove* OOM 
protection from normal postgres backends. We at least would be playing 
nice, and not engaging in an arms race.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Setting oom_adj on linux?

2010-01-09 Thread Tom Lane
Alex Hunsaker bada...@gmail.com writes:
 On Sat, Jan 9, 2010 at 14:06, Peter Eisentraut pete...@gmx.net wrote:
 If word of this gets out, every
 server process on Linux will excuse itself from the OOM killer.  And
 then the kernel guys will add another setting to override the process
 preference.

 ... maybe it will convince them its time to fix the damn thing.
 Although postgres really is kind of special in this regard.

Yeah.  If they had saner handling of shared-memory accounting, maybe
there wouldn't be a need for us to kluge around the OOM logic.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Robert Haas
On Sat, Jan 9, 2010 at 12:00 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Peter Eisentraut pete...@gmx.net writes:
 On fre, 2010-01-08 at 12:04 -0300, Alvaro Herrera wrote:
 Do .gitignore files have the same format as .cvsignore?

 The format is the same, but while cvsignore files currently list a few
 dozen files, the proposed gitignore would list all files that are ever
 build anywhere.

 The charter of the .cvsignore files is to ignore files that are not in
 the repository but are nonetheless left behind after make distclean.
 Any git-oriented replacement should behave the same IMO.

Oh.  Never mind.  That doesn't seem useful enough to be worth spending
time on.  What I want is to ignore all of the build products, so that
when I do 'git status' in my working tree, I only see the the files
I've actually added/changed.  Now that you mention it, I think I had
the same complaint about the .cvsignore files back when I was using
CVS.  It seems like an odd charter.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Andrew Dunstan



Robert Haas wrote:

On Sat, Jan 9, 2010 at 12:00 PM, Tom Lane t...@sss.pgh.pa.us wrote:
  

Peter Eisentraut pete...@gmx.net writes:


On fre, 2010-01-08 at 12:04 -0300, Alvaro Herrera wrote:
  

Do .gitignore files have the same format as .cvsignore?


The format is the same, but while cvsignore files currently list a few
dozen files, the proposed gitignore would list all files that are ever
build anywhere.
  

The charter of the .cvsignore files is to ignore files that are not in
the repository but are nonetheless left behind after make distclean.
Any git-oriented replacement should behave the same IMO.



Oh.  Never mind.  That doesn't seem useful enough to be worth spending
time on.  What I want is to ignore all of the build products, so that
when I do 'git status' in my working tree, I only see the the files
I've actually added/changed.  Now that you mention it, I think I had
the same complaint about the .cvsignore files back when I was using
CVS.  It seems like an odd charter.

  


Use a vpath build, and you'll keep those artifacts out of your source tree.

cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Robert Haas
On Sat, Jan 9, 2010 at 5:18 PM, Andrew Dunstan and...@dunslane.net wrote:
 Robert Haas wrote:
 On Sat, Jan 9, 2010 at 12:00 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Peter Eisentraut pete...@gmx.net writes:
 On fre, 2010-01-08 at 12:04 -0300, Alvaro Herrera wrote:
 Do .gitignore files have the same format as .cvsignore?
 The format is the same, but while cvsignore files currently list a few
 dozen files, the proposed gitignore would list all files that are ever
 build anywhere.

 The charter of the .cvsignore files is to ignore files that are not in
 the repository but are nonetheless left behind after make distclean.
 Any git-oriented replacement should behave the same IMO.


 Oh.  Never mind.  That doesn't seem useful enough to be worth spending
 time on.  What I want is to ignore all of the build products, so that
 when I do 'git status' in my working tree, I only see the the files
 I've actually added/changed.  Now that you mention it, I think I had
 the same complaint about the .cvsignore files back when I was using
 CVS.  It seems like an odd charter.

 Use a vpath build, and you'll keep those artifacts out of your source tree.

I suppose that's one answer, but of what use is it to ignore only the
'make distclean' leftovers?

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] mailing list archiver chewing patches

2010-01-09 Thread Tim Bunce
On Sat, Jan 09, 2010 at 02:17:27AM -0300, Alvaro Herrera wrote:
 Alvaro Herrera wrote:
  Andrew Dunstan wrote:
   
   Tim Bunce's recent patch has been mangled apparently by the list
   archives. He sent it as an attachment, and that's how I have it in
   my mailbox, so why isn't it appearing as such in the web archive so
   that it can be nicely downloaded? See 
   http://archives.postgresql.org/message-id/20100108124613.gl2...@timac.local.
   It's happened to other people as well:
   http://archives.postgresql.org/message-id/4b02d3e4.1040...@hut.fi
   
   Reviewers and others shouldn't have to cp patches from web pages,
   especially when it will be horribly line wrapped etc. Can we stop
   this happening somehow?
  
  Try this
  
  http://archives.postgresql.org/msgtxt.php?id=20100108124613.gl2...@timac.local

That looks like it dumps the raw message. That'll cause problems for any
messages using quoted-printable encoding. I'd hazard a guess it also
won't do thing right thing for non-charset=us-ascii emails/attachments.

 This was previously broken for a lot of emails, but I just fixed some of
 it, and it seems to work for the vast majority of our emails (and
 certainly for all emails that matter).
 
 The other point related to this is that each email should have a link
 pointing to its text/plain version.  This used to be present, but it got
 broken (I think) at the same time that the anti-email-harvesting measure
 got broken.  I'm going to look at that next.

 Let me know if you find something broken with this style of link.

What's needed is a) a download link for each attachment, regardless of the
kind of attachment, and b) the download link should download the content
of the attachment in a way that's directly usable.

For example, see 
http://archives.postgresql.org/pgsql-hackers/2010-01/msg00589.php
Looking at the raw version of the original message

http://archives.postgresql.org/msgtxt.php?id=757953.70187...@web29001.mail.ird.yahoo.com
That message has a patch as an attachment:
Content-Type: application/octet-stream; name=patch_bit.patch
Content-Transfer-Encoding: base64
Content-Disposition: attachment; filename=patch_bit.patch

It gets a link in the archive (because it's a non-text content-type I presume):
http://archives.postgresql.org/pgsql-hackers/2010-01/bin5ThVOJC3jI.bin
but the link doesn't work well.  The url ends with .bin and the http
response content-type is Content-Type: application/octet-stream so
downloaders get a .bin file instead of the original .patch file.

It seems that people wanting to send in a patch have two options: send
it as text/(something) so it's readable on the archive web page but not
copy-n-paste'able because of wordwrapping, or set it as
application/octet-stream so it's downloadable but not readable on the
web page.

Let me know if I've misunderstood anything.

Some sugestions:
- Provide links for all attachments, whether text/* or not.
- For text/* types show the content inline verbatim, don't wrap the text.
- If the attachment has a Content-Disposition with a filename then
  append that to the url. It could simply be a fake 'path info':
.../2010-01/bin5ThVOJC3jI.bin/patch_bit.patch
- Instead of Description: Binary data on the web page, give the
  values of the Content-Type and Content-Disposition headers.

Tim.

p.s. For background... I'm writing an email to the dbi-users 
dbi-announce mailing lists (~2000  ~5000 users last time I checked)
asking anyone who might be interested to help review the plperl feature
patch and encouraging them to contribute to the commitfest review
process for other patches.  It's important that it's *very* easy for
these new-comers to follow simple instructions to get involved.
I was hoping to be able to use a archives.postgresql.org url to the
message with the patch to explain what's the patch does _and_ provide a
download link. It seems I'll have to upload the patch somewhere else.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] mailing list archiver chewing patches

2010-01-09 Thread Andrew Dunstan



Tim Bunce wrote:

It seems that people wanting to send in a patch have two options: send
it as text/(something) so it's readable on the archive web page but not
copy-n-paste'able because of wordwrapping, or set it as
application/octet-stream so it's downloadable but not readable on the
web page.


  


That is assuming that the MUA gives you the option of specifying the 
attachment MIME type. Many (including mine) do not. It would mean an 
extra step - I'd have to gzip each patch or something like that. That 
would be unfortunate,as well as imposing extra effort, because it would 
make the patch not display inline in many MUAs (again, like mine).


cheers

andrew



--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 On Sat, Jan 9, 2010 at 5:18 PM, Andrew Dunstan and...@dunslane.net wrote:
 Robert Haas wrote:
 What I want is to ignore all of the build products

 Use a vpath build, and you'll keep those artifacts out of your source tree.

 I suppose that's one answer, but of what use is it to ignore only the
 'make distclean' leftovers?

Well, it fits my workflow, which is make distclean - cvs update -
fresh build.  If you want to update without cleaning the build tree,
using a separate build tree seems like a much better answer.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Peter Eisentraut
On lör, 2010-01-09 at 17:19 -0500, Robert Haas wrote:
  Oh.  Never mind.  That doesn't seem useful enough to be worth spending
  time on.  What I want is to ignore all of the build products, so that
  when I do 'git status' in my working tree, I only see the the files
  I've actually added/changed.  Now that you mention it, I think I had
  the same complaint about the .cvsignore files back when I was using
  CVS.  It seems like an odd charter.
 
  Use a vpath build, and you'll keep those artifacts out of your source tree.
 
 I suppose that's one answer, but of what use is it to ignore only the
 'make distclean' leftovers?

That charter was established before make maintainer-clean was invented.



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Robert Haas
On Sat, Jan 9, 2010 at 5:44 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 On Sat, Jan 9, 2010 at 5:18 PM, Andrew Dunstan and...@dunslane.net wrote:
 Robert Haas wrote:
 What I want is to ignore all of the build products

 Use a vpath build, and you'll keep those artifacts out of your source tree.

 I suppose that's one answer, but of what use is it to ignore only the
 'make distclean' leftovers?

 Well, it fits my workflow, which is make distclean - cvs update -
 fresh build.  If you want to update without cleaning the build tree,
 using a separate build tree seems like a much better answer.

Oh, I see.  With git, I clean the tree with git clean -dfx rather
than make distclean.  That just blows away all the files that aren't
in the repository.  I find that faster and more reliable than make
distclean, although of course you have to be careful not to be leave
anything in the tree that you were planning to hold on to...

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Peter Eisentraut
On lör, 2010-01-09 at 17:12 -0500, Robert Haas wrote:
 Oh.  Never mind.  That doesn't seem useful enough to be worth spending
 time on.  What I want is to ignore all of the build products, so that
 when I do 'git status' in my working tree, I only see the the files
 I've actually added/changed.

Shouldn't it be possible to just build the master branch once and then
take the result of git status to produce the list of files to be
ignored?  That should reduce the maintenance effort.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Robert Haas
On Sat, Jan 9, 2010 at 5:53 PM, Peter Eisentraut pete...@gmx.net wrote:
 On lör, 2010-01-09 at 17:12 -0500, Robert Haas wrote:
 Oh.  Never mind.  That doesn't seem useful enough to be worth spending
 time on.  What I want is to ignore all of the build products, so that
 when I do 'git status' in my working tree, I only see the the files
 I've actually added/changed.

 Shouldn't it be possible to just build the master branch once and then
 take the result of git status to produce the list of files to be
 ignored?  That should reduce the maintenance effort.

Oh, for sure.  That's what I would do, although I would say we should
ignore *.o in the relevant directories rather than listing out the
individual files.  But if Tom only wants the things that remain after
'make distclean' to be ignored, then it's not actually going to solve
any problem that I have, because, as mentioned upthread, I use
'git-clean -dfx' to clean my tree.  What I'm interested in is being
able to run 'git status' on a tree in which I've run a build without
getting a lot of extra output, and that will require ignoring all the
build products.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes:
 ... What I'm interested in is being
 able to run 'git status' on a tree in which I've run a build without
 getting a lot of extra output, and that will require ignoring all the
 build products.

I'm fairly hesitant to set up ignore files that list *all* the build
products (or even all the non-.o ones) because of the probability of
error --- in particular, the likelihood that this would mask an omission
in a make clean rule.  The current charter for .cvsignore is
relatively safe and low-maintenance because there are so few built files
that are supposed to remain around in a distribution tree.  What you're
talking about would require a great deal more maintenance effort, and
I don't see the point compared to using a VPATH build.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Robert Haas
On Sat, Jan 9, 2010 at 6:20 PM, Tom Lane t...@sss.pgh.pa.us wrote:
 Robert Haas robertmh...@gmail.com writes:
 ... What I'm interested in is being
 able to run 'git status' on a tree in which I've run a build without
 getting a lot of extra output, and that will require ignoring all the
 build products.

 I'm fairly hesitant to set up ignore files that list *all* the build
 products (or even all the non-.o ones) because of the probability of
 error --- in particular, the likelihood that this would mask an omission
 in a make clean rule.  The current charter for .cvsignore is
 relatively safe and low-maintenance because there are so few built files
 that are supposed to remain around in a distribution tree.  What you're
 talking about would require a great deal more maintenance effort, and
 I don't see the point compared to using a VPATH build.

That seems to be a common POV, so I think we should just let it go.

...Robert

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Re: CVS HEAD: Error accessing system column from plpgsql trigger function

2010-01-09 Thread Dean Rasheed
2010/1/9 Tom Lane t...@sss.pgh.pa.us:
 Dean Rasheed dean.a.rash...@googlemail.com writes:
 I wonder if it might be better to have plpgsql_parse_dblword() ignore
 plpgsql_LookupIdentifiers, and always do the lookups.

 Not if you'd like things to still work.


OK, I admit that I'm totally new that area of code, so I'm not seeing
it - what does it break?
[regression tests still pass BTW]

Regards,
Dean

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initial refactoring of plperl.c - updated

2010-01-09 Thread Tim Bunce
On Sat, Jan 09, 2010 at 11:16:18PM +0200, Peter Eisentraut wrote:
 On fre, 2010-01-08 at 12:46 +, Tim Bunce wrote:
  *** 45,50 
  --- 45,55 

include $(top_srcdir)/src/Makefile.shlib

  + plperl.o: perlchunks.h
  + 
  + perlchunks.h: plc_*.pl
  +   $(PERL) text2macro.pl --strip='^(\#.*|\s*)$$' plc_*.pl 
  perlchunks.htmp
  +   mv perlchunks.htmp perlchunks.h

all: all-lib
 
 What's the reason for the temp file here?

Defensive. If the text2macro.pl program fails/dies then you'd be left
with a broken output file with a newer timestamp, so the next make
wouldn't rerun text2macro.pl.

Tim.

p.s. In the makefile for perl we use a little utility called mv_if_diff
instead of a plain mv so that any downstream dependencies only get
rebuilt if the contents have changed.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initial refactoring of plperl.c - updated

2010-01-09 Thread Tim Bunce
On Sat, Jan 09, 2010 at 11:49:22PM +, Tim Bunce wrote:
 On Sat, Jan 09, 2010 at 11:16:18PM +0200, Peter Eisentraut wrote:
  On fre, 2010-01-08 at 12:46 +, Tim Bunce wrote:
   *** 45,50 
   --- 45,55 
 
 include $(top_srcdir)/src/Makefile.shlib
 
   + plperl.o: perlchunks.h
   + 
   + perlchunks.h: plc_*.pl
   +   $(PERL) text2macro.pl --strip='^(\#.*|\s*)$$' plc_*.pl 
   perlchunks.htmp
   +   mv perlchunks.htmp perlchunks.h
 
 all: all-lib
  
  What's the reason for the temp file here?
 
 Defensive. If the text2macro.pl program fails/dies then you'd be left
 with a broken output file with a newer timestamp, so the next make
 wouldn't rerun text2macro.pl.

An alternative would be for text2macro.pl to write to a temp file and
rename at the end.

Tim.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Feature patch 1 for plperl [PATCH]

2010-01-09 Thread Andrew Dunstan



Peter Eisentraut wrote:

On fre, 2010-01-08 at 15:01 +, Tim Bunce wrote:
  

I didn't get any significant feedback from the earlier draft so here's
the finished 'feature patch 1' for plperl.



I think it would help if you could split this up into about 6 to 10
single-feature patches.
  


I think that's a bit excessive. I'd suggest three patches:

   * the new utility functions (quote_literal, decode_bytea etc.) These
 should be fairly uncontroversial, but account for a large part of
 the patch volume.
   * the code relating to library load, interpreter initialization and
 termination
   * the remainder (function naming, better error checking, enabling
 use/require if a lib is already loaded, cleanup and optimization)

We could ask Tim to break up the last, but they are all fairly small 
items, and I at least wouldn't be bothered by having them combined. And 
having to handle 6 to 10 patches all hitting the same body of code 
doesn't sound terrible pleasant either.


cheers

andrew

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initial refactoring of plperl.c - updated

2010-01-09 Thread Peter Eisentraut
On sön, 2010-01-10 at 00:03 +, Tim Bunce wrote:
 On Sat, Jan 09, 2010 at 11:49:22PM +, Tim Bunce wrote:
  On Sat, Jan 09, 2010 at 11:16:18PM +0200, Peter Eisentraut wrote:
   On fre, 2010-01-08 at 12:46 +, Tim Bunce wrote:
*** 45,50 
--- 45,55 
  
  include $(top_srcdir)/src/Makefile.shlib
  
+ plperl.o: perlchunks.h
+ 
+ perlchunks.h: plc_*.pl
+   $(PERL) text2macro.pl --strip='^(\#.*|\s*)$$' plc_*.pl 
perlchunks.htmp
+   mv perlchunks.htmp perlchunks.h
  
  all: all-lib
   
   What's the reason for the temp file here?
  
  Defensive. If the text2macro.pl program fails/dies then you'd be left
  with a broken output file with a newer timestamp, so the next make
  wouldn't rerun text2macro.pl.
 
 An alternative would be for text2macro.pl to write to a temp file and
 rename at the end.

Sounds better.  I think any program should be written such that it
doesn't produce an output file at all if it cannot produce a correct
output file.  So use a temp file or a trap or something like that.  The
makefile should not have to clean up after everyone.


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] mailing list archiver chewing patches

2010-01-09 Thread Alvaro Herrera
Tim Bunce wrote:

   Try this
   
   http://archives.postgresql.org/msgtxt.php?id=20100108124613.gl2...@timac.local
 
 That looks like it dumps the raw message. That'll cause problems for any
 messages using quoted-printable encoding. I'd hazard a guess it also
 won't do thing right thing for non-charset=us-ascii emails/attachments.

Yeah.  Grab it and open it as an mbox.

 What's needed is a) a download link for each attachment, regardless of the
 kind of attachment, and b) the download link should download the content
 of the attachment in a way that's directly usable.

Yeah, well, that's a bit outside what I am able to do, unless you can
get a MHonArc expert somewhere who can help us figure out how to
set it up for these requirements.

-- 
Alvaro Herrerahttp://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Add .gitignore files to CVS?

2010-01-09 Thread Greg Smith

Peter Eisentraut wrote:

Probably someone to actually track the open items that are mentioned
every time this discussion happens.
  


http://wiki.postgresql.org/wiki/Switching_PostgreSQL_from_CVS_to_Git now 
has what I believe the state of the world to be in this area.


--
Greg Smith2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
g...@2ndquadrant.com  www.2ndQuadrant.com


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Testing with concurrent sessions

2010-01-09 Thread Craig Ringer

On 8/01/2010 1:39 AM, Greg Sabino Mullane wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: RIPEMD160



Using DBI/DBD::Pg would raise another issue - what version of libpq
would it be using? Not the one in the build being tested, that's for
sure.


Er...why not? That's what psql uses.


Because you'd have to build DBD::Pg against the new libpq, as you do 
psql. That means you need DBD::Pg sources and the build environment for 
Perl (headers etc) not just a working Perl runtime. Big difference.


There's no guarantee the user's machine has the same major version of 
libpq and thus no guarantee that DBD::Pq can be redirected to use your 
custom libpq by LD_LIBRARY_PATH. It might also override the library 
search path with rpath linking. Building your own would be pretty much 
unavoidable unless you're prepared to either require the user to provide 
a matching version of DBD::Pg or have the tests running with whatever 
random version happens to be lying around.


Using whatever DBD::Pg version happens to be present on the machine 
would be a bit of a nightmare for reproducibility of test results, and 
would be really unattractive for use in the standard tests. make check 
fails on my some-random-distro would become painfully common on the 
GENERAL list...


Is bundling a Perl module in the source tree and building it as part of 
the Pg build a reasonable choice? Personally, I don't think so.


Additionally, a dedicated testing tool like some folks have been talking 
about would be really handy for users who want to test their schema. 
I've had to write my own (in Java, or I'd be offering it) for this 
purpose, as psql is completely unsuitable for concurrent-run testing and 
I needed to show that my locking was safe and deadlock-free in some of 
the more complex stored procs I have.


--
Craig Ringer

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


[HACKERS] Congrats Alvaro!

2010-01-09 Thread Devrim GÜNDÜZ

Alvaro, one of our hackers and committers and my colleague more than 4
years, had a new baby today. 

Congrats Alvaro for his second daughter !

-committers, please commit your patches for our new baby elephant!
-- 
Devrim GÜNDÜZ, RHCE
Command Prompt - http://www.CommandPrompt.com 
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz


signature.asc
Description: This is a digitally signed message part


Re: [HACKERS] damage control mode

2010-01-09 Thread Josh Berkus
Peter,

 Just to clarify: I am for sticking to the agreed dates.  If some things
 are not ready by the necessary date plus/minus one, they won't make the
 release.  If it's obvious earlier that something won't make the date, it
 shouldn't be committed, and maybe put on the backburner right now.  But
 AFAICT, that's independent of when it was submitted.  Some things that
 were submitted just the other day might be almost ready, some things
 that were first submitted many months ago are still not ready.

In that case, Robert's comments about patches to bounce on Day 1 of the
commitfest are still valid, regardless of patch size.  That is, if
we're getting patches which seem very unlikely to make the cut by
Feburary 15 (like KNNGiST, which currently doesn't even apply), then it
makes sense for the CFM to notify those authors as soon as possible that
their patch is probably last in line to get reviewed.

(btw, I'd have prefered the no large patches rule, but it did not get
a firm consensus)

In any case, the CFM has sole authority for allocating the efforts of
reviewers who volunteer for assingment (the RRR).  So the CFM can
certainly assign people to review only on patches they think will make
it, and leave high-risk patches for last.

For example, if I were Robert, I probably wouldn't assign any RRR to
KNNGiST until all patches with a high chance of commit were clear.  Of
course, if the PostGIS community got motivated and put Paul and others
on reviewing it to get it through, then great.  Not every reviewer cares
about all patches equally.

That's completely aside from the concept of owing anyone a review.
The last commitfest is really about, what can we get committed for
8.5?  This would be the main difference between the last commitfest and
the other 3; during the other commitfests we can afford to devote
resources to reviewing patches just because someone has been a good
hacker; in CF4, we really have to be pragmatic about what's going to
make it in, or not.

I'll also say: if we can't make time-based releases work, we're probably
dead as a project.  MySQL and Ingres both tried feature-based releases,
and look where they are now.

--Josh Berkus



-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Congrats Alvaro!

2010-01-09 Thread Marc G. Fournier


Okay *scratch head* ... if he just had a baby girl today, what was he 
doing answering emails??  Priorities folks :)


Congrats Alvaro ... hopefully she came out healthy and without a trunk? :)

On Sun, 10 Jan 2010, Devrim G?ND?Z wrote:



Alvaro, one of our hackers and committers and my colleague more than 4
years, had a new baby today.

Congrats Alvaro for his second daughter !

-committers, please commit your patches for our new baby elephant!
--
Devrim G?ND?Z, RHCE
Command Prompt - http://www.CommandPrompt.com
devrim~gunduz.org, devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr
http://www.gunduz.org  Twitter: http://twitter.com/devrimgunduz




Marc G. FournierHub.Org Hosting Solutions S.A.
scra...@hub.org http://www.hub.org

Yahoo:yscrappySkype: hub.orgICQ:7615664MSN:scra...@hub.org

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Congrats Alvaro!

2010-01-09 Thread Jaime Casanova
2010/1/9 Devrim GÜNDÜZ dev...@gunduz.org:

 Congrats Alvaro for his second daughter !


+1

-- 
Atentamente,
Jaime Casanova
Soporte y capacitación de PostgreSQL
Asesoría y desarrollo de sistemas
Guayaquil - Ecuador
Cel. +59387171157

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Congrats Alvaro!

2010-01-09 Thread Oleg Bartunov

Alvaro !

My best wishes to your wife and children !

Oleg
On Sun, 10 Jan 2010, Devrim G?ND?Z wrote:



Alvaro, one of our hackers and committers and my colleague more than 4
years, had a new baby today.

Congrats Alvaro for his second daughter !

-committers, please commit your patches for our new baby elephant!



Regards,
Oleg
_
Oleg Bartunov, Research Scientist, Head of AstroNet (www.astronet.ru),
Sternberg Astronomical Institute, Moscow University, Russia
Internet: o...@sai.msu.su, http://www.sai.msu.su/~megera/
phone: +007(495)939-16-83, +007(495)939-23-83

--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Initial refactoring of plperl.c - updated

2010-01-09 Thread Tom Lane
Tim Bunce tim.bu...@pobox.com writes:
 On Sat, Jan 09, 2010 at 11:16:18PM +0200, Peter Eisentraut wrote:
 What's the reason for the temp file here?

 Defensive. If the text2macro.pl program fails/dies then you'd be left
 with a broken output file with a newer timestamp, so the next make
 wouldn't rerun text2macro.pl.

Doesn't make forcibly remove the target file if the command fails?

[ rummages in manual... ]  It does if you specify .DELETE_ON_ERROR,
which we do (in Makefile.global); so this bit of complication is
a waste.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] win32 socket definition

2010-01-09 Thread Tom Lane
Magnus Hagander mag...@hagander.net writes:
 On Wed, Jan 6, 2010 at 22:42, Tom Lane t...@sss.pgh.pa.us wrote:
 Can't think of one, but you could try grepping for the socket-related
 syscalls to see what variables are referenced there.

 Found two more by going over it again that way.

 Unless there are objections, I will apply this version tomorrow.

There's another copy of ListenSocket[] in the BackendParameters struct.
I also wonder about postmaster.c's habit of using -1 for empty slots
in ListenSocket ... how safe is that for Win64?

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] Feature patch 1 for plperl [PATCH]

2010-01-09 Thread Tom Lane
Andrew Dunstan and...@dunslane.net writes:
 Peter Eisentraut wrote:
 I think it would help if you could split this up into about 6 to 10
 single-feature patches.

 ... having to handle 6 to 10 patches all hitting the same body of code 
 doesn't sound terrible pleasant either.

Indeed.  That sounds like rather a mess.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Robert Treat
On Saturday 09 January 2010 16:32:29 Robert Haas wrote:
 On Sat, Jan 9, 2010 at 4:01 PM, Peter Eisentraut pete...@gmx.net wrote:
  On lör, 2010-01-09 at 14:12 -0500, Robert Haas wrote:
  If we accept large patches at the very end of the development cycle,
  that's when people will submit them.  You've previously criticized the
  high proportion of the release cycle that is set aside for CommitFest
  and beta, so I'm surprised to see you advocating for a policy that
  seems likely to lengthen the time for which the tree is closed.
 
  Just to clarify: I am for sticking to the agreed dates.  If some things
  are not ready by the necessary date plus/minus one, they won't make the
  release.  If it's obvious earlier that something won't make the date, it
  shouldn't be committed, and maybe put on the backburner right now.  But
  AFAICT, that's independent of when it was submitted.  Some things that
  were submitted just the other day might be almost ready, some things
  that were first submitted many months ago are still not ready.

 The portion of the schedule I'm worried about is the one where we go
 to beta by March 7th.

 http://archives.postgresql.org/pgsql-hackers/2009-09/msg01251.php

 I think we can get all the remaining large patches committed by
 February 15th if Tom doesn't start working on the remaining open items
 until February 15th - but then I do not think that we will have a beta
 on March 7th.


1) The goal of any CF should not be that everything submitted gets committed, 
but that everything gets reviewed.

2) ISTM what had the most agreement was that the large/ugly patches that show 
up late should have no expectation of being committed (though we will try to 
review them), and also that they would be first in line for being punted should 
we start missing dates.

3) Our biggist problem wrt oscillating between a time based and feature based 
release has not been releasing the software, but on never even settling on a 
feature set to get into beta with. 

Given that, if we follow the normal CF process, by Feb 15 we should have a 
clear indication of what is and is not ready for commit, and my suspicion is 
that those large patches you are most worried about will have taken care of 
themselves (one way or the other)

But I don't see much sense in worrying about it now; the 2 weeks between end 
of CF and Beta are when we need to be cut-throat. Given that this time the 
must-have feature is already in the tree, I think you will find people coming 
around quickly to the side of pushing things out rather than fighting to get 
things in.  

Just my .02

-- 
Robert Treat
Conjecture: http://www.xzilla.net
Consulting: http://www.omniti.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Tom Lane
Robert Treat xzi...@users.sourceforge.net writes:
 ... I don't see much sense in worrying about it now; the 2 weeks between end 
 of CF and Beta are when we need to be cut-throat. Given that this time the 
 must-have feature is already in the tree, I think you will find people 
 coming 
 around quickly to the side of pushing things out rather than fighting to get 
 things in.  

I think the other Robert's main point is that getting to beta in only
two weeks is ridiculously optimistic (which I'm afraid I agree with).
I believe that what he's proposing is tossing enough stuff overboard
so that we can finish the January CF in much less than a month, and
thereby have more time for alpha-level testing and stabilization of
the tree.

Now the other approach we could take is that we'll ship *something*
on 7 Mar, even if it's less stable than what we've traditionally
considered to be beta quality.  I don't think that really helps
much though; it just means we need more time in beta.

regards, tom lane

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] damage control mode

2010-01-09 Thread Robert Treat
On Sunday 10 January 2010 01:38:07 Tom Lane wrote:
 Robert Treat xzi...@users.sourceforge.net writes:
  ... I don't see much sense in worrying about it now; the 2 weeks between
  end of CF and Beta are when we need to be cut-throat. Given that this
  time the must-have feature is already in the tree, I think you will
  find people coming around quickly to the side of pushing things out
  rather than fighting to get things in.

 I think the other Robert's main point is that getting to beta in only
 two weeks is ridiculously optimistic (which I'm afraid I agree with).
 I believe that what he's proposing is tossing enough stuff overboard
 so that we can finish the January CF in much less than a month, and
 thereby have more time for alpha-level testing and stabilization of
 the tree.


I agree with your summary, although I'm not sure it needs to be supported at 
this point. While it hasn't been stated explicitly, I suspect most reviewers 
will be reviewing with the idea of is there any chance that this is ready for 
commit in the back of thier heads, and things that aren't will likely get 
pushed off quickly.

 Now the other approach we could take is that we'll ship *something*
 on 7 Mar, even if it's less stable than what we've traditionally
 considered to be beta quality.  I don't think that really helps
 much though; it just means we need more time in beta.


There are three reasons I'd probably be comfortable with that; 1) the CF 
process means we've likely had more eyes on the code going in than in past 
releases. 2) the alpha releases mean we should have already had more review 
than in previous releases. 3) so far we're still looking good on pg_migrator, 
which should make it easier for people to test the release once we get into 
beta (which should help speed that cycle up). 

But really if beta slips because we don't like the looks of our open issues 
list, thats signicantly better than the last couple releases where we held 
everything up just to get things into CVS months after feature freeze had 
passed us by. 

-- 
Robert Treat
Conjecture: http://www.xzilla.net
Consulting: http://www.omniti.com

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers