Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
On lör, 2010-05-01 at 17:26 -0400, Bruce Momjian wrote: I am unclear why it would be in /bin if it requires 15 steps to run and is run only once by only some users. It seems natural for /contrib, like pgcrypto. Well, pg_resetxlog is also rarely run by most users. It started in contrib but was later moved to bin in order to show that it is fully supported. Also, I think the 15 steps are a bit inflated. Several of those steps are about building and installing various pieces of software. If you count that way, using PostgreSQL itself might also require about 12 steps. In a packaged environment that allows side-by-side installation of major versions (such as Debian or Windows), you need about 4 or 5 manual steps, and with a small script layer you need only 1 or 0. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] XML Todo List
Hackers, I'm interested in tackling some of the todo items in XML category. Being new to postgres hacking I'm hoping I chose an item that isn't more than I can chew in the first sitting. One item that has caught my eye that I (naively) hope isn't a huge todo is: xpath_exists() is needed. It checks, whether or not the path specified exists in the XML value. (W/o this function we need to use weird array_dims(xpath(...)) IS NOT NULL syntax.) Is any one else working on the XML todos who might have some friendly pointers to help me on my way or am I just better off getting some code together for general review? Thanks, -- Mike Fowler Registered Linux user: 379787 I could be a genius if I just put my mind to it, and I, I could do anything, if only I could get 'round to it -PULP 'Glory Days' -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Further Hot Standby documentation required
Recent changes to parameters aren't yet sufficiently well documented and I'd like to see this improved by the authors of those patches. I accept the behaviour changes, but we need full docs to explain them. * hot_standby doesn't mention that wal_level = hot_standby is also required, nor is there a xref. * wal_level = 'hot_standby' doesn't mention that the second parameter also needs to be set, nor is there a xref. * wal_level doesn't describe what the impacts are on a standby if the level is changed on the primary, nor is there a caution or a warning of any kind. For example, if a standby is setup with hot_standby = on and the primary is set wal_level = archive, does the standby start working if the primary changes wal_level = hot_standby? What happens if the primary is set wal_level = hot_standby and is then changed to archive? * wal_level doesn't explicitly describe that the levels are in sequence and that hot_standby is a superset of archive. The comment so at least archive level must be used to enable streaming replication. can be misinterpreted to mean that hot_standby level cannot be used with SR, unless the at least archive level is successfully interpreted with the understanding that minimal archive hot_standby. So would like to see that made explicit. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] TOAST code ignores freespace (was Tweak TOAST code)
On Wed, 2009-07-22 at 01:21 +, Tom Lane wrote: Tweak TOAST code so that columns marked with MAIN storage strategy are not forced out-of-line unless that is necessary to make the row fit on a page. Previously, they were forced out-of-line if needed to get the row down to the default target size (1/4th page). A comment from Selena made me notice this patch from last year. I notice that this patch might result in longer rows in the heap, which in many cases is good. For updates, it can result in a row too large to fit on the current block and for us to move block to one in which it will fit. Not commenting further on that patch, but I notice that when we UPDATE the toasting algorithm takes no account of the available freespace on the current block. If we are updating and the space available would make a difference to the row length chosen, it seems like it would be more beneficial to trim the row and encourage HOT updates. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: pg_start_backup and pg_stop_backup Re: [HACKERS] Re: [COMMITTERS] pgsql: Make CheckRequiredParameterValues() depend upon correct
On Fri, 2010-04-30 at 13:41 -0500, Kevin Grittner wrote: Heikki Linnakangas heikki.linnakan...@enterprisedb.com wrote: Yeah, min_wal_segments or something would make sense. Surely it would confuse people to see they have fewer than min_wal_segments WAL segments. That does sound like a reasonable argument, though it also applies to wal_keep_segments, so isn't an argument either way. The user will be equally confused to see fewer WAL files than they have asked to keep. min_wal_segments is much clearer, IMHO. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] standbycheck was:(Re: [HACKERS] testing hot standby
On Sat, 2010-05-01 at 09:05 -0500, Jaime Casanova wrote: maybe we should be using the tables that exists in the regression database or adding hs_setup_primary in installcheck to prepare the regression database to run standbycheck in the standby server This can definitely use some improvement though not yet. I've documented what's there a little better and fixed up the test results. Thanks for your input in this area. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Robert Haas wrote: On Sat, May 1, 2010 at 5:46 PM, Bruce Momjian br...@momjian.us wrote: Agreed, we're not holding up 9.0 for it. ?I think the main bit of work that would be needed to put it into contrib would be to SGML-ize the docs. ?Don't know if Bruce has got the time to get that done. Creating the SGML docs is trivial, especially compared to the 9.0 release notes SGML. ?;-) ?It will take only an hour --- I am basically going to merge the README and the INSTALL file, remove mentions about migrating to 9.0, and add SGML markup. ?I labored on README and the INSTALL files for a long time and can't figure out how to improve them. Oh, and I will remove the C code that was used to migrate _to_ pre-9.0 databases. ?People can use the pgfoundry version for such cases. I have specifically not created a pgfoundry release of pg_migrator that migrates to 9.0. ?(It worked for the 8.5 numbering.) I wonder if this is just going to lead to us maintaining two versions of pg_migrator, which wouldn't be awesome. I don't think it's going to be practical to retain all the migration code for every pair of versions forever, but I'm reluctant to start changing things just because we're sucking the thing into our main tree. Especially things that sound suspiciously like features. Tom's idea basically was that each version of pg_migrator would only support it current major version as a _target_. We would have to backpatch fixes to pg_migrator to previous major versions, and to pgfoundry if necessary. However, there isn't much code churn in pg_migrator anymore (as there was months before), so we should be OK. We already do such backpatching for all our other core code. I can easily keep all the code in each version of pg_migrator. However, pg_dump only supports loads into the _target_ major version, just like pg_migrator would do. I am unclear on which direction to take, but both are easy for me. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TOAST code ignores freespace (was Tweak TOAST code)
Simon Riggs si...@2ndquadrant.com writes: Not commenting further on that patch, but I notice that when we UPDATE the toasting algorithm takes no account of the available freespace on the current block. If we are updating and the space available would make a difference to the row length chosen, it seems like it would be more beneficial to trim the row and encourage HOT updates. That doesn't strike me as a terribly good idea: it would make the behavior of TOAST significantly more difficult to predict. Also, what happens if we force a row to a smaller size and then it doesn't fit anyway (eg because someone else inserted another row on the page while we were busy doing this)? Spend even more cycles to un-toast back to the normal size, to be consistent with ordinary cross-page updates? Pretty much every previous discussion of tweaking the TOAST behavior has focused on giving the user more control (indeed, the patch you mention could be seen as doing that). What you're suggesting here would give the user less control, as well as less predictability. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Andrew Dunstan wrote: Robert Haas wrote: I don't think it's going to be practical to retain all the migration code for every pair of versions forever, I thought the idea was just to support migration from version N to version N+1. Oh, I will also support many older _source_ versions, like 8.3 and 8.4. The question is whether a pg_migrator in /contrib should support multiple _target_ versions. (Again, I assumed this discussion would be necessary, and is best done during beta.) -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] TOAST code ignores freespace (was Tweak TOAST code)
On Sun, 2010-05-02 at 10:34 -0400, Tom Lane wrote: Simon Riggs si...@2ndquadrant.com writes: Not commenting further on that patch, but I notice that when we UPDATE the toasting algorithm takes no account of the available freespace on the current block. If we are updating and the space available would make a difference to the row length chosen, it seems like it would be more beneficial to trim the row and encourage HOT updates. That doesn't strike me as a terribly good idea: it would make the behavior of TOAST significantly more difficult to predict. Also, what happens if we force a row to a smaller size and then it doesn't fit anyway (eg because someone else inserted another row on the page while we were busy doing this)? Spend even more cycles to un-toast back to the normal size, to be consistent with ordinary cross-page updates? Pretty much every previous discussion of tweaking the TOAST behavior has focused on giving the user more control (indeed, the patch you mention could be seen as doing that). What you're suggesting here would give the user less control, as well as less predictability. As long as we've considered it, I'm happy either way. You know I'm happier with more user control. -- Simon Riggs www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Bruce Momjian br...@momjian.us writes: Andrew Dunstan wrote: I thought the idea was just to support migration from version N to version N+1. Oh, I will also support many older _source_ versions, like 8.3 and 8.4. Really? Nobody else has bought into that, and it's not only pg_migrator that would have to go out of its way to support such cases. You're talking about cross-multi-version compatibility of on-disk formats too. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Robert Haas wrote: On Sat, May 1, 2010 at 11:31 PM, Andrew Dunstan and...@dunslane.net wrote: Robert Haas wrote: ?I don't think it's going to be practical to retain all the migration code for every pair of versions forever, I thought the idea was just to support migration from version N to version N+1. I think that would be good enough. But right now we can do better. No reason to rip it out, is there? Again, we are talking about removing _target_ support for 8.4 in pg_migrator 9.0. It is hard to see why someone would use 9.0 /contrib pg_migrator to migrate from 8.3 to 8.4. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Tom Lane wrote: Andrew Dunstan and...@dunslane.net writes: Robert Haas wrote: I don't think it's going to be practical to retain all the migration code for every pair of versions forever, I thought the idea was just to support migration from version N to version N+1. Yeah. I think trying to do more than that is just going to make things messy. For example, we added features to pg_dump and the core server since 8.4 to help pg_migrator do its thing. Trying to make the same pg_migrator code support cases with and without those features available is going to complicate the code, not to mention the documentation, enormously. As a summary, let me list the migrations pg_migrator supports: 8.3 - 8.4 8.4 - 9.0 8.3 - 9.0 Surprisingly, it is 8.3 - 8.4 that has the most restrictions because it doesn't have access to the features we added in Postgres 9.0. Tom is right that the code could be cleaned up if we removed 8.3 - 8.4, but more importantly the documentation would be clearer. To the extent that future bug fixes are relevant to multiple versions of pg_migrator, we could just apply them to multiple branches, same as we manage such fixes for the core code. I don't see that trying to have a single version of pg_migrator is going to make things easier anywhere. Yea, that is probably right. We can enforce in pg_migrator that you can only migrate to the matching major version you are using in /contrib. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Show schema in COPY error CONTEXT strings
Use case: when running a process that populates many inherited
tables across schemas, having one fail gives the unhelpful
error message:
ERROR: invalid input syntax for integer: abc
CONTEXT: COPY foo, line 1, column a: abc
Unhelpful because foo does not uniquely identifies the table
or statement in question, which was actually: COPY alpha.foo FROM STDIN;
where 'alpha' was one of scores of schemas being populated. This
patch changes the output to:
ERROR: invalid input syntax for integer: abc
CONTEXT: COPY alpha.foo, line 1, column a: abc
I had to change the initial table in test/regress/sql/copy2.sql
from a temp table to a real table, as I could not find an easy
way to represent a wild card temp schema name inside of the
test/regres/expected/copy2.out file.
--
Greg Sabino Mullane g...@endpoint.com
End Point Corporation
PGP Key: 0x14964AC8
Index: src/backend/commands/copy.c
===
RCS file: /projects/cvsroot/pgsql/src/backend/commands/copy.c,v
retrieving revision 1.327
diff -c -r1.327 copy.c
*** src/backend/commands/copy.c 28 Apr 2010 16:10:41 - 1.327
--- src/backend/commands/copy.c 2 May 2010 15:40:52 -
***
*** 119,124
--- 119,125
bool *force_notnull_flags;/* per-column CSV FNN flags */
/* these are just for error messages, see copy_in_error_callback */
+ const char *cur_schemaname; /* schema name for error messages */
const char *cur_relname;/* table name for error messages */
int cur_lineno; /* line number for
error messages */
const char *cur_attname;/* current att for error messages */
***
*** 1569,1580
{
/* can't usefully display the data */
if (cstate-cur_attname)
! errcontext(COPY %s, line %d, column %s,
! cstate-cur_relname,
cstate-cur_lineno,
! cstate-cur_attname);
else
! errcontext(COPY %s, line %d,
! cstate-cur_relname,
cstate-cur_lineno);
}
else
{
--- 1570,1582
{
/* can't usefully display the data */
if (cstate-cur_attname)
! errcontext(COPY %s.%s, line %d, column %s,
! cstate-cur_schemaname,
cstate-cur_relname,
! cstate-cur_lineno,
cstate-cur_attname);
else
! errcontext(COPY %s.%s, line %d,
! cstate-cur_schemaname,
cstate-cur_relname,
! cstate-cur_lineno);
}
else
{
***
*** 1584,1600
char *attval;
attval = limit_printout_length(cstate-cur_attval);
! errcontext(COPY %s, line %d, column %s: \%s\,
! cstate-cur_relname,
cstate-cur_lineno,
! cstate-cur_attname, attval);
pfree(attval);
}
else if (cstate-cur_attname)
{
/* error is relevant to a particular column, value is
NULL */
! errcontext(COPY %s, line %d, column %s: null input,
! cstate-cur_relname,
cstate-cur_lineno,
! cstate-cur_attname);
}
else
{
--- 1586,1602
char *attval;
attval = limit_printout_length(cstate-cur_attval);
! errcontext(COPY %s.%s, line %d, column %s: \%s\,
! cstate-cur_schemaname,
cstate-cur_relname,
! cstate-cur_lineno,
cstate-cur_attname, attval);
pfree(attval);
}
else if (cstate-cur_attname)
{
/* error is relevant to a particular column, value is
NULL */
! errcontext(COPY %s.%s, line %d, column %s: null input,
! cstate-cur_schemaname,
cstate-cur_relname,
! cstate-cur_lineno,
cstate-cur_attname);
}
else
{
***
*** 1604,1611
char *lineval;
lineval =
limit_printout_length(cstate-line_buf.data);
! errcontext(COPY %s, line %d: \%s\,
! cstate-cur_relname,
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Andrew Dunstan wrote: I thought the idea was just to support migration from version N to version N+1. Oh, I will also support many older _source_ versions, like 8.3 and 8.4. Really? Nobody else has bought into that, and it's not only pg_migrator that would have to go out of its way to support such cases. You're talking about cross-multi-version compatibility of on-disk formats too. Well, it works. I have a test suite that I run regularly. Because of the way pg_migrator works it is pretty painless to support multiple _source_ major versions. The binary format issue is relevant, but until we have a way to remove the old binary format, I don't see much value in supporting just one source version. For example, we don't have any system now to remove the HEAP_MOVED_OFF and HEAP_MOVED_IN heap bits so effectively major versions have to support them forever. Now, if we develop a system where a version would _remove_ the old data format, we would then specify that pg_migrator can only migrate _from_ one major version, and you would have to run a script to remove the old data format. For example, migrating from 9.0 to 9.2 would requiring migrating from 9.0 to 9.1 with pg_migrator, updating the data pages to 9.1 format, then using pg_migrator again to migrate from 9.1 to 9.2, but of course, we are not there yet. My guess is that when that happens we would just document/enforce it in pg_migrator, but I don't see why we would arbitrarily restrict pg_migrator at this time. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Show schema in COPY error CONTEXT strings
Greg Sabino Mullane g...@turnstep.com writes: Use case: when running a process that populates many inherited tables across schemas, having one fail gives the unhelpful error message: ERROR: invalid input syntax for integer: abc CONTEXT: COPY foo, line 1, column a: abc Unhelpful because foo does not uniquely identifies the table or statement in question, which was actually: COPY alpha.foo FROM STDIN; where 'alpha' was one of scores of schemas being populated. This patch changes the output to: ERROR: invalid input syntax for integer: abc CONTEXT: COPY alpha.foo, line 1, column a: abc We're really not going to address this type of complaint on a one-error-message-at-a-time basis. See prior discussions --- a more realistic (and standards compliant) approach will probably involve adding fields to the verbose form of the error message. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
On May 2, 2010, at 12:01 PM, Bruce Momjian br...@momjian.us wrote: Tom Lane wrote: Bruce Momjian br...@momjian.us writes: Andrew Dunstan wrote: I thought the idea was just to support migration from version N to version N+1. Oh, I will also support many older _source_ versions, like 8.3 and 8.4. Really? Nobody else has bought into that, and it's not only pg_migrator that would have to go out of its way to support such cases. You're talking about cross-multi-version compatibility of on-disk formats too. Well, it works. I have a test suite that I run regularly. Because of the way pg_migrator works it is pretty painless to support multiple _source_ major versions. The binary format issue is relevant, but until we have a way to remove the old binary format, I don't see much value in supporting just one source version. For example, we don't have any system now to remove the HEAP_MOVED_OFF and HEAP_MOVED_IN heap bits so effectively major versions have to support them forever. Now, if we develop a system where a version would _remove_ the old data format, we would then specify that pg_migrator can only migrate _from_ one major version, and you would have to run a script to remove the old data format. For example, migrating from 9.0 to 9.2 would requiring migrating from 9.0 to 9.1 with pg_migrator, updating the data pages to 9.1 format, then using pg_migrator again to migrate from 9.1 to 9.2, but of course, we are not there yet. My guess is that when that happens we would just document/enforce it in pg_migrator, but I don't see why we would arbitrarily restrict pg_migrator at this time. Yeah. It's not uncommon to want to upgrade by more than one release at a time, and I haven't heard any reason why we should arbitrarily refuse to support that. Of course we may need to do that eventually for some specific reason, but it seems like we should only consider imposing such a restriction in the face of a tangible need. ...Robert -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Robert Haas wrote: My guess is that when that happens we would just document/enforce it in pg_migrator, but I don't see why we would arbitrarily restrict pg_migrator at this time. Yeah. It's not uncommon to want to upgrade by more than one release at a time, and I haven't heard any reason why we should arbitrarily refuse to support that. Of course we may need to do that eventually for some specific reason, but it seems like we should only consider imposing such a restriction in the face of a tangible need. Yea, we will need it one day, and if pg_migrator was more tied to system table changes and such, it would be smart to do it now, but if you look at the C code you will see that most of the effort is related to compatibility with the _target_ major version, not the _source_ major version, and by definition, the source major version never changes as we release new major versions. (Remember, pg_dump already does the heavy lifting of moving our database schema to the new major version.) A lot of understanding pg_migrator is understanding the source/target matrix of compatibility --- something we as a community have not thought about very much at this level. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Robert Haas robertmh...@gmail.com writes: Yeah. It's not uncommon to want to upgrade by more than one release at a time, and I haven't heard any reason why we should arbitrarily refuse to support that. Of course we may need to do that eventually for some specific reason, but it seems like we should only consider imposing such a restriction in the face of a tangible need. I wasn't suggesting that we should arbitrarily refuse to support cases that would work otherwise. What I *was* saying is that the community has not bought into doing any extra work to support cross-multiple-version migrations, and I don't think it will do so. It would be a mistake to give users the impression that such cases can be expected to work in future. In particular we should not provide a documentation table that is set up to give the impression that multi-version upgrades are going to be supported. The docs should be written to make it clear that one-version-at-a-time is the normal case, and any cases where you can take a shortcut should be documented as exceptions. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: Yeah. It's not uncommon to want to upgrade by more than one release at a time, and I haven't heard any reason why we should arbitrarily refuse to support that. Of course we may need to do that eventually for some specific reason, but it seems like we should only consider imposing such a restriction in the face of a tangible need. I wasn't suggesting that we should arbitrarily refuse to support cases that would work otherwise. What I *was* saying is that the community has not bought into doing any extra work to support cross-multiple-version migrations, and I don't think it will do so. It would be a mistake to give users the impression that such cases can be expected to work in future. In particular we should not provide a documentation table that is set up to give the impression that multi-version upgrades are going to be supported. The docs should be written to make it clear that one-version-at-a-time is the normal case, and any cases where you can take a shortcut should be documented as exceptions. Well, that is going to make the documentation more complicated than it already is. Why mention a process in 9.0 that no one needs to use? I am not writing the docs for some hypothetical release, but for 9.0. When we have some restriction, we can document that. My guess is that 9.2 will be able to upgrade from 8.3, 8.4, 9.0, and 9.1, but that going to 9.5 will require you to go to 9.2 first, run some script, then upgrade to 9.5; again all hypothetical. I think we can require people using pg_migrator to read matching documentation for that major version of pg_migrator, and pg_migrator will enforce any restrictions we come up with in the future. For example, I assume there will be some major version of Postgres where pg_migrator will not work at all. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Peter Eisentraut wrote: On l?r, 2010-05-01 at 17:26 -0400, Bruce Momjian wrote: I am unclear why it would be in /bin if it requires 15 steps to run and is run only once by only some users. It seems natural for /contrib, like pgcrypto. Well, pg_resetxlog is also rarely run by most users. It started in contrib but was later moved to bin in order to show that it is fully supported. Yea, it is like pg_resetxlog. The only counter-argument I can think of is that we decided that everyone should have the pg_resetxlog binary available, and not have to scramble around to install it in case of a problem, but I will admit that is a thin argument. Also, I think the 15 steps are a bit inflated. Several of those steps are about building and installing various pieces of software. If you count that way, using PostgreSQL itself might also require about 12 steps. In a packaged environment that allows side-by-side installation of major versions (such as Debian or Windows), you need about 4 or 5 manual steps, and with a small script layer you need only 1 or 0. Well, that is good news. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Further Hot Standby documentation required
Simon Riggs wrote: Recent changes to parameters aren't yet sufficiently well documented and I'd like to see this improved by the authors of those patches. I accept the behaviour changes, but we need full docs to explain them. * hot_standby doesn't mention that wal_level = hot_standby is also required, nor is there a xref. * wal_level = 'hot_standby' doesn't mention that the second parameter also needs to be set, nor is there a xref. * wal_level doesn't describe what the impacts are on a standby if the level is changed on the primary, nor is there a caution or a warning of any kind. For example, if a standby is setup with hot_standby = on and the primary is set wal_level = archive, does the standby start working if the primary changes wal_level = hot_standby? What happens if the primary is set wal_level = hot_standby and is then changed to archive? * wal_level doesn't explicitly describe that the levels are in sequence and that hot_standby is a superset of archive. The comment so at least archive level must be used to enable streaming replication. can be misinterpreted to mean that hot_standby level cannot be used with SR, unless the at least archive level is successfully interpreted with the understanding that minimal archive hot_standby. So would like to see that made explicit. I totally agree. Also, there is the new requirement to set wal_level for PITR, which is not documented as a necessary setup step yet. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Bruce Momjian wrote: Well, that is going to make the documentation more complicated than it already is. Why mention a process in 9.0 that no one needs to use? I am not writing the docs for some hypothetical release, but for 9.0. When we have some restriction, we can document that. My guess is that 9.2 will be able to upgrade from 8.3, 8.4, 9.0, and 9.1, but that going to 9.5 will require you to go to 9.2 first, run some script, then upgrade to 9.5; again all hypothetical. I think we can require people using pg_migrator to read matching documentation for that major version of pg_migrator, and pg_migrator will enforce any restrictions we come up with in the future. For example, I assume there will be some major version of Postgres where pg_migrator will not work at all. I do think we are going to have to mention new pg_migrator restrictions in the major release notes for future versions. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Bruce Momjian wrote: For example, I assume there will be some major version of Postgres where pg_migrator will not work at all. We need to be thinking more now about such a contingency. Postgres use in very large installations is now at such a level that requiring a pg_dump/pg_restore is really not an option for many users. If pg_migrator is not always going to work then we need to be addressing that now, or else it is at best a stop-gap. ISTM some sort of page layout versioning scheme might be at least part of what we'll need in the medium term. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] pg_migrator to /contrib in a later 9.0 beta
Andrew Dunstan wrote: Bruce Momjian wrote: For example, I assume there will be some major version of Postgres where pg_migrator will not work at all. We need to be thinking more now about such a contingency. Postgres use in very large installations is now at such a level that requiring a pg_dump/pg_restore is really not an option for many users. If pg_migrator is not always going to work then we need to be addressing that now, or else it is at best a stop-gap. ISTM some sort of page layout versioning scheme might be at least part of what we'll need in the medium term. Yes, my bet is that when we do need to change the page format, we will either use a conversion-on-read process or some external tool to do the conversion. pg_migrator does have code to do page conversions but it has never been used. If you are using pg_migrator in copy mode, we might be able to do page conversion during the copy, but for link mode, that is not possible. -- Bruce Momjian br...@momjian.ushttp://momjian.us EnterpriseDB http://enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Invalidating dependent views and functions
Robert Haas wrote: On Fri, Apr 30, 2010 at 3:33 AM, Scott Bailey arta...@comcast.net wrote: Proposal: Add an invalid flag to pg_class. Invalid objects would be ignored when doing dependency checks for DDL statements. And an exception would be thrown when an invalid object is called. This is similar to what Oracle does. And most Oracle tools have find and compile invalid objects with a statement like: ALTER VIEW foo RECOMPILE; ALTER PACKAGE bar RECOMPILE BODY; Keep in mind that our implementation is apparently quite different from Oracle's. Of course I have no idea what they do under the hood, but we don't even store the original text of the view. Instead, we store a parsed version of the view text that refers to the target objects logically rather than by name. That has some advantages; for example, you can rename a column in some other table that the view uses, and nothing breaks. You can rename a whole table that is used by the view, and nothing breaks. Even if we added storage for the text of the view, recompiling it might result in some fairly astonishing behavior - you might suddenly be referring to tables or columns that were quite different from the ones you originally targeted, if the old ones were renamed out of the way and new, eponymous ones were added. I'm familiar with the view-dependency-hell problem you mention, having fought with it (succesfully, I'm pleased to say, using a big Perl script to manage things - and also - obligatory dig here - to work around our lack of support for CREATE IF NOT EXISTS) on many occasions, but I don't have any brilliant ideas about how to solve it. I would like to eventually support ALTER VIEW ... DROP COLUMN; note that we do now support ADDING columns to a view using CREATE OR REPLACE as long as all the new ones are at the end. But neither of those things is going to help with a case like yours, when you want to change the type of the column. I'm not really sure what to do about that case. ...Robert I've been using the source in information_schema.views rather than storing the original source. Oracle does indeed store the original source code for your objects. I don't know what they use to recompile. But my inclination is that they use the original source. If you alter a table/column name I believe it will invalidate any dependent views which will need manually edited before they will compile successfully. As for Oracle's approach being stupid and not user friendly, OK, maybe they could automatically try to recompile. But even a manual process is better than no help at all. Scott -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
