Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
> "Kevin Grittner" wrote: > Tom Lane wrote: > >> Yeah, it would definitely be nicer if BEGIN; SET TRANSACTION LEVEL >> would work too. Maybe the place to put the check is where we >> establish the transaction snapshot. > > That makes sense, > I'll take a shot at it sometime today, Attached. With a default setting in postgresql.conf it handles Robert's test case nicely, and does what Tom asks for above: test=# set default_transaction_isolation = 'serializable'; SET test=# begin; BEGIN test=# select 1; ERROR: can not create a serializable snapshot during recovery test=# rollback; ROLLBACK test=# begin; BEGIN test=# set transaction isolation level repeatable read; SET test=# select 1; ?column? -- 1 (1 row) test=# commit; COMMIT But if you set it in the postgresql.conf file, it's not pretty: kevin@kevin-desktop:~$ psql -p 5433 test psql: FATAL: can not create a serializable snapshot during recovery Ideas? -Kevin hs-default-serializable-v1.diff Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plpython crash (PG 92)
Peter Eisentraut writes: > On tor, 2012-04-26 at 17:32 +0500, Asif Naeem wrote: >> PFA test case. It used simple select statement to retrieve data via >> plpython. It crashes latest pg 9.2 with the following stack trace i.e. >> Apparently it is being crashed because of invalid related pointer value of >> pfree() *header->context->methods->free_p. It is reproducible with latest >> version of python i.e. Python-2.7.3 and Python-3.2.3. Thanks. > This is because of this code: I traced through this test case. The situation is: (1) PLy_spi_execute_fetch_result is executed with CurrentMemoryContext being the SPI Proc context for the current SPI invocation, so that's where CreateTupleDescCopy creates the tupledesc that's placed into the PLyResultObject. (2) As we fall out of the SETOF function with the first result row, PLy_exec_function does SPI_finish() (plpy_exec.c:146). That causes the SPI contexts to go away. The PLyResultObject's tupdesc pointer is now pointing at freed memory, and in a cassert build, that memory has been actively wiped. (3) The main executor calls back to plpython for the next value-per-call result. Control goes to the PyIter_Next call at plpy_exec.c:108, which decides that we're done iterating (since this example only returns one row), and evidently it tries to deallocate the PLyResultObject immediately. Whether that happens immediately or later, though, you're screwed because FreeTupleDesc will be invoked on garbage. I'm inclined to think that the best fix is for PLy_spi_execute_fetch_result to copy the tupledesc into TopMemoryContext, not the current context. This is a tad scary from a memory leakage standpoint, but I suppose that if python fails to recover the PLyResultObject, this isn't the only memory that's going to be leaked. This area appears to be shy a regression test case or two, in any event. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
On Apr 27, 2012, at 21:24, Tom Lane wrote: > Bruce Momjian writes: >> On Fri, Apr 27, 2012 at 09:10:54PM -0400, Tom Lane wrote: >>> How would that help? The bottleneck is packaging, which is the same >>> work whatever we call it. > >> We release the alpha with no packaging. > > Think we'd lose a lot of potential testers that way. > It seems that even if you published before PGCon a reasonable number of possible testers would be there and/or responding to any feedback is going to be delayed since developers are going to be there. I could see the goal being having a package-ready commit before PGCon but schedule the official release until after. No matter what you call it those who want a head start can self-compile while others will just wait until the packagers are done. Either way the same codebase will be in the wild (so probably don't want to call it alpha). The original reason for pre-conference is so the developers can feel less bad talking about 9.3 features (and 9.2 post-mortem) since the beta for 9.2 will be completed. David J. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
Bruce Momjian writes: > On Fri, Apr 27, 2012 at 09:10:54PM -0400, Tom Lane wrote: >> How would that help? The bottleneck is packaging, which is the same >> work whatever we call it. > We release the alpha with no packaging. Think we'd lose a lot of potential testers that way. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
On Fri, Apr 27, 2012 at 09:10:54PM -0400, Tom Lane wrote: > Bruce Momjian writes: > > On Fri, Apr 27, 2012 at 07:19:55PM -0400, Tom Lane wrote: > >> Ugh. Maybe the whole idea of getting a beta out before PGCon is doomed. > >> Still, if we don't try for this schedule, we're looking at at least two > >> more weeks' slip, because we're surely not going to wrap during PGCon. > > > Should we go for an alpha? > > How would that help? The bottleneck is packaging, which is the same > work whatever we call it. We release the alpha with no packaging. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
Bruce Momjian writes: > On Fri, Apr 27, 2012 at 07:19:55PM -0400, Tom Lane wrote: >> Ugh. Maybe the whole idea of getting a beta out before PGCon is doomed. >> Still, if we don't try for this schedule, we're looking at at least two >> more weeks' slip, because we're surely not going to wrap during PGCon. > Should we go for an alpha? How would that help? The bottleneck is packaging, which is the same work whatever we call it. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xReader, double-effort
On 04/27/2012 06:48 PM, Kevin Grittner wrote: That's largely my fault. It may not have come out that way, but I was trying more to point out the complexity of the story rather than assign blame. I find it hard to point at any obvious "don't do that again" target here. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
> Do we want to do the normal package Thursday, release Monday process? I > assumed we did. Given that in the last 2 years, our betas have been "news", I think we need to. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
> Ugh. Maybe the whole idea of getting a beta out before PGCon is doomed. > Still, if we don't try for this schedule, we're looking at at least two > more weeks' slip, because we're surely not going to wrap during PGCon. We could do it in person! -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
On Fri, Apr 27, 2012 at 07:19:55PM -0400, Tom Lane wrote: > Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= writes: > > (As a side note, RPMs *may not* be ready, because I (and Magnus) will be > > at PGDay Turkey on 12th, and will be busy over the whole weekend). > > Ugh. Maybe the whole idea of getting a beta out before PGCon is doomed. > Still, if we don't try for this schedule, we're looking at at least two > more weeks' slip, because we're surely not going to wrap during PGCon. Should we go for an alpha? -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
Devrim =?ISO-8859-1?Q?G=DCND=DCZ?= writes: > (As a side note, RPMs *may not* be ready, because I (and Magnus) will be > at PGDay Turkey on 12th, and will be busy over the whole weekend). Ugh. Maybe the whole idea of getting a beta out before PGCon is doomed. Still, if we don't try for this schedule, we're looking at at least two more weeks' slip, because we're surely not going to wrap during PGCon. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
Hi, On Fri, 2012-04-27 at 19:02 -0400, Tom Lane wrote: > The website crew might have a problem though. Magnus is flying on 13th and 14th, Dave, Stefan and Thom are flying on 14th. I'm flying on 15th -- it seems I have some cycles for updating website, I hope. Not sure about the others. (As a side note, RPMs *may not* be ready, because I (and Magnus) will be at PGDay Turkey on 12th, and will be busy over the whole weekend). Regards, -- Devrim GÜNDÜZ Principal Systems Engineer @ EnterpriseDB: http://www.enterprisedb.com PostgreSQL Danışmanı/Consultant, Red Hat Certified Engineer Community: devrim~PostgreSQL.org, devrim.gunduz~linux.org.tr http://www.gunduz.org Twitter: http://twitter.com/devrimgunduz signature.asc Description: This is a digitally signed message part
Re: [HACKERS] 9.2 release notes, beta time?
Josh Berkus writes: > On 4/27/12 2:29 PM, Bruce Momjian wrote: >> Someone said one week as not enough, so I assumed a wrap on May 10 with >> beta release on May 14. > Doing a release on May 14 will be a challenge, thanks to people being on > airplanes to pgCon. We might want to verify availability from the > packagers. Packagers wouldn't be the issue; their work should be done by the Saturday at latest. The website crew might have a problem though. The main gating factor here is whether Dave Page's crew can get the Windows installers ready any sooner; given his previous comments, I don't think that moving the wrap up a few days would change anything. We could wrap next week if we were OK with not having Windows installers at the announcement, but I bet you'll say that's a no-go. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)
Andres Freund wrote: > Something like registering/deregistering also doesn't fit that > well with the way walsender works as far as I understand it. If you look at the diagrams on the xReader Wiki page, the lines labeled "XLOG stream" are the ones using walsender/walreceiver. The green arrows represent normal connections to the database, to run queries to retrieve metadata needed to interpret the WAL records, and the lines labeled "Listener n" are expected to use the pg protocol to connect, but won't be talking page-oriented WAL -- they will be dealing with logical interpretation of the WAL. The sort of data which could be fed to a database which doesn't have the same page images. Like Slony et al do. Perhaps, given other points you made, the library for interpreting the WAL records could be shared, and hopefully a protocol for the clients, although that seems a lot more muddy to me at this point. If we can share enough code, there may be room for both approaches with minimal code duplication. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plpython crash (PG 92)
Peter Eisentraut writes: > I must have been confused about the tuple descriptor APIs. ob->tupdesc > is created using CreateTupleDescCopy(), which copies the refcount of the > original tuple descriptor, Um, surely not. That would be nonsensical, and anyway a look at the code shows it isn't doing that. > thus causing a failure when the (seemingly still referenced) tupdesc > is freed. Is this behavior correct and useful? I think there must be some mistake in your analysis. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Future In-Core Replication
On Fri, Apr 27, 2012 at 4:11 AM, Simon Riggs wrote: > What I'm hoping to do is to build a basic prototype of logical > replication using WAL translation, so we can inspect it to see what > the downsides are. It's an extremely non-trivial problem and so I > expect there to be mountains to climb. There are other routes to > logical replication, with messages marshalled in a similar way to > Slony/Londiste/Bucardo/Mammoth(?). So there are options, with > measurements to be made and discussions to be had. I'll note that the latest version of Slony, expected to be 2.2 (which generally seems to work, but we're stuck at the moment waiting to get free cycles to QA it) has made a substantial change to its data representation. The triggers used to cook data into a sort of "fractional WHERE clause," transforming an I/U/D into a string that you'd trivially combine with the string INSERT INTO/UPDATE/DELETE FROM to get the logical update. If there was need to do anything fancier, you'd be left having to have a "fractional SQL parser" to split the data out by hand. New in 2.2 is that the log data is split out into an array of text values which means that if someone wanted to do some transformation, such as filtering on value, or filtering out columns, they could modify the application-of-updates code to query for the data that they want to fiddle with. No parser needed. It's doubtless worthwhile to take a peek at that to make sure it informs your data representation appropriately. It's important to have data represented in a fashion that is amenable to manipulation, and that decidedly wasn't the case pre-2.2. I wonder if a meaningful transport mechanism might involve combining: a) A trigger that indicates that some data needs to be captured in a "logical" form (rather than the presently pretty purely physical form of WAL) b) Perhaps a way of capturing logical updates in WAL c) One of the old ideas that fell through was to try to capture commit timestamps via triggers. Doing it directly turned out to be too controversial to get in. Perhaps that's something that could be captured via some process that parses WAL. Something seems wrong about that in that it mixes together updates of multiple forms into WAL, physical *and* logical, and perhaps that implies that there should be an altogether separate "logical updates log." (LUL?) That still involves capturing updates in a duplicative fashion, e.g. - WAL + LUL, which seems somehow wrong. Or perhaps I'm tilting at a windmill here. With Slony/Londiste/Bucardo, we're capturing "LUL" in some tables, meaning that it gets written both to the tables' data files as well as WAL. Adding a binary LUL eliminates those table files and attendant WAL updates, thus providing some savings. [Insert a LULCATS joke here...] Perhaps I've just had too much coffee... -- When confronted by a difficult problem, solve it by reducing it to the question, "How would the Lone Ranger handle this?" -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xReader, double-effort
Greg Smith wrote: > On 04/26/2012 12:57 PM, Robert Haas wrote: >> I don't recall seeing any previous postings about the 2Q project, >> either. We should try to keep these discussions on-list. > > First brought up two months ago to the list it seemed most > relevant to at the time: > http://archives.postgresql.org/pgsql-cluster-hackers/2012-02/msg1.php > and the solid commentary started coming out yesterday, right near > the approach of May as promised. There hasn't been much posted on > hackers about this out of respect for the extended CommitFest. > > I personally suggested Aakash vent ideas out on pgsql-students (or > here) at the beginning of the month: > http://archives.postgresql.org/pgsql-students/2012-04/msg2.php > but the next time I heard about the subject was the acceptance > notice. That's largely my fault. Aakash's original proposal was naively assuming that logical records were available in our xlog stream and wanted to work on tools to do something useful with that data. There was some confusion among the various core team members, committers, and other contributors who volunteered to review the proposals about whether he was talking about analyzing the server log or xlog. We had a wide-ranging discussion on the freenode #postgresql channel about it, and brainstormed to the current idea -- at which point Aakash had about a week and a half to rewrite his proposal along the lines discussed on irc. A week after that the acceptance notices came out. If we had had any idea that 2Q was working on something along these lines, we would have coordinated or guided him in another direction. The post you cite from a couple months ago was way too vague to have even an inkling that it had anything to do with providing feeds of the logical interpretation of the WAL stream. I don't recall having heard the idea even mentioned before. The timing in regard to the release process seems to have had an effect on both groups here, since I suggested he read up on walsender and walreceiver and a few other areas, put together a Wiki page with the plan, and *then* describe the preliminary plan to the community (a month or so before starting to code), so that details could be hashed out in public before time was spent coding. Had there not been so many people heads-down trying to get the release out, I might have advised him to post a hand-wavy description of the effort before he got the Wiki page set up and reviewed the relevant code. As Josh points out, his first choice was to write utilities to *do* something with the logical xlog information, and he only switched over to working on *producing* it when he heard that no such thing was currently available. I think there's a lot of room to coordinate these efforts so that there *isn't* a waste of effort as long as communication is open enough. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A problem with dump/restore of views containing whole row references
Abbas Butt writes:
> On Fri, Apr 27, 2012 at 11:21 PM, Tom Lane wrote:
>> More generally, it seems rather inelegant to be forcibly adding a cast
>> when in most cases the existing notation is not wrong. AFAICS the
>> plain "relname" notation is only ambiguous if there is a column of the
>> same name as the relation. I wonder whether we should instead address
>> this by not letting the parser strip the "no op" cast in the first
>> place.
> You mean that the parser should not strip the "no op" cast in all cases or
> in the case only when the parser somehow detects a column of the same name
> as the relation?
On reflection that's the wrong thing anyway. While (AFAICS) one could
only initially create this type of situation by using an explicit cast
as in your example, the ambiguity could be introduced after the fact by
a rename or ALTER TABLE ADD COLUMN, which wouldn't even have to affect
the troublesome table itself --- a column matching the table's name
anywhere in the FROM clause would create the same ambiguity. So there's
no guarantee that there ever was a cast there.
So I think that your patch is the right approach, if wrong in detail.
What we have to do is stop using the ambiguous table-name-only syntax,
and instead always print tabname.*, and then add a cast to prevent
expansion of the "*" if we are at top level of a SELECT targetlist.
Attached is a patch that I think does this correctly. I renamed the
flag parameter (and flipped its sense) since it is no longer controlling
whether or not a "*" gets printed. One thing I like about this is that
whole-row Vars are no longer ever special in terms of naming; looking
at the code with a fresh eye, I wonder whether we didn't have other bugs
here in cases such as where a schema qualification is needed. Omitting
the star is just asking for trouble ...
regards, tom lane
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 3beed37dd26d6fc51a4365b3f4086b97f71cc16c..7ad99a0ec32579760f6e10278ec8585a0b2ab855 100644
*** a/src/backend/utils/adt/ruleutils.c
--- b/src/backend/utils/adt/ruleutils.c
*** static void get_rule_orderby(List *order
*** 209,215
static void get_rule_windowclause(Query *query, deparse_context *context);
static void get_rule_windowspec(WindowClause *wc, List *targetList,
deparse_context *context);
! static char *get_variable(Var *var, int levelsup, bool showstar,
deparse_context *context);
static RangeTblEntry *find_rte_by_refname(const char *refname,
deparse_context *context);
--- 209,215
static void get_rule_windowclause(Query *query, deparse_context *context);
static void get_rule_windowspec(WindowClause *wc, List *targetList,
deparse_context *context);
! static char *get_variable(Var *var, int levelsup, bool istoplevel,
deparse_context *context);
static RangeTblEntry *find_rte_by_refname(const char *refname,
deparse_context *context);
*** get_target_list(List *targetList, depars
*** 3074,3084
* "foo.*", which is the preferred notation in most contexts, but at
* the top level of a SELECT list it's not right (the parser will
* expand that notation into multiple columns, yielding behavior
! * different from a whole-row Var). We want just "foo", instead.
*/
if (tle->expr && IsA(tle->expr, Var))
{
! attname = get_variable((Var *) tle->expr, 0, false, context);
}
else
{
--- 3074,3085
* "foo.*", which is the preferred notation in most contexts, but at
* the top level of a SELECT list it's not right (the parser will
* expand that notation into multiple columns, yielding behavior
! * different from a whole-row Var). We need to call get_variable
! * directly so that we can tell it to do the right thing.
*/
if (tle->expr && IsA(tle->expr, Var))
{
! attname = get_variable((Var *) tle->expr, 0, true, context);
}
else
{
*** get_utility_query_def(Query *query, depa
*** 3803,3815
* the Var's varlevelsup has to be interpreted with respect to a context
* above the current one; levelsup indicates the offset.
*
! * If showstar is TRUE, whole-row Vars are displayed as "foo.*";
! * if FALSE, merely as "foo".
*
! * Returns the attname of the Var, or NULL if not determinable.
*/
static char *
! get_variable(Var *var, int levelsup, bool showstar, deparse_context *context)
{
StringInfo buf = context->buf;
RangeTblEntry *rte;
--- 3804,3823
* the Var's varlevelsup has to be interpreted with respect to a context
* above the current one; levelsup indicates the offset.
*
! * If istoplevel is TRUE, the Var is at the top level of a SELECT's
! * targetlist, which means we need special treatment of whole-row Vars.
! * Instead of the normal "tab.*", we'll print "tab.*::typename", which is a
! * dirty hack to prevent "tab.*" from being expanded into multipl
Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)
Hi Kevin, Hi Aakash, On Saturday, April 28, 2012 12:18:38 AM Kevin Grittner wrote: > Andres Freund wrote: > > In the current, prototypal, state there is one component thats > > integrated into the server (because it needs information thats > > only available there). > The xReader design was based on the idea that it would be nice not > to cause load on the master machine, and that by proxying the WAL > stream to the HS, using synchronous replication style to write from > xReader to the HS, you could use the HS for a source for that data > with it being at exactly the right point in time to query it. Yes, that does make sense for some workloads. I don't think its viable for everything though, thats why were not aiming for that ourselves atm. > I'm not convinced that I would rather see the logic fixed inside the > master as opposed to being deployable on the master's machine, the > slave machine, or even on its own machine in between. I don't think that you can do everything apart from the master. We currently need shared memory for coordination between the moving parts, thats why we have it inside the master. It also have the advantage of being easier to setup. > > That component is layered ontop of a totally generic xlog > > reading/parsing library that doesn't care at all where its > > running. > That's cool. > > Its also used in another cluster to read the received (filtered) > > stream. > I don't quite follow what you're saying there. To interpret the xlog back into something that can be used for replication you need to read it again. After filtering we again write valid WAL, so we can use the same library on the sending|filtering side and on the receiving side. But thats actually off topic for this thread ;) > > I took a *very* short glance over the current wiki description of > > xReader and from that it seems to me it would benefit from trying > > to make it architecturally more similar to the rest of pg. > We're planning on using existing protocol to talk between pieces. > Other than breaking it out so that it can run somewhere other than > inside the server, and allowing clients to connect to xReader to > listen to WAL events of interest, are you referring to anything > else? It sounds like the xReader is designed to be one multiplexing process. While this definitely has some advantages resource-usage-wise it doesn't seem to be fitting the rest of the design that well. The advantages might outweigh everything else, but I am not sure about that. Something like registering/deregistering also doesn't fit that well with the way walsender works as far as I understand it. Greetings, Andres -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Future In-Core Replication
On 04/27/2012 11:33 AM, Simon Riggs wrote: Well, they all sound similar. My info was that Mammoth was not WAL-based. Mammoth was transaction log based but not WAL based. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xReader, double-effort
On 04/26/2012 12:57 PM, Robert Haas wrote: I don't recall seeing any previous postings about the 2Q project, either. We should try to keep these discussions on-list. First brought up two months ago to the list it seemed most relevant to at the time: http://archives.postgresql.org/pgsql-cluster-hackers/2012-02/msg1.php and the solid commentary started coming out yesterday, right near the approach of May as promised. There hasn't been much posted on hackers about this out of respect for the extended CommitFest. I personally suggested Aakash vent ideas out on pgsql-students (or here) at the beginning of the month: http://archives.postgresql.org/pgsql-students/2012-04/msg2.php but the next time I heard about the subject was the acceptance notice. -- Greg Smith 2ndQuadrant USg...@2ndquadrant.com Baltimore, MD PostgreSQL Training, Services, and 24x7 Support www.2ndQuadrant.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)
Andres Freund wrote: > In the current, prototypal, state there is one component thats > integrated into the server (because it needs information thats > only available there). The xReader design was based on the idea that it would be nice not to cause load on the master machine, and that by proxying the WAL stream to the HS, using synchronous replication style to write from xReader to the HS, you could use the HS for a source for that data with it being at exactly the right point in time to query it. I'm not convinced that I would rather see the logic fixed inside the master as opposed to being deployable on the master's machine, the slave machine, or even on its own machine in between. > That component is layered ontop of a totally generic xlog > reading/parsing library that doesn't care at all where its > running. That's cool. > Its also used in another cluster to read the received (filtered) > stream. I don't quite follow what you're saying there. > I plan to submit the XLogReader (thats what its called atm) > before everything else, so everybody can take a look as soon as > possible. Great! That will allow more discussion and planning. > I took a *very* short glance over the current wiki description of > xReader and from that it seems to me it would benefit from trying > to make it architecturally more similar to the rest of pg. We're planning on using existing protocol to talk between pieces. Other than breaking it out so that it can run somewhere other than inside the server, and allowing clients to connect to xReader to listen to WAL events of interest, are you referring to anything else? > I also would suggest reviewing how the current walreceiver/sender, > and their protocol, work. Of course! The first "inch-stone" in the GSoC project plan basically consists of creating an executable that functions as a walreceiver and a walsender to just pass things through from the master to the slave. We build from there by allowing clients to connect (again, over existing protocol) and register for events of interest, and then recognizing different WAL records to generate events. The project was just going to create a simple client to dump the information to disk, but with the time saved by adopting what you've already done, that might leave more time for generating a useful client. Aakash, when you get a chance, could you fill in the "inch-stones" from the GSoC proposal page onto the Wiki page? I think the descriptions of those interim steps would help people understand your proposal better. Obviously, some of the particulars of tasks and the dates may need adjustment based on the new work which is expected to appear before you start, but what's there now would be a helpful reference. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
Robert Haas writes: > It seems we need another signal for the new mode, and the obvious > candidate is SIGUSR2. But what shall the mapping look like? > [Choice #1] SIGUSR2 -> slow, SIGTERM -> smart, SIGINT -> fast, SIGQUIT > -> immediate > [Choice #2] SIGTERM -> slow, SIGUSR2 -> smart, SIGINT -> fast, SIGQUIT > -> immediate SIGTERM needs to correspond to a fairly aggressive shutdown mode, since (at least on some systems) init will send that during the system shutdown sequence, shortly before escalating to SIGKILL. So I think choice #2 is not sensible at all. If we were willing to consider wholesale breakage of any scripts that send these signals directly, I'd almost consider that it should be SIGUSR2, SIGINT, SIGTERM, SIGQUIT. But that might be more churn than we want. Keeping SIGTERM attached to the default/"smart" shutdown mode seems like a reasonable compromise. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
On Fri, Apr 27, 2012 at 02:43:13PM -0700, Josh Berkus wrote: > On 4/27/12 2:29 PM, Bruce Momjian wrote: > > On Fri, Apr 27, 2012 at 02:16:02PM -0700, Josh Berkus wrote: > >> > I suspect that's a bit aggressive. We have a couple of issues that I > think we should fix before beta1: > >> > >> Hmmm. So are we looking at May 7? Or later? > >> > >> I'd like to do some press work for this. > > > > Someone said one week as not enough, so I assumed a wrap on May 10 with > > beta release on May 14. > > Doing a release on May 14 will be a challenge, thanks to people being on > airplanes to pgCon. We might want to verify availability from the > packagers. Do we want to do the normal package Thursday, release Monday process? I assumed we did. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
On 4/27/12 2:29 PM, Bruce Momjian wrote: > On Fri, Apr 27, 2012 at 02:16:02PM -0700, Josh Berkus wrote: >> I suspect that's a bit aggressive. We have a couple of issues that I think we should fix before beta1: >> >> Hmmm. So are we looking at May 7? Or later? >> >> I'd like to do some press work for this. > > Someone said one week as not enough, so I assumed a wrap on May 10 with > beta release on May 14. Doing a release on May 14 will be a challenge, thanks to people being on airplanes to pgCon. We might want to verify availability from the packagers. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)
On Friday, April 27, 2012 11:04:04 PM Kevin Grittner wrote: > [replaced bad email address for Josh (which was my fault)] > > Aakash Goel wrote: > > All, the wiki page is now up at > > > > http://wiki.postgresql.org/wiki/XReader. > > Note that the approach Aakash is taking doesn't involve changes to > the backend code, it is strictly a standalone executable to which > functions as a proxy to a hot standby and to which clients like > replications systems connect. There is a possible additional > configuration which wouldn't require a hot standby, if time permits. > I am not clear on whether 2nd Quadrant's code takes this approach > or builds it into the server. I think we need to know that much > before we can get very far in discussion. In the current, prototypal, state there is one component thats integrated into the server (because it needs information thats only available there). That component is layered ontop of a totally generic xlog reading/parsing library that doesn't care at all where its running. Its also used in another cluster to read the received (filtered) stream. I plan to submit the XLogReader (thats what its called atm) before everything else, so everybody can take a look as soon as possible. I took a *very* short glance over the current wiki description of xReader and from that it seems to me it would benefit from trying to make it architecturally more similar to the rest of pg. I also would suggest reviewing how the current walreceiver/sender, and their protocol, work. Andres -- Andres Freund http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
On Fri, Apr 27, 2012 at 02:16:02PM -0700, Josh Berkus wrote: > > >> I suspect that's a bit aggressive. We have a couple of issues that I > >> think we should fix before beta1: > > Hmmm. So are we looking at May 7? Or later? > > I'd like to do some press work for this. Someone said one week as not enough, so I assumed a wrap on May 10 with beta release on May 14. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Request to add options to tools/git_changelog
On Thu, Apr 26, 2012 at 03:19:04PM -0400, Bruce Momjian wrote:
> Also consider that A is usually the big, clear commit message, and B,C,D
> are just minor adjustments with more brief commits, which might require
> adjusting the release note item for feature A. When they are in
> newest-first order, that is much harder.
Updated, attached patch applied. Thanks.
--
Bruce Momjian http://momjian.us
EnterpriseDB http://enterprisedb.com
+ It's impossible for everything to be true. +
*** /pg/tools/git_changelog 2012-01-01 17:48:55.0 -0500
--- /rtmp/pggitlog 2012-04-27 17:06:19.0 -0400
***
*** 40,48
# Might want to make this parameter user-settable.
my $timestamp_slop = 600;
my $post_date = 0;
my $since;
! Getopt::Long::GetOptions('post-date' => \$post_date,
'since=s' => \$since) || usage();
usage() if @ARGV;
--- 40,57
# Might want to make this parameter user-settable.
my $timestamp_slop = 600;
+ my $details_after = 0;
my $post_date = 0;
+ my $master_only = 0;
+ my $oldest_first = 0;
my $since;
! my @output_buffer;
! my $output_line = '';
!
! Getopt::Long::GetOptions('details-after' => \$details_after,
! 'master-only' => \$master_only,
! 'post-date' => \$post_date,
! 'oldest-first' => \$oldest_first,
'since=s' => \$since) || usage();
usage() if @ARGV;
***
*** 179,195
last if !defined $best_branch;
my $winner =
$all_commits_by_branch{$best_branch}->[$position{$best_branch}];
! printf "Author: %s\n", $winner->{'author'};
! foreach my $c (@{$winner->{'commits'}}) {
! printf "Branch: %s", $c->{'branch'};
! if (defined $c->{'last_tag'}) {
! printf " Release: %s", $c->{'last_tag'};
! }
! printf " [%s] %s\n", substr($c->{'commit'}, 0, 9), $c->{'date'};
}
! print "\n";
! print $winner->{'message'};
! print "\n";
$winner->{'done'} = 1;
for my $branch (@BRANCHES) {
my $leader = $all_commits_by_branch{$branch}->[$position{$branch}];
--- 188,204
last if !defined $best_branch;
my $winner =
$all_commits_by_branch{$best_branch}->[$position{$best_branch}];
!
! # check for master-only
! if (! $master_only || ($winner->{'commits'}[0]->{'branch'} eq 'master' &&
! @{$winner->{'commits'}} == 1)) {
! output_details($winner) if (! $details_after);
! output_str("%s", $winner->{'message'} . "\n");
! output_details($winner) if ($details_after);
! unshift(@output_buffer, $output_line) if ($oldest_first);
! $output_line = '';
}
!
$winner->{'done'} = 1;
for my $branch (@BRANCHES) {
my $leader = $all_commits_by_branch{$branch}->[$position{$branch}];
***
*** 200,205
--- 209,216
}
}
+ print @output_buffer if ($oldest_first);
+
sub push_commit {
my ($c) = @_;
my $ht = hash_commit($c);
***
*** 258,268
return $gm - $tzoffset;
}
sub usage {
print STDERR <{'author'} =~ m{^(.*?)\s*<[^>]*>$};
+ # output only author name, not email address
+ output_str("(%s)\n", $1);
+ } else {
+ output_str("Author: %s\n", $item->{'author'});
+ }
+ foreach my $c (@{$item->{'commits'}}) {
+ output_str("Branch: %s ", $c->{'branch'}) if (! $master_only);
+ if (defined $c->{'last_tag'}) {
+ output_str("Release: %s ", $c->{'last_tag'});
+ }
+ output_str("[%s] %s\n", substr($c->{'commit'}, 0, 9), $c->{'date'});
+ }
+ output_str("\n");
+ }
+
sub usage {
print STDERR <
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] 9.2 release notes, beta time?
>> I suspect that's a bit aggressive. We have a couple of issues that I >> think we should fix before beta1: Hmmm. So are we looking at May 7? Or later? I'd like to do some press work for this. -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] Re: xReader, double-effort (was: Temporary tables under hot standby)
[replaced bad email address for Josh (which was my fault)] Aakash Goel wrote: > All, the wiki page is now up at > http://wiki.postgresql.org/wiki/XReader. Note that the approach Aakash is taking doesn't involve changes to the backend code, it is strictly a standalone executable to which functions as a proxy to a hot standby and to which clients like replications systems connect. There is a possible additional configuration which wouldn't require a hot standby, if time permits. I am not clear on whether 2nd Quadrant's code takes this approach or builds it into the server. I think we need to know that much before we can get very far in discussion. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Fri, Apr 27, 2012 at 1:57 PM, Robert Haas wrote: > I think there is no point at all in having a discussion about this > unless we can first agree that the overwhelming majority of people who > have commented on this issue on this list are unhappy with the current > default behavior. count me in. the current behavior sucks. merlin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xReader, double-effort (was: Temporary tables under hot standby)
All, the wiki page is now up at http://wiki.postgresql.org/wiki/XReader. On Sat, Apr 28, 2012 at 1:19 AM, Aakash Goel wrote: > Sure Kevin, will get the wiki page ready asap, and reply back. Thanks. > > > On Thu, Apr 26, 2012 at 8:10 PM, Kevin Grittner < > kevin.gritt...@wicourts.gov> wrote: > >> [resending because of postgresql.org bounces on first try] >> >> Simon Riggs wrote: >> > Kevin Grittner wrote: >> >> >> The GSoC xReader project is intended to be a major step toward >> >> that, by providing a way to translate the WAL stream to a series >> >> of notifications of logical events to clients which register with >> >> xReader. >> > >> > This is already nearly finished in prototype and will be published >> > in May. Andres Freund is working on it, copied here. >> >> URL? >> >> > It looks like there is significant overlap there. >> >> Hard for me to know without more information. It sounds like there >> is at least some overlap. I hope that can involve cooperation, with >> the efforts of Andres forming the basis of Aakash's GSoC effort. >> That might leave him more time to polish up the user filters. >> >> Aakash: It seems like we need that Wiki page rather sooner than >> later. Can you get to that quickly? I would think that just >> copying the text from your approved GSoC proposal would be a very >> good start. If you need help figuring out how to embed the images >> from your proposal, let me know. >> >> -Kevin >> > >
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Friday, April 27, 2012 10:17:59 PM Peter Eisentraut wrote: > On fre, 2012-04-27 at 20:39 +0200, Andres Freund wrote: > > I think the current smart mode is rather useful. There is quite some > > stuff that you cannot do inside a transaction - or it doesn't make > > sense - which still needs to shutdown gracefully. E.g. transaction > > managers. > Could you elaborate on that? What would happen to the transaction > manager if you terminate any idle, not-in-a-transaction database backend > sessions it has established? In the few cases where I investigated it TMs don't use transactions themselves (which I think is correct, they don't need them), so terminating any idle session - which the TM would appear as, as its not using txns - would leave prepared transactions in a limbo state till the database is up again, instead of waiting till all prepared transactions are either aborted or committed. It may also choose to coordinate to abort all transactions, but all that is hard if the database shuts you out. I actually also have co-maintained other software where some processes have an idle connection open on which some shutdown stuff will happen. Obviously all those will need to handle the case where the connection was aborted, but that may result in suboptimal behaviour. Requiring such processes to keep open a transaction doesn't seem to be a good design choice in the pg world. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] plpython crash (PG 92)
On tor, 2012-04-26 at 17:32 +0500, Asif Naeem wrote:
> PFA test case. It used simple select statement to retrieve data via
> plpython. It crashes latest pg 9.2 with the following stack trace i.e.
> Apparently it is being crashed because of invalid related pointer value of
> pfree() *header->context->methods->free_p. It is reproducible with latest
> version of python i.e. Python-2.7.3 and Python-3.2.3. Thanks.
This is because of this code:
static void
PLy_result_dealloc(PyObject *arg)
{
PLyResultObject *ob = (PLyResultObject *) arg;
Py_XDECREF(ob->nrows);
Py_XDECREF(ob->rows);
Py_XDECREF(ob->status);
if (ob->tupdesc)
{
FreeTupleDesc(ob->tupdesc); // <-- dies here
ob->tupdesc = NULL;
}
arg->ob_type->tp_free(arg);
}
I must have been confused about the tuple descriptor APIs. ob->tupdesc
is created using CreateTupleDescCopy(), which copies the refcount of the
original tuple descriptor, thus causing a failure when the (seemingly
still referenced) tupdesc is freed. Is this behavior correct and
useful?
The dominant coding practice appears to be to not explicitly free copied
tuple descriptors, so maybe that should be done here as well.
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On fre, 2012-04-27 at 20:39 +0200, Andres Freund wrote: > I think the current smart mode is rather useful. There is quite some > stuff that you cannot do inside a transaction - or it doesn't make > sense - which still needs to shutdown gracefully. E.g. transaction > managers. Could you elaborate on that? What would happen to the transaction manager if you terminate any idle, not-in-a-transaction database backend sessions it has established? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
Heikki Linnakangas wrote: > Just thinking out loud here.. In the spirit of kicking around ideas... For those writing service scripts where you want a time limit on how long a stop can take, so that the service script doesn't prevent OS shutdown within a bounded time, it would also be nice to add an escalation time limit; so if there isn't a shutdown withing k seconds at one level it goes to the next. If the building is on fire and you need to power down all equipment before the fire department cuts power and starts spraying water (a situation we had at a courthouse a year or two ago), you really don't want the OS waiting for anything for more than a limited number of seconds before escalating to immediate. We do that in our sh scripts now, by using kill and sleep instead of trusting pg_ctl, but it seems like it would be better to have pg_ctl know how to do that. maybe?: --escalate-after=seconds -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Fri, Apr 27, 2012 at 3:00 PM, Robert Haas wrote: > On Fri, Apr 27, 2012 at 2:48 PM, Tom Lane wrote: >> I'm not necessarily opposed to commandeering the name "smart" for the >> new behavior, so that what we have to find a name for is the old "smart" >> behavior. How about >> >> slow - allow existing sessions to finish (old "smart") >> smart - allow existing transactions to finish (new) >> fast - kill active queries >> immediate - unclean shutdown > > I could live with that. Really, I'd like to have fast just be the > default. But the above compromise would still be a big improvement > over what we have now, assuming the new smart becomes the default. So right now, we have a mapping from signals to shutdown types that looks like this: [Current] SIGTERM -> smart, SIGINT -> fast, SIGQUIT -> immediate It seems we need another signal for the new mode, and the obvious candidate is SIGUSR2. But what shall the mapping look like? [Choice #1] SIGUSR2 -> slow, SIGTERM -> smart, SIGINT -> fast, SIGQUIT -> immediate [Choice #2] SIGTERM -> slow, SIGUSR2 -> smart, SIGINT -> fast, SIGQUIT -> immediate In other words, should we retain the existing behavior for SIGTERM and make SIGUSR2 have the new behavior (choice #2)? Or shall we preserve the invariant that SIGTERM invokes the default shutdown mode, and move the current default behavior off into SIGUSR2 land (choice #1)? -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] xReader, double-effort (was: Temporary tables under hot standby)
Sure Kevin, will get the wiki page ready asap, and reply back. Thanks. On Thu, Apr 26, 2012 at 8:10 PM, Kevin Grittner wrote: > [resending because of postgresql.org bounces on first try] > > Simon Riggs wrote: > > Kevin Grittner wrote: > > >> The GSoC xReader project is intended to be a major step toward > >> that, by providing a way to translate the WAL stream to a series > >> of notifications of logical events to clients which register with > >> xReader. > > > > This is already nearly finished in prototype and will be published > > in May. Andres Freund is working on it, copied here. > > URL? > > > It looks like there is significant overlap there. > > Hard for me to know without more information. It sounds like there > is at least some overlap. I hope that can involve cooperation, with > the efforts of Andres forming the basis of Aakash's GSoC effort. > That might leave him more time to polish up the user filters. > > Aakash: It seems like we need that Wiki page rather sooner than > later. Can you get to that quickly? I would think that just > copying the text from your approved GSoC proposal would be a very > good start. If you need help figuring out how to embed the images > from your proposal, let me know. > > -Kevin >
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On 27.04.2012 21:56, Tom Lane wrote:
Magnus Hagander writes:
On Fri, Apr 27, 2012 at 20:48, Tom Lane wrote:
I'm not necessarily opposed to commandeering the name "smart" for the
new behavior, so that what we have to find a name for is the old "smart"
behavior. How about
slow- allow existing sessions to finish (old "smart")
How about "wait" instead of "slow"?
I kinda liked "slow" vs "fast", but if you think that's too cute ...
("wait" doesn't seem very good, though, since all these except immediate
are waiting, just for different things.)
All the modes indeed wait (except for immediate), so I think it would
make sense to define the modes in terms of *what* they wait for.
wait sessions - allow existing sessions to finish (old "smart")
wait transactions - allow existing transactions to finish (new)
wait checkpoint - kill active queries
wait none - unclean shutdown
Hmm, the latter two are perhaps a bit confusing. So maybe:
wait_sessions - allow existing sessions to finish (old "smart")
wait_transactions - allow existing transactions to finish (new)
fast- kill active queries
immediate - unclean shutdown
Just thinking out loud here..
--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] psql omits row count under "\x auto"
On Thu, Apr 26, 2012 at 09:25:25PM +0300, Peter Eisentraut wrote:
> On m??n, 2012-04-23 at 12:30 -0400, Noah Misch wrote:
> > I've been enjoying "\x auto" in .psqlrc, but I noticed the row count footer
> > missing when it chooses ordinary output:
>
> > Looks like the logic in printQuery() needs further treatment.
>
> Hmm, this looks a bit tricky, because at the time we add the footer we
> don't yet know which output format will be used. I don't have a good
> idea how to fix that at the moment.
I fiddled with this and settled on moving the default_footer boolean setting
and the interpretation thereof down from the printQuery() level to the
printTable() level. That permits delaying the decision until we determine
whether the final output format is indeed vertical.
*** a/src/bin/psql/command.c
--- b/src/bin/psql/command.c
***
*** 2407,2418 do_pset(const char *param, const char *value, printQueryOpt
*popt, bool quiet)
else if (strcmp(param, "footer") == 0)
{
if (value)
! popt->default_footer = ParseVariableBool(value);
else
! popt->default_footer = !popt->default_footer;
if (!quiet)
{
! if (popt->default_footer)
puts(_("Default footer is on."));
else
puts(_("Default footer is off."));
--- 2407,2418
else if (strcmp(param, "footer") == 0)
{
if (value)
! popt->topt.default_footer = ParseVariableBool(value);
else
! popt->topt.default_footer = !popt->topt.default_footer;
if (!quiet)
{
! if (popt->topt.default_footer)
puts(_("Default footer is on."));
else
puts(_("Default footer is off."));
*** a/src/bin/psql/describe.c
--- b/src/bin/psql/describe.c
***
*** 1130,1135 describeOneTableDetails(const char *schemaname,
--- 1130,1136
retval = false;
+ myopt.default_footer = false;
/* This output looks confusing in expanded mode. */
myopt.expanded = false;
***
*** 2363,2368 describeRoles(const char *pattern, bool verbose)
--- 2364,2371
const char align = 'l';
char **attr;
+ myopt.default_footer = false;
+
initPQExpBuffer(&buf);
if (pset.sversion >= 80100)
***
*** 3362,3368 describeOneTSParser(const char *oid, const char *nspname,
const char *prsname)
sprintf(title, _("Text search parser \"%s\""), prsname);
myopt.title = title;
myopt.footers = NULL;
! myopt.default_footer = false;
myopt.translate_header = true;
myopt.translate_columns = translate_columns;
--- 3365,3371
sprintf(title, _("Text search parser \"%s\""), prsname);
myopt.title = title;
myopt.footers = NULL;
! myopt.topt.default_footer = false;
myopt.translate_header = true;
myopt.translate_columns = translate_columns;
***
*** 3393,3399 describeOneTSParser(const char *oid, const char *nspname,
const char *prsname)
sprintf(title, _("Token types for parser \"%s\""), prsname);
myopt.title = title;
myopt.footers = NULL;
! myopt.default_footer = true;
myopt.translate_header = true;
myopt.translate_columns = NULL;
--- 3396,3402
sprintf(title, _("Token types for parser \"%s\""), prsname);
myopt.title = title;
myopt.footers = NULL;
! myopt.topt.default_footer = true;
myopt.translate_header = true;
myopt.translate_columns = NULL;
***
*** 3725,3731 describeOneTSConfig(const char *oid, const char *nspname,
const char *cfgname,
myopt.nullPrint = NULL;
myopt.title = title.data;
myopt.footers = NULL;
! myopt.default_footer = false;
myopt.translate_header = true;
printQuery(res, &myopt, pset.queryFout, pset.logfile);
--- 3728,3734
myopt.nullPrint = NULL;
myopt.title = title.data;
myopt.footers = NULL;
! myopt.topt.default_footer = false;
myopt.translate_header = true;
printQuery(res, &myopt, pset.queryFout, pset.logfile);
*** a/src/bin/psql/print.c
--- b/src/bin/psql/print.c
***
*** 44,49 static char *decimal_point;
--- 44,52
static char *grouping;
static char *thousands_sep;
+ static char default_footer[100];
+ static printTableFooter default_footer_cell = { default_footer, NULL };
+
/* Line style control structures */
const printTextFormat pg_asciiformat =
{
***
*** 278,283 print_separator(struct separator sep, FIL
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Fri, Apr 27, 2012 at 2:48 PM, Tom Lane wrote: > I'm not necessarily opposed to commandeering the name "smart" for the > new behavior, so that what we have to find a name for is the old "smart" > behavior. How about > > slow - allow existing sessions to finish (old "smart") > smart - allow existing transactions to finish (new) > fast - kill active queries > immediate - unclean shutdown I could live with that. Really, I'd like to have fast just be the default. But the above compromise would still be a big improvement over what we have now, assuming the new smart becomes the default. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Fri, Apr 27, 2012 at 2:29 PM, Tom Lane wrote: >> This idea appeared to have some support. I'd like to suggest that we >> take this a step further. Instead of adding a fourth mode, I'd like >> to suggest that we redefine "smart" to have the behavior described >> above. > > No, I'm not happy with that. Smart shutdown is defined to not affect > current sessions. I'm fine with having a fourth mode that acts as you > suggest (and, probably, even with making it the default); but not with > taking away a behavior that people may well be relying on. I think there is no point at all in having a discussion about this unless we can first agree that the overwhelming majority of people who have commented on this issue on this list are unhappy with the current default behavior. If we are not going to change the default behavior, then there is zero point in talking about this. So I am nervous about your use of the word "probably", because I do not want to do a bunch of work on this just to add a fourth shutdown mode without changing the default to something that does not suck. I would like to get some agreement that we ARE going to change the default behavior, and then we can argue about what exactly we're going to change it to. >> This is based on the theory that (1) people who like smart >> shutdown like it because it allows currently-running transactions to >> complete without error, > > I think they like it because it allows currently-running *sessions* > to complete without error. You have no real basis for asserting that > relocating that goalpost won't change the game. I'm not asserting that. What I am asserting is that the vast majority of users will consider the revised game to be more fun than the original one. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
Magnus Hagander writes:
> On Fri, Apr 27, 2012 at 20:48, Tom Lane wrote:
>> I'm not necessarily opposed to commandeering the name "smart" for the
>> new behavior, so that what we have to find a name for is the old "smart"
>> behavior. How about
>>
>>slow- allow existing sessions to finish (old "smart")
> How about "wait" instead of "slow"?
I kinda liked "slow" vs "fast", but if you think that's too cute ...
("wait" doesn't seem very good, though, since all these except immediate
are waiting, just for different things.)
>>smart - allow existing transactions to finish (new)
> and still default, right?
Right.
>>fast- kill active queries
>>immediate - unclean shutdown
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Fri, Apr 27, 2012 at 20:48, Tom Lane wrote: > Simon Riggs writes: >> On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane wrote: >>> No, I'm not happy with that. Smart shutdown is defined to not affect >>> current sessions. I'm fine with having a fourth mode that acts as you >>> suggest (and, probably, even with making it the default); but not with >>> taking away a behavior that people may well be relying on. > >> Agreed, but not sure what to call the new mode: "smarter"? > > I'm not necessarily opposed to commandeering the name "smart" for the > new behavior, so that what we have to find a name for is the old "smart" > behavior. How about > > slow - allow existing sessions to finish (old "smart") How about "wait" instead of "slow"? > smart - allow existing transactions to finish (new) and still default, right? > fast - kill active queries > immediate - unclean shutdown -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
Simon Riggs writes: > On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane wrote: >> No, I'm not happy with that. Smart shutdown is defined to not affect >> current sessions. I'm fine with having a fourth mode that acts as you >> suggest (and, probably, even with making it the default); but not with >> taking away a behavior that people may well be relying on. > Agreed, but not sure what to call the new mode: "smarter"? I'm not necessarily opposed to commandeering the name "smart" for the new behavior, so that what we have to find a name for is the old "smart" behavior. How about slow- allow existing sessions to finish (old "smart") smart - allow existing transactions to finish (new) fast- kill active queries immediate - unclean shutdown regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Friday, April 27, 2012 08:38:10 PM Simon Riggs wrote: > On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane wrote: > > Robert Haas writes: > >> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera > >> > >> wrote: > >>> It occurs to me that we may need a new mode, which disconnects sessions > >>> that are not in a transaction (or as soon as they are) but leaves > >>> in-progress transactions alone; this could be the new default. Of > >>> course, this is much more difficult to implement than the current > >>> modes. > >> > >> This idea appeared to have some support. I'd like to suggest that we > >> take this a step further. Instead of adding a fourth mode, I'd like > >> to suggest that we redefine "smart" to have the behavior described > >> above. > > > > No, I'm not happy with that. Smart shutdown is defined to not affect > > current sessions. I'm fine with having a fourth mode that acts as you > > suggest (and, probably, even with making it the default); but not with > > taking away a behavior that people may well be relying on. > > Agreed, but not sure what to call the new mode: "smarter"? graceful? Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
Hi, On Friday, April 27, 2012 07:42:59 PM Robert Haas wrote: > On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera > wrote: > > It occurs to me that we may need a new mode, which disconnects sessions > > that are not in a transaction (or as soon as they are) but leaves > > in-progress transactions alone; this could be the new default. Of > > course, this is much more difficult to implement than the current modes. > > This idea appeared to have some support. I'd like to suggest that we > take this a step further. Instead of adding a fourth mode, I'd like > to suggest that we redefine "smart" to have the behavior described > above. This is based on the theory that (1) people who like smart > shutdown like it because it allows currently-running transactions to > complete without error, and will find it acceptable to have idle > transactions terminated immediately and other sessions terminated > after the command completes; and (2) people who dislike smart shutdown > (such as me) dislike it primarily because a completely idle session > that someone's forgotten to close can prevent shutdown indefinitely. > Either part of this theory could be wrong, of course, although I'm > pretty sure #2 holds for me personally at the least. I think the current smart mode is rather useful. There is quite some stuff that you cannot do inside a transaction - or it doesn't make sense - which still needs to shutdown gracefully. E.g. transaction managers. Andres -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Fri, Apr 27, 2012 at 7:29 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera >> wrote: >>> It occurs to me that we may need a new mode, which disconnects sessions >>> that are not in a transaction (or as soon as they are) but leaves >>> in-progress transactions alone; this could be the new default. Of >>> course, this is much more difficult to implement than the current modes. > >> This idea appeared to have some support. I'd like to suggest that we >> take this a step further. Instead of adding a fourth mode, I'd like >> to suggest that we redefine "smart" to have the behavior described >> above. > > No, I'm not happy with that. Smart shutdown is defined to not affect > current sessions. I'm fine with having a fourth mode that acts as you > suggest (and, probably, even with making it the default); but not with > taking away a behavior that people may well be relying on. Agreed, but not sure what to call the new mode: "smarter"? -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A problem with dump/restore of views containing whole row references
On Fri, Apr 27, 2012 at 11:21 PM, Tom Lane wrote: > Andrew Dunstan writes: > > Right, what I'm asking is whether or not we actually want that side > > effect in all cases, and specifically in this case where it's clearly > > not necessary. > > We could dodge that case by only changing the behavior when showstar is > false; there is no need to change it otherwise. The patch has assorted > other bugs too, in particular its schema-name treatment seems completely > wrong (hint: RelationIsVisible is not the same as TypeIsVisible, and > it's at best shaky to assume that a relation's name is the same as its > rowtype's name anyway). > > More generally, it seems rather inelegant to be forcibly adding a cast > when in most cases the existing notation is not wrong. AFAICS the > plain "relname" notation is only ambiguous if there is a column of the > same name as the relation. I wonder whether we should instead address > this by not letting the parser strip the "no op" cast in the first > place. > You mean that the parser should not strip the "no op" cast in all cases or in the case only when the parser somehow detects a column of the same name as the relation? > >regards, tom lane > -- Abbas EnterpriseDB Corporation The Enterprise PostgreSQL Company
Re: [HACKERS] Future In-Core Replication
On Fri, Apr 27, 2012 at 6:59 PM, Josh Berkus wrote: > >> What I'm hoping to do is to build a basic prototype of logical >> replication using WAL translation, so we can inspect it to see what >> the downsides are. > > Sounds like Mammoth. You take a look at that? Well, they all sound similar. My info was that Mammoth was not WAL-based. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
Robert Haas writes: > On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera > wrote: >> It occurs to me that we may need a new mode, which disconnects sessions >> that are not in a transaction (or as soon as they are) but leaves >> in-progress transactions alone; this could be the new default. Of >> course, this is much more difficult to implement than the current modes. > This idea appeared to have some support. I'd like to suggest that we > take this a step further. Instead of adding a fourth mode, I'd like > to suggest that we redefine "smart" to have the behavior described > above. No, I'm not happy with that. Smart shutdown is defined to not affect current sessions. I'm fine with having a fourth mode that acts as you suggest (and, probably, even with making it the default); but not with taking away a behavior that people may well be relying on. > This is based on the theory that (1) people who like smart > shutdown like it because it allows currently-running transactions to > complete without error, I think they like it because it allows currently-running *sessions* to complete without error. You have no real basis for asserting that relocating that goalpost won't change the game. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A problem with dump/restore of views containing whole row references
Andrew Dunstan writes: > Right, what I'm asking is whether or not we actually want that side > effect in all cases, and specifically in this case where it's clearly > not necessary. We could dodge that case by only changing the behavior when showstar is false; there is no need to change it otherwise. The patch has assorted other bugs too, in particular its schema-name treatment seems completely wrong (hint: RelationIsVisible is not the same as TypeIsVisible, and it's at best shaky to assume that a relation's name is the same as its rowtype's name anyway). More generally, it seems rather inelegant to be forcibly adding a cast when in most cases the existing notation is not wrong. AFAICS the plain "relname" notation is only ambiguous if there is a column of the same name as the relation. I wonder whether we should instead address this by not letting the parser strip the "no op" cast in the first place. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Future In-Core Replication
> What I'm hoping to do is to build a basic prototype of logical > replication using WAL translation, so we can inspect it to see what > the downsides are. Sounds like Mammoth. You take a look at that? -- Josh Berkus PostgreSQL Experts Inc. http://pgexperts.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Fri, Apr 27, 2012 at 1:46 PM, Magnus Hagander wrote: > On Fri, Apr 27, 2012 at 19:42, Robert Haas wrote: >> On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera >> wrote: >>> It occurs to me that we may need a new mode, which disconnects sessions >>> that are not in a transaction (or as soon as they are) but leaves >>> in-progress transactions alone; this could be the new default. Of >>> course, this is much more difficult to implement than the current modes. >> >> This idea appeared to have some support. I'd like to suggest that we >> take this a step further. Instead of adding a fourth mode, I'd like >> to suggest that we redefine "smart" to have the behavior described > > +1762329! Thanks. :-) >> above. This is based on the theory that (1) people who like smart >> shutdown like it because it allows currently-running transactions to >> complete without error, and will find it acceptable to have idle >> transactions terminated immediately and other sessions terminated > > Uh, I don't think it's ok to terminate an idle transaction > immediately. An idle *session* is ok, though - maybe that's what you > mean? Yes, exactly. What the patch does is arrange things so that, when smart shutdown is requested, we terminate each session as soon as it is both (1) idle and (2) not in a transaction. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Fri, Apr 27, 2012 at 19:42, Robert Haas wrote: > On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera > wrote: >> It occurs to me that we may need a new mode, which disconnects sessions >> that are not in a transaction (or as soon as they are) but leaves >> in-progress transactions alone; this could be the new default. Of >> course, this is much more difficult to implement than the current modes. > > This idea appeared to have some support. I'd like to suggest that we > take this a step further. Instead of adding a fourth mode, I'd like > to suggest that we redefine "smart" to have the behavior described +1762329! > above. This is based on the theory that (1) people who like smart > shutdown like it because it allows currently-running transactions to > complete without error, and will find it acceptable to have idle > transactions terminated immediately and other sessions terminated Uh, I don't think it's ok to terminate an idle transaction immediately. An idle *session* is ok, though - maybe that's what you mean? Because every transaction that's *doing* multiple things will be idle for milliseconds every now and then. -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] smart shutdown at end of transaction (was: Default mode for shutdown)
On Wed, Dec 15, 2010 at 10:11 AM, Alvaro Herrera wrote: > It occurs to me that we may need a new mode, which disconnects sessions > that are not in a transaction (or as soon as they are) but leaves > in-progress transactions alone; this could be the new default. Of > course, this is much more difficult to implement than the current modes. This idea appeared to have some support. I'd like to suggest that we take this a step further. Instead of adding a fourth mode, I'd like to suggest that we redefine "smart" to have the behavior described above. This is based on the theory that (1) people who like smart shutdown like it because it allows currently-running transactions to complete without error, and will find it acceptable to have idle transactions terminated immediately and other sessions terminated after the command completes; and (2) people who dislike smart shutdown (such as me) dislike it primarily because a completely idle session that someone's forgotten to close can prevent shutdown indefinitely. Either part of this theory could be wrong, of course, although I'm pretty sure #2 holds for me personally at the least. Patch is attached. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company smart-shutdown-at-eoxact.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
Tom Lane wrote: > Yeah, it would definitely be nicer if BEGIN; SET TRANSACTION LEVEL > would work too. Maybe the place to put the check is where we > establish the transaction snapshot. That makes sense, and doesn't seem like it would be hard, from what I recall of that code. I know I've fallen down on a couple other things which I meant to look at lately, but this one should be small enough to fit in between the brickbats the non-PostgreSQL world has been throwing at me lately. I'll take a shot at it sometime today, although it may be late. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enable_indexonly
On Fri, Apr 27, 2012 at 17:56, Andrew Dunstan wrote: > > > On 04/27/2012 11:45 AM, Simon Riggs wrote: >> >> On Fri, Apr 27, 2012 at 4:41 PM, Thom Brown wrote: >>> >>> On 27 April 2012 16:08, Simon Riggs wrote: I notice that there isn't a parameter called enable_indexonly (or similar). ISTM that such a major new feature should be controlled by a planner method parameter, just as all the existing planner methods are. This will help us evaluate index only scans in production, and turn them off if they have negative impacts. >>> >>> It already exists and is called "enable_indexonlyscan". >> >> Hmm. Forgive me, I pressed the wrong button and looked at current docs >> rather than dev docs. >> >> (Easier when they used to look different...) > > > Maybe we should have the stylesheet watermark the dev docs pages. Are you offering to write the patch? ;) (if someone can provide a CSS that will look good and work cross browser, I can do the website code side of things that enables it for the devel docs) -- Magnus Hagander Me: http://www.hagander.net/ Work: http://www.redpill-linpro.com/ -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enable_indexonly
On Fri, Apr 27, 2012 at 4:56 PM, Andrew Dunstan wrote: >> (Easier when they used to look different...) > > > Maybe we should have the stylesheet watermark the dev docs pages. +1 Many users have >1 version in production. This would help -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
"Kevin Grittner" writes: > Tom Lane wrote: >> Couldn't we check and throw an error at the place in transaction >> startup where default_transaction_isolation is copied to the >> active variable? > Wouldn't that leave users stuck if the postgresql.conf set the > default to serializable? Nobody would be able to start a > transaction, even to change the default, would they? I was assuming "BEGIN TRANSACTION LEVEL ..." would still work; if not, it's a non-starter. I haven't looked at the code to see if the sequence of operations is amenable to that though. > Robert's suggestion might be the least of the various evils. Yeah, it would definitely be nicer if BEGIN; SET TRANSACTION LEVEL would work too. Maybe the place to put the check is where we establish the transaction snapshot. regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A problem with dump/restore of views containing whole row references
On 04/27/2012 12:02 PM, Abbas Butt wrote:
However, is this a change we really want to make?:
pg_get_triggerdef
---
- CREATE TRIGGER modified_any BEFORE UPDATE OF a ON main_table
FOR EACH ROW WHEN (old.* IS DISTINCT FROM new.*) EXECUTE PROCEDURE
trigger_func('modified_any')
+
pg_get_triggerdef
+--
+ CREATE TRIGGER modified_any BEFORE UPDATE OF a ON main_table
FOR EACH ROW WHEN (old.*::main_table IS DISTINCT FROM
new.*::main_table) EXECUTE PROCEDURE trigger_func('modified_any')
Maybe we need to be a bit more selective about when the cast is
supplied. It's not adding any extra disambiguation (or clarity) here.
I ran the regression and found that my patch is causing a diff in the
trigger test case, thats why I changed the expected output of the test
case accordingly. This is a side effect of the change I did to fix the
bug.
Right, what I'm asking is whether or not we actually want that side
effect in all cases, and specifically in this case where it's clearly
not necessary.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A problem with dump/restore of views containing whole row references
On Fri, Apr 27, 2012 at 6:25 PM, Andrew Dunstan wrote:
>
>
> On 04/27/2012 08:25 AM, Abbas Butt wrote:
>
>>
>> The notation "relation.*" represents a whole-row reference.
>> While parsing a whole-row reference is transformed into a Var with varno
>> set to the correct range table entry,
>> and varattno == 0 to signal that it references the whole tuple. (For
>> reference see comments of function makeWholeRowVar)
>> While deparsing we need to take care of this case.
>> The attached patch provides deparsing of a whole-row reference.
>> A whole row reference will be deparsed either into alias.*::relation or
>> relation.*::relation depending on alias
>>
>>
> I agree there's a bug, although it's easily worked around: in the case of
> your example:
>
> CREATE VIEW v2 AS
>SELECT p AS price FROM price p;
>
> would do the trick.
>
> However, is this a change we really want to make?:
>
> pg_get_triggerdef
> --**--**
> --**--**
> --**-
> - CREATE TRIGGER modified_any BEFORE UPDATE OF a ON main_table FOR EACH
> ROW WHEN (old.* IS DISTINCT FROM new.*) EXECUTE PROCEDURE
> trigger_func('modified_any')
> +
>pg_get_triggerdef
> +-**--**
> --**--**
> --**--**---
> + CREATE TRIGGER modified_any BEFORE UPDATE OF a ON main_table FOR EACH
> ROW WHEN (old.*::main_table IS DISTINCT FROM new.*::main_table) EXECUTE
> PROCEDURE trigger_func('modified_any')
>
>
> Maybe we need to be a bit more selective about when the cast is supplied.
> It's not adding any extra disambiguation (or clarity) here.
>
I ran the regression and found that my patch is causing a diff in the
trigger test case, thats why I changed the expected output of the test case
accordingly. This is a side effect of the change I did to fix the bug.
>
> cheers
>
> andrew
>
>
--
--
Abbas
Architect
EnterpriseDB Corporation
The Enterprise PostgreSQL Company
Phone: 92-334-5100153
Website: www.enterprisedb.com
EnterpriseDB Blog: http://blogs.enterprisedb.com/
Follow us on Twitter: http://www.twitter.com/enterprisedb
This e-mail message (and any attachment) is intended for the use of
the individual or entity to whom it is addressed. This message
contains information from EnterpriseDB Corporation that may be
privileged, confidential, or exempt from disclosure under applicable
law. If you are not the intended recipient or authorized to receive
this for the intended recipient, any use, dissemination, distribution,
retention, archiving, or copying of this communication is strictly
prohibited. If you have received this e-mail in error, please notify
the sender immediately by reply e-mail and delete this message.
Re: [HACKERS] enable_indexonly
On 04/27/2012 11:45 AM, Simon Riggs wrote: On Fri, Apr 27, 2012 at 4:41 PM, Thom Brown wrote: On 27 April 2012 16:08, Simon Riggs wrote: I notice that there isn't a parameter called enable_indexonly (or similar). ISTM that such a major new feature should be controlled by a planner method parameter, just as all the existing planner methods are. This will help us evaluate index only scans in production, and turn them off if they have negative impacts. It already exists and is called "enable_indexonlyscan". Hmm. Forgive me, I pressed the wrong button and looked at current docs rather than dev docs. (Easier when they used to look different...) Maybe we should have the stylesheet watermark the dev docs pages. cheers andrew -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enable_indexonly
On Fri, Apr 27, 2012 at 4:41 PM, Thom Brown wrote: > On 27 April 2012 16:08, Simon Riggs wrote: >> I notice that there isn't a parameter called enable_indexonly (or similar). >> >> ISTM that such a major new feature should be controlled by a planner >> method parameter, just as all the existing planner methods are. >> >> This will help us evaluate index only scans in production, and turn >> them off if they have negative impacts. > > It already exists and is called "enable_indexonlyscan". Hmm. Forgive me, I pressed the wrong button and looked at current docs rather than dev docs. (Easier when they used to look different...) -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] enable_indexonly
On 27 April 2012 16:08, Simon Riggs wrote: > I notice that there isn't a parameter called enable_indexonly (or similar). > > ISTM that such a major new feature should be controlled by a planner > method parameter, just as all the existing planner methods are. > > This will help us evaluate index only scans in production, and turn > them off if they have negative impacts. It already exists and is called "enable_indexonlyscan". -- Thom -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] enable_indexonly
I notice that there isn't a parameter called enable_indexonly (or similar). ISTM that such a major new feature should be controlled by a planner method parameter, just as all the existing planner methods are. This will help us evaluate index only scans in production, and turn them off if they have negative impacts. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
Tom Lane wrote: > Robert Haas writes: >> Or, maybe there's a way to throw an error when serializable mode >> is used rather than when it's requested. > > Couldn't we check and throw an error at the place in transaction > startup where default_transaction_isolation is copied to the > active variable? Wouldn't that leave users stuck if the postgresql.conf set the default to serializable? Nobody would be able to start a transaction, even to change the default, would they? If that's the case, we might as well refuse to start. Robert's suggestion might be the least of the various evils. -Kevin -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
On Fri, Apr 27, 2012 at 11:02 AM, Tom Lane wrote: > Robert Haas writes: >> Or, maybe there's a way to throw an error when serializable mode is >> used rather than when it's requested. > > Couldn't we check and throw an error at the place in transaction startup > where default_transaction_isolation is copied to the active variable? Yeah, possibly. Although the user might still use SET TRANSACTION ISOLATION, so it would ideally be nice to postpone throwing the error until after that opportunity is past. Not sure if that's feasible, though. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
Robert Haas writes: > Or, maybe there's a way to throw an error when serializable mode is > used rather than when it's requested. Couldn't we check and throw an error at the place in transaction startup where default_transaction_isolation is copied to the active variable? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
On Fri, Apr 27, 2012 at 10:21 AM, Kevin Grittner wrote: > My first thought was that if we can detect that we are in HS, we > should probably throw an ERROR on an attempt to set > default_transaction_isolation = 'serializable'. I think that would result in the server failing to start. We could throw a warning there and give repeatable read. Or, maybe there's a way to throw an error when serializable mode is used rather than when it's requested. So in the above example SELECT 1; would say, hey, somehow I ended up in serializable mode under HS, abort, abort! -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Request to add options to tools/git_changelog
On Thu, Apr 26, 2012 at 03:19:04PM -0400, Bruce Momjian wrote: > On Thu, Apr 26, 2012 at 02:05:23PM -0400, Tom Lane wrote: > > Bruce Momjian writes: > > > I agree adding rarely-used options to a tool doesn't make sense, but the > > > question is what percentage of the git_changelog userbase am I? > > > > 50% I think. The only thing that's really concerning me here is that > > the reverse-sort option seems likely to be bug-inducing, and I really > > don't grasp that it has real value. But whatever. > > Well, newest first would show this: > > add feature D to feature ABC > add feature C to feature AB > add feature B to feature A > add feature A > > More logical (oldest-first) is: > > add feature A > add feature B to feature A > add feature C to feature AB > add feature D to feature ABC > > Also consider that A is usually the big, clear commit message, and B,C,D > are just minor adjustments with more brief commits, which might require > adjusting the release note item for feature A. When they are in > newest-first order, that is much harder. Oh, one more thing. The contributor names appended to each release note item usually has to be listed A,B,C,D because A is usually the most significant contribution. -- Bruce Momjian http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. + -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
Robert Haas wrote:
> When I do this:
>
> rhaas=# set default_transaction_isolation = 'serializable';
> SET
> rhaas=# begin;
> BEGIN
> rhaas=# select 1;
>
> Then I get this:
>
> TRAP: FailedAssertion("!(!RecoveryInProgress())", File:
> "predicate.c", Line: 1637)
> LOG: server process (PID 290) was terminated by signal 6: Abort
> trap
>
> The root of the problem here seems to be that we're imagining that
> it's possible to prevent serializable mode from being used under
> HS from within the check function for the transaction_isolation
> GUC - see check_XactIsoLevel. However, because there's a second
> GUC (default_transaction_isolation) that can also be used to
> change the initial setting, it doesn't work.
>
> I'm not exactly sure what the best way to fix this is. I assume
> that this problem also exists in 9.1, but I haven't checked.
My first thought was that if we can detect that we are in HS, we
should probably throw an ERROR on an attempt to set
default_transaction_isolation = 'serializable'. But that raises the
question about what to do with environments where people want the
master to be running with that default (set from postgresql.conf) --
fail-over could be, well, "interesting". We haven't run into this
situation yet because we don't use our production postgresql.conf on
our hot standbys -- when we make a base backup we rename
postgresql.conf to postgresql.conf.production and copy in a special
configuration file. For those who want the same configuration on
both, and who want to use serializable transactions on the master, I
don't see a really clean solution. Does anyone else?
We didn't want to allow SERIALIZABLE to be set while still allowing
read-only anomalies like this (assuming T3 was on the HS):
http://wiki.postgresql.org/wiki/SSI#Deposit_Report
-Kevin
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] A problem with dump/restore of views containing whole row references
On 04/27/2012 08:25 AM, Abbas Butt wrote:
The notation "relation.*" represents a whole-row reference.
While parsing a whole-row reference is transformed into a Var with
varno set to the correct range table entry,
and varattno == 0 to signal that it references the whole tuple. (For
reference see comments of function makeWholeRowVar)
While deparsing we need to take care of this case.
The attached patch provides deparsing of a whole-row reference.
A whole row reference will be deparsed either into alias.*::relation
or relation.*::relation depending on alias
I agree there's a bug, although it's easily worked around: in the case
of your example:
CREATE VIEW v2 AS
SELECT p AS price FROM price p;
would do the trick.
However, is this a change we really want to make?:
pg_get_triggerdef
---
- CREATE TRIGGER modified_any BEFORE UPDATE OF a ON main_table FOR EACH ROW
WHEN (old.* IS DISTINCT FROM new.*) EXECUTE PROCEDURE
trigger_func('modified_any')
+
pg_get_triggerdef
+--
+ CREATE TRIGGER modified_any BEFORE UPDATE OF a ON main_table FOR EACH ROW
WHEN (old.*::main_table IS DISTINCT FROM new.*::main_table) EXECUTE PROCEDURE
trigger_func('modified_any')
Maybe we need to be a bit more selective about when the cast is
supplied. It's not adding any extra disambiguation (or clarity) here.
cheers
andrew
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] default_transaction_isolation = serializable causes crash under Hot Standby
When I do this:
rhaas=# set default_transaction_isolation = 'serializable';
SET
rhaas=# begin;
BEGIN
rhaas=# select 1;
Then I get this:
TRAP: FailedAssertion("!(!RecoveryInProgress())", File: "predicate.c",
Line: 1637)
LOG: server process (PID 290) was terminated by signal 6: Abort trap
The root of the problem here seems to be that we're imagining that
it's possible to prevent serializable mode from being used under HS
from within the check function for the transaction_isolation GUC - see
check_XactIsoLevel. However, because there's a second GUC
(default_transaction_isolation) that can also be used to change the
initial setting, it doesn't work.
I'm not exactly sure what the best way to fix this is. I assume that
this problem also exists in 9.1, but I haven't checked.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
[HACKERS] A problem with dump/restore of views containing whole row references
Hi,
This is the version I used to run the following commands
select version();
version
PostgreSQL 9.2devel on x86_64-unknown-linux-gnu, compiled by gcc
(Ubuntu/Linaro 4.4.4-14ubuntu5) 4.4.5, 64-bit
(1 row)
Run these commands
CREATE TABLE price (id INT PRIMARY KEY, active BOOLEAN NOT NULL, price
NUMERIC);
insert into price values (1,false,42), (10,false,100), (11,true,17.99);
create view v2 as select price.*::price from price;
select * from v2;
price
--
(1,f,42)
(10,f,100)
(11,t,17.99)
(3 rows)
\d+ v2;
View "public.v2"
Column | Type | Modifiers | Storage | Description
+---+---+--+-
price | price | | extended |
View definition:
SELECT price AS price
FROM price;
Note the output from the view, also note the "Type" in view defination.
Now take dump of this database.
./pg_dump --file=/home/user_name/d.sql --format=p --inserts -p test
The dump file is attached with the mail. (d.sql)
Now lets restore this dump.
./createdb test2 -p
./psql -p -f /home/user_name/d.sql test2
./psql test2 -p
psql (9.2devel)
Type "help" for help.
test2=# select * from v2;
price
---
42
100
17.99
(3 rows)
test2=# \d+ v2
View "public.v2"
Column | Type | Modifiers | Storage | Description
+-+---+-+-
price | numeric | | main|
View definition:
SELECT price.price
FROM price;
In the database test2 the view was not restored correctly.
The output of the view as well as the Type in its defination is wrong.
The cause of the problem is as follows
The notation "relation.*" represents a whole-row reference.
While parsing a whole-row reference is transformed into a Var with varno
set to the correct range table entry,
and varattno == 0 to signal that it references the whole tuple. (For
reference see comments of function makeWholeRowVar)
While deparsing we need to take care of this case.
The attached patch provides deparsing of a whole-row reference.
A whole row reference will be deparsed either into alias.*::relation or
relation.*::relation depending on alias
--
Abbas
EnterpriseDB Corporation
The Enterprise PostgreSQL Company
diff --git a/src/backend/utils/adt/ruleutils.c b/src/backend/utils/adt/ruleutils.c
index 3beed37..272d1a5 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -4007,7 +4007,47 @@ get_variable(Var *var, int levelsup, bool showstar, deparse_context *context)
}
if (attnum == InvalidAttrNumber)
+ {
attname = NULL;
+ /*
+ * The notation "relation.*" represents a whole-row reference.
+ * While parsing a whole-row reference is transformed into
+ * a Var with varno set to the correct range table entry,
+ * and varattno == 0 to signal that it references the whole tuple.
+ * For reference see comments of function makeWholeRowVar
+ * While deparsing we need to take care of this case
+ * This block of code is deparsing a whole-row reference.
+ * A whole row reference will be deparsed either into alias.*::relation
+ * or relation.*::relation depending on alias
+ * Explicit typecasting to relation is needed because
+ * input of anonymous composite types is not implemented
+ */
+
+ if (rte->relid != InvalidOid && refname && var->varattno == 0 &&
+ var->varno >= 1 && var->varno <= list_length(dpns->rtable))
+ {
+ char *rel_name;
+ List *schemalist;
+
+ schemalist = NULL;
+ if (schemaname)
+list_make1(schemaname);
+
+ /* This relation name is required for explicit type casting later */
+ rel_name = generate_relation_name(rte->relid, schemalist);
+
+ /* Add name space qualification if required */
+ if (schemaname)
+appendStringInfo(buf, "%s.", quote_identifier(schemaname));
+ appendStringInfoString(buf, quote_identifier(refname));
+ appendStringInfoString(buf, ".*::");
+ appendStringInfoString(buf, quote_identifier(rel_name));
+
+ pfree(rel_name);
+
+ return attname;
+ }
+ }
else
attname = get_rte_attribute_name(rte, attnum);
diff --git a/src/test/regress/expected/triggers.out b/src/test/regress/expected/triggers.out
index 0e7177e..cb39881 100644
--- a/src/test/regress/expected/triggers.out
+++ b/src/test/regress/expected/triggers.out
@@ -387,9 +387,9 @@ SELECT pg_get_triggerdef(oid, false) FROM pg_trigger WHERE tgrelid = 'main_table
(1 row)
SELECT pg_get_triggerdef(oid, true) FROM pg_trigger WHERE tgrelid = 'main_table'::regclass AND tgname = 'modified_any';
- pg_get_triggerdef
-
Re: [HACKERS] patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap
On Fri, Apr 27, 2012 at 1:51 AM, Josh Berkus wrote: > Now, the other issue I'd be worried about for this optimization is what > happens when the nulls become non-trailing? For example, this pattern: > > 1. Out of 700 columns, columns 301+ are all Null, so we map them away. > 2. User updates column 688 to non-null > 3. Suddenly we have a MUCH larger row which will no longer fit on the page. > > If your application had a lot of that kind of update pattern, I'd be > concerned that this would be a deoptimzation. Currently, we have a long row before and a long row after. Jamie's proposals would give us a short row before and a long row after. Since we don't ever update in place, we're much more likely to fit on the same page with this optimisation than without it. I guess we can check that with a performance test. (Perhaps a more obvious optimisation would be to use a compressed NULL bitmap. That would respond better in a wider range of use cases than just truncation of trailing zeroes.) -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Future In-Core Replication
On Fri, Apr 27, 2012 at 12:40 AM, Robert Haas wrote: > On Thu, Apr 26, 2012 at 6:48 PM, Josh Berkus wrote: >> So the idea is that you'll present briefly your intentions for 9.3 at >> the developer meeting, and then have this in-depth afterwards? Sounds >> great. > > I really, really do not want the developer meeting to turn into a > series of presentations. That's what we had last year, and it was > boring and unproductive. Furthermore, it excludes from the > conversation everyone who doesn't fit into a room in Ottawa. I think > that plans should be presented here, on pgsql-hackers, and the > developer meeting should be reserved for discussion of issues with > which everyone who will be there is already familiar. If a > presentation is required, the developer meeting is the wrong forum, > IMHO of course. I agree. Obviously, the word presentation is the issue here. If one person speaks and everybody else is silent that is not a good use of anybody's time. On any topic, I expect the introducer of that topic to set the scene, establish the major questions and encourage discussion. I would follow that form myself: I want to hear the wisdom of others and alter my plans accordingly. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Future In-Core Replication
On Fri, Apr 27, 2012 at 1:26 AM, Josh Berkus wrote: > Simon, > >> I'm beginning to work on advanced additions to in-core replication for >> PostgreSQL. > ... >> Those are the basic requirements that I am trying to address. There >> are a great many important details, but the core of this is probably >> what I would call "logical replication", that is shipping changes to >> other nodes in a way that does not tie us to the same physical >> representation that recovery/streaming replication does now. Of >> course, non-physical replication can take many forms. > > So, I'm a bit confused. You talk about this as "additions to in-core > replication", but then you talk about implementing logical replication, > which would NOT be an addition to Binary replication. Can you explain > what you mean? The key point is that there is a specific objective of including additional features in-core. That places some restrictions, but also offers some opportunities. Tight integration allows performance improvements, as well as ease of use etc.. I'm not sure what you mean by "would not be an addition to binary replication". Yes, for reasons most elegantly explained by Robert here [http://rhaas.blogspot.co.uk/2011/02/case-for-logical-replication.html], physical/binary replication puts too many restrictions on us and we cannot solve all of the problems that way. I was unaware of Robert's post, but it sets the scene clearly. So the future of in-core replication, IMHO, is some form of non-physical replication. There are various options there, but anything that goes in would reuse significant parts of the existing replication setup that already works so well. Put that another way: the infrastructure for the secure and efficient transport of replication messages is already in place. Reuse is also what makes something useful be achievable in a reasonable timescale. What we need to consider is the form of those new non-physicalWAL messages, how they are built on the sender and how they are handled at the receiving end. What I'm hoping to do is to build a basic prototype of logical replication using WAL translation, so we can inspect it to see what the downsides are. It's an extremely non-trivial problem and so I expect there to be mountains to climb. There are other routes to logical replication, with messages marshalled in a similar way to Slony/Londiste/Bucardo/Mammoth(?). So there are options, with measurements to be made and discussions to be had. It will take time for people to believe this is possible and longer to analyse and agree the options. -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments
Sorry for broken message. > >From I said that the former (spinlock) could be dropped, but the > latter (read as volatile) should be needed. I said that the former (spinlock) could be dropped from the view of functionarity, but the latter (read as volatile) should be needed. > >From the view of maintenancibility (suspicious-proof > expression?), it may be preferable that the manner to read shared > memory be uniform whole source code if no particular reasons. regards, -- Kyotaro Horiguchi NTT Open Source Software Center == My e-mail address has been changed since Apr. 1, 2012. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] [BUG] Checkpointer on hot standby runs without looking checkpoint_segments
Thank you for sugestion. > This still makes catching up in standby mode slower, as you get > many more restartpoints. The reason for ignoring > checkpoint_segments during recovery was to avoid that. I may have a misunderstanding around there, or your intention. I understand that standby creates no WAL archive, and can not recover from WAL archive, and both master and standby keeps WAL segment no longer than about them for about 2 * 1h, spans two maximum checkpoint_timeout intervals and some more. Could you please tell me whether the above is correct? If you meant crash recovery with the word 'recovery', there's WALs no more than for 2+ hours, far less than days, or weeks long. Otherwise, if you meant archive recovery, this patch does not change the behavior of archive recovery as far as I intended. This patch intended to change the behavior of standby under WAL shipping. If it is correct and the patch works correctly, your anxiety below should disappear, I hope. And if not correct, I *MUST* avoid such negative impacts on the functions out of the target - governing checkpoint progress on standby server shipping WALs from its master. > Maybe it's still better than what we have currently, I'm not > sure, but at least it needs to be discussed. Would be good to > do some performance testing of recovery with various > checkpoint_segments and _timeout settings, with and without > this patch. regards, -- Kyotaro Horiguchi NTT Open Source Software Center == My e-mail address has been changed since Apr. 1, 2012. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
