Re: [HACKERS] [GENERAL] [SQL] pg_multixact issues

On 2014-09-18 22:52:57 +0530, Dev Kumkar wrote: > On Thu, Sep 18, 2014 at 6:20 PM, Dev Kumkar wrote: > > > On Thu, Sep 18, 2014 at 4:03 PM, Andres Freund > > wrote: > > > >> I don't think that's relevant for you. > >> > >> Did you upgrade the database using pg_upgrade? > >> > > > > That's correc

Re: [HACKERS] [GENERAL] [SQL] pg_multixact issues

On Fri, Sep 19, 2014 at 1:03 PM, Andres Freund wrote: > Yes: Learning some patience. You'd given the previous answer two hours > before this one. Nobody is paid to work on this list... Apologies for the delay, was working/troubleshooting same issue and was away from my emails. :( Regards...

Re: [HACKERS] pg_xlogdump --stats

Hi. I've attached two patches here: 0001-Make-pg_xlogdump-record-stats-display-summary-statis.patch is my earlier patch to pg_xlogdump, rebased to master. It introduces the new rm_identify callback, but doesn't touch rm_desc. Other than rebasing to master after the INT64_FORMAT changes, I haven't

Re: [HACKERS] pg_xlogdump --stats

On 2014-09-19 13:24:11 +0530, Abhijit Menon-Sen wrote: > diff --git a/configure.in b/configure.in > index 1392277..c3c458c 100644 > --- a/configure.in > +++ b/configure.in > @@ -1637,7 +1637,7 @@ fi > # If we found "long int" is 64 bits, assume snprintf handles it. If > # we found we need to use

Re: [HACKERS] pg_xlogdump --stats

At 2014-09-19 13:24:11 +0530, a...@2ndquadrant.com wrote: > > Good enough? Not quite. I've attached a small additional patch that shifts the responsibility of adding rm_name to the output from xlog_outrec to xlog_outdesc. Now we get WAL_DEBUG output like this: LOG: INSERT @ 0/16C51D0: prev 0/16C

Re: [HACKERS] pg_xlogdump --stats

At 2014-09-19 10:44:48 +0200, and...@2ndquadrant.com wrote: > > > # snprintfs that use %lld, %qd, or %I64d as the format. If none of these > > -# work, fall back to our own snprintf emulation (which we know uses %lld). > > +# works, fall back to our own snprintf emulation (which we know uses %lld

Re: [HACKERS] pg_xlogdump --stats

Hi, On 2014-09-19 14:38:29 +0530, Abhijit Menon-Sen wrote: > > b) I'm not against it, but I wonder if the best way to add the > >SizeOfXLogRecord to the record size. It's just as much part of the > >FPI. And this means that the record length will be > 0 even if all > >the record data h

[HACKERS] GCC memory barriers are missing "cc" clobbers

Hi, barrier.h defines memory barriers for x86 as: 32bit: #define pg_memory_barrier() \ __asm__ __volatile__ ("lock; addl $0,0(%%esp)" : : : "memory") 64bit: #define pg_memory_barrier() \ __asm__ __volatile__ ("lock; addl $0,0(%%rsp)" : : : "memory") But addl

Re: [HACKERS] pg_xlogdump --stats

I've attached the revised patch, split up differently: 1. Introduce rm_identify, change rm_desc, glue the two together in xlog.c 2. Introduce pg_xlogdump --stats[=record]. The requested changes (16, filter, z:, UNKNOWN) are included. The grammar nitpicking and rebase cruft is^Ware^Wain't inclu

Re: [HACKERS] pg_xlogdump --stats

At 2014-09-19 15:39:37 +0530, a...@2ndquadrant.com wrote: > > I hope I didn't miss anything this time. But of course I did. The attached fixup makes the output of pg_xlogdump match that of xlog_outdesc for unidentifiable records (UNKNOWN (%x)). Sorry for the inconvenience. -- Abhijit diff --git a

Re: [HACKERS] GCC memory barriers are missing "cc" clobbers

Hi, On 2014-09-19 12:00:16 +0200, Andres Freund wrote: > > barrier.h defines memory barriers for x86 as: > 32bit: > #define pg_memory_barrier() \ > __asm__ __volatile__ ("lock; addl $0,0(%%esp)" : : : "memory") > 64bit: > #define pg_memory_barrier() \ > __asm__ _

Re: [HACKERS] Scaling shared buffer eviction

On Tue, Sep 16, 2014 at 10:21 PM, Robert Haas wrote: > On Tue, Sep 16, 2014 at 8:18 AM, Amit Kapila > wrote: > >> In most cases performance with patch is slightly less as compare >> to HEAD and the difference is generally less than 1% and in a case >> or 2 close to 2%. I think the main reason fo

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

Hello, >Maybe. Let's get the basic patch done first; then we can argue about that Please find attached patch to compress FPW using pglz compression. All backup blocks in WAL record are compressed at once before inserting it into WAL buffers . Full_page_writes GUC has been modified to accept thr

Re: [HACKERS] jsonb format is pessimal for toast compression

On 09/18/2014 09:27 PM, Heikki Linnakangas wrote: On 09/18/2014 07:53 PM, Josh Berkus wrote: On 09/16/2014 08:45 PM, Tom Lane wrote: We're somewhat comparing apples and oranges here, in that I pushed my approach to something that I think is of committable quality (and which, not incidentally, f

Re: [HACKERS] GCC memory barriers are missing "cc" clobbers

Andres Freund writes: > On 2014-09-19 12:00:16 +0200, Andres Freund wrote: >> But addl sets condition flags. So this really also needs a "cc" clobber? >> Or am I missing something? > What I missed is that x86 has an implied "cc" clobber for every inline > assembly statement. So forget that. Whil

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

>Please find attached patch to compress FPW using pglz compression. Please refer the updated patch attached. The earlier patch added few duplicate lines of code in guc.c file. compress_fpw_v1.patch Thank you, Rahila

Re: [HACKERS] jsonb format is pessimal for toast compression

Heikki Linnakangas writes: > Tom: You mentioned earlier that your patch fixes some existing bugs. > What were they? What I remember at the moment (sans caffeine) is that the routines for assembling jsonb values out of field data were lacking some necessary tests for overflow of the size/offset f

Re: [HACKERS] Anonymous code block with parameters

On Thu, Sep 18, 2014 at 5:22 PM, Hannu Krosing wrote: > Though it would be even nicer to have fully in-line type definition > > SELECT (tup).* FROM > ( > SELECT CASE WHEN .. THEN ROW(1,2,3)::(a int, b text, c int2) > WHEN .. THEN ROW(2,3,4) > ELSE ROW (3,4,5) END AS t

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

Rahila Syed writes: > Please find attached patch to compress FPW using pglz compression. Patch not actually attached AFAICS (no, a link is not good enough). regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Final Patch for GROUPING SETS

On Fri, Sep 19, 2014 at 4:45 AM, Andrew Gierth wrote: > GroupAggregate (cost=1122.39..1197.48 rows=9 width=8) >Group Key: two, four >Group Key: two >Group Key: () > "Grouping Sets": [ > ["two", "four"], > ["two"], > [] +1 looks good to me. > (yaml for

Re: [HACKERS] pg_xlogdump --stats

Hi, On 2014-09-19 15:39:37 +0530, Abhijit Menon-Sen wrote: > I've attached the revised patch, split up differently: > > 1. Introduce rm_identify, change rm_desc, glue the two together in >xlog.c > 2. Introduce pg_xlogdump --stats[=record]. I've pushed these after some fixing up. As we previ

Re: [HACKERS] GCC memory barriers are missing "cc" clobbers

On 2014-09-19 09:58:01 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-09-19 12:00:16 +0200, Andres Freund wrote: > >> But addl sets condition flags. So this really also needs a "cc" clobber? > >> Or am I missing something? > > > What I missed is that x86 has an implied "cc" clobber fo

Re: [HACKERS] GCC memory barriers are missing "cc" clobbers

Andres Freund writes: > On 2014-09-19 09:58:01 -0400, Tom Lane wrote: >> While it might not be buggy as it stands, I think we should add the "cc" >> rather than rely on it being implicit. One reason is that people will >> look at the x86 cases when developing code for other architectures, and >>

Re: [HACKERS] GCC memory barriers are missing "cc" clobbers

On 2014-09-19 10:58:56 -0400, Tom Lane wrote: > Andres Freund writes: > > I don't really see any need to backpatch though, do you? > Well, I'd make it the same in all branches which have that code, which > is not very far back is it? It was already introduced in 9.2 - no idea whether that's "far

[HACKERS] identify_locking_dependencies is broken for schema-only dumps

This can lead to deadlocks during parallel restore. Test case: create table bar (a int primary key, b int); create table baz (z int, a int references bar); create view foo as select a, b, sum(1) from bar group by a union all select z, a, 0 from baz; I dumped this with: pg_dump -Fc -s -f test.dmp

Re: [HACKERS] Final Patch for GROUPING SETS

> "Marti" == Marti Raudsepp writes: >> (yaml format) >> Grouping Sets: >> - - "two" >> - "four" >> - - "two" >> - Marti> Now this is weird. You're telling me. Also, feeding it to an online yaml-to-json converter gives the result as [["two","four"],["two"],null] which is not

Re: [HACKERS] Final Patch for GROUPING SETS

On 2014-09-19 16:35:52 +0100, Andrew Gierth wrote: > Marti> But is anyone actually using YAML output format, or was it > Marti> implemented simply "because we can"? > > Until someone decides to dike it out, I think we are obligated to make > it produce something resembling correct output. I vot

Re: [HACKERS] Final Patch for GROUPING SETS

> "Andrew" == Andrew Gierth writes: Andrew> You're telling me. Also, feeding it to an online yaml-to-json Andrew> converter gives the result as [["two","four"],["two"],null] Andrew> which is not quite the same as the json version. An Andrew> alternative would be: Oh, another YAML alterna

Re: [HACKERS] RLS Design

On Sun, Sep 14, 2014 at 11:38 AM, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: >> On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost wrote: >> > If we want to be able to disable RLS w/o dropping the policies, then I >> > think we have to completely de-couple the two and users w

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

On Fri, Sep 19, 2014 at 11:05 PM, Rahila Syed wrote: > >>Please find attached patch to compress FPW using pglz compression. > Please refer the updated patch attached. The earlier patch added few > duplicate lines of code in guc.c file. > > compress_fpw_v1.patch >

Re: [HACKERS] Final Patch for GROUPING SETS

On 19/09/14 17:52, Andres Freund wrote: On 2014-09-19 16:35:52 +0100, Andrew Gierth wrote: Marti> But is anyone actually using YAML output format, or was it Marti> implemented simply "because we can"? Until someone decides to dike it out, I think we are obligated to make it produce somethin

Re: [HACKERS] RLS Design

On 2014-09-19 11:53:06 -0400, Robert Haas wrote: > On Sun, Sep 14, 2014 at 11:38 AM, Stephen Frost wrote: > > * Robert Haas (robertmh...@gmail.com) wrote: > >> On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost wrote: > >> > If we want to be able to disable RLS w/o dropping the policies, then I > >>

[HACKERS] CreateEventTrigStmt copy fix

Hi hackers, I was trying to create event trigger inside DO statement inside an extension SQL script and noticed that the new event trigger has empty evtevent field. After some tinkering with gdb I found out that the memory context switches sometimes clear the eventname in CreateEventTrigStmt s

Re: [HACKERS] RLS Design

On 14 September 2014 16:38, Stephen Frost wrote: > * Robert Haas (robertmh...@gmail.com) wrote: > > On Thu, Sep 11, 2014 at 3:08 PM, Stephen Frost > wrote: > > > If we want to be able to disable RLS w/o dropping the policies, then I > > > think we have to completely de-couple the two and users w

Re: [HACKERS] RLS Design

Thom, Thanks! * Thom Brown (t...@linux.com) wrote: > On 14 September 2014 16:38, Stephen Frost wrote: > # create policy visible_colours on colours for all to joe using (visible = > true); > CREATE POLICY [...] > > insert into colours (name, visible) values ('transparent',false); > ERROR: new ro

Re: [HACKERS] RLS Design

Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Sep 14, 2014 at 11:38 AM, Stephen Frost wrote: > > Alright, updated patch attached which does just that (thanks to Adam > > for the updates for this and testing pg_dump- I just reviewed it and > > added some documentation updates and

Re: [HACKERS] RLS Design

On 19 September 2014 17:32, Stephen Frost wrote: > Thom, > > Thanks! > > * Thom Brown (t...@linux.com) wrote: > > On 14 September 2014 16:38, Stephen Frost wrote: > > # create policy visible_colours on colours for all to joe using (visible > = > > true); > > CREATE POLICY > [...] > > > insert in

Re: [HACKERS] RLS Design

On 2014-09-19 12:38:39 -0400, Stephen Frost wrote: > I would not (nor do I feel that I did..) have committed it over a > specific request to not do so from another committer. I had been hoping > that there would be another review coming from somewhere, but there is > always a trade-off between wai

Re: [HACKERS] RLS Design

On Fri, Sep 19, 2014 at 12:38 PM, Stephen Frost wrote: >> This patch, on the other hand, was massively revised after the start >> of the CommitFest after many months of inactivity and committed with >> no thorough review by anyone who was truly independent of the >> development effort. It was the

Re: [HACKERS] RLS Design

Thom, * Thom Brown (t...@linux.com) wrote: > On 19 September 2014 17:32, Stephen Frost wrote: > > * Thom Brown (t...@linux.com) wrote: > > > On 14 September 2014 16:38, Stephen Frost wrote: > > > # create policy visible_colours on colours for all to joe using (visible > > = > > > true); > > > CR

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

On Tue, Sep 16, 2014 at 4:55 PM, Peter Geoghegan wrote: > On Tue, Sep 16, 2014 at 1:45 PM, Robert Haas wrote: >> Even though our testing seems to indicate that the memcmp() is >> basically free, I think it would be good to make the effort to avoid >> doing memcmp() and then strcoll() and then str

Re: [HACKERS] Minor improvement in lock.sgml

On Tue, Sep 16, 2014 at 7:20 AM, Etsuro Fujita wrote: > Here is a patch to a bit improve the reference page for the LOCK > command. I think it'd be better for the isolation level to be in > capitals and wrapped in the tags. It's done that way elsewhere in the same page, so committed. Overall,

Re: [HACKERS] Support for N synchronous standby servers

On Tue, Sep 16, 2014 at 2:19 PM, Michael Paquier wrote: > On Tue, Sep 16, 2014 at 5:25 AM, Robert Haas wrote: >> On Mon, Sep 15, 2014 at 3:00 PM, Michael Paquier >> wrote: >>> At least a set of hooks has the merit to say: do what you like with >>> your synchronous node policy. >> >> Sure. I dun

Re: [HACKERS] Final Patch for GROUPING SETS

On 09/19/2014 08:52 AM, Andres Freund wrote: >> Until someone decides to dike it out, I think we are obligated to make >> > it produce something resembling correct output. > I vote for ripping it out. There really isn't any justification for it > and it broke more than once. (a) I personally use i

Re: [HACKERS] CreateEventTrigStmt copy fix

On Fri, Sep 19, 2014 at 11:09 AM, Petr Jelinek wrote: > Hi hackers, > > I was trying to create event trigger inside DO statement inside an extension > SQL script and noticed that the new event trigger has empty evtevent field. > After some tinkering with gdb I found out that the memory context swi

Re: [HACKERS] removing volatile qualifiers from lwlock.c

On Wed, Sep 17, 2014 at 7:45 AM, Andres Freund wrote: > I just tried this on my normal x86 workstation. I applied your lwlock > patch and ontop I removed most volatiles (there's a couple still > required) from xlog.c. Works for 100 seconds. Then I reverted the above > commits. Breaks within second

Re: [HACKERS] Final Patch for GROUPING SETS

> "Josh" == Josh Berkus writes: Josh> (b) If we're going to discuss ripping out YAML format, please Josh> let's do that as a *separate* patch and discussion, +infinity >> Grouping Sets: >> - ["two","four"] >> - ["two"] >> - [] >> >> Would that be better? (It's not consistent

Re: [HACKERS] Turning off HOT/Cleanup sometimes

On Mon, Sep 15, 2014 at 5:13 PM, Simon Riggs wrote: > On 15 September 2014 17:09, Robert Haas wrote: >> Do we really want to disable HOT for all catalog scans? > > The intention of the patch is that catalog scans are treated > identically to non-catalog scans. The idea here is that HOT cleanup >

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

On Fri, Sep 19, 2014 at 9:59 AM, Robert Haas wrote: > OK, good point. So committed as-is, then, except that I rewrote the > comments, which I felt were excessively long for the amount of code. Thanks! I look forward to hearing your thoughts on the open issues with the patch as a whole. -- Pete

Re: [HACKERS] removing volatile qualifiers from lwlock.c

On 2014-09-19 13:58:17 -0400, Robert Haas wrote: > On Wed, Sep 17, 2014 at 7:45 AM, Andres Freund wrote: > > I just tried this on my normal x86 workstation. I applied your lwlock > > patch and ontop I removed most volatiles (there's a couple still > > required) from xlog.c. Works for 100 seconds.

Re: [HACKERS] Anonymous code block with parameters

On Fri, Sep 19, 2014 at 9:26 AM, Merlin Moncure wrote: > On Thu, Sep 18, 2014 at 5:22 PM, Hannu Krosing wrote: >> Though it would be even nicer to have fully in-line type definition >> >> SELECT (tup).* FROM >> ( >> SELECT CASE WHEN .. THEN ROW(1,2,3)::(a int, b text, c int2) >>

Re: [HACKERS] Anonymous code block with parameters

On 2014-09-19 8:20 PM, Merlin Moncure wrote: actually, this trick *only* works during json serialization -- it allows control over the column names that row() masks over. trying to expand (tup).* still gives the dreaded "ERROR: record type has not been registered". That's because this works:

Re: [HACKERS] Turning off HOT/Cleanup sometimes

On 19 September 2014 13:04, Robert Haas wrote: > What I'm thinking about is that the smarts to enable pruning is all in > the executor nodes. So anything that updates the catalog without > going through the executor will never be subject to pruning. That > includes nearly all catalog-modifying

Re: [HACKERS] Turning off HOT/Cleanup sometimes

On September 19, 2014 10:16:35 PM CEST, Simon Riggs wrote: >On 19 September 2014 13:04, Robert Haas wrote: > >> What I'm thinking about is that the smarts to enable pruning is all >in >> the executor nodes. So anything that updates the catalog without >> going through the executor will never be

Re: [HACKERS] Turning off HOT/Cleanup sometimes

Andres Freund writes: > On September 19, 2014 10:16:35 PM CEST, Simon Riggs > wrote: >> Are you saying this is a problem or a benefit? (and please explain >> why). > I have no idea what Robert is thinking of, but I'd imagine its horrible for > workloads with catalog bloat. Like ones involving

Re: [HACKERS] Turning off HOT/Cleanup sometimes

On Fri, Sep 19, 2014 at 4:30 PM, Andres Freund wrote: > On September 19, 2014 10:16:35 PM CEST, Simon Riggs > wrote: >>On 19 September 2014 13:04, Robert Haas wrote: >> >>> What I'm thinking about is that the smarts to enable pruning is all >>in >>> the executor nodes. So anything that updates

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

On Thu, Sep 11, 2014 at 8:34 PM, Peter Geoghegan wrote: > On Tue, Sep 9, 2014 at 2:25 PM, Robert Haas wrote: >>> I like that I don't have to care about every combination, and can >>> treat abbreviation abortion as the special case with the extra step, >>> in line with how I think of the optimizat

Re: [HACKERS] Turning off HOT/Cleanup sometimes

On 2014-09-19 17:29:08 -0400, Robert Haas wrote: > > I generally have serious doubts about disabling it generally for > > read workloads. I imagine it e.g. will significantly penalize > > workloads where its likely that a cleanup lock can't be acquired > > every time... > > I share that doubt. Bu

Re: [HACKERS] Turning off HOT/Cleanup sometimes

On 19 September 2014 15:35, Tom Lane wrote: > Having said that, this could reasonably be considered outside the scope > of a patch that's trying to improve the behavior for user queries. > But if the patch author doesn't want to expand the scope like that, > ISTM he ought to ensure that the behav

Re: [HACKERS] Turning off HOT/Cleanup sometimes

On 2014-09-19 16:35:19 -0400, Tom Lane wrote: > Andres Freund writes: > > On September 19, 2014 10:16:35 PM CEST, Simon Riggs > > wrote: > >> Are you saying this is a problem or a benefit? (and please explain > >> why). > > > I have no idea what Robert is thinking of, but I'd imagine its horrib

Re: [HACKERS] B-Tree support function number 3 (strxfrm() optimization)

On Fri, Sep 19, 2014 at 2:35 PM, Robert Haas wrote: > Also, shouldn't you go back and fix up > those abbreviated keys to point to datum1 again if you abort? Probably not - it appears to make very little difference to unoptimized pass-by-reference types whether or not datum1 can be used (see my si

Re: [HACKERS] RLS Design

Thom, Also, I seem to get an error message with the following: > > # create policy nice_colours ON colours for all to joe using (visible = > true) with check (name in ('blue','green','yellow')); > CREATE POLICY > > \c - joe > > > insert into colours (name, visible) values ('blue',false); > ERROR:

Re: [HACKERS] pg_xlogdump --stats

Andres Freund wrote: > Hi, > > On 2014-09-19 15:39:37 +0530, Abhijit Menon-Sen wrote: > > I've attached the revised patch, split up differently: > > > > 1. Introduce rm_identify, change rm_desc, glue the two together in > >xlog.c > > 2. Introduce pg_xlogdump --stats[=record]. > > I've pushed

Re: [HACKERS] pg_xlogdump --stats

On 2014-09-19 19:34:08 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > Hi, > > > > On 2014-09-19 15:39:37 +0530, Abhijit Menon-Sen wrote: > > > I've attached the revised patch, split up differently: > > > > > > 1. Introduce rm_identify, change rm_desc, glue the two together in > > >xl

Re: [HACKERS] Commitfest status

CF3 is actually over for a couple of days, wouldn't it be better to bounce back patches marked as "waiting on author" and work on the rest needing review? -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql

Re: [HACKERS] Add CREATE support to event triggers

On Tue, Aug 26, 2014 at 11:14 PM, Michael Paquier wrote: > And a last one before lunch, closing the review for all the basic things... > Patch 13: Could you explain why this is necessary? > +extern PGDLLIMPORT bool creating_extension; > It may make sense by looking at the core features (then why i

Re: [HACKERS] pg_shmem_allocations view

On Mon, Aug 18, 2014 at 1:12 PM, Robert Haas wrote: > On Mon, Aug 18, 2014 at 1:27 PM, Tom Lane wrote: >> Robert Haas writes: >>> I thought you were printing actual pointer addresses. If you're just >>> printing offsets relative to wherever the segment happens to be >>> mapped, I don't care abo

Re: [HACKERS] Support for N synchronous standby servers

On Fri, Sep 19, 2014 at 12:18 PM, Robert Haas wrote: > On Tue, Sep 16, 2014 at 2:19 PM, Michael Paquier > wrote: >> On Tue, Sep 16, 2014 at 5:25 AM, Robert Haas wrote: >>> On Mon, Sep 15, 2014 at 3:00 PM, Michael Paquier >>> wrote: At least a set of hooks has the merit to say: do what you

Re: [HACKERS] RLS Design

> "Adam" == Brightwell, Adam > writes: Adam> At any rate, this appears to be a previously existing issue Adam> with WITH CHECK OPTION. Thoughts? It's definitely an existing issue; you can reproduce it more simply, no need to mess with different users. The issue as far as I can tell