Re: [HACKERS] GIN improvements part 1: additional information

2013-12-18 Thread Oleg Bartunov
Guys, before digging deep into the art of comp/decomp world I'd like to know if you familiar with results of http://wwwconference.org/www2008/papers/pdf/p387-zhangA.pdf paper and some newer research ? Do we agree in what we really want ? Basically, there are three main features: size, compression

Re: [HACKERS] [9.3 bug] disk space in pg_xlog increases during archive recovery

2013-12-18 Thread Fujii Masao
On Mon, Dec 16, 2013 at 9:22 PM, MauMau wrote: > Hi, Fujii san, > >> On Wed, Aug 7, 2013 at 7:03 AM, Fujii Masao wrote: >>> >>> On second thought, probably we cannot remove the restored WAL files early >>> because they might be required for fast promotion which is new feature in >>> 9.3. >>> In f

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Tatsuo Ishii
> I found that the psql tab-completion for ALTER SYSTEM SET has not been > implemented yet. > Attached patch does that. Barring any objections, I will commit this patch. Good catch! Best regards, -- Tatsuo Ishii SRA OSS, Inc. Japan English: http://www.sraoss.co.jp/index_en.php Japanese: http://ww

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-12-18 Thread Haribabu kommi
On 19 December 2013 05:31 Bruce Momjian wrote: > On Wed, Dec 11, 2013 at 10:22:32AM +, Haribabu kommi wrote: > > The make_absolute_path() function moving to port is changed in > similar > > way as Bruce Momjian approach. The psprintf is used to store the > error > > string which Occurred in the

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Fujii Masao
On Thu, Dec 19, 2013 at 12:08 PM, Amit Kapila wrote: > On Wed, Dec 18, 2013 at 8:25 PM, Tatsuo Ishii wrote: Is there any reason for the function returns int as it always returns 0 or 1. Maybe returns bool is better? >>> >> >> I have committed your patches. Thanks. > > Thank you very muc

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-12-18 Thread Peter Geoghegan
On Thu, Dec 12, 2013 at 4:18 PM, Peter Geoghegan wrote: > Both of these revisions have identical ad-hoc test cases included as > new files - see testcase.sh and upsert.sql. My patch doesn't have any > unique constraint violations, and has pretty consistent performance, > while yours has many uniqu

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Amit Kapila
On Wed, Dec 18, 2013 at 11:30 AM, Michael Paquier wrote: > On Wed, Dec 18, 2013 at 11:22 AM, Amit Kapila wrote: >> On Fri, Dec 13, 2013 at 7:57 PM, Heikki Linnakangas >> wrote: >>> Thanks, committed with some minor changes: >> >> Should this patch in CF app be moved to Committed Patches or is th

Re: [HACKERS] row security roadmap proposal

2013-12-18 Thread Craig Ringer
On 12/18/2013 11:14 PM, Robert Haas wrote: > To be clear, I wasn't advocating for a declarative approach; I think > predicates are fine. There are usability issues to worry about either > way, and my concern is that we address those. A declarative approach > would certainly be valuable in that,

Re: [HACKERS] clang's -Wmissing-variable-declarations shows some shoddy programming

2013-12-18 Thread Bruce Momjian
On Sat, Dec 14, 2013 at 04:52:28PM +0100, Andres Freund wrote: > Hi, > > Compiling postgres with said option in CFLAGS really gives an astounding > number of warnings. Except some bison/flex generated ones, none of them > looks acceptable to me. > Most are just file local variables with a missing

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Amit Kapila
On Wed, Dec 18, 2013 at 8:25 PM, Tatsuo Ishii wrote: >>> Is there any reason for the function returns int as it always returns >>> 0 or 1. Maybe returns bool is better? >> > > I have committed your patches. Thanks. Thank you very much. With Regards, Amit Kapila. EnterpriseDB: http://www.enterpr

Re: [HACKERS] preserving forensic information when we freeze

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 5:54 PM, Andres Freund wrote: >> if (frz->frzflags & XLH_FREEZE_XVAC) >> + { >> HeapTupleHeaderSetXvac(tuple, FrozenTransactionId); >> + /* If we somehow haven't hinted the tuple previously, do it >> now. */ >> + HeapTupleHea

Re: [HACKERS] Autoconf 2.69 update

2013-12-18 Thread Peter Eisentraut
On Thu, 2013-11-14 at 22:00 -0500, Peter Eisentraut wrote: > I'm proposing that we upgrade our Autoconf to 2.69 This has been done. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 6:07 PM, Cédric Villemain wrote: > In the case of effective_io_concurrency, however, this may not work as well as > expected, IIRC it is used to prefetch heap blocks, hopefully the requested > blocks are contiguous but if there are too much holes it is enough to fill the >

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Andreas Karlsson
On 12/18/2013 01:02 PM, Alexander Korotkov wrote: My idea for a solution was to modify tuplesort to allow storing the already sorted keys in either memtuples or the sort result file, but setting a field so it does not sort thee already sorted tuples again. This would allow the res

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Alvaro Herrera
Robert Haas escribió: > I'm not inclined to wait for the next CommitFest to commit this, > because it's a very simple patch and has already had a lot more field > testing than most patches get before they're committed. And it's just > a contrib module, so the damage it can do if there is in fact

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mercredi 18 décembre 2013 18:40:09, Robert Haas a écrit : > Now that we have dynamic background workers, I've been thinking that > it might be possible to write a background worker to do asynchronous > prefetch on systems where we don't have OS support. We could store a > small ring buffer in s

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-12-18 Thread Bruce Momjian
On Wed, Dec 11, 2013 at 10:22:32AM +, Haribabu kommi wrote: > The make_absolute_path() function moving to port is changed in similar way as > Bruce Momjian approach. The psprintf is used to store the error string which > Occurred in the function. But psprintf is not used for storing the absolut

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Josh Berkus wrote: > On 12/18/2013 11:26 AM, Jim Nasby wrote: >> Another possibility is to allow for two different types of >> assertions, one based on SSI and one based on locking. > > The locking version would have to pretty much lock on a table > basis (or even a whole-database basis) every ti

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
On 12/19/13, 12:01 AM, David Johnston wrote: Marko Tiikkaja-4 wrote On 2013-12-18 22:32, Andrew Dunstan wrote: You're not really free to assume it - you'll need an exception handler for the other-than-1 case, or your code might blow up. This seems to be codifying a bad pattern, which should be

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Bruce Momjian
On Wed, Dec 18, 2013 at 03:41:24PM -0500, Robert Haas wrote: > On the other hand, there's not much value in adding monitoring > features that are going to materially harm performance, and a lot of > the monitoring features that get proposed die on the vine for exactly > that reason. I think the ro

Re: [HACKERS] Assertion failure in base backup code path

2013-12-18 Thread Andres Freund
Hi Magnus, It looks to me like the path to do_pg_start_backup() outside of a transaction context comes from your initial commit of the base backup facility. The problem is that you're not allowed to do anything leading to a syscache/catcache lookup in those contexts. Greetings, Andres Freund --

Re: [HACKERS] preserving forensic information when we freeze

2013-12-18 Thread Andres Freund
Hi, On 2013-12-17 16:00:14 -0500, Robert Haas wrote: > @@ -5874,19 +5858,27 @@ heap_prepare_freeze_tuple(HeapTupleHeader tuple, > TransactionId cutoff_xid, > void > heap_execute_freeze_tuple(HeapTupleHeader tuple, xl_heap_freeze_tuple *frz) > { > + tuple->t_infomask = frz->t_infomask; > +

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread David Johnston
Marko Tiikkaja-4 wrote > On 2013-12-18 22:32, Andrew Dunstan wrote: >> You're not really free to assume it - you'll need an exception handler >> for the other-than-1 case, or your code might blow up. >> >> This seems to be codifying a bad pattern, which should be using >> array_lower() and array_up

Re: [HACKERS] shared memory message queues

2013-12-18 Thread Andres Freund
On 2013-12-18 15:23:23 -0500, Robert Haas wrote: > It sounds like most people who have looked at this stuff are broadly > happy with it, so I'd like to push on toward commit soon, but it'd be > helpful, Andres, if you could review the comment additions to > shm-mq-v2.patch and see whether those add

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
Applying your patch plus adding -fno-omit-frame-pointer, I got 54526.90 notpm. The profile (part) below: # Samples: 610K of event 'cycles' # Event count (approx.): 6686532056450 # # Overhead Command Shared Object Symbol # .. . ...

Re: [HACKERS] Extension Templates S03E11

2013-12-18 Thread Greg Stark
Yeah I think this whole discussion is happening at the wrong level. The problem you're having, despite appearances, is not that people disagree about the best way to accomplish your goals. The problem is that not everyone is convinced your goals are a good idea. Either they just don't understand t

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
On 2013-12-18 22:32, Andrew Dunstan wrote: You're not really free to assume it - you'll need an exception handler for the other-than-1 case, or your code might blow up. This seems to be codifying a bad pattern, which should be using array_lower() and array_upper() instead. That's the entire po

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 04:19 PM, Marko Tiikkaja wrote: On 2013-12-18 22:13, Andrew Dunstan wrote: On 12/18/2013 03:27 PM, Marko Tiikkaja wrote: Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the arr

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
On 2013-12-18 22:19, I wrote: On 2013-12-18 22:13, Andrew Dunstan wrote: On 12/18/2013 03:27 PM, Marko Tiikkaja wrote: Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the array's lower bound

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
On 2013-12-18 22:13, Andrew Dunstan wrote: On 12/18/2013 03:27 PM, Marko Tiikkaja wrote: Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the array's lower bound isn't 1. In other words, does

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Josh Berkus
On 12/18/2013 11:26 AM, Jim Nasby wrote: > The flip-side is that now you can get serialization failures, and I > think there's a ton of software that has no clue how to deal with that. > So now you don't get to use assertions at all unless you re-engineer > your application (but see below). Well,

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 04:09 PM, Heikki Linnakangas wrote: On 12/18/2013 11:04 PM, Andrew Dunstan wrote: On 12/18/2013 02:45 PM, Andres Freund wrote: On 2013-12-18 16:39:58 -0300, Alvaro Herrera wrote: Andres Freund wrote: It would only force serialization for transactions that modify tables covered

Re: [HACKERS] array_length(anyarray)

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 03:27 PM, Marko Tiikkaja wrote: Hi, Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the array's lower bound isn't 1. In other words, does the right thing when used with the

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 11:04 PM, Andrew Dunstan wrote: On 12/18/2013 02:45 PM, Andres Freund wrote: On 2013-12-18 16:39:58 -0300, Alvaro Herrera wrote: Andres Freund wrote: It would only force serialization for transactions that modify tables covered by the assert, that doesn't seem to bad. Anything c

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Stephen Frost
* Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Dec 18, 2013 at 8:47 AM, Stephen Frost wrote: > > Agreed. My other thought on this is that there's a lot to be said for > > having everything you need available through one tool- kinda like how > > Emacs users rarely go outside of it.. :) An

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 02:45 PM, Andres Freund wrote: On 2013-12-18 16:39:58 -0300, Alvaro Herrera wrote: Andres Freund wrote: It would only force serialization for transactions that modify tables covered by the assert, that doesn't seem to bad. Anything covered by an assert shoulnd't be modified frequ

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 8:47 AM, Stephen Frost wrote: > Agreed. My other thought on this is that there's a lot to be said for > having everything you need available through one tool- kinda like how > Emacs users rarely go outside of it.. :) And then there's also the > consideration that DBAs may

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Jim Nasby wrote: > On 12/18/13, 1:42 PM, Kevin Grittner wrote: >> Jim Nasby wrote: >> >>> This is another case where it would be very useful to restrict >>> what relations a transaction (or in this case, a substransaction) >>> can access. If we had the ability to make that restriction then >>> we

[HACKERS] array_length(anyarray)

2013-12-18 Thread Marko Tiikkaja
Hi, Attached is a patch to add support for array_length(anyarray), which only works for one-dimensional arrays, returns 0 for empty arrays and complains if the array's lower bound isn't 1. In other words, does the right thing when used with the arrays people use 99% of the time. I'll add th

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
Hi, Applied your patch (but not using -fno-omit-frame-pointer). It seems recover the perf loss: 55659.72 notpm. FWIW, the profile is below. I will do a run with the flag.. Samples: 598K of event 'cycles', Event count (approx.): 6568957160059 + 4.03% postgres postgres [.

Re: [HACKERS] SQL objects UNITs

2013-12-18 Thread Jim Nasby
On 12/18/13, 4:22 AM, Dimitri Fontaine wrote: ALTER UNIT name SET SCHEMA ; FWIW, with the "units" that we've developed we use schemas to differentiate between public objects and "internal" (private or protected) objects. So single-schema stuff becomes a PITA. Of course, since extensions

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
HI On Wed, Dec 18, 2013 at 3:03 PM, Andres Freund wrote: > > That looks like a postgres compiled without > -fno-omit-frame-pointer. Without that hierarchical profiles are > meaningless. Very new perfs can also do it using dwarf, but it's unusabl > slow... > > Let me complete current test without t

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Andres Freund
Hi, On 2013-12-18 14:59:45 -0500, Dong Ye wrote: > ~20 minutes each run with binary. > Try your patch now.. > You are right I used -g in perf record. But what I reported was flat (meant > as a start). > > Expand GetMultiXactIdMembers: > > 3.82% postgres postgres [.

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
~20 minutes each run with binary. Try your patch now.. You are right I used -g in perf record. But what I reported was flat (meant as a start). Expand GetMultiXactIdMembers: 3.82% postgres postgres [.] GetMultiXactIdMembers | |-

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Jim Nasby
On 12/18/13, 1:42 PM, Kevin Grittner wrote: Jim Nasby wrote: This is another case where it would be very useful to restrict what relations a transaction (or in this case, a substransaction) can access. If we had the ability to make that restriction then we could force assertions that aren't pl

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andres Freund
On 2013-12-18 16:39:58 -0300, Alvaro Herrera wrote: > Andres Freund wrote: > > It would only force serialization for transactions that modify tables > > covered by the assert, that doesn't seem to bad. Anything covered by an > > assert shoulnd't be modified frequently, otherwise you'll run into maj

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Jim Nasby wrote: > This is another case where it would be very useful to restrict > what relations a transaction (or in this case, a substransaction) > can access. If we had the ability to make that restriction then > we could force assertions that aren't plain SQL to explicitly > specify what ta

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Alvaro Herrera
Andres Freund wrote: > On 2013-12-18 13:44:15 -0300, Alvaro Herrera wrote: > > Heikki Linnakangas wrote: > > > > > Ah, I see. You don't need to block anyone else from modifying the > > > table, you just need to block anyone else from committing a > > > transaction that had modified the table. So t

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Jim Nasby
On 12/18/13, 10:44 AM, Alvaro Herrera wrote: It might prove useful to note that any given assertion involves tables A, B, C. If a transaction doesn't modify any of those tables then there's no need to run the assertion when the transaction commits, skipping acquisition of the assertion lock. Pr

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Jim Nasby
On 12/18/13, 11:57 AM, Josh Berkus wrote: On 12/18/2013 08:44 AM, Alvaro Herrera wrote: Another thought: at the initial run of the assertion, note which tables it locked, and record this as an OID array in the catalog row for the assertion; consider running the assertion only when those tables a

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andres Freund
On 2013-12-18 13:44:15 -0300, Alvaro Herrera wrote: > Heikki Linnakangas wrote: > > > Ah, I see. You don't need to block anyone else from modifying the > > table, you just need to block anyone else from committing a > > transaction that had modified the table. So the locks shouldn't > > interfere

Re: [HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Andres Freund
Hello, On 2013-12-18 10:24:56 -0800, Dong Ye wrote: > It seems that 0ac5ad5134f2769ccbaefec73844f8504c4d6182 is the culprit > commit. How long does a run take to verify the problem? Could you retry with the patch attached to http://www.postgresql.org/message-id/20131201114514.gg18...@alap2.anaraz

[HACKERS] 9.3 regression with dbt2

2013-12-18 Thread Dong Ye
Hi, We recently observed ~15% performance regression with dbt2 from PG 9.3. We narrowed down on testing master between 9.2 cut and 9.3 cut. It seems that 0ac5ad5134f2769ccbaefec73844f8504c4d6182 is the culprit commit. We did several runs and perf profiling comparing it against its parent (f9

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Josh Berkus
On 12/18/2013 08:44 AM, Alvaro Herrera wrote: > Another thought: at the initial run of the assertion, note which tables > it locked, and record this as an OID array in the catalog row for the > assertion; consider running the assertion only when those tables are > touched. This doesn't work if the

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 7:05 PM, Cédric Villemain wrote: > Le mardi 17 décembre 2013 21:14:44, Josh Berkus a écrit : >> On 12/17/2013 06:34 AM, Robert Haas wrote: >> > On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila > wrote: >> >> I have used pg_prewarm during some of work related to Buffer Managem

Re: [HACKERS] Dynamic Shared Memory stuff

2013-12-18 Thread Robert Haas
On Tue, Dec 10, 2013 at 6:26 PM, Tom Lane wrote: > Noah Misch writes: >> On Tue, Dec 10, 2013 at 07:50:20PM +0200, Heikki Linnakangas wrote: >>> Let's not add more cases like that, if we can avoid it. > >> Only if we can avoid it for a modicum of effort and feature compromise. >> You're asking fo

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mardi 17 décembre 2013 21:14:44, Josh Berkus a écrit : > On 12/17/2013 06:34 AM, Robert Haas wrote: > > On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila wrote: > >> I have used pg_prewarm during some of work related to Buffer Management > >> and other performance related work. It is quite useful

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Cédric Villemain
Le mardi 17 décembre 2013 17:45:51, Robert Haas a écrit : > On Tue, Dec 17, 2013 at 11:02 AM, Jim Nasby wrote: > > On 12/17/13, 8:34 AM, Robert Haas wrote: > >> On Tue, Dec 17, 2013 at 12:09 AM, Amit Kapila > >> > >> wrote: > >>> I have used pg_prewarm during some of work related to Buffer Manag

Re: [HACKERS] RFC: Async query processing

2013-12-18 Thread Florian Weimer
On 11/04/2013 02:51 AM, Claudio Freire wrote: On Sun, Nov 3, 2013 at 3:58 PM, Florian Weimer wrote: I would like to add truly asynchronous query processing to libpq, enabling command pipelining. The idea is to to allow applications to auto-tune to the bandwidth-delay product and reduce the num

Re: [HACKERS] 9.3 reference constraint regression

2013-12-18 Thread Alvaro Herrera
Alvaro Herrera wrote: > Andres Freund wrote: > > > I have to say, it makes me really uncomfortable to take such > > shortcuts. In other branches we are doing liveliness checks with > > MultiXactIdIsRunning() et al. Why isn't that necessary here? And how > > likely is that this won't cause breakage

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Alvaro Herrera
Heikki Linnakangas wrote: > Ah, I see. You don't need to block anyone else from modifying the > table, you just need to block anyone else from committing a > transaction that had modified the table. So the locks shouldn't > interfere with regular table locks. A ShareUpdateExclusiveLock on > the as

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Josh Berkus wrote: > Serializable or not, *what* do we lock for assertions?  It's not > rows.  Tables?  Which tables?  What if the assertion is an > interpreted language function?  Does the SSI reference counter > really take care of all of this? The simple answer is that, without adding any add

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Sawada Masahiko
2013/12/14 0:14 "Tom Lane" : > > Heikki Linnakangas writes: > > I'm not totally satisfied with the name of the GUC, wal_log_hintbits. > > Me either; at the very least, it's short an underscore: wal_log_hint_bits > would be more readable. But how about just "wal_log_hints"? > +1 too. it's readble

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Kevin Grittner
Heikki Linnakangas wrote: >On 12/18/2013 01:39 PM, Andres Freund wrote: >> On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: >>> Here's another idea that doesn't involve SSI: >>> >>> At COMMIT, take a new snapshot and check that the assertion still passes >>> with that snapshot. Now, there's

Re: [HACKERS] 9.3 reference constraint regression

2013-12-18 Thread Alvaro Herrera
Andres Freund wrote: > I have to say, it makes me really uncomfortable to take such > shortcuts. In other branches we are doing liveliness checks with > MultiXactIdIsRunning() et al. Why isn't that necessary here? And how > likely is that this won't cause breakage somewhere down the line because >

Re: [HACKERS] Logging WAL when updating hintbit

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 10:22 PM, Amit Kapila wrote: >> Me either; at the very least, it's short an underscore: wal_log_hint_bits >> would be more readable. But how about just "wal_log_hints"? > > +1 for wal_log_hints, it sounds better. +1. -- Robert Haas EnterpriseDB: http://www.enterprisedb.

Re: [HACKERS] row security roadmap proposal

2013-12-18 Thread Robert Haas
On Wed, Dec 18, 2013 at 3:30 AM, Craig Ringer wrote: > In my view the proposed patch doesn't offer a significant improvement in > declarative security, beyond what we can get by just adding update > support to s.b. views and using search_path to control whether a user > sees the view or the base t

Re: [HACKERS] row security roadmap proposal

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 1:27 PM, Simon Riggs wrote: > Not sure I'd say required, but its certainly desirable to have > updateable security barrier views in themselves. And it comes across > to me as a cleaner and potentially more performant way of doing the > security checks for RLS. Yes, that's

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 9:03 PM, KONDO Mitsumasa wrote: > (2013/12/18 5:33), Robert Haas wrote: >> Sounds like it might be worth dusting the patch off again... > > I'd like to request you to add all_index option and usage_count option. > When all_index option is selected, all index become rewarm n

Re: SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Alvaro Herrera
Stephen Frost escribió: > * Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: > > Basically with building `UNIT` we realise with hindsight that we failed to > > build a proper `EXTENSION` system, and we send that message to our users. > > Little difficult to draw conclusions about what out 'hindsi

Re: [HACKERS] pg_rewarm status

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 12:35 PM, Jeff Janes wrote: > Since it doesn't use directIO, you can't warm the PG buffers without also > warming FS cache as a side effect. That is why I like 'buffer' as the > default--if the data fits in shared_buffers, it warm those, otherwise it at > least warms the F

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Tatsuo Ishii
>> Is there any reason for the function returns int as it always returns >> 0 or 1. Maybe returns bool is better? > > No, return type should be bool, I have changed the same in attached patch. Confirmed. >> 2) initdb.c >> >> + strcpy(tempautobuf, "# Do not edit this file manually! \n"); >>

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 01:45 PM, Alexander Korotkov wrote: On Tue, Dec 17, 2013 at 2:49 AM, Heikki Linnakangas wrote: On 12/17/2013 12:22 AM, Alexander Korotkov wrote: 2) Storage would be easily extendable to hold additional information as well. Better compression shouldn't block more serious impro

Re: SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Stephen Frost
* Dimitri Fontaine (dimi...@2ndquadrant.fr) wrote: > Here's my attempt: > > # Inline Extension, Extension Templates > > The problem with *Inline Extension* is the dump and restore policy. The > contents of an extensions are not be found in a `pg_dump` script, ever. You keep coming back to this a

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 06:00 AM, Heikki Linnakangas wrote: If you don't force everything to run in SSI mode, then you have to somehow figure out what parts do need to run in SSI mode to enforce the assertion. For example, if the assertion refers tables A and B, perhaps you can get away without predi

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Andrew Dunstan
On 12/18/2013 03:35 AM, Tatsuo Ishii wrote: 3) initdb.c It seems the memory allocated for autoconflines[0] and autoconflines[1] by pg_strdup is never freed. (I think there's similar problem with "conflines" as well, though it was not introduced by the patch). Why would we care? initdb does

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Stephen Frost
* Craig Ringer (cr...@2ndquadrant.com) wrote: > On 12/12/2013 02:51 AM, Tom Lane wrote: > > The thing that I'm wondering is why the database would be the right place > > to be measuring it at all. If you've got a network usage problem, > > aggregate usage across everything on the server is probabl

Re: [HACKERS] sepgsql: label regression test failed

2013-12-18 Thread Sergey Muraviov
# semodule -l | grep sepgslq sepgsql-regtest 1.07 Full list of modules is in attachment. 2013/12/18 Kohei KaiGai > Could you show me semodule -l on your environment? > I believe security policy has not been changed between F19 and F20... > > Thanks, > > 2013/12/18 Sergey Muraviov : > > Hi > >

Re: [HACKERS] sepgsql: label regression test failed

2013-12-18 Thread Kohei KaiGai
Could you show me semodule -l on your environment? I believe security policy has not been changed between F19 and F20... Thanks, 2013/12/18 Sergey Muraviov : > Hi > > I've tried to test postgres 9.3.2 and 9.4devel with selinux on Fedora 20 and > met with a label regression test failure. > > PS >

[HACKERS] sepgsql: label regression test failed

2013-12-18 Thread Sergey Muraviov
Hi I've tried to test postgres 9.3.2 and 9.4devel with selinux on Fedora 20 and met with a label regression test failure. PS I've got some warning during build process. -- Best regards, Sergey Muraviov regression.out Description: Binary data regression.diffs Description: Binary data warni

Re: [HACKERS] ALTER SYSTEM SET command to change postgresql.conf parameters

2013-12-18 Thread Amit Kapila
On Wed, Dec 18, 2013 at 2:05 PM, Tatsuo Ishii wrote: > Hi, > > I have looked into this because it's marked as "ready for committer". Thank you. > It looks like working as advertised. Great! However I have noticed a > few minor issues. > > 1) validate_conf_option > > +/* > + * Validates configur

Re: [HACKERS] commit fest 2013-11 final report

2013-12-18 Thread Robert Haas
On Tue, Dec 17, 2013 at 7:14 PM, Peter Eisentraut wrote: > On 12/17/13, 10:19 AM, Tom Lane wrote: >> Perhaps we should just move all the Needs Review and RFC patches forward >> to the next fest, so we don't forget about them? > > This was done the last few times, but it has caused some controversy

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 11:47 PM, Andreas Karlsson wrote: > On 12/14/2013 10:59 AM, Alexander Korotkov wrote: > >> This patch allows to use index for order-by if order-by clause and index >> has non-empty common prefix. So, index gives right ordering for first n >> order-by columns. In order to pr

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 01:50 PM, Andres Freund wrote: On 2013-12-18 13:46:59 +0200, Heikki Linnakangas wrote: On 12/18/2013 01:39 PM, Andres Freund wrote: On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: Here's another idea that doesn't involve SSI: At COMMIT, take a new snapshot and check that

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 7:04 PM, Andres Freund wrote: > Hi, > > > > > Limit (cost=69214.06..69214.08 rows=10 width=16) (actual > > > > time=0.097..0.099 rows=10 loops=1) > > > >-> Sort (cost=69214.06..71714.06 rows=100 width=16) (actual > > > > time=0.096..0.097 rows=10 loops=1) > > >

Re: [HACKERS] PoC: Partial sort

2013-12-18 Thread Alexander Korotkov
On Sat, Dec 14, 2013 at 6:39 PM, Martijn van Oosterhout wrote: > On Sat, Dec 14, 2013 at 06:21:18PM +0400, Alexander Korotkov wrote: > > > Is that actually all that beneficial when sorting with a bog standard > > > qsort() since that doesn't generally benefit from data being > pre-sorted? > > > I

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andres Freund
On 2013-12-18 13:46:59 +0200, Heikki Linnakangas wrote: > On 12/18/2013 01:39 PM, Andres Freund wrote: > >On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: > >>Here's another idea that doesn't involve SSI: > >> > >>At COMMIT, take a new snapshot and check that the assertion still passes > >>w

Re: [HACKERS] hstore ng index for <@ ?

2013-12-18 Thread Alexander Korotkov
Hi! On Wed, Dec 18, 2013 at 2:35 PM, Kaare Rasmussen wrote: > In many ways the new hstore (and perhaps json) format looks very exciting. > But will there ever be (GIN/GIST) index support for <@ ? It looks not hard to do with GiST. About GIN I don't have promising ideas: likely we can't effecti

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 01:39 PM, Andres Freund wrote: On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: Here's another idea that doesn't involve SSI: At COMMIT, take a new snapshot and check that the assertion still passes with that snapshot. Now, there's a race condition, if another transaction i

Re: [HACKERS] GIN improvements part 1: additional information

2013-12-18 Thread Alexander Korotkov
On Tue, Dec 17, 2013 at 2:49 AM, Heikki Linnakangas wrote: > On 12/17/2013 12:22 AM, Alexander Korotkov wrote: > >> On Mon, Dec 16, 2013 at 3:30 PM, Heikki Linnakangas < >> hlinnakan...@vmware.com >> >>> wrote: >>> >> >> On 12/12/2013 06:44 PM, Alexander Korotkov wrote: >>> >>> When values are

Re: [HACKERS] Example query causing param_info to be set in plain rel path

2013-12-18 Thread Ashutosh Bapat
I got an example where paths for plain rel require param_info i.e. plain rel scans require to take care of the lateral references. Here's the example from PG regression explain verbose select v.* from (int8_tbl x left join (select q1,(select coalesce(q2,0)) q2 from int8_tbl) y on x.q2 = y.q1)

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Andres Freund
On 2013-12-18 13:07:51 +0200, Heikki Linnakangas wrote: > Here's another idea that doesn't involve SSI: > > At COMMIT, take a new snapshot and check that the assertion still passes > with that snapshot. Now, there's a race condition, if another transaction is > committing at the same time, and per

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 02:59 AM, Josh Berkus wrote: On 12/17/2013 01:42 PM, Kevin Grittner wrote: It works fine as long as you set default_transaction_isolation = 'serializable' and never override that. :-) Of course, it sure would be nice to have a way to prohibit overrides, but that's another issue.

Re: [HACKERS] [PATCH] SQL assertions prototype

2013-12-18 Thread Heikki Linnakangas
On 12/18/2013 02:59 AM, Josh Berkus wrote: On 12/17/2013 01:42 PM, Kevin Grittner wrote: Josh Berkus wrote: Going back over this patch, I haven't seen any further discussion of the point Heikki raises above, which seems like a bit of a showstopper. Heikki, did you have specific ideas on how t

Re: [HACKERS] Extension Templates S03E11

2013-12-18 Thread Dimitri Fontaine
Tom Lane writes: > I keep telling you this, and it keeps not sinking in. How can you say that? I've been spending a couple of years on designing and implementing and arguing for a complete feature set where dealing with modules is avoided at all costs. The problem we have now is that I'm being t

[HACKERS] hstore ng index for <@ ?

2013-12-18 Thread Kaare Rasmussen
Hi In many ways the new hstore (and perhaps json) format looks very exciting. But will there ever be (GIN/GIST) index support for <@ ? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

SQL objects UNITs (was: [HACKERS] Extension Templates S03E11)

2013-12-18 Thread Dimitri Fontaine
Simon Riggs writes: > On 17 December 2013 23:42, Tom Lane wrote: >>> We aim to have the simplest implementation that meets the stated need >>> and reasonable extrapolations of that. Text in a catalog table is the >>> simplest implementation. That is not a reason to reject it, especially >>> when

Re: [HACKERS] stats for network traffic WIP

2013-12-18 Thread Craig Ringer
On 12/12/2013 02:51 AM, Tom Lane wrote: > The thing that I'm wondering is why the database would be the right place > to be measuring it at all. If you've got a network usage problem, > aggregate usage across everything on the server is probably what you > need to be worried about, and PG can't te

Re: [HACKERS] Problem with displaying "wide" tables in psql

2013-12-18 Thread Sergey Muraviov
Hello 2013/12/18 Sameer Thakur > On Wed, Dec 11, 2013 at 11:13 PM, Sergey Muraviov > wrote: > > Hi. > > > > I've improved the patch. > > It works in expanded mode when either format option is set to wrapped > (\pset > > format wrapped), or we have no pager, or pager doesn't chop long lines > (s

Re: [HACKERS] Proposal: variant of regclass

2013-12-18 Thread Tatsuo Ishii
>> For the pgpool-II use case, I'm happy to follow you because pgpool-II >> always does a grammatical check (using raw parser) on a query first >> and such syntax problem will be pointed out, thus never reaches to >> the state where calling toregclass. >> >> One concern is, other users expect toreg

  1   2   >