Re: [HACKERS] Backup throttling

2013-12-05 Thread Antonin Houska
On 12/02/2013 02:23 PM, Boszormenyi Zoltan wrote: > Hi, > > I am reviewing your patch. Thanks. New version attached. > > * Does it follow the project coding guidelines? > > Yes. A nitpicking: this else branch below might need brackets > because there is also a comment in that branch: > > +

[HACKERS] [bug fix] pg_ctl always uses the same event source

2013-12-05 Thread MauMau
Hello, I've removed a limitation regarding event log on Windows with the attached patch. I hesitate to admit this is a bug fix and want to regard this an improvement, but maybe it's a bug fix from users' perspective. Actually, I received problem reports from some users. [Problem] pg_ctl a

Re: [HACKERS] Regression tests failing if not launched on db "regression"

2013-12-05 Thread Tom Lane
Michael Paquier writes: > It happens that the following regression tests are failing if they are > run on a database not named "regression": This does not seem like a bug to me, although maybe we'd better update the documentation to specify that you need to use a DB named regression.

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-05 08:58:55 -0500, Tom Lane wrote: > Andres Freund writes: > > I don't think we can get rid of that dance in record_image_eq - it very > > well could used on records originally generated when those bits haven't > > been guaranteed to be zeroed. > > No, you're failing to think about the

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Andres Freund
On 2013-12-05 15:57:22 +0200, Heikki Linnakangas wrote: > As a side-note, I've been thinking that we don't really need same-address > mapping for shared_buffers either. Getting rid of it wouldn't buy us > anything right now, but if we wanted e.g to make shared_buffers changeable > without a restart

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Tom Lane
Andres Freund writes: > On 2013-12-04 18:48:44 -0500, Robert Haas wrote: >> And record_image_eq does a rather elaborate dance around here, calling >> the appropriate GET_x_BYTES macro depending on the type-width. If we >> can really count on the high-order bits to be zero, that's all >> completel

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Heikki Linnakangas
On 12/05/2013 06:32 AM, Robert Haas wrote: During development of the dynamic shared memory facility, Noah and I spent a lot of time arguing about whether it was practical to ensure that a dynamic shared memory segment got mapped at the same address in every backend that used it. My vote goes fo

[HACKERS] Feature request: Logging SSL connections

2013-12-05 Thread Dr. Andreas Kunert
Hello, we were really missing the information in our log files if (and which of) our users are using SSL during their connections. The attached patch is a very simple solution to this problem - it just tests if the ssl pointer in Port is null. If no, it adds "SSL" to the logfile, otherwise i

Re: [HACKERS] Changes in Trigger Firing

2013-12-05 Thread Sameer Kumar
One scenario where I can forsee an issue is when someone uses the tablename with-in the trigger function on which the trigger was fired. I am not sure how and what issue might crop up but this will be one of my test cases. -- View this message in context: http://postgresql.1045698.n5.nabble.com

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-05 Thread Andres Freund
Hi, On 2013-12-05 10:42:35 -0300, Alvaro Herrera wrote: > I intend to apply these two to 9.3 and master, and > then apply your freeze fix on top (which I'm cleaning up a bit -- will > resend later.) I sure hope it get's cleaned up - it's an evening's hack, with a good glass of wine ontop. Do you

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Stephen Frost
* Peter Geoghegan (p...@heroku.com) wrote: > On Wed, Dec 4, 2013 at 11:07 AM, Josh Berkus wrote: > > But you know what? 2.6, overall, still performs better than any kernel > > in the 3.X series, at least for Postgres. > > What about the fseek() scalability issue? Not to mention that the 2.6 whi

Re: [HACKERS] pgsql: Fix a couple of bugs in MultiXactId freezing

2013-12-05 Thread Alvaro Herrera
Andres Freund wrote: > On 2013-12-03 19:55:40 -0300, Alvaro Herrera wrote: > > I added a new isolation spec to test this specific case, and noticed > > something that seems curious to me when that test is run in REPEATABLE > > READ mode: when the UPDATE is aborted, the concurrent FOR UPDATE gets a

Re: [HACKERS] Changes in Trigger Firing

2013-12-05 Thread Sameer Kumar
I have been finally able to get the right set of files. I going with below approach: 1) Add a new column in pg_trigger called tgiscascaded 2) Change pg_trigger.h for this 3) Made changes in trigger.c to insert this values 4) Corresponding changes made in - reltrigger.h

Re: [HACKERS] shared memory message queues

2013-12-05 Thread Andres Freund
Hi, Planned to look at this for a while... Not a detailed review, just some thoughts. I'll let what I read sink in and possibly comment later. On 2013-10-31 12:21:31 -0400, Robert Haas wrote: > The attached patches attempt to rectify some of these problems. Well, I wouldn't call it problems. Jus

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Heikki Linnakangas
On 12/05/2013 07:30 AM, Tom Lane wrote: Peter Eisentraut writes: On Wed, 2013-12-04 at 20:27 -0500, Tom Lane wrote: Lazy people? I'm not in a hurry to drop it; it's not costing us much to just sit there, other than in this connection which we see how to fix. Actually, I think it probably c

Re: [HACKERS] logical changeset generation v6.7

2013-12-05 Thread Kyotaro HORIGUCHI
Hello, > Will send the rebased version as soon as I've addressed your comments. Thank you. > > = 0001: > > > > - You assined HeapTupleGetOid(tuple) into relid to read in > >several points but no modification. Nevertheless, you left > >HeapTupleGetOid not replaced there. I think 're

Re: [HACKERS] [bug fix] pg_ctl fails with config-only directory

2013-12-05 Thread MauMau
From: "Amit Kapila" On Wed, Dec 4, 2013 at 7:57 PM, MauMau wrote: * Approach 1 When postgres starts, it removes Administrator privileges from its own process. But is this possible at all? Windows security API is complex and provides many functions. It seems difficult to understand them. I

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Robert Haas
On Thu, Dec 5, 2013 at 4:56 AM, Andres Freund wrote: > Hi Robert, > > On 2013-12-04 23:32:27 -0500, Robert Haas wrote: >> But I'm also learning painfully that this kind of thing only goes so >> far. For example, I spent some time looking at what it would take to >> provide a dynamic shared memory

Re: [HACKERS] tracking commit timestamps

2013-12-05 Thread Andres Freund
On 2013-12-02 02:39:55 -0500, Jaime Casanova wrote: > === performance === > > i expected a regression on performance with the module turned on > because of the new XLOG records and wrote of files in pg_committs but > the performance drop is excessive. > Master 437.835674 tps > Patch, g

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Pavel Golub
Hello, Andres. You wrote: AF> On 2013-12-04 20:25:53 -0500, Tom Lane wrote: >> Tatsuo Ishii writes: >> > I would like to add a variant of regclass, which is exactly same as >> > current regclass except it does not raise an error when the target >> > table is not found. Instead it returns Invalid

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Andres Freund
On 2013-12-04 18:48:44 -0500, Robert Haas wrote: > * When a type narrower than Datum is stored in a Datum, we place it in the > * low-order bits and are careful that the DatumGetXXX macro for it discards > * the unused high-order bits (as opposed to, say, assuming they are zero). > * This is ne

Re: [HACKERS] [bug fix] "pg_ctl stop" times out when it should respond quickly

2013-12-05 Thread MauMau
From: "Tom Lane" If you're going to do a postmaster_is_alive check, why bother with repeated get_pgpid()? As I said yesterday, I removed get_pgpid() calls. I'll add this patch to 2014-1 commitfest this weekend if it is not committed until then. Regards MauMau pg_stop_fail_v2.patch Descri

Re: [HACKERS] Get more from indices.

2013-12-05 Thread Etsuro Fujita
I wrote: > Kyotaro HORIGUCHI wrote: > > Another is, you changed pathkeys expantion to be all-or-nothing decision. > > While this change should simplify the code slightly, it also dismisses > > the oppotunity for partially-extended pathkeys. Could you let me know > > the > reason > > why you did so.

Re: [HACKERS] Get more from indices.

2013-12-05 Thread Etsuro Fujita
Kyotaro HORIGUCHI wrote: > Thank you, but it seems to me too simplified. You made two major functional > changes. Thank you for the comments! > One is, you put the added code for getrelation_info() out of the block for > the condition (info->relam == BTREE_AM_OID) (though amcanorder would be > pr

Re: [HACKERS] better atomics - v0.2

2013-12-05 Thread Andres Freund
On 2013-11-19 10:37:35 -0500, Tom Lane wrote: > Andres Freund writes: > > The only animal we have that doesn't support quiet inlines today is > > HP-UX/ac++, and I think - as in patch 1 in the series - we might be able > > to simply suppress the warning there. > > Or just not worry about it, if i

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Pavel Stehule
2013/12/5 Andres Freund > On 2013-12-05 11:54:20 +0100, Pavel Stehule wrote: > > 2013/12/5 Andres Freund > > We can introduce some assert polymorphic function > > > > CREATE OR REPLACE FUNCTION notnull(any, message text) RETURNS any, that > can > > be used for check inside SQL > > Uh. How is tha

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Andres Freund
On 2013-12-04 20:55:08 -0500, Peter Eisentraut wrote: > On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: > >If an application wants to allow these connection parameters to be > > used, it would need to do PQenableStartServer() first. If it doesn't, > > those connection parameters will be r

Re: [HACKERS] Proof of concept: standalone backend with full FE/BE protocol

2013-12-05 Thread Simon Riggs
On 5 December 2013 01:55, Peter Eisentraut wrote: > On Thu, 2013-11-14 at 12:11 +0530, Amit Kapila wrote: >>If an application wants to allow these connection parameters to be >> used, it would need to do PQenableStartServer() first. If it doesn't, >> those connection parameters will be rejecte

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Andres Freund
On 2013-12-05 11:54:20 +0100, Pavel Stehule wrote: > 2013/12/5 Andres Freund > We can introduce some assert polymorphic function > > CREATE OR REPLACE FUNCTION notnull(any, message text) RETURNS any, that can > be used for check inside SQL Uh. How is that going to help applications that upgraded

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Pavel Stehule
2013/12/5 Andres Freund > On 2013-12-04 20:25:53 -0500, Tom Lane wrote: > > Tatsuo Ishii writes: > > > I would like to add a variant of regclass, which is exactly same as > > > current regclass except it does not raise an error when the target > > > table is not found. Instead it returns Invalid

Re: [HACKERS] Proposal: variant of regclass

2013-12-05 Thread Andres Freund
On 2013-12-04 20:25:53 -0500, Tom Lane wrote: > Tatsuo Ishii writes: > > I would like to add a variant of regclass, which is exactly same as > > current regclass except it does not raise an error when the target > > table is not found. Instead it returns InvalidOid (0). > > I've sometimes thought

Re: [HACKERS] Performance optimization of btree binary search

2013-12-05 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 5:28 PM, Peter Geoghegan wrote: > I'm also curious about the impact on insertion into primary key > indexes. Presently, we hold an exclusive buffer lock for the duration > of a couple of operations when checkUnique != UNIQUE_CHECK_NO. > _bt_binsrch() is one such operation. T

Re: [HACKERS] Changes in Trigger Firing

2013-12-05 Thread Sameer Kumar
> > > I read somewhere that "the best editor is the > one you master" (1) :) > 1: http://www.postgresql.org/message-id/m2wrs6giyp@hi-media.com > > Thanks, I am using eclipse now. Any comments about the utility of this feature? Or is it just me who thinks this can be useful? I think users/dev

Re: [HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-12-05 Thread Pavel Stehule
2013/12/5 Dean Rasheed > On 5 December 2013 01:33, Peter Eisentraut wrote: > > Can someone in this thread clarify the commit fest situation? I see two > > entries that appear to be the same: > > > > https://commitfest.postgresql.org/action/patch_view?id=1174 > > https://commitfest.postgresql.or

Re: [HACKERS] Re: [BUGS] BUG #7873: pg_restore --clean tries to drop tables that don't exist

2013-12-05 Thread Pavel Stehule
Hello 2013/12/5 Peter Eisentraut > Can someone in this thread clarify the commit fest situation? I see two > entries that appear to be the same: > > https://commitfest.postgresql.org/action/patch_view?id=1174 > https://commitfest.postgresql.org/action/patch_view?id=1175 > > I think the first o

Re: [HACKERS] Time-Delayed Standbys

2013-12-05 Thread Simon Riggs
On 5 December 2013 08:51, Magnus Hagander wrote: > Not recalling the older thread, but it seems the "breaks on clock drift", I > think we can fairly easily make that situation "good enough". Just have > IDENTIFY_SYSTEM return the current timestamp on the master, and refuse to > start if the time

Re: [HACKERS] same-address mappings vs. relative pointers

2013-12-05 Thread Andres Freund
Hi Robert, On 2013-12-04 23:32:27 -0500, Robert Haas wrote: > But I'm also learning painfully that this kind of thing only goes so > far. For example, I spent some time looking at what it would take to > provide a dynamic shared memory equivalent of palloc/pfree, a facility > that I feel fairly s

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Andres Freund
On 2013-12-05 11:33:29 +0200, Metin Doslu wrote: > > Is your workload bigger than RAM? > > RAM is bigger than workload (more than a couple of times). > > I think a good bit of the contention > > you're seeing in that listing is populating shared_buffers - and might > > actually vanish once you're

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Metin Doslu
> Is your workload bigger than RAM? RAM is bigger than workload (more than a couple of times). > I think a good bit of the contention > you're seeing in that listing is populating shared_buffers - and might > actually vanish once you're halfway cached. > From what I've seen so far the bigger prob

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Andres Freund
On 2013-12-05 11:15:20 +0200, Metin Doslu wrote: > > - When we increased NUM_BUFFER_PARTITIONS to 1024, this problem is > > disappeared for 8 core machines and come back with 16 core machines on > > Amazon EC2. Would it be related with PostgreSQL locking mechanism? > > If we build with -DLWLOCK_ST

Re: [HACKERS] Parallel Select query performance and shared buffers

2013-12-05 Thread Metin Doslu
> - When we increased NUM_BUFFER_PARTITIONS to 1024, this problem is > disappeared for 8 core machines and come back with 16 core machines on > Amazon EC2. Would it be related with PostgreSQL locking mechanism? If we build with -DLWLOCK_STATS to print locking stats from PostgreSQL, we see tons of

Re: [HACKERS] Time-Delayed Standbys

2013-12-05 Thread Magnus Hagander
On Thu, Dec 5, 2013 at 1:45 AM, Simon Riggs wrote: > On 3 December 2013 18:46, Robert Haas wrote: > > On Tue, Dec 3, 2013 at 12:36 PM, Fabrízio de Royes Mello > > wrote: > >> On Tue, Dec 3, 2013 at 2:33 PM, Christian Kruse < > christ...@2ndquadrant.com> > >> wrote: > >>> > >>> Hi Fabrizio, > >>

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread Peter Geoghegan
On Wed, Dec 4, 2013 at 11:07 AM, Josh Berkus wrote: > I also wasn't exaggerating the reception I got when I tried to talk > about IO and PostgreSQL at LinuxCon and other events. The majority of > Linux hackers I've talked to simply don't want to be bothered with > PostgreSQL's performance needs,

Re: [HACKERS] Why we are going to have to go DirectIO

2013-12-05 Thread KONDO Mitsumasa
(2013/12/04 16:39), Claudio Freire wrote: On Wed, Dec 4, 2013 at 4:28 AM, Tatsuo Ishii wrote: Can we avoid the Linux kernel problem by simply increasing our shared buffer size, say up to 80% of memory? It will be swap more easier. Is that the case? If the system has not enough memory, the ke

Re: [HACKERS] Regression tests failing if not launched on db "regression"

2013-12-05 Thread Michael Paquier
On Thu, Dec 5, 2013 at 5:12 PM, Michael Paquier wrote: > Those tests are failing because some relations of information_schemas > contain information that are database-dependent. I forgot to mention that it is our QE team at VMware that reported me a portion of this failure, and I just had to dig a

[HACKERS] Regression tests failing if not launched on db "regression"

2013-12-05 Thread Michael Paquier
Hi all, It happens that the following regression tests are failing if they are run on a database not named "regression": - updatable_views - foreign_data - sequence Those tests are failing because some relations of information_schemas contain information that are database-dependent. Please see the

Re: [HACKERS] Get more from indices.

2013-12-05 Thread Kyotaro HORIGUCHI
Hello, > > I've modified the patch to work in such a way. Also, as ISTM the patch > > is more complicated than what the patch really does, I've simplified the > > patch. > > I've revised the patch a bit. Please find attached the patch. Thank you, but it seems to me too simplified. You made tw

<    1   2