Re: [HACKERS] issue with gininsert under very high load

On 14/02/14 00:49, Tom Lane wrote: Andres Freund writes: On 2014-02-13 16:15:42 -0500, Tom Lane wrote: Something like the attached? Can somebody who's seen this problem confirm this improves matters? Hm. Won't that possiby lead to the fast tuple list growing unboundedly? I think we would nee

Re: [HACKERS] HBA files w/include support?

Alvaro Herrera writes: > Bruce Momjian wrote: > >> On Thu, Feb 13, 2014 at 08:24:27PM -0600, Jerry Sievers wrote: >> > Greetings Hackers. >> > >> > I'm aware of how a pg_hba.conf file can refer to other files for >> > including @lists of users, etc. >> > >> > But there is currently no support f

Re: [HACKERS] Per table autovacuum vacuum cost limit behaviour strange

I hadn't noticed this thread. I will give this a look. Thanks for providing a patch. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to

Re: [HACKERS] HBA files w/include support?

Bruce Momjian wrote: > On Thu, Feb 13, 2014 at 08:24:27PM -0600, Jerry Sievers wrote: > > Greetings Hackers. > > > > I'm aware of how a pg_hba.conf file can refer to other files for > > including @lists of users, etc. > > > > But there is currently no support for being able to pull in entire file

Re: [HACKERS] Long paths for tablespace leads to uninterruptible hang in Windows

On Tue, Jan 7, 2014 at 12:33:33PM +0530, Amit Kapila wrote: > On Thu, Oct 31, 2013 at 8:58 AM, Amit Kapila wrote: > > On Wed, Oct 16, 2013 at 1:44 AM, Amit Kapila > > wrote: > >> On Tue, Oct 15, 2013 at 6:28 PM, Magnus Hagander > >> wrote: > >>> On Tue, Oct 15, 2013 at 2:55 PM, Robert Haas

Re: [HACKERS] Changeset Extraction v7.0 (was logical changeset generation)

On Sun, 2014-01-19 at 15:31 +0100, Stefan Kaltenbrunner wrote: > > /* followings are for client encoding only */ > > PG_SJIS,/* Shift JIS > > (Winindows-932) */ > > while you have that file open: s/Winindows-932/Windows-932 maybe? done --

Re: [HACKERS] HBA files w/include support?

On Thu, Feb 13, 2014 at 08:24:27PM -0600, Jerry Sievers wrote: > Greetings Hackers. > > I'm aware of how a pg_hba.conf file can refer to other files for > including @lists of users, etc. > > But there is currently no support for being able to pull in entire file > segments as can be done for post

Re: [HACKERS] [SQL] Comparison semantics of CHAR data type

On Wed, Oct 16, 2013 at 02:17:11PM -0400, Bruce Momjian wrote: > > > You can see the UTF8 case is fine because \n is considered greater > > > than space, but in the C locale, where \n is less than space, the > > > false return value shows the problem with > > > internal_bpchar_pattern_compare() tri

[HACKERS] HBA files w/include support?

Greetings Hackers. I'm aware of how a pg_hba.conf file can refer to other files for including @lists of users, etc. But there is currently no support for being able to pull in entire file segments as can be done for postgresql.conf via the include directive. In the environment that I'm managing,

Re: [HACKERS] narwhal and PGDLLIMPORT

Hiroshi Inoue writes: > Rebuild with --disable-auto-import causes errors in contrib on > both machines. Errors occur in pg_buffercache, pg_stat_statements, > postgres_fdw and test_shm_mq. Yeah, that's the idea: we want to get the same failures as on MSVC. I'm going to put in PGDLLIMPORT macros t

Re: [HACKERS] narwhal and PGDLLIMPORT

(2014/02/13 9:51), Hiroshi Inoue wrote: > (2014/02/12 12:28), Inoue, Hiroshi wrote: >> (2014/02/12 8:30), Tom Lane wrote: >>> I wrote: Hiroshi Inoue writes: > I tried MINGW port with the attached change and successfully built > src and contrib and all pararell regression tests were OK

Re: [HACKERS] [bug fix] psql \copy doesn't end if backend is killed

"MauMau" writes: > If the backend is terminated with SIGKILL while psql is running "\copy > table_name from file_name", the \copy didn't end forever. I expected \copy > to be cancelled because the corresponding server process vanished. I just noticed this CF entry pertaining to the same proble

Re: [HACKERS] [PATCH] Store Extension Options

On Sun, Feb 9, 2014 at 2:22 PM, Fabrízio de Royes Mello < fabriziome...@gmail.com> wrote: > > > On Sat, Jan 11, 2014 at 2:47 AM, Peter Eisentraut wrote: > > > > On Sat, 2014-01-11 at 00:48 -0200, Fabrízio de Royes Mello wrote: > > > > Now, if bdr is installed but the validation doesn't happen unle

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

Greg Stark writes: > On Thu, Feb 13, 2014 at 7:52 PM, Tom Lane wrote: >> That's what's bothering me, too. On the other hand, if we can't think of >> a scenario where it'd be necessary to replay the high-offset update, then >> I'm disinclined to mess with the code further. > And the whole point

Re: [HACKERS] issue with gininsert under very high load

On 2014-02-13 18:49:21 -0500, Tom Lane wrote: > Andres Freund writes: > > On 2014-02-13 16:15:42 -0500, Tom Lane wrote: > >> Something like the attached? Can somebody who's seen this problem confirm > >> this improves matters? > > > Hm. Won't that possiby lead to the fast tuple list growing unbo

Re: [HACKERS] issue with gininsert under very high load

Andres Freund writes: > On 2014-02-13 16:15:42 -0500, Tom Lane wrote: >> Something like the attached? Can somebody who's seen this problem confirm >> this improves matters? > Hm. Won't that possiby lead to the fast tuple list growing unboundedly? > I think we would need to at least need to stop

Re: [HACKERS] Another pgindent gripe

On Thu, Feb 13, 2014 at 06:14:17PM -0500, Tom Lane wrote: > I noticed that current pgindent wants to do this to a recently-added > comment in psql/copy.c: > > *** handleCopyIn(PGconn *conn, FILE *copystr > *** 627,633 > /* >

[HACKERS] Another pgindent gripe

I noticed that current pgindent wants to do this to a recently-added comment in psql/copy.c: *** handleCopyIn(PGconn *conn, FILE *copystr *** 627,633 /* * This code erroneously assumes '\.' on a line

Re: [HACKERS] issue with gininsert under very high load

On 2014-02-13 16:15:42 -0500, Tom Lane wrote: > Heikki Linnakangas writes: > > Perhaps we should use a lock to enforce that only one process tries to > > clean up the pending list at a time. > > Something like the attached? Can somebody who's seen this problem confirm > this improves matters? >

Re: [HACKERS] truncating pg_multixact/members

Alvaro Herrera escribió: > So here are two patches -- the first one, for 9.3 and HEAD, introduce > the new aging variables and use them throughout vacuum and autovacuum, > including per-table options; the second one adjusts the struct > declarations to avoid the ABI break in VacuumStmt and StdRdOp

Re: [HACKERS] issue with gininsert under very high load

Heikki Linnakangas writes: > Perhaps we should use a lock to enforce that only one process tries to > clean up the pending list at a time. Something like the attached? Can somebody who's seen this problem confirm this improves matters? (ginInsertCleanup's header comment also needs to be rewritt

Re: [HACKERS] [BUGS] BUG #9210: PostgreSQL string store bug? not enforce check with correct characterSET/encoding

dig...@126.com writes: > select t, t::bytea from convert_from('\xeec1', 'sql_ascii') as g(t); > [ fails to check that string is valid in database encoding ] Hm, yeah. Normal input to the database goes through pg_any_to_server(), which will apply a validation step if the source encoding is SQL_ASC

Re: [HACKERS] truncating pg_multixact/members

Alvaro Herrera escribió: > So here are two patches -- the first one, for 9.3 and HEAD, introduce > the new aging variables and use them throughout vacuum and autovacuum, > including per-table options; the second one adjusts the struct > declarations to avoid the ABI break in VacuumStmt and StdRdOp

Re: [HACKERS] Auto-tuning work_mem and maintenance_work_mem

On Fri, Oct 11, 2013 at 03:39:51PM -0700, Kevin Grittner wrote: > Josh Berkus wrote: > > On 10/11/2013 01:11 PM, Bruce Momjian wrote: > >> In summary, I think we need to: > >> > >> *  decide on new defaults for work_mem and maintenance_work_mem > >> *  add an initdb flag to allow users/packagers t

Re: [HACKERS] New hook after raw parsing, before analyze

David Beck writes: > I have table like data structures in the source system for the FDW I work on. > These tables are sometimes too big and the source system is able to filter > and join them with limitations, thus it is not optimal to transfer the data > to Postgres. > At the same time I want t

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

On Thu, Feb 13, 2014 at 7:52 PM, Tom Lane wrote: >> The scenario I could come up with that didn't require a broken base backup >> was that there was an earlier truncate or vacuum. So the sequence is high >> offset reference, truncate, growth, crash. All possibly on a single >> database. > > That's

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

Greg Stark writes: >> I think what you're arguing is that we should see WAL records filling the >> rest of segment 1 before we see any references to segment 2, but if that's >> the case then how did we get into the situation you reported? Or is it >> just that it was a broken base backup to start

Re: [HACKERS] how set GUC_check_errhint_string in call_string_check_hook()

Amit Langote writes: > On Thu, Feb 13, 2014 at 5:59 PM, amul sul wrote: >> 2. Can I pass hint message in above ereport(), how? > It could be done by setting the check hook related global variable > "GUC_check_errhint_string" inside your hook function. You should use the macro GUC_check_errhint(

Re: [HACKERS] truncating pg_multixact/members

On 2014-02-13 14:40:39 -0300, Alvaro Herrera wrote: > Andres Freund escribió: > > On 2014-02-12 17:40:44 -0300, Alvaro Herrera wrote: > > > > > Also, AutoVacOpts (used as part of reloptions) gained three extra > > > > > fields. Since this is in the middle of StdRdOptions, it'd be somewhat > > > >

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

On 02/13/2014 06:47 PM, Heikki Linnakangas wrote: On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: The behavior where we prefer a segment from archive with lower TLI over a file with higher TLI in pg_xlog actually changed in commit a068c391ab0. Arguably changing it wasn't a good idea, but the p

Re: [HACKERS] truncating pg_multixact/members

Alvaro Herrera escribió: > Yes, that's what I did --- see the attached patch, which I would apply > on top of the code for master and would be only in 9.3. (Of course, these changes affect other parts of the code, in particular autovacuum.c and reloptions.c. But that's not important here). --

Re: [HACKERS] truncating pg_multixact/members

Andres Freund escribió: > On 2014-02-12 17:40:44 -0300, Alvaro Herrera wrote: > > > > Also, AutoVacOpts (used as part of reloptions) gained three extra > > > > fields. Since this is in the middle of StdRdOptions, it'd be somewhat > > > > more involve to put these at the end of that struct. This m

Re: [HACKERS] Same double precision operations, different results

=?ISO-8859-1?Q?Daniel_V=E1zquez?= writes: > Please can someone tell me why this behavior? Thx!! You're confusing numeric and double precision; in particular this calculation is *not* being done in float8, but numeric: > # select (0.766238989559398 * 0.766238989559398 * 1 + 0.642555686986733 * >

[HACKERS] Same double precision operations, different results

Please can someone tell me why this behavior? Thx!! I know it's calculating over coordinates on same points, but the operations of values (all double precision) differs even forcing with cast (*) Here some double precision calculations and obtaining values #Select #cos(radians(39.9826557

Re: [HACKERS] truncating pg_multixact/members

On 2014-02-12 17:40:44 -0300, Alvaro Herrera wrote: > > > Also, AutoVacOpts (used as part of reloptions) gained three extra > > > fields. Since this is in the middle of StdRdOptions, it'd be somewhat > > > more involve to put these at the end of that struct. This might be a > > > problem if someb

Re: [HACKERS] issue with gininsert under very high load

On 02/13/2014 10:58 AM, Heikki Linnakangas wrote: Perhaps we should use a lock to enforce that only one process tries to clean up the pending list at a time. Is that going to serialize all these inserts? It will serialize the cleanup process, which moves entries from the pending list to th

[HACKERS] nextVictimBuffer in README

While reading through src/backend/storage/buffer/README and looking at the code that it describes, I noticed that the case is wrong for nextVictimBuffer. It's no big deal really, but the attached trivial patch makes the README match the code. -- Vik *** a/src/backend/storage/buffer/README --- b

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

On 02/13/2014 02:42 PM, Heikki Linnakangas wrote: The behavior where we prefer a segment from archive with lower TLI over a file with higher TLI in pg_xlog actually changed in commit a068c391ab0. Arguably changing it wasn't a good idea, but the problem your test script demonstrates can be fixed b

Re: [HACKERS] issue with gininsert under very high load

On 02/13/2014 05:40 PM, Andrew Dunstan wrote: On 02/12/2014 04:04 PM, Heikki Linnakangas wrote: On 02/12/2014 10:50 PM, Andres Freund wrote: On February 12, 2014 9:33:38 PM CET, Tom Lane wrote: Andres Freund writes: On 2014-02-12 14:39:37 -0500, Andrew Dunstan wrote: On investigation I fo

Re: [HACKERS] old warning in docs

On Mon, Oct 7, 2013 at 08:43:27AM -0700, David Fetter wrote: > On Mon, Oct 07, 2013 at 07:51:44AM -0400, Andrew Dunstan wrote: > > > > Given that we have not supported releases older than 8.3 for quite a > > while, do we need to keep this in extend.sgml any longer? > > > > > > > > C

Re: [HACKERS] issue with gininsert under very high load

On 02/12/2014 04:04 PM, Heikki Linnakangas wrote: On 02/12/2014 10:50 PM, Andres Freund wrote: On February 12, 2014 9:33:38 PM CET, Tom Lane wrote: Andres Freund writes: On 2014-02-12 14:39:37 -0500, Andrew Dunstan wrote: On investigation I found that a number of processes were locked wai

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

On Feb10, 2014, at 17:38 , Andres Freund wrote: > On 2014-02-10 11:11:28 -0500, Tom Lane wrote: >> Andres Freund writes: >>> So what we need to do is to acquire a write barrier between the >>> assignments to lwWaitLink and lwWaiting, i.e. >>>proc->lwWaitLink = NULL; >>>pg_write_ba

Re: [HACKERS] Terminating pg_basebackup background streamer

On 2/13/14, 6:25 AM, Magnus Hagander wrote: > Are you sure you made that test after my fixup patch (the one suggested > by Andres)? Because that one was at least supposed to add signal.h... works now -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

On 02/13/2014 04:07 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-02-13 <52fcd02c.3060...@vmware.com> Is removing the "test ! -f" part and hence overwriting files in the archive safe, i.e. are the files the same? No. Not in general, anyway. If the old master keeps running, even for a m

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

Re: Heikki Linnakangas 2014-02-13 <52fcd02c.3060...@vmware.com> > >Is removing the "test ! -f" part and hence overwriting files in the > >archive safe, i.e. are the files the same? > > No. Not in general, anyway. If the old master keeps running, even > for a moment, after the partial file was copi

Re: [HACKERS] Weaker shmem interlock w/o postmaster.pid

On Wed, Feb 12, 2014 at 06:06:40PM -0500, Bruce Momjian wrote: > On Wed, Sep 11, 2013 at 02:10:45PM -0400, Robert Haas wrote: > > On Tue, Sep 10, 2013 at 11:33 PM, Noah Misch wrote: > > > I'm thinking to preserve postmaster.pid at immediate shutdown in all > > > released > > > versions, but I'm l

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

On 02/13/2014 03:53 PM, Christoph Berg wrote: Is removing the "test ! -f" part and hence overwriting files in the archive safe, i.e. are the files the same? No. Not in general, anyway. If the old master keeps running, even for a moment, after the partial file was copied, it will have created m

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

Re: Heikki Linnakangas 2014-02-13 <52fcca40.3060...@vmware.com> > I was testing this with streaming replication; 9.1 and 9.2 behave > the same in that scenario. But they differ when doing archive > recovery. > > Is this an argument for back-patching the "don't archive last > segment from old timel

Re: [HACKERS] Prevent pg_basebackup -Fp -D -?

On Thu, Feb 13, 2014 at 4:10 AM, Bruce Momjian wrote: > On Thu, Oct 3, 2013 at 06:50:57AM +0200, Magnus Hagander wrote: > > > > On Oct 3, 2013 2:47 AM, "Michael Paquier" > wrote: > > > > > > On Wed, Oct 2, 2013 at 11:31 PM, Magnus Hagander > wrote: > > > > Right now, if you use > > > > > > > >

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

On 02/12/2014 06:10 PM, Ants Aasma wrote: On Wed, Feb 12, 2014 at 4:04 PM, knizhnik wrote: Even if reordering was not done by compiler, it still can happen while execution. There is no warranty that two subsequent assignments will be observed by all CPU cores in the same order. The x86 memory

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

On 02/13/2014 02:42 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-02-13 <52fc9468.4050...@vmware.com> With 9.1, it works, but 9.2 and 9.3 don't archive anything until I remove the "test ! -f" part. (An alternative fix would be to declare the behavior ok and adjust that example in the con

Re: [HACKERS] New hook after raw parsing, before analyze

Thanks for the link. I want flexibility. Here is a situation: my hook knows the size of tableA and tableB on the legacy side. It should be able to decide wether to offload the join/filter onto the legacy side or not. At the same time it can start transferring the data to real Postgres tables in

Re: [HACKERS] gaussian distribution pgbench

Sorry, previos attached patch has small bug. Please use latest one. > 134 - return min + (int64) (max - min + 1) * rand; > 134 + return min + (int64)((max - min + 1) * rand); Regards, -- Mitsumasa KONDO NTT Open Source Software Center *** a/contrib/pgbench/pgbench.c --- b/contrib/pgbench/pgbench

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

Re: Heikki Linnakangas 2014-02-13 <52fc9468.4050...@vmware.com> > >With 9.1, it works, but 9.2 and 9.3 don't archive anything until I > >remove the "test ! -f" part. (An alternative fix would be to declare > >the behavior ok and adjust that example in the config.) > > Hmm, the behavior is the same

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

On 02/13/2014 01:37 PM, Kyotaro HORIGUCHI wrote: At Thu, 13 Feb 2014 10:11:22 +0200, Heikki Linnakangas wrote in <52fc7e2a.9060...@vmware.com> On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: Wouldn't it be better to not archive the old segment, and instead switch to a new segment after writi

Re: [HACKERS] New hook after raw parsing, before analyze

See the discussion of Custom-Scan API. https://commitfest.postgresql.org/action/patch_view?id=1282 I believe my third patch is what you really want to do... > This rewritten query would be handled by the FDW table that I previously > added to the catalog. > > The reason I want this new hook is t

Re: [HACKERS] gaussian distribution pgbench

Hi Febien, Thank you very much for your very detail and useful comments! I read your comment, I agree most of your advice:) Attached patch is fixed for your comment. That are... - Remove redundant long-option. - We can use "--gaussian=NUM -S" or "--gaussian=NUMN -N" options. - Add sentence

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

Hello, I might have been misunderstood your words. At Thu, 13 Feb 2014 10:11:22 +0200, Heikki Linnakangas wrote in <52fc7e2a.9060...@vmware.com> > On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: > > Wouldn't it be better to not archive the old segment, and instead > > switch > > to

Re: [HACKERS] Terminating pg_basebackup background streamer

On Wed, Feb 12, 2014 at 10:53 PM, Peter Eisentraut wrote: > On 2/12/14, 4:34 PM, Magnus Hagander wrote: > > On Wed, Feb 12, 2014 at 10:28 PM, Peter Eisentraut > > wrote: > > > > On 2/12/14, 12:47 PM, Magnus Hagander wrote: > > > Since there were no other objection

Re: [HACKERS] Memory ordering issue in LWLockRelease, WakeupWaiters, WALInsertSlotRelease

eFrom: "Andres Freund" could you try if you get more readable dumps by using disassemble/m? That might at least print line numbers if you have debug info installed. Please find the attached file. I hope this will reveal something. Regards MauMau Dump of assembler code for function LWLockRel

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

On Thu, Feb 13, 2014 at 10:20:46AM +0530, Amit Kapila wrote: > > Why not *only* prefix/suffix? > > To represent prefix/suffix match, we atleast need a way to tell > that the offset and len of matched bytes and then how much > is the length of unmatched bytes we have copied. > I agree that a simple

[HACKERS] New hook after raw parsing, before analyze

Hello Hackers, I work on a foreign data wrapper for a legacy system. I generally find the hook system very useful and flexible way to extend Postgres. The post parse analyze hook almost fits what I need, but I have a few use cases where I would need to tap right into the parsed queries but befor

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

> I think what you're arguing is that we should see WAL records filling the > rest of segment 1 before we see any references to segment 2, but if that's > the case then how did we get into the situation you reported? Or is it > just that it was a broken base backup to start with? The scenario I c

Re: [HACKERS] how set GUC_check_errhint_string in call_string_check_hook()

Amul, On Thu, Feb 13, 2014 at 5:59 PM, amul sul wrote: > I have basic questions, > > 1. Condition testing of GUC_check_errdetail_string or other GUC_* is > needed, even if we resting it? A few lines before this conditional test, there is a call to the hook function (using the check_hook funct

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

On 02/12/2014 01:24 PM, Christoph Berg wrote: Re: Heikki Linnakangas 2014-01-13 <52d3caff.3010...@vmware.com> Actually, why is the partially-filled 00010002 file archived in the first place? Looking at the code, it's been like that forever, but it seems like a bad idea. If the or

[HACKERS] how set GUC_check_errhint_string in call_string_check_hook()

Dear all, I need help to send hint message to erreport()  in  call_string_check_hook function [src/backend/utils/misc/guc.c:8092] In   call_string_check_hook(), following parameter are resets   /* Reset variables that might be set by hook */ 8100         GUC_check_errcode_value = ERRCODE_INVALI

Re: [HACKERS] Recovery inconsistencies, standby much larger than primary

Hi all, On 02/12/2014 08:27 PM, Greg Stark wrote: On Wed, Feb 12, 2014 at 6:55 PM, Tom Lane wrote: Greg Stark writes: For what it's worth I've confirmed the bug in wal-e caused the initial problem. Huh? Bug in wal-e? What bug? WAL-E actually didn't restore a whole 1GB file due to a tra

Re: [HACKERS] [BUG] Archive recovery failure on 9.3+.

On 02/13/2014 08:44 AM, Kyotaro HORIGUCHI wrote: Wouldn't it be better to not archive the old segment, and instead switch to a new segment after writing the end-of-recovery checkpoint, so that the segment on the new timeline is archived sooner? It would be better to zero-fill and switch segment