[HACKERS] Parse more than bind and execute when connect to database by jdbc

2013-11-18 Thread wangshuo
Hi everyone, Finally we found , the JDBC function we ever modified contributed to this phenomenon, thanks of all. Yours, Wang Shuo HighGo Software Co.,Ltd. November 18, 2013 -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to you

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread David Rowley
On Wed, Oct 30, 2013 at 3:32 AM, Andres Freund wrote: > On 2013-10-25 09:26:29 -0400, Robert Haas wrote: > > > In any case, it's very far from obvious to me that CLUSTER ought > > > to throw away information by default, which is what you're proposing. > > > > I find it odd to referring to this as

Re: [HACKERS] writable FDWs / update targets confusion

2013-11-18 Thread Albe Laurenz
Tom Lane wrote: >> Tom, could you show us a rope if there is one? > > What is it you actually need to fetch? > > IIRC, the idea was that most FDWs would do the equivalent of fetching the > primary-key columns to use in an update. If that's what you need, then > AddForeignUpdateTargets should ide

Re: [HACKERS] freeze cannot be finished

2013-11-18 Thread Heikki Linnakangas
Committed, thanks for the report! On 16.11.2013 22:05, Миша Тюрин wrote: Hello! Could anyone review patch suggested by Jeff Janes ? Initial thread http://www.postgresql.org/message-id/flat/1384356585.995240612%40f50.i.mail.ru Thanks in advance! On Wed, Nov 13, 2013 at 3:53 PM, Sergey Burl

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 15.11.2013 21:00, Simon Riggs wrote: On 15 November 2013 15:48, Peter Eisentraut wrote: Also, you set this to "returned with feedback" in the CF app. Please verify whether that was intentional. Not sure that was me, if so, corrected. It was me, sorry. I figured this needs such a large r

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 15.11.2013 20:21, Andres Freund wrote: On 2013-11-15 20:08:30 +0200, Heikki Linnakangas wrote: It's pretty hard to review the this without seeing the "other" implementation you're envisioning to use this API. But I'll try: We've written a distributed sequence implementation against it, so i

Re: [HACKERS] additional json functionality

2013-11-18 Thread Hannu Krosing
On 11/18/2013 05:19 AM, Andrew Dunstan wrote: > > On 11/17/2013 08:49 PM, Josh Berkus wrote: >>> Now, if it turns out that the new hstore is not dealing with json input >>> and output, we could have json, jstore and hstore. >> Jstore isn't the worst name suggestion I've heard on this thread. The >

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-18 Thread Amit Khandekar
On 18 October 2013 17:07, Rajeev rastogi wrote: > From the following mail, copy behaviour between stdin and normal file > having some inconsistency. > > > http://www.postgresql.org/message-id/ce85a517.4878e%tim.k...@gmail.com > > > > The issue was that if copy execute "from stdin", then it goes

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Haribabu kommi
On 18 November 2013 11:19 Haribabu kommi wrote: > On 17 November 2013 00:55 Fujii Masao wrote: > > On Sat, Nov 16, 2013 at 2:27 PM, Haribabu kommi > > wrote: > > > on 15 November 2013 17:26 Magnus Hagander wrote: > > > > > >>On Fri, Nov 15, 2013 at 12:10 PM, Haribabu kommi > > >> wrote: > > > > >

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Andres Freund
On 2013-11-18 10:54:42 +0200, Heikki Linnakangas wrote: > On 15.11.2013 20:21, Andres Freund wrote: > >Well, it exposes log_sequence_tuple() - together with the added "am > >private" column of pg_squence that allows to do quite a bit of different > >things. I think unless we really implement plugga

Re: [HACKERS] pg_stat_statements: calls under-estimation propagation

2013-11-18 Thread Sameer Thakur
Hello, Please find v10 of patch attached. This patch addresses following review comments 1. Removed errcode and used elogs for error "pg_stat_statements schema is not supported by its binary" 2. Removed comments and other code formatting not directly relevant to patch functionality 3. changed posit

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 18.11.2013 11:48, Andres Freund wrote: On 2013-11-18 10:54:42 +0200, Heikki Linnakangas wrote: On 15.11.2013 20:21, Andres Freund wrote: Well, it exposes log_sequence_tuple() - together with the added "am private" column of pg_squence that allows to do quite a bit of different things. I thin

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Haribabu kommi
On 18 October 2013 13:35 KONDO Mitsumasa wrote: > Hi, > > I submit improvement of pg_stat_statement usage patch in CF3. > > In pg_stat_statement, I think buffer hit ratio is very important value. > However, it is difficult to calculate it, and it need complicated SQL. > This patch makes it more

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/18 20:16), Haribabu kommi wrote: On 18 October 2013 13:35 KONDO Mitsumasa wrote: This patch conflicts pg_stat_statement_min_max_exectime patch which I submitted, and pg_stat_statement_min_max_exectime patch also adds new columns which are min_time and max_time. So I'd like to change it

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Andres Freund
On 2013-11-18 12:50:21 +0200, Heikki Linnakangas wrote: > On 18.11.2013 11:48, Andres Freund wrote: > I don't think the sequence AM should be in control of 'cached'. The caching > is done outside the AM. And log_cnt probably should be passed to the _alloc > function directly as an argument, ie. the

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Simon Riggs
On 18 November 2013 07:50, Heikki Linnakangas wrote: > It doesn't go far enough, it's still too *low*-level. The sequence AM > implementation shouldn't need to have direct access to the buffer page at > all. > I don't think the sequence AM should be in control of 'cached'. The caching > is done

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 18.11.2013 13:48, Simon Riggs wrote: On 18 November 2013 07:50, Heikki Linnakangas wrote: It doesn't go far enough, it's still too *low*-level. The sequence AM implementation shouldn't need to have direct access to the buffer page at all. I don't think the sequence AM should be in contro

Re: [HACKERS] REINDEX CONCURRENTLY 2.0

2013-11-18 Thread Andres Freund
On 2013-11-18 19:52:29 +0900, Michael Paquier wrote: > On Sat, Nov 16, 2013 at 5:09 AM, Andres Freund wrote: > > On 2013-11-15 11:40:17 +0900, Michael Paquier wrote: > >> - 20131114_3_reindex_concurrently.patch, providing the core feature. > >> Patch 3 needs to have patch 2 applied first. Regressi

Re: [HACKERS] Sequence Access Method WIP

2013-11-18 Thread Heikki Linnakangas
On 14.11.2013 22:10, Simon Riggs wrote: Includes test extension which allows sequences without gaps - "gapless". I realize this is just for demonstration purposes, but it's worth noting that it doesn't actually guarantee that when you use the sequence to populate a column in the table, the co

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Haribabu kommi
On 17 November 2013 12:25 Amit Kapila wrote: > On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi > wrote: > > On 16 November 2013 09:43 Amit Kapila wrote: > >> On Fri, Nov 15, 2013 at 10:18 PM, Peter Eisentraut > >> wrote: > >> > On 11/14/13, 2:50 AM, Amit Kapila wrote: > >> >> Find the rebased ver

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Fujii Masao
On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi wrote: > > On 18 November 2013 11:19 Haribabu kommi wrote: >> On 17 November 2013 00:55 Fujii Masao wrote: >> > On Sat, Nov 16, 2013 at 2:27 PM, Haribabu kommi >> > wrote: >> > > on 15 November 2013 17:26 Magnus Hagander wrote: >> > > >> > >>On Fri,

Re: [HACKERS] CREATE TABLE IF NOT EXISTS AS

2013-11-18 Thread Fabrízio de Royes Mello
On Sun, Nov 17, 2013 at 6:05 PM, David E. Wheeler wrote: > On Nov 16, 2013, at 4:40 PM, Tom Lane wrote: > > >> Co-worker asked a question I could not answer: Why is IF NOT EXISTS not > supported by CREATE TABLE AS? > > > > That's an even worse idea than plain CREATE IF NOT EXISTS (which was > > p

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-18 Thread Marko Kreen
On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote: > On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen wrote: > > I am bit suspicious of performance impact of this patch, but think > > that it's still worthwhile as it decreases code style where single > > string argument is given to printf-s

Re: [HACKERS] additional json functionality

2013-11-18 Thread Peter Eisentraut
On 11/15/13, 6:15 PM, Josh Berkus wrote: > Thing is, I'm not particularly concerned about *Merlin's* specific use > case, which there are ways around. What I am concerned about is that we > may have users who have years of data stored in JSON text fields which > won't survive an upgrade to binary J

[HACKERS] Force optimizer to use hash/nl/merge join?

2013-11-18 Thread Zhan Li
Hi All, Is there any way to force the optimizer to use a specific join operator? For example, in SQL Server, I can do this way select * from (A inner hash join B on A.a = B.b) inner loop join C on A.a = C.c I did some search but didn't find PostgreSQL had similar join hints except for enable_* s

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Amit Kapila
On Mon, Nov 18, 2013 at 6:28 PM, Haribabu kommi wrote: > On 17 November 2013 12:25 Amit Kapila wrote: >> On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi >> >> >> Find the rebased version attached with this mail. >> >> > >> > ereport(ERROR, >> > >> (errcode(ERRCODE_CONFIG_FI

Re: [HACKERS] Force optimizer to use hash/nl/merge join?

2013-11-18 Thread Amit Kapila
On Mon, Nov 18, 2013 at 7:58 PM, Zhan Li wrote: > Hi All, > > Is there any way to force the optimizer to use a specific join operator? For > example, in SQL Server, I can do this way > > select * from (A inner hash join B on A.a = B.b) inner loop join C on A.a = > C.c > > I did some search but did

Re: [HACKERS] inherit support for foreign tables

2013-11-18 Thread Robert Haas
On Thu, Nov 14, 2013 at 12:28 AM, Tom Lane wrote: >> 2) Allow foreign tables to have CHECK constraints >> Like NOT NULL, I think we don't need to enforce the check duroing >> INSERT/UPDATE against foreign table. > > Really? It's one thing to say that somebody who adds a CHECK constraint > to a fo

Re: [HACKERS] additional json functionality

2013-11-18 Thread Merlin Moncure
On Sun, Nov 17, 2013 at 10:19 PM, Andrew Dunstan wrote: > I don't think any name that doesn't begin with "json" is acceptable. I could > live with "jsonb". It has the merit of brevity, but maybe it's a tad too > close to "json" to be the right answer. I think that seems right. Couple thoughts:

[HACKERS] Review: pre-commit triggers

2013-11-18 Thread Ian Lawrence Barwick
Review for Andrew Dunstan's patch in CF 2013-11: https://commitfest.postgresql.org/action/patch_view?id=1312 This review is more from a usage point of view, I would like to spend more time looking at the code but only so many hours in a day, etcetera; I hope this is useful as-is. Submission r

Re: [HACKERS] additional json functionality

2013-11-18 Thread Andrew Dunstan
On 11/18/2013 09:38 AM, Merlin Moncure wrote: On Sun, Nov 17, 2013 at 10:19 PM, Andrew Dunstan wrote: I don't think any name that doesn't begin with "json" is acceptable. I could live with "jsonb". It has the merit of brevity, but maybe it's a tad too close to "json" to be the right answer. I

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Heikki Linnakangas
On 14.10.2013 07:12, Peter Geoghegan wrote: On Wed, Oct 9, 2013 at 1:11 PM, Peter Geoghegan wrote: Unfortunately, I have a very busy schedule in the month ahead, including travelling to Ireland and Japan, so I don't think I'm going to get the opportunity to work on this too much. I'll try and p

Re: [HACKERS] inherit support for foreign tables

2013-11-18 Thread Tom Lane
Robert Haas writes: > On Thu, Nov 14, 2013 at 12:28 AM, Tom Lane wrote: >>> 2) Allow foreign tables to have CHECK constraints >>> Like NOT NULL, I think we don't need to enforce the check duroing >>> INSERT/UPDATE against foreign table. >> Really? > I think it's been previously proposed that we

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Haribabu kommi
On 18 November 2013 18:45 Fujii Masao wrote: > On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi > wrote: > > > > On 18 November 2013 11:19 Haribabu kommi wrote: > >> On 17 November 2013 00:55 Fujii Masao wrote: > >> > On Sat, Nov 16, 2013 at 2:27 PM, Haribabu kommi > >> > wrote: > >> > > on 15 Nov

Re: ALTER SYSTEM SET command to change postgresql.conf parameters (RE: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review])

2013-11-18 Thread Haribabu kommi
On 18 November 2013 20:01 Amit Kapila wrote: > On Mon, Nov 18, 2013 at 6:28 PM, Haribabu kommi > wrote: > > On 17 November 2013 12:25 Amit Kapila wrote: > >> On Sat, Nov 16, 2013 at 4:35 PM, Haribabu kommi > >> >> >> Find the rebased version attached with this mail. > >> >> > > >> >

Re: [HACKERS] Small improvement to json out functions by using cstring_to_text_with_len instead of cstring_to_text

2013-11-18 Thread Robert Haas
On Thu, Nov 14, 2013 at 2:18 AM, David Rowley wrote: > Hi, > > Here's a small patch which should speedup json out functions a little bit by > removing a call to strlen for which could be a long string. > The length of the string is already known by the StringInfoData, so there's > no point in lett

Re: [HACKERS] Can we add sample systemd service file to git repo?

2013-11-18 Thread Will Crawford
On 12 November 2013 20:54, Nigel Heron wrote: > On Tue, Nov 12, 2013 at 11:47 AM, Devrim GÜNDÜZ wrote: ... >> http://svn.pgrpms.org/browser/rpm/redhat/9.3/postgresql/F-19/postgresql-9.3.service >> is an example of what we use in the RPMs. (if website fails, please just >> reload) >> >> Attached

Re: [HACKERS] Freezing without write I/O

2013-11-18 Thread Andres Freund
On 2013-09-25 12:31:20 +0300, Heikki Linnakangas wrote: > Hmm, some of those are trivial, but others, rewrite_heap_tuple() are > currently only passed the HeapTuple, with no reference to the page where the > tuple came from. Instead of changing code to always pass that along with a > tuple, I think

Re: [HACKERS] nested hstore patch

2013-11-18 Thread Andres Freund
Hi, On 2013-11-12 22:35:31 +0400, Teodor Sigaev wrote: > Attatched patch adds nesting feature, types (string, boll and numeric > values), arrays and scalar to hstore type. I took a quick peek at this: * You cannot simply catch and ignore errors by doing + PG_TRY(); + { +

Re: [HACKERS] appendPQExpBufferVA vs appendStringInfoVA

2013-11-18 Thread Heikki Linnakangas
On 18.11.2013 15:40, Marko Kreen wrote: On Mon, Nov 18, 2013 at 06:18:01PM +1300, David Rowley wrote: On Mon, Nov 18, 2013 at 1:01 AM, Marko Kreen wrote: I am bit suspicious of performance impact of this patch, but think that it's still worthwhile as it decreases code style where single string

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 09:22:58PM +1300, David Rowley wrote: > So now I'm wondering what the next move should be for this patch? > > a. Are we waiting on Robert's patch to be committed before we can apply > Thomas's > cluster with freeze as default? > b. Are we waiting on me reviewing one or bot

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Andres Freund
On 2013-11-18 11:39:44 -0500, Bruce Momjian wrote: > On Mon, Nov 18, 2013 at 09:22:58PM +1300, David Rowley wrote: > > So now I'm wondering what the next move should be for this patch? > > > > a. Are we waiting on Robert's patch to be committed before we can apply > > Thomas's > > cluster with fr

Re: [HACKERS] Review: pre-commit triggers

2013-11-18 Thread Andrew Dunstan
On 11/18/2013 09:39 AM, Ian Lawrence Barwick wrote: Review for Andrew Dunstan's patch in CF 2013-11: https://commitfest.postgresql.org/action/patch_view?id=1312 This review is more from a usage point of view, I would like to spend more time looking at the code but only so many hours in a da

Re: [HACKERS] Turning recovery.conf into GUCs

2013-11-18 Thread Andres Freund
Hi, On 2013-11-15 22:38:05 -0500, Jaime Casanova wrote: > sorry, i clearly misunderstood you. attached a rebased patch with > those functions removed. * --write-standby-enable seems to loose quite some functionality in comparison to --write-recovery-conf since it doesn't seem to set primary_c

Re: [HACKERS] additional json functionality

2013-11-18 Thread Josh Berkus
On 11/18/2013 06:13 AM, Peter Eisentraut wrote: > On 11/15/13, 6:15 PM, Josh Berkus wrote: >> Thing is, I'm not particularly concerned about *Merlin's* specific use >> case, which there are ways around. What I am concerned about is that we >> may have users who have years of data stored in JSON tex

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Andres Freund
On 2013-11-18 15:01:42 +, Haribabu kommi wrote: > > /* > + * Returns the malloced string of containing current working directory. > + * The caller has to take care of freeing the memory. > + * On failure exits with error code. > + */ > +static char * > +get_current_working_dir() > +{ > +

Re: [HACKERS] New option for pg_basebackup, to specify a different directory for pg_xlog

2013-11-18 Thread Fujii Masao
On Tue, Nov 19, 2013 at 12:01 AM, Haribabu kommi wrote: > On 18 November 2013 18:45 Fujii Masao wrote: >> On Mon, Nov 18, 2013 at 6:31 PM, Haribabu kommi >> wrote: >> > >> > On 18 November 2013 11:19 Haribabu kommi wrote: >> >> On 17 November 2013 00:55 Fujii Masao wrote: >> >> > On Sat, Nov 16,

Re: [HACKERS] additional json functionality

2013-11-18 Thread Josh Berkus
Merlin, > *) Aside from the text in and out routines, how is 'jsbonb' different > from the coming 'nested hstore'? Enough to justify two code bases? In/out functions and defaults are all different. Otherwise, the two types will be accessing the same code base, so no duplication. Think of is a

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Josh Berkus
On 11/18/2013 08:39 AM, Bruce Momjian wrote: > If we do add FREEZE, all we would be doing is delaying the time when all > CLUSTER operations will use FREEZE, and hence debugging will be just as > difficult. My point is that will full knowledge, everyone would use > FREEZE unless they expect MVCC b

Re: [HACKERS] Review: pre-commit triggers

2013-11-18 Thread Dimitri Fontaine
Andrew Dunstan writes: > Given that, do we want to keep the bar on these operating in single user > mode? I can easily drop it and just document this way out of difficulty. Currently Event Triggers are disabled in single user mode, in parts because operating them require accessing to a catalog in

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Fujii Masao
On Mon, Nov 18, 2013 at 8:36 PM, KONDO Mitsumasa wrote: > (2013/11/18 20:16), Haribabu kommi wrote: >> >> On 18 October 2013 13:35 KONDO Mitsumasa wrote: >>> >>> This patch conflicts pg_stat_statement_min_max_exectime patch which I >>> submitted, and pg_stat_statement_min_max_exectime patch also a

Re: [HACKERS] additional json functionality

2013-11-18 Thread Merlin Moncure
On Mon, Nov 18, 2013 at 12:10 PM, Josh Berkus wrote: > Merlin, > >> *) Aside from the text in and out routines, how is 'jsbonb' different >> from the coming 'nested hstore'? Enough to justify two code bases? > > In/out functions and defaults are all different. Otherwise, the two > types will be

[HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
Three times in the last two weeks, we have experience data corruption secondary servers using streaming replication on client systems. The versions involved are 9.0.14, 9.2.5, and 9.3.1. Each incident was separate; two cases they were for the same client (9.0.14 and 9.3.1), one for a different

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao wrote: > The same idea was proposed before but not committed because > Itagaki thought that pg_stat_statements view should report only raw values. > Please read the following thread. I have the same feeling with him. > Anyway we should listen to more o

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Josh Berkus
On 11/18/2013 10:58 AM, Christophe Pettus wrote: > Three times in the last two weeks, we have experience data corruption > secondary servers using streaming replication on client systems. The > versions involved are 9.0.14, 9.2.5, and 9.3.1. Each incident was separate; > two cases they were fo

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 10:58 AM, Christophe Pettus wrote: > As a note, P1 was created from another system (let's call it P0) using just > WAL shipping (no streaming replication), and no data corruption was observed. As another data point, P0 was running 9.0.13, rather than 9.0.14. -- -- Christophe

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
Hi, On 2013-11-18 10:58:26 -0800, Christophe Pettus wrote: > INCDIDENT #1: 9.0.14 -- A new secondary (S1) was initialized using > rsync off of an existing, correct primary (P1) for the base backup, > and using WAL-E for WAL segment shipping. Both the primary and > secondary were running 9.0.14.

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:28 AM, Andres Freund wrote: > Could you detail how exactly the base backup was created? Including the > *exact* logic for copying? 0. Before any of this began, P1 was archiving WAL segments to AWS-S3. 1. pg_start_backup('', true) on P1. 2. Using rsync -av on P1, the entire

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 11:38:43 -0800, Christophe Pettus wrote: > > On Nov 18, 2013, at 11:28 AM, Andres Freund wrote: > > Could you detail how exactly the base backup was created? Including the > > *exact* logic for copying? > > 0. Before any of this began, P1 was archiving WAL segments to AWS-S3. > 1.

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund wrote: > Without deleting any data, including pg_xlog/, backup.label, anything? Correct. > Did you have hot_standby enabled on all of those machines? Even on the > 9.0.13 cluster? Yes. > That was just recovery command and primary conninfo? Correct

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund wrote: > Did you have hot_standby enabled on all of those machines? Even on the > 9.0.13 cluster? Actually, it's a bit more complex than this: 1. We don't know about P0, the 9.0.13 machine. I assume it was, but it was managed elsewhere. 2. P1 never

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 11:47 AM, Andres Freund wrote: > Without deleting any data, including pg_xlog/, backup.label, anything? One more correction: After rsync finished and the pg_base_backup() was issued, the contents of pg_xlog/ on S1 were deleted. -- -- Christophe Pettus x...@thebuild.com

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 12:00 PM, Christophe Pettus wrote: > One more correction: After rsync finished and the pg_base_backup() was > issued, the contents of pg_xlog/ on S1 were deleted. pg_stop_backup(), sorry. -- -- Christophe Pettus x...@thebuild.com -- Sent via pgsql-hackers mailing li

Re: [HACKERS] CLUSTER FREEZE

2013-11-18 Thread Kevin Grittner
Josh Berkus wrote: > On 11/18/2013 08:39 AM, Bruce Momjian wrote: >> If we do add FREEZE, all we would be doing is delaying the time >> when all CLUSTER operations will use FREEZE, and hence debugging >> will be just as difficult.  My point is that will full >> knowledge, everyone would use FREEZE

Re: [HACKERS] Changing pg_dump default file format

2013-11-18 Thread Bruce Momjian
On Thu, Nov 7, 2013 at 02:40:17PM -0500, Peter Eisentraut wrote: > On 11/7/13, 1:00 PM, Josh Berkus wrote: > > If we wanted to change the defaults, I think it would be easier to > > create a separate bin name (e.g. pg_backup) than to change the existing > > parameters for pg_dump. > > Note the fo

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Alexander Korotkov
On Fri, Nov 15, 2013 at 11:42 PM, Alexander Korotkov wrote: > On Fri, Nov 15, 2013 at 11:39 PM, Rod Taylor wrote: > >> >> >> >> On Fri, Nov 15, 2013 at 2:26 PM, Alexander Korotkov > > wrote: >> >>> On Fri, Nov 15, 2013 at 11:18 PM, Rod Taylor wrote: >>> 2%. It's essentially sentenc

Re: [HACKERS] Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Sat, Nov 9, 2013 at 02:15:52PM -0500, Robert Haas wrote: > On Fri, Nov 8, 2013 at 5:36 PM, Tom Lane wrote: > > [ I'm so far behind ... ] > > > > Bruce Momjian writes: > >> Applied. Thank you for all your suggestions. > > > > I thought the suggestion had been to issue a *warning*. How did th

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 10:58:26 -0800, Christophe Pettus wrote: > After promotion, it was discovered that there was significant data > loss on S1. Rows that were present on P1 were missing on S1, and some > rows were duplicated (including duplicates that violated primary key > and other unique constraints)

Re: [HACKERS] unaccent module - two params function should be immutable

2013-11-18 Thread Bruce Momjian
On Fri, Nov 8, 2013 at 06:00:53PM -0500, Tom Lane wrote: > Bruce Momjian writes: > > [ mark unaccent functions immutable ] > > > Applied. > > This patch is flat out wrong and needs to be reverted. > > The functions were correctly marked (by you!) in commit > c0577c92a84cc477a88fe6868c16c4a7e33

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 12:57 PM, Andres Freund wrote: > Were there any kind of patterns in the lost data? What kind of workload > are they running? I have an idea what the issue might be... On the P1 > S1 case, the data corrupted was data modified in the last few minutes before the switchover. I

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
Hi, Afaics it's likely a combination/interaction of bugs and fixes between: * the initial HS code * 5a031a5556ff83b8a9646892715d7fef415b83c3 * f44eedc3f0f347a856eea8590730769125964597 But that'd mean nobody noticed it during 9.3's beta... Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
Great! If there's any further data I can supply to help, let me know. On Nov 18, 2013, at 2:15 PM, Andres Freund wrote: > Hi, > > Afaics it's likely a combination/interaction of bugs and fixes between: > * the initial HS code > * 5a031a5556ff83b8a9646892715d7fef415b83c3 > * f44eedc3f0f347a856e

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Andres Freund
On 2013-11-18 14:25:58 -0800, Christophe Pettus wrote: > Great! If there's any further data I can supply to help, let me know. Trying to reproduce the issue with and without hot_standby=on would be very helpful, but I guess that's time consuming? Greetings, Andres Freund -- Andres Freund

Re: [HACKERS] Data corruption issues using streaming replication on 9.0.14/9.2.5/9.3.1

2013-11-18 Thread Christophe Pettus
On Nov 18, 2013, at 2:26 PM, Andres Freund wrote: > Trying to reproduce the issue with and without hot_standby=on would be > very helpful, but I guess that's time consuming? I've been working on it, but I haven't gotten it to fail yet. I'll keep at it. -- -- Christophe Pettus x...@thebuild

[HACKERS] Review: HStore Gin Speedup

2013-11-18 Thread Antonin Houska
Following are my initial comments for https://commitfest.postgresql.org/action/patch_view?id=1203 1. It no longer applies, probably due to commit a18167510f4c385329697588ce5132cbf95779c3 error: contrib/hstore/hstore--1.1.sql: No such file or directory 2. Compatibility with HStore v2.0 (https:/

Re: Custom Scan APIs (Re: [HACKERS] Custom Plan node)

2013-11-18 Thread Jim Mlodgenski
On Mon, Nov 18, 2013 at 7:25 AM, Kohei KaiGai wrote: > The attached patches are the revised custom-scan APIs. > My initial review on this feature: - The patches apply and build, but it produces a warning: ctidscan.c: In function ‘CTidInitCustomScanPlan’: ctidscan.c:362:9: warning: unused variabl

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 6:44 AM, Heikki Linnakangas wrote: > I think it's important to recap the design goals of this. Seems reasonable to list them out. > * It should be usable and perform well for both large batch updates and > small transactions. I think that that's a secondary goal, a quest

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
On Fri, Nov 15, 2013 at 2:42 PM, Alexander Korotkov wrote: > On Fri, Nov 15, 2013 at 11:39 PM, Rod Taylor wrote: > >> >> The patched index is 58% of the 9.4 master size. 212 MB instead of 365 MB. >> > > Good. That's meet my expectations :) > You mention that both master and patched versions was c

[HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread David Johnston
Bruce Momjian wrote > Considering we are doing this outside of a transaction, and WARNING or > ERROR is pretty much the same, from a behavioral perspective. > > Should we change this and LOCK to be a warning? >From the calling application's perspective an error and a warning are definitely behavi

[HACKERS] More legacy code: pg_ctl

2013-11-18 Thread Josh Berkus
Folks, Speaking of legacy code with bad default behaviors: pg_ctl. The current utility is designed to fathfully reproduce the rather hackish shell script we originally had for postgres startup. This results in a couple of unintuitive defaults which give sysadmins and config management developer

Re: [HACKERS] More legacy code: pg_ctl

2013-11-18 Thread Josh Berkus
On 11/18/2013 05:09 PM, Josh Berkus wrote: > Folks, > > Speaking of legacy code with bad default behaviors: pg_ctl. The current > utility is designed to fathfully reproduce the rather hackish shell > script we originally had for postgres startup. This results in a > couple of unintuitive defaul

Re: [HACKERS] [PATCH] configure: allow adding a custom string to PG_VERSION

2013-11-18 Thread Peter Eisentraut
On Tue, 2013-11-05 at 18:29 +0200, Oskari Saarenmaa wrote: > This can be used to tag custom built packages with an extra version string > such as the git describe id or distribution package release version. I think this is a reasonable feature, and the implementation is OK, but I don't see why the

Re: [HACKERS] INSERT...ON DUPLICATE KEY LOCK FOR UPDATE

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 4:37 PM, Peter Geoghegan wrote: > You're right that the value locking is scary. I think we need to very > carefully consider it, once I have buy-in on the basic approach. I > really do think it's the least-worst approach described to date. It > isn't like we can't discuss m

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/19 3:56), Peter Geoghegan wrote: On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao wrote: The same idea was proposed before but not committed because Itagaki thought that pg_stat_statements view should report only raw values. Please read the following thread. I have the same feeling with h

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 05:05:45PM -0800, David Johnston wrote: > Bruce Momjian wrote > > Considering we are doing this outside of a transaction, and WARNING or > > ERROR is pretty much the same, from a behavioral perspective. > > > > Should we change this and LOCK to be a warning? > > >From the

Re: [HACKERS] [PATCH] configure: allow adding a custom string to PG_VERSION

2013-11-18 Thread Michael Paquier
On Tue, Nov 19, 2013 at 10:48 AM, Peter Eisentraut wrote: > I think this is a reasonable feature, and the implementation is OK, but > I don't see why the format of the extra version information needs to be > exactly > > PG_VERSION="$PACKAGE_VERSION ($withval)" > > I'd imagine, for example, tha

[HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread David Johnston
Bruce Momjian wrote > On Mon, Nov 18, 2013 at 05:05:45PM -0800, David Johnston wrote: >> Bruce Momjian wrote >> > Considering we are doing this outside of a transaction, and WARNING or >> > ERROR is pretty much the same, from a behavioral perspective. >> > >> > Should we change this and LOCK to be

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread KONDO Mitsumasa
(2013/11/19 11:12), KONDO Mitsumasa wrote: (2013/11/19 3:56), Peter Geoghegan wrote: On Mon, Nov 18, 2013 at 10:49 AM, Fujii Masao wrote: The same idea was proposed before but not committed because Itagaki thought that pg_stat_statements view should report only raw values. Please read the foll

[HACKERS] Remove use of obsolescent Autoconf macros

2013-11-18 Thread Peter Eisentraut
According to the Autoconf documentation, the following macros are obsolescent because no reasonably current system has the defect that they are testing for. - AC_C_CONST - AC_C_STRINGIZE - AC_C_VOLATILE - AC_FUNC_MEMCMP Therefore, I propose to remove their use, with the attached patch. >From 21d

Re: [HACKERS] Transaction-lifespan memory leak with plpgsql DO blocks

2013-11-18 Thread Dilip kumar
On 13 November 2013 03:17 David Johnston wrote, > > Having had this same thought WRT the "FOR UPDATE in LOOP" bug posting > the lack of a listing of outstanding bugs does leave some gaps. I > would imagine people would appreciate something like: > > Frequency: Rare > Severity: Low > Fix Complex

Re: [HACKERS] Improvement of pg_stat_statement usage about buffer hit ratio

2013-11-18 Thread Peter Geoghegan
On Mon, Nov 18, 2013 at 6:12 PM, KONDO Mitsumasa wrote: > I confirmed that Itagaki-san and Mr Cerdic disscution. He said that raw > values be just simple. However, were his changes just simple? I cannot > understand his aesthetics sense and also you, too:-( > It's too complicated, and do you know

[HACKERS] Standalone synchronous master

2013-11-18 Thread Rajeev rastogi
This patch implements the following TODO item: Add a new "eager" synchronous mode that starts out synchronous but reverts to asynchronous after a failure timeout period This would require some type of command to be executed to alert administrators of this change. http://archives.postgresql.org/p

Re: [HACKERS] Add \i option to bring in the specified file as a quoted literal

2013-11-18 Thread Piotr Marcinczyk
Dnia 2013-11-13, śro o godzinie 10:26 -0500, Bruce Momjian pisze: > On Wed, Nov 13, 2013 at 08:58:07AM +0530, Amit Kapila wrote: > > On Tue, Nov 12, 2013 at 9:37 PM, Bruce Momjian wrote: > > > On Wed, Oct 23, 2013 at 10:31:39AM +0530, Amit Kapila wrote: > > >> On Tue, Oct 22, 2013 at 3:04 AM, Piot

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
I checked out master and put together a test case using a small percentage of production data for a known problem we have with Pg 9.2 and text search scans. A small percentage in this case means 10 million records randomly selected; has a few billion records. Tests ran for master successfully an

Re: [HACKERS] GIN improvements part2: fast scan

2013-11-18 Thread Rod Taylor
I tried again this morning using gin-packed-postinglists-16.patch and gin-fast-scan.6.patch. No crashes. It is about a 0.1% random sample of production data (10,000,000 records) with the below structure. Pg was compiled with debug enabled in both cases. Table "public.kp" Column | Type |

Re: [HACKERS] pg_upgrade: delete_old_cluster.sh issues

2013-11-18 Thread Bruce Momjian
On Tue, Nov 12, 2013 at 10:35:58AM +, Marc Mamin wrote: > Hello, > > IMHO, there is a serious issue in the script to clean the old data directory > when running pg_upgrade in link mode. > > in short: When working with symbolic links, the first step in > delete_old_cluster.sh > is to delete

Re: [HACKERS] Re: Suggestion: Issue warning when calling SET TRANSACTION outside transaction block

2013-11-18 Thread Bruce Momjian
On Mon, Nov 18, 2013 at 06:30:32PM -0800, David Johnston wrote: > > Personally, I am fine with changing them all to WARNING. > > Error makes more sense if the goal is internal consistency. That goal > should be subservient to backward compatibility. Changing LOCK to warning > is less problematic

[HACKERS]

2013-11-18 Thread Rajeev rastogi
On 18 November 2013, Amit Khandekar wrote: >> On 18 October 2013 17:07, Rajeev rastogi >> mailto:rajeev.rast...@huawei.com>> wrote: >>From the following mail, copy behaviour between stdin and normal file having >>some inconsistency. >> >> http://www.postgresql.org/message-id/ce85a517.4878

Re: [HACKERS] COPY table FROM STDIN doesn't show count tag

2013-11-18 Thread Rajeev rastogi
On 18 November 2013, Amit Khandekar wrote: >> On 18 October 2013 17:07, Rajeev rastogi >> mailto:rajeev.rast...@huawei.com>> wrote: >>From the following mail, copy behaviour between stdin and normal file having >>some inconsistency. >> >> http://www.postgresql.org/message-id/ce85a517.4878e

Re: [HACKERS] LISTEN / NOTIFY enhancement request for Postgresql

2013-11-18 Thread Sev Zaslavsky
Thank you all for considering my feature request. Dimitri's suggestion is a very good one - I feel it will accomplish the goal of allowing more granularity in the "Listen". We might also want to add a flag in postgresql.conf to disable this enhancement so that we don't break existing code.

  1   2   >