Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-16 Thread Michael Paquier
On Fri, Nov 17, 2017 at 11:14 AM, Andres Freund <and...@anarazel.de> wrote: > On 2017-11-17 11:09:56 +0900, Michael Paquier wrote: >> when redirection_done is switched to true because the first process >> generating a message to the syslogger pipe needs to open it first if &

Re: [GENERAL] postmaster deadlock while logging after syslogger exited

2017-11-16 Thread Michael Paquier
On Fri, Nov 17, 2017 at 10:50 AM, Andres Freund wrote: > On 2017-11-06 15:35:03 -0500, Tom Lane wrote: >> David Pacheco writes: >> > I ran into what appears to be a deadlock in the logging subsystem. It >> > looks like what happened was that the syslogger

Re: [GENERAL] PostgreSQL walsender process doesn't exist after "pg_ctl stop -m fast"

2017-11-13 Thread Michael Paquier
On Tue, Nov 14, 2017 at 12:05 AM, Tom Lane wrote: > y39chen writes: >> We encounter one problem that PostgreSQL walsender process doesn't exist >> after "pg_ctl stop -m fast". >> Uses PostgreSQL 9.6.2 > > There was a fix in 9.6.4 that's at least

Re: [GENERAL] archive_command not being executed

2017-11-13 Thread Michael Paquier
On Tue, Nov 14, 2017 at 8:56 AM, Jeremy Schneider wrote: > From my reading of the docs and commit logs, standby databases > couldn't archive their WALs until 9.5. pg_receivexlog is available in 9.3. You could leverage your archives with it easily, by for example

Re: [GENERAL] Postgresql and github

2017-11-09 Thread Michael Paquier
On Thu, Nov 9, 2017 at 8:27 PM, Poul Kristensen wrote: > How come that Postgresql is not present in a github with latest release? > > It would be very convenient to deploy PG using Ansible. > > Oracle(latest release) is available through github. You are looking for that, which

Re: [GENERAL] Naming conventions for column names

2017-11-07 Thread Michael Paquier
On Wed, Nov 8, 2017 at 12:41 AM, Alvaro Herrera wrote: > Sachin Kotwal wrote: >> 3. Notify or highlight these changes in release notes because this can >> break some existing tools and user code. > > Notifying people when their tools no longer work with a new server is >

Re: [GENERAL] Fwd: standby stop replicating, then picked back up

2017-11-07 Thread Michael Paquier
On Wed, Nov 8, 2017 at 5:17 AM, Laurenz Albe wrote: > chris kim wrote: >> I had a standby hang for a while, not replicating, but then it fixed >> itself but I'm not sure why it happened in the first place. What would I >> look into to see why this happened, or any

Re: [GENERAL] [HACKERS] SSL and Encryption

2017-11-02 Thread Michael Paquier
On Fri, Nov 3, 2017 at 3:19 AM, Craig Ringer wrote: > This is probably off topic for pgsql-hackers. > > For password crypto please go read the SCRAM thread and the PostgreSQL > 10 release notes. The SCRAM discussion is spread across two threads mainly with hundreds of

Re: [GENERAL] PG 10 - Trouble with overlap of range partition of two dimensions

2017-10-31 Thread Michael Paquier
On Tue, Oct 31, 2017 at 8:36 PM, Stephen Froehlich wrote: > CREATE TABLE lotsa_data_20171027_src3 PARTITION OF lotsa_data > FOR VALUES FROM ('2017-10-26 18:00:00-06', 3) TO ('2017-10-27 > 17:59:59.999-06', 3); > ERROR: partition " lotsa_data_20171027_src1" would

Re: [GENERAL] Backup strategy using 'wal_keep_segments'

2017-10-31 Thread Michael Paquier
On Tue, Oct 31, 2017 at 9:53 AM, Rhhh Lin wrote: > I would actually be an advocate for using a proper archive_command in order > to facilitate a proper (Per the documentation) PITR and backup strategy. You should avoid using your own fancy archive command. There are

Re: [GENERAL] starting PG command line options vs postgresql.con

2017-10-30 Thread Michael Paquier
On Mon, Oct 30, 2017 at 2:08 PM, David G. Johnston wrote: > On Mon, Oct 30, 2017 at 6:48 AM, rakeshkumar464 > wrote: >> >> I would prefer using postgresql.conf. what is the consensus in this forum >> regarding command line vs

Re: [GENERAL] Can't build ODBC -- odbc_config not found

2017-10-27 Thread Michael Paquier
On Thu, Oct 26, 2017 at 11:40 PM, Chris Albertson wrote: > I just tried to build Postgres ODBC from source (psqlodbc-10.00..tar.gz) > > I type "./configure" > > Then get this message: > configure: error: odbc_config not found (required for unixODBC build) > > So it

Re: [GENERAL] Allow only certain query on replication slave

2017-10-24 Thread Michael Paquier
On Tue, Oct 24, 2017 at 6:12 AM, basti wrote: > I have a Postgres slave (wal replication) and want no query on it, expect > > - SELECT pg_last_xlog_receive_location() > - SELECT pg_last_xlog_replay_location() > > When I set hot_standby = off in postgres.conf nobody

Re: [GENERAL] Postgres 9.6 fails to start on VMWare

2017-10-22 Thread Michael Paquier
On Sun, Oct 22, 2017 at 11:13 PM, Martin Moore wrote: > I’ve migrated a running Debian Jessie system from a Google Compute instance > to a VMWare ESXi 6.5 system. How did you actually do this migration? It is really easy to finish with a corrupted instance if not doing

Re: [GENERAL] Backup strategy using 'wal_keep_segments'

2017-10-22 Thread Michael Paquier
On Mon, Oct 23, 2017 at 5:57 AM, Rhhh Lin wrote: > Is this approach feasible? Assuming obviously, we have sufficient disk space > to facilitate 1000 WAL files etc. You expose yourself to race conditions with such methods if a checkpoint has the bad idea to recycle past

Re: [GENERAL] Weird performance difference

2017-10-22 Thread Michael Paquier
On Sat, Oct 21, 2017 at 9:38 AM, Tom Lane wrote: > Also try explicitly ANALYZE'ing the foreign tables. I do not > believe auto-analyze will touch foreign tables ... Autovacuum and autoanalyze only process relations and matviews, discarding the rest when scanning pg_class.

Re: [GENERAL] could not fdatasync log file: Input/output error

2017-10-17 Thread Michael Paquier
On Wed, Oct 18, 2017 at 8:02 AM, said assemlal wrote: > Thanks for your response. > > We are currently running postgresql-9.4.14 > I see there are some tools to check if the indexes/pages are not corrupted. > But is there a faster way to check if a PGDATA instance is

Re: [GENERAL] wal_retrieve_retry_interval

2017-10-16 Thread Michael Paquier
On Mon, May 29, 2017 at 3:58 PM, Ludovic Vaugeois-Pepin wrote: > < 2017-05-29 22:42:17.026 CEST > DEBUG: switched WAL source from > archive to stream after failure > ... (15 seconds later) ... > < 2017-05-29 22:42:32.042 CEST > DEBUG: switched WAL source from > stream to

Re: [GENERAL] could not fdatasync log file: Input/output error

2017-10-16 Thread Michael Paquier
On Mon, Oct 16, 2017 at 11:47 PM, said assemlal wrote: > Just before we restart the server today, I found only one line as: > > PANIC: could not fdatasync log file 000101760083: Input/output > error > the database system is in recovery mode Ouch. I would not

Re: [GENERAL] Using cp to back up a database?

2017-10-09 Thread Michael Paquier
On Mon, Oct 9, 2017 at 11:09 PM, Achilleas Mantzios wrote: > On 09/10/2017 16:51, Larry Rosenman wrote: > > If you want a consistent database (you *REALLY* do), pg_dump is the correct > tool. > > In all pg conferences I have been, ppl scream : do not use pg_dump for

Re: [GENERAL] Regarding scram authentication libpq version 10 or more

2017-10-06 Thread Michael Paquier
On Fri, Oct 6, 2017 at 8:56 PM, Durgamahesh Manne wrote: > i have already read complete info about New Postgres 10 in postgres.org and > i have tried to access server by adding the > scram-sha-256 authentication in hba.conf and also have already set >

Re: [GENERAL] Regarding scram authentication libpq version 10 or more

2017-10-06 Thread Michael Paquier
On Fri, Oct 6, 2017 at 5:44 PM, Durgamahesh Manne wrote: > This is regarding scram authentication libpq version 10. From which site i > can download to configure libpq in order to use scram authentication in > postgres 10 > > i got below error while i check to access

Re: [GENERAL] pg_rewind copy so much data

2017-09-29 Thread Michael Paquier
On Fri, Sep 29, 2017 at 6:22 PM, Hung Phan wrote: > Thanks for your help. Do you have any more ideas about my case? I cannot > find any reason for that. If pg_rewind just examines WAL for changed blocks > from the common checkpoint, why did it copy all data ? As I

Re: [GENERAL] pg_rewind copy so much data

2017-09-28 Thread Michael Paquier
On Fri, Sep 29, 2017 at 1:06 PM, Hung Phan <hungphan...@gmail.com> wrote: > I used tablespace to store data and it seems to be that pg_rewind copied > everthing in the tablespace. Today I found an article posted by you (Michael > Paquier) and you said that there was no tablespace

Re: [GENERAL] Logical decoding client has the power to crash the server

2017-09-21 Thread Michael Paquier
On Fri, Sep 22, 2017 at 5:44 AM, Igor Neyman wrote: > I think the difference between pg_current_wal_lsn() and confirmed_flush_lsn > form pg_catalog.pg_replication_slots for specific replication slot: > > SELECT (pg_current_wal_lsn() - confirmed_flush_lsn) AS lsn_distance

Re: [GENERAL] 10 beta 4 foreign table partition check constraint broken?

2017-09-21 Thread Michael Paquier
On Fri, Sep 15, 2017 at 10:43 PM, Paul Jones wrote: > Is this a bug in Postgres 10b4? Looks like neither partition ranges > nor check constraints are honored in 10b4 when inserting into > partitions that are foreign tables. Here is what you are looking for in the documentation:

Re: [GENERAL] Logical decoding client has the power to crash the server

2017-09-20 Thread Michael Paquier
On Thu, Sep 21, 2017 at 1:09 PM, Meel Velliste wrote: > In this situation, neither us, nor our customer has the power to install the > required monitoring of pg_xlog. The database hosting provider would have to > do it. In most cases (e.g. Amazon RDS) the hosting provider does

Re: [GENERAL] VM-Ware Backup of VM safe?

2017-09-20 Thread Michael Paquier
On Wed, Sep 20, 2017 at 5:45 PM, Albe Laurenz wrote: > Thomas Güttler wrote: >> We run a PostgreSQL 9.6 server in a virtual machine. >> >> The virtual machine is managed by the customer. >> >> He does backup the VM. >> >> Is this enough, is this safe? > > I don't know

Re: [GENERAL] Logical decoding client has the power to crash the server

2017-09-20 Thread Michael Paquier
On Wed, Sep 20, 2017 at 3:14 PM, Meel Velliste wrote: > From what I understand about logical decoding, there is no limit to how many > log entries will be retained by the server if nobody reads them from the > logical slot. This means that a client that fails to read from the

Re: [GENERAL] pg_rewind issue

2017-09-18 Thread Michael Paquier
On Mon, Sep 18, 2017 at 1:36 PM, James Sewell wrote: > When Iook at timeline 15 I see this: > > 20 C74/4500no recovery target specified+ This refers to timeline 20. Are there other entries in this history file? Isn't timeline 15 a direct parent of timeline

Re: [GENERAL] "Canceling authentication due to timeout" with idle transaction and reindex

2017-09-15 Thread Michael Paquier
On Fri, Sep 15, 2017 at 7:25 PM, s19n wrote: > Is this expected? I am failing to see the relation between an idle > transaction in the 'postgres' database, a reindex operation and subsequent > logins. REINDEX DATABASE processes as well system indexes, and takes an exclusive

Re: [GENERAL] pg_rewind copy so much data

2017-09-15 Thread Michael Paquier
On Fri, Sep 15, 2017 at 2:57 PM, Hung Phan wrote: > [...] Please do not top-post. This breaks the logic of the thread. > I use ver 9.5.3. You should update to the latest minor version available, there have been quite a couple of bug fixes in Postgres since this 9.5.3. >

Re: [GENERAL] looking for a globally unique row ID

2017-09-14 Thread Michael Paquier
On Thu, Sep 14, 2017 at 4:45 PM, Rafal Pietrak wrote: > Can anybody help me find a way to implement an ID which: > > 1. guarantees being unique across multiple tables. > > 2. guarantees its uniqueness not only during INSERT, but also during the > lifetime of the

Re: [GENERAL] Postgres DB is failed due to pg_Xlog is continues full.

2017-09-13 Thread Michael Paquier
On Thu, Sep 14, 2017 at 12:44 PM, John R Pierce wrote: > prior to that error, something else catastrophic must have happened to the > system, that error is more of a side effect. recovering a database server > that far gone which is running such an obsolete version will

Re: [GENERAL] pg_rewind copy so much data

2017-09-13 Thread Michael Paquier
On Wed, Sep 13, 2017 at 2:21 PM, Hung Phan wrote: > If I run pg_rewind with debug option, it just show additional bunch of files > copied in directories like base or pg_tblspc. I claim that there is no data > inserted of modified from the first step. The only difference

Re: [GENERAL] pg_rewind copy so much data

2017-09-12 Thread Michael Paquier
On Wed, Sep 13, 2017 at 12:41 PM, Hung Phan wrote: > I have tested pg_rewind (ver 9.5) with the following scenario: > > - one master and one slave (total size of each server is more than 4GB) > - set wal_log_hint=on and restart both > - stop master, promote slave > - start

Re: [GENERAL] WAL & ready files retained after turning off log shipping

2017-09-12 Thread Michael Paquier
On Tue, Sep 12, 2017 at 11:43 PM, Ron Johnson wrote: > On 09/07/2017 09:32 AM, Tom Lane wrote: >> >> Ron Johnson writes: >>> >>> On 09/07/2017 09:08 AM, Tom Lane wrote: Manual cleanup shouldn't be very hard, fortunately. Run pg_controldata

Re: [GENERAL] contrecord is requested

2017-09-11 Thread Michael Paquier
On Tue, Sep 12, 2017 at 5:27 AM, Scott Marlowe wrote: > So we have a db we're trying to rewind and get synced to the master. > pg_rewind says it doesn't need rewinding, and when we try to bring it > up, it gets this error: > > "contrecord is requested by 2E7/4028" > >

Re: [GENERAL] pg_rewind issue

2017-09-07 Thread Michael Paquier
On Thu, Sep 7, 2017 at 9:06 PM, James Sewell wrote: > A client has have been having problems with pg_rewind. > > They have two PostgreSQL (Oracle Enterprise Linux 7, 9.6.4) nodes in > streaming replication and follow these steps: > > 1. Stop the master > 2. Promote the

Re: [GENERAL] WAL & ready files retained after turning off log shipping

2017-09-07 Thread Michael Paquier
On Thu, Sep 7, 2017 at 11:08 PM, Tom Lane wrote: > Manual cleanup shouldn't be very hard, fortunately. Run pg_controldata > to see where the last checkpoint is, and delete WAL files whose names > indicate they are before that (but not the one including the checkpoint!). > If

Re: [GENERAL] CREATE TABLE LIKE including all not including storage parameters?

2017-09-05 Thread Michael Paquier
On Tue, Sep 5, 2017 at 6:45 PM, Achilleas Mantzios wrote: > Am I doing something wrong here? >From the documentation: https://www.postgresql.org/docs/devel/static/sql-createtable.html STORAGE settings for the copied column definitions will be copied only if

Re: [GENERAL] RemoveIPC problem

2017-08-31 Thread Michael Paquier
On Fri, Sep 1, 2017 at 12:10 PM, scott ribe wrote: > Yeah, I was kind of thinking that PG detects the semaphore not existing, > bails immediately, restarts clean, thus no problem. I just wanted to hear > from people, like you, that know way more than I do about the

Re: [GENERAL] Table create time

2017-08-31 Thread Michael Paquier
On Fri, Sep 1, 2017 at 12:20 AM, Melvin Davidson wrote: > >you could just create an event trigger looking for CREATE TABLE as > >filter_value: > > I have tried that. Unfortunately, I have been unable to extract the table > name from the event because TG_TABLE_NAME is not >

Re: [GENERAL] Table create time

2017-08-31 Thread Michael Paquier
On Thu, Aug 31, 2017 at 10:21 PM, Melvin Davidson wrote: > Wolfgang, as David said, a column in pg_class for the creation time of a > table does not exist. I long ago requested that feature as it is > in other DB's (Oracle & MS SQL Server), but the main reason that it was

Re: [GENERAL] Unlogged Crash Detection

2017-08-29 Thread Michael Paquier
On Tue, Aug 29, 2017 at 11:09 PM, Andres Freund wrote: > Huh, but that's not particularly meaningful, is it? That'll just as well > be the case for a freshly created relation, no? I have assumed that the OP has some control on the timing of the relations, using an event

Re: [GENERAL] Unlogged Crash Detection

2017-08-29 Thread Michael Paquier
On Tue, Aug 29, 2017 at 6:06 PM, Gersner wrote: > I see, interesting. Please do not top-post. This is not the recommended way of dealing with threads on this mailing list. > We have lots of unlogged tables, upon a crash we want to create a > feedback/alert that data

Re: [GENERAL] Unlogged Crash Detection

2017-08-29 Thread Michael Paquier
On Tue, Aug 29, 2017 at 5:17 PM, Gersner wrote: > Is there a reliable way to distinguish between an empty unlogged table to an > unlogged table which has been truncated due to a crash? Why do you want to make such a difference? At the beginning of a crash recovery all the, the

Re: [GENERAL] Extension coverage

2017-08-28 Thread Michael Paquier
On Sat, Aug 26, 2017 at 6:28 PM, Gabriel Furstenheim Milerud wrote: > Not sure I follow. Do you have an example that I could check? > I have the impression that my problem is that no .gcda files are created. If > I just run the lcov part: > lcov -d . -c -o lcov.info > >

Re: [GENERAL] Retrieving query results

2017-08-28 Thread Michael Paquier
On Sun, Aug 27, 2017 at 12:12 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Michael Paquier <michael.paqu...@gmail.com> writes: >> On Fri, Aug 25, 2017 at 8:10 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> I think the real problem occurs where we re

Re: [GENERAL] Retrieving query results

2017-08-26 Thread Michael Paquier
On Fri, Aug 25, 2017 at 8:10 AM, Tom Lane wrote: > I think the real problem occurs where we realloc the array bigger. > tupArrSize needs to be kept to no more than INT_MAX --- and, ideally, > it should reach that value rather than dying on the iteration after > it reaches 2^30

Re: [GENERAL] Extension coverage

2017-08-25 Thread Michael Paquier
On Sat, Aug 26, 2017 at 2:59 AM, Gabriel Furstenheim Milerud wrote: > The I built my extension against this postgres server > >$ cd $EXTENSION_SOURCE_DIR >$ make install && make installcheck > > the extension is built fine and the test run. However, I see no

Re: [GENERAL] Retrieving query results

2017-08-24 Thread Michael Paquier
On Thu, Aug 24, 2017 at 11:56 PM, Tom Lane wrote: > I haven't tried it, but it sure looks like it would, if you don't hit > OOM first. pqAddTuple() isn't doing anything to guard against integer > overflow. The lack of reports implies that no one has ever tried to > retrieve

Re: [GENERAL] pg_rewind - restore new slave failed to startup during recovery

2017-08-22 Thread Michael Paquier
On Tue, Aug 22, 2017 at 11:39 PM, Magnus Hagander <mag...@hagander.net> wrote: > On Tue, Aug 22, 2017 at 3:06 AM, Michael Paquier <michael.paqu...@gmail.com> > wrote: >> That flow looks correct to me. No I think that you should trigger >> manually a checkpoint after

Re: [GENERAL] Retrieving query results

2017-08-22 Thread Michael Paquier
On Wed, Aug 23, 2017 at 3:19 AM, Igor Korot wrote: > [quote] > PQntuples > > Returns the number of rows (tuples) in the query result. Because it > returns an integer result, large result sets might overflow the return > value on 32-bit operating systems. > > int

Re: [GENERAL] [BUGS] Fwd: PostgreSQL 9.4.13 is facing issue in shutting down

2017-08-21 Thread Michael Paquier
On Tue, Aug 22, 2017 at 1:58 PM, Abhijit Gharami wrote: > Recently we have updated our PostgreSQL version from 9.4.12 to 9.4.13. With > version 9.4.13 while trying to stop the PostgreSQL server, database is > facing issues in shutting down. The problem probably occurs

Re: [GENERAL] pg_rewind - restore new slave failed to startup during recovery

2017-08-21 Thread Michael Paquier
On Tue, Aug 22, 2017 at 9:52 AM, Dylan Luong wrote: > I have 1 master and 1 slave wal streaming replication setup and the > Application connects via a load balancer (LTM) where the all connections are > redirected to the master member (master db). > > We have

Re: [GENERAL] Deleting unwanted wal files

2017-08-17 Thread Michael Paquier
On Fri, Aug 18, 2017 at 12:43 AM, krishna chaitanya wrote: > Thanks for your reply, Please do not top-post, this is not the style of this mailing list. > but will pg_basebackup generate a .backup file when > scheduled in cron job so that i can give that as input to

Re: [GENERAL] Deleting unwanted wal files

2017-08-17 Thread Michael Paquier
On Thu, Aug 17, 2017 at 8:06 PM, krish050591 wrote: > Hi, if i'm using pg_basebackup utility for taking my database backup and also > enabled wal level archiving, how will i detect the unwanted wal files and > how will it delete them ? Have you heard of pg_archivecleanup?

Re: [GENERAL] Begginers question

2017-08-16 Thread Michael Paquier
On Wed, Aug 16, 2017 at 2:32 PM, Alex Samad wrote: > 1) why did it fill up this time and not previously > I add this > archive_command = '/bin/true' > wal_keep_segments = 1000 # <<< I'm guessing its this > > 2) how do I fix up, can I just remove the files from the pg_xlog

Re: [GENERAL] WAL replication wrong collate

2017-08-15 Thread Michael Paquier
On Tue, Aug 15, 2017 at 4:45 AM, basti wrote: > i have fixed. pg_update has create a wrong cluster Let's be sure that we are not talking about a bug here, because you are giving no details so it is hard to know if what you are seeing is caused by an incorrect operation,

Re: [GENERAL] WAL replication wrong collate

2017-08-14 Thread Michael Paquier
On Tue, Aug 15, 2017 at 3:52 AM, basti wrote: > master and slave had set the same locales. > I dont unterstand that i can create a database in en_us.utf8 and then when i > did the basebackup it's change to c locale. > I cant find any option for pg_basebackup to set

Re: [GENERAL] pg_stat_statements -- Historical Query

2017-08-09 Thread Michael Paquier
On Thu, Aug 10, 2017 at 6:23 AM, anand086 wrote: > I was looking for a way to maintain historical query details in Postgres to > answer questions like > > What was the sql call rate between time X and Y? > Did the execution count increase for the query increase between time X

Re: [GENERAL] Lifetime of PQexecPrepared() returned value

2017-08-04 Thread Michael Paquier
On Fri, Aug 4, 2017 at 9:12 PM, Igor Korot wrote: > Am I missing something? How do I fix the crash? Based on what I can see here, I see nothing wrong. Now it is hard to reach any conclusion with the limited information you are providing. -- Michael -- Sent via

Re: [GENERAL] standby database crash

2017-08-01 Thread Michael Paquier
On Mon, Jul 31, 2017 at 11:15 PM, Seong Son (US) wrote: > So my questions are, could an old WAL segment being resent through the > network cause crash like this? Shouldn’t Postgresql be able to handle out > of order WAL segments instead of just crashing? When the

Re: [GENERAL] vacuum on streaming replication

2017-07-31 Thread Michael Paquier
On Mon, Jul 31, 2017 at 7:28 AM, Andreas Kretschmer wrote: > The standby is read only, vacuum runs on the master and replicated to the > standby. Analyse as well. Please note as well that if hot_standby_feedback is enabled, the cleanup done by VACUUM on the primary is

Re: [GENERAL] Fwd: getting error while parsing log file using pgbadger

2017-07-26 Thread Michael Paquier
On Wed, Jul 26, 2017 at 2:01 PM, PAWAN SHARMA wrote: > For root user its working fine > > [root@abc :/opt/PostgreSQL/9.5/data/pgaudit]# > #-> pgbadger -f stderr postgres-2017-07-26_00.csv -o abc.html > [==> ] Parsed 1873673 bytes of 1991998

Re: [GENERAL] Fwd: getting error while parsing log file using pgbadger

2017-07-26 Thread Michael Paquier
On Wed, Jul 26, 2017 at 10:59 AM, PAWAN SHARMA wrote: > > Hi All, > > I am facing below error while parsing log file. > > [postgres@abc pgaudit]$ pgbadger -f stderr postgres-2017-07-25_121445.csv > Can't locate Text/CSV_XS.pm in @INC (@INC contains:

Re: [GENERAL] Monitoring of a hot standby with a largely idle master

2017-07-26 Thread Michael Paquier
On Mon, Jul 24, 2017 at 9:08 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Sun, Jul 16, 2017 at 8:47 AM, Michael Paquier <michael.paqu...@gmail.com> > wrote: >> What do you think about the patch attached? > > Looks OK. Should it mention specifically

Re: [GENERAL] pg_dump not dropping event trigger

2017-07-23 Thread Michael Paquier
On Sun, Jul 23, 2017 at 5:48 PM, Tom Lane wrote: > Greg Atkins writes: >> would you like a bug report to track this? > > No, it's already dealt with. In any case, your original email was good > enough --- we track bugs these days more by message-ID

Re: [GENERAL] Logging at schema level

2017-07-21 Thread Michael Paquier
On Fri, Jul 21, 2017 at 8:21 AM, John R Pierce wrote: > if you have per schema logging, where should that get logged ? > > you could implement per DATABASE logging, if you A) add the database name to > the log_prefix, and B) feed your logs to a program that understands this

Re: [GENERAL] Streaming Replication archive_command is really needed?

2017-07-21 Thread Michael Paquier
On Fri, Jul 21, 2017 at 8:15 AM, Andreas Kretschmer <andr...@a-kretschmer.de> wrote: > Am 21.07.2017 um 08:01 schrieb Michael Paquier: >> "No" is not completely exact and lacks in details. There are two cases >> where having an archive is helpful: >> 1)

Re: [GENERAL] Streaming Replication archive_command is really needed?

2017-07-21 Thread Michael Paquier
On Thu, Jul 20, 2017 at 10:07 PM, Leonardo M. Ramé wrote: > El 20/07/17 a las 16:57, Andreas Kretschmer escribió: >> On 20 July 2017 21:46:09 GMT+02:00, "Leonardo M. Ramé" >> wrote: >>> >>> Hi, I wonder if archive_mode=on and archive_command parameters in

Re: [GENERAL] Logging at schema level

2017-07-20 Thread Michael Paquier
On Fri, Jul 21, 2017 at 7:10 AM, Nikhil wrote: > I am using postgresql schema feature for multi-tenancy. can we get > postgresql logs at schema level. Currently it is for the whole database > server (pg_log) Not directly. All the log-related parameters can be controlled

Re: [GENERAL] Two-phase commit case studies

2017-07-19 Thread Michael Paquier
On Wed, Jul 19, 2017 at 7:10 PM, Gavin Wahl wrote: > I've read the documentation for two-phase commit and it sounds interesting, > but > I'm having trouble thinking of how to actually put it into production. How are > people using it? Do you use a XA transaction manager or

Re: [GENERAL] Monitoring of a hot standby with a largely idle master

2017-07-16 Thread Michael Paquier
On Fri, Jul 14, 2017 at 9:11 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Thu, Jul 13, 2017 at 10:38 AM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> >> On Thu, Jul 13, 2017 at 7:23 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> >

Re: [GENERAL] What is exactly a schema?

2017-07-14 Thread Michael Paquier
On Fri, Jul 14, 2017 at 12:00 PM, marcelo wrote: > The question is not trivial. Could I maintain two or three separate/distinct > "versions" of same database using one schema for every of them? > Could some tables (in the public schema) be shared among all the schemas?

Re: [GENERAL] Monitoring of a hot standby with a largely idle master

2017-07-13 Thread Michael Paquier
On Thu, Jul 13, 2017 at 7:23 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Thu, Jul 13, 2017 at 1:15 AM, Michael Paquier <michael.paqu...@gmail.com> > wrote: >> >> On Thu, Jul 13, 2017 at 5:26 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: >> > >&

Re: [GENERAL] Associating a basebackup and it's .backup WAL file

2017-07-13 Thread Michael Paquier
On Thu, Jul 13, 2017 at 10:30 AM, cen wrote: > Given a basebackup base.tar.gz and an archive of WAL files, is there any way > to find out which .backup WAL file is associated with the basebackup from > command line? Not from what Postgres ships directly. Without any custom

Re: [GENERAL] Monitoring of a hot standby with a largely idle master

2017-07-13 Thread Michael Paquier
On Thu, Jul 13, 2017 at 5:26 AM, Jeff Janes wrote: > > I think that none of the recovery information functions > (https://www.postgresql.org/docs/9.6/static/functions-admin.html#FUNCTIONS-RECOVERY-INFO-TABLE) > can distinguish a hot standby which is connected to an idle

Re: [GENERAL] debugging SSL connection problems

2017-07-11 Thread Michael Paquier
On Tue, Jul 11, 2017 at 6:32 AM, Magnus Hagander wrote: > On Mon, Jul 10, 2017 at 11:19 PM, Jeff Janes wrote: >> Is there a way to get libpq to hand over the certificate it gets from the >> server, so I can inspect it with other tools that give better

Re: [GENERAL] pg_start/stop_backup non-exclusive scripts to snapshot

2017-07-05 Thread Michael Paquier
On Wed, Jul 5, 2017 at 10:47 PM, Melvin Davidson wrote: > Your criticism is noted, however, I have used it many times in the past with > absolutely no problem. Plug off the server on which is stored the backup just after your script finishes, you have a good chance to be

Re: [GENERAL] duplicate key value violates unique constraint and duplicated records

2017-07-02 Thread Michael Paquier
On Mon, Jul 3, 2017 at 10:08 AM, Steven Chang wrote: > Hello : Please avoid top-posting. >PG VERSION : PPAS 9.3 , enterprisedb >os version : 2.6.32-358.el6.x86_64 This is EnterpriseDB's fork of Postgres. Until it can be proved that a corruption has

Re: [GENERAL] Suddenly - LOG: could not open file "postmaster.pid": No such file or directory

2017-06-22 Thread Michael Paquier
On Fri, Jun 23, 2017 at 1:43 PM, Tom Lane wrote: > Muhammad Hanif Abdul Hamid writes: >> Within this week, I have faced two times "postmaster.pid" went missing in a >> sudden. Nobody deleted it or shutdown the server. > > I'd lay very long odds that

Re: [GENERAL] Error with DefineSavepoint:Unexpected state STARTED

2017-06-19 Thread Michael Paquier
On Tue, Jun 20, 2017 at 11:12 AM, Computer Study wrote: > The UI application is to add/remove some permissions through a listbox for > a certain user. For any add/remove, it will first delete all records for > this certain user in the table, then insert the values

Re: [GENERAL] Replication slot and pg_rewind

2017-06-05 Thread Michael Paquier
On Tue, Jun 6, 2017 at 1:52 PM, Bhattacharyya, Subhro wrote: > Our expectation is that slave will be able to sync with the new master with > the help of whatever WALs are present in the new master due to replication > slots. > Can pg_rewind still work without WAL

Re: [GENERAL] Replication slot and pg_rewind

2017-06-05 Thread Michael Paquier
On Tue, Jun 6, 2017 at 12:03 PM, Bhattacharyya, Subhro wrote: > We are using the replication slot and pg_rewind feature of postgresql 9.6 > Our cluster consists of 1 master and 1 slave node. > > The replication slot feature allows the master to keep as much WAL as is >

Re: [GENERAL] pg_rewind - enable wal_log_hints or data-checksums

2017-06-04 Thread Michael Paquier
On Mon, Jun 5, 2017 at 9:37 AM, Dylan Luong wrote: > pg_rewind requires that the target server either has the wal_log_hints > option enabled in postgresql.conf or data checksums enabled when the cluster > was initialized with initdb. Yes, this is to make sure that you

Re: [GENERAL] wal_retrieve_retry_interval

2017-05-29 Thread Michael Paquier
On Sun, May 28, 2017 at 9:41 AM, Ludovic Vaugeois-Pepin wrote: > On Sun, May 28, 2017 at 9:49 AM, Ludovic Vaugeois-Pepin > wrote: >> It really is set at 3s on all servers (master and standbies) earlier in the >> "deployment" process at the same time

Re: [GENERAL] wal_retrieve_retry_interval

2017-05-27 Thread Michael Paquier
On Sat, May 27, 2017 at 2:40 PM, Ludovic Vaugeois-Pepin wrote: > Say, with 9.6.2, a hot_standby fails to connect to a replication slot: > FATAL: could not start WAL streaming: ERROR: replication slot "test3" > does not exist > or > FATAL: could not connect to the

Re: [GENERAL] Re: Is there possibility btree_redo with XLOG_BTREE_DELETE done between standby_redo and the end of backup

2017-05-25 Thread Michael Paquier
On Thu, May 25, 2017 at 5:23 AM, y39chen wrote: > My doubt is Standby is redoing the records in WAL from master. how accept > connection in standby side while recovering would trigger > btree_xlog_delete_get_latestRemovedXid() and panic happen. You should look at the

Re: [GENERAL] Is there possibility btree_redo with XLOG_BTREE_DELETE done between standby_redo and the end of backup

2017-05-24 Thread Michael Paquier
On Wed, May 24, 2017 at 11:47 AM, Tom Lane wrote: > Wup, sorry, I quoted the wrong thread mentioning > btree_xlog_delete_get_latestRemovedXid. The right one is > > https://www.postgresql.org/message-id/flat/00F0B2CEF6D0CEF8A90119D4%40eje.credativ.lan > > and the patch is > >

Re: [GENERAL] Is there possibility btree_redo with XLOG_BTREE_DELETE done between standby_redo and the end of backup

2017-05-24 Thread Michael Paquier
On Wed, May 24, 2017 at 11:18 AM, Tom Lane wrote: > y39chen writes: >> is there possibility btree_redo with XLOG_BTREE_DELETE info done between >> standby_redo and the end of backup? I have PostgreSQL 9.3.14 which have >> some patches to use and easy

Re: [GENERAL] pgAdmin4 needs information of v10 SCRAM authentication

2017-04-24 Thread Michael Paquier
On Mon, Apr 24, 2017 at 3:04 PM, Akshay Joshi wrote: >I have gone through this, but still facing issue to encrypt/change the > database server password. In pgAdmin4 we have "Change Password" feature where > user will enter the old and new password for the

Re: [GENERAL] pg_basebackup ----xlog-method=stream

2017-04-21 Thread Michael Paquier
On Sat, Apr 22, 2017 at 3:02 AM, Ian Harding wrote: > Am I misunderstanding how this works? I have WAL archiving set up, so the > files are available, but I wanted them included in the backup. Please note that if you have a WAL archive available, you may not even need to

Re: [GENERAL] pgAdmin4 needs information of v10 SCRAM authentication

2017-04-21 Thread Michael Paquier
On Fri, Apr 21, 2017 at 3:43 PM, Akshay Joshi wrote: >Thanks Michael, will check this. One thing I forgot to mention... Both StoredKey and ServerKey are now encoded in hex, but there is still an open item related to the handling of psql's \password on which I

Re: [GENERAL] pgAdmin4 needs information of v10 SCRAM authentication

2017-04-21 Thread Michael Paquier
On Fri, Apr 21, 2017 at 3:27 PM, Akshay Joshi wrote: > from passlib.hash import scram > hash = scram.encrypt(data['newPassword']) -- This function provide password > for all the supported digest like [md5, sha-1, sha-256, sha-512]. Didn't work > I have tried with

Re: [GENERAL] financial formulae

2017-04-13 Thread Michael Paquier
On Thu, Apr 13, 2017 at 8:44 PM, FarjadFarid(ChkNet) wrote: > Is there a package/library, for postgresql server, covering financial > formula equivalent to excel financial formulae? > > Especially considering the quality and precision. You will need to be more

Re: [GENERAL] Suggestion to improve select pg_reload_conf()

2017-04-03 Thread Michael Paquier
On Mon, Apr 3, 2017 at 4:39 PM, Achilleas Mantzios wrote: > On 03/04/2017 10:31, Thomas Kellerer wrote: >> >> I would like to suggest an improvement to the select pg_reload_conf() >> function. >> >> Currently this will only return true or false indicating if

Re: [GENERAL] Index loading methods

2017-03-27 Thread Michael Paquier
On Mon, Mar 27, 2017 at 8:36 PM, pinker wrote: > If PostgreSQL decides to use an index, does he every time load the whole > B-tree into memory? or maybe loads only specific subtree or some chunks of > index? src/backend/access/nbtree/README provides details about the algorithm of

Re: [GENERAL] browser interface to forums please?

2017-03-26 Thread Michael Paquier
On Sun, Mar 26, 2017 at 2:50 PM, John R Pierce wrote: > On 3/25/2017 10:31 PM, George Neuner wrote: >> >> I also dislike having my email full of list posts. > > I have a 'postgres' folder in Thunderbird, and all posts "To:" or CC: > pgsql-*@postgresql.org get moved there,

  1   2   3   4   5   6   >