Re: [GENERAL] Log storage

2017-10-19 Thread Condor
On 18-10-2017 09:18, Ivan Sagalaev wrote: Hello everyone, An inaugural poster here, sorry if I misidentified a list for my question. I am planning to use PostgreSQL as a storage for application logs (lines of text) with the following properties: - Ingest logs at high rate: 3K lines per secon

Re: [GENERAL] Log storage

2017-10-18 Thread Laurent Laborde
Friendly greetings ! You may want to take a look at a postgresql "fork" called pipelinedb : https://www.pipelinedb.com/ https://github.com/pipelinedb/pipelinedb I'm not working for them, not using it, but i happen to know it exist :) *hugs* -- Laurent "ker2x" Laborde

Re: [GENERAL] Log storage

2017-10-18 Thread legrand legrand
I saw a similar project on oracle that was storing (long) messages (clob). Partionning by creation date was in place, as btree indexes to access data per id. It was working fine for inserts, as for sélect, but purges (delete) where not freeing space. In fact rétention was not the same for all rec

Re: [GENERAL] Log storage

2017-10-18 Thread Ivan Sagalaev
Can't get to hard data right now, but those are app logs that try to be no more than ~100 bytes characters long for readability, and also HTTP logs with long-ish request lines which might put it in the neighborhood of 2K characters. On 10/18/2017 02:30 AM, legrand legrand wrote: What is the

Re: [GENERAL] Log storage

2017-10-18 Thread legrand legrand
What is the (min, max, avg) size of the inserted text ? - PAscal SQLeo projection manager Senior Oracle dba migrating towards PostgreSQL -- Sent from: http://www.postgresql-archive.org/PostgreSQL-general-f1843780.html -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org)

[GENERAL] Log storage

2017-10-17 Thread Ivan Sagalaev
Hello everyone, An inaugural poster here, sorry if I misidentified a list for my question. I am planning to use PostgreSQL as a storage for application logs (lines of text) with the following properties: - Ingest logs at high rate: 3K lines per second minimum, but the more the better as it w

Re: [GENERAL] Log shipping in v8.4.7

2017-08-27 Thread Christoph Moench-Tegeder
## Ron Johnson (ron.l.john...@cox.net): > > On today's LANs, total archiving time is dominated by connection > > startup time (how long does it take to transfer 16MB on a 10GbE link? > > See...). > > And if we've only got a WAN link from one DC to another 360 miles away? Well... TCP handshake wi

Re: [GENERAL] Log shipping in v8.4.7

2017-08-27 Thread Ron Johnson
On 08/27/2017 02:23 PM, Christoph Moench-Tegeder wrote: ## Ron Johnson (ron.l.john...@cox.net): Everything I've read says that you should use "rsync -a". Is there any reason why we can't/shouldn't use "rsync -az" so as to reduce transfer time? On today's LANs, total archiving time is dominate

Re: [GENERAL] Log shipping in v8.4.7

2017-08-27 Thread Christoph Moench-Tegeder
## Ron Johnson (ron.l.john...@cox.net): > Everything I've read says that you should use "rsync -a". Is there > any reason why we can't/shouldn't use "rsync -az" so as to reduce > transfer time? On today's LANs, total archiving time is dominated by connection startup time (how long does it take t

[GENERAL] Log shipping in v8.4.7

2017-08-27 Thread Ron Johnson
Hi, (Yes, its old. Nothing I can do about that.) Everything I've read says that you should use "rsync -a". Is there any reason why we can't/shouldn't use "rsync -az" so as to reduce transfer time? Also, does that change require a full restart (difficult with production systems)? Thanks

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-12 Thread Chris Richards
I have another application that consumes all of the huge pages; they aren't for pgsql. :) I've modified the configuration file from "try" to "off" and munmap is no more. Mischief managed. Thanks for your help. On Tue, Oct 11, 2016 at 6:15 PM, Tom Lane wrote: > Chris Richards writes: > > Oh th

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-11 Thread Tom Lane
Chris Richards writes: > Oh the email spam :P No problem. I'm starting to think that this isn't actually a Postgres bug, but rather something funny with your kernel. The code in sysv_shmem is certainly as fragile as can be, but I'm darned if I can find an existing code path that would actually

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-11 Thread Chris Richards
Oh the email spam :P I did another reboot test to validate the error. It does exist; it does not spam munmap error like it did with the aptitude install. I then marked the log file, shutdown (1 munmap during), marked it again and then started it (1 munmap during). MARK PRE-SHUTDOWN 2016-10-11 20

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-11 Thread Chris Richards
One extra bit, if this helps: chrisr@ff1:~$ cat /proc/meminfo MemTotal:8163104 kB MemFree: 2183692 kB MemAvailable:3648680 kB Buffers: 170080 kB Cached: 1231708 kB SwapCached:0 kB Active: 1083596 kB Inactive: 442312 kB Active(anon)

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-11 Thread Chris Richards
What more would you like--I'm happy to oblige? The upgrade steps I listed installed into a new directory, new conf file. I didn't even migrate my (skeletal) 9.3 database. Here's the 9.5 command-line. I've attached the referenced config file. $ chrisr@ff1:~$ ps ax | grep bin/post 10 ?S

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-10 Thread Tom Lane
Andres Freund writes: > On 2016-10-10 18:21:48 -0400, Tom Lane wrote: >> Chris Richards writes: >>> LOG: munmap(0x7fff8000) failed: Invalid argument >> [ digs in code... ] One theory is that PGSharedMemoryDetach is getting >> called more than once, but I'm not sure how that would happen.

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-10 Thread Andres Freund
On 2016-10-10 18:21:48 -0400, Tom Lane wrote: > Chris Richards writes: > > Setting up postgresql-9.5 (9.5.4-1.pgdg14.04+2) ... > > Creating new cluster 9.5/main ... > > config /etc/postgresql/9.5/main > > data /var/lib/postgresql/9.5/main > > locale en_US.UTF-8 > > LOG: munmap(0x7fff8

Re: [GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-10 Thread Tom Lane
Chris Richards writes: > Setting up postgresql-9.5 (9.5.4-1.pgdg14.04+2) ... > Creating new cluster 9.5/main ... > config /etc/postgresql/9.5/main > data /var/lib/postgresql/9.5/main > locale en_US.UTF-8 > LOG: munmap(0x7fff8000) failed: Invalid argument > [... snip 14 or so repeats .

[GENERAL] LOG: munmap(0x7fff80000000) failed: Invalid argument

2016-10-10 Thread Chris Richards
Howdy. I have an Ubuntu 14.04 LTS, configured with huge pages (4x1GB, disabled transparent) and a later kernel than what ships with 14.04. root@ff2:~# uname -a Linux ff2 3.16.7-ckt11-061515+ #1 SMP Mon Jun 15 18:47:13 CDT 2015 x86_64 x86_64 x86_64 GNU/Linux It had postgresql-9.3 on it and I inst

Re: [GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-28 Thread Jeff Janes
On Sun, Aug 28, 2016 at 5:18 PM, Jim Nasby wrote: > On 8/25/16 7:45 PM, Ahsan Ali wrote: > > Please don't top-post; it's harder to read. > >> On Thu, Aug 25, 2016 at 5:29 PM, John R Pierce > > wrote: >> so there were 1818 postgres client processes at the time it co

Re: [GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-28 Thread Jim Nasby
On 8/25/16 7:45 PM, Ahsan Ali wrote: Please don't top-post; it's harder to read. On Thu, Aug 25, 2016 at 5:29 PM, John R Pierce mailto:pie...@hogranch.com>> wrote: so there were 1818 postgres client processes at the time it coudln't create a new process. thats certainly a larger number

Re: [GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-25 Thread Ahsan Ali
we have a pooling on the application level. however we never had this issues before this start happning since last couple of days in past we had over 2300 sessions but no issues. On Thu, Aug 25, 2016 at 5:29 PM, John R Pierce wrote: > On 8/25/2016 5:10 PM, Ahsan Ali wrote: > >> yes it is older h

Re: [GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-25 Thread John R Pierce
On 8/25/2016 5:10 PM, Ahsan Ali wrote: yes it is older however we do apply security patches now a then. redhat doesn't really support mixing packages from different releases, they only test things with all packages from the same snapshot. "yum update" should bring the whole system up to cur

Re: [GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-25 Thread Ahsan Ali
yes it is older however we do apply security patches now a then. regarding max connection its the application design however it does not have that many active session. postgres=# select count(*) from pg_stat_activity; count --- 1818 Please let me know if you like to see any other logs and

Re: [GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-25 Thread John R Pierce
On 8/25/2016 3:54 PM, Ahsan Ali wrote: Red Hat Enterprise Linux Server release 6.3 (Santiago) that was released in June 2012, you're missing 4+ years of bug fixes, 6.8 is current. max_connections = 3000 thats insanely high for most purposes unless you have several 100 CPU cores. otherwi

Re: [GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-25 Thread Ahsan Ali
Hi John, Thanks for replying. Below are all the details I am using psql (PostgreSQL) 9.5.2 Error we got in postgresql log* 2016-08-25 11:15:55 PDT [60739]: [10282-1] LOG: could not fork new process for connection: Cannot allocate memory 2016-08-25 11:15:55 PDT [60739]: [10283-1] LOG: co

Re: [GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-25 Thread John R Pierce
On 8/25/2016 11:49 AM, Ahsan Ali wrote: I having my production server service intruption because of this "Cannot allocate memory" error in the db log. could you paste the whole error message? what version of postgres is this? what OS (if linux, distribution) version is this? older version

[GENERAL] LOG: could not fork new process for connection: Cannot allocate memory

2016-08-25 Thread Ahsan Ali
Hi Support I having my production server service intruption because of this "Cannot allocate memory" error in the db log. because of this error even DBA's cant login to the db server unless we kill one of the existing sessions. Please help me resolve this issue. 1. $ grep Commit /proc/meminf

Re: [GENERAL] Log all queries before migration ?

2016-08-03 Thread Melvin Davidson
woch, 3. August 2016 15:29 > > To: pgsql-general@postgresql.org > > Subject: Re: [GENERAL] Log all queries before migration ? > > > > Hi guys, > > > > In fact, I would like to copy a table and then, when everything is > finish, execute all inserts and updates s

Re: [GENERAL] Log all queries before migration ?

2016-08-03 Thread Charles Clavadetscher
Hello > -Original Message- > From: pgsql-general-ow...@postgresql.org > [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of ben.play > Sent: Mittwoch, 3. August 2016 15:29 > To: pgsql-general@postgresql.org > Subject: Re: [GENERAL] Log all queries before migrat

Re: [GENERAL] Log all queries before migration ?

2016-08-03 Thread ben.play
Hi guys, In fact, I would like to copy a table and then, when everything is finish, execute all inserts and updates since the beginning of the copy. The biggest problem with the log at this time it's the query. All my queries are not runnable. When I log, I see this kind of queries : "duratio

Re: [GENERAL] Log all queries before migration ?

2016-07-29 Thread David G. Johnston
On Fri, Jul 29, 2016 at 7:35 AM, ben.play wrote: > Hi guys, > > We have some storage problem on our dedicated server and we have to > migrate. > Do you know how can I log all queries in order to have no downtime during > migration ? > > ​I don't understand how logging of queries has anything to d

Re: [GENERAL] Log all queries before migration ?

2016-07-29 Thread CS DBA
log_min_duration_statement = 0 On 07/29/2016 05:35 AM, ben.play wrote: Hi guys, We have some storage problem on our dedicated server and we have to migrate. Do you know how can I log all queries in order to have no downtime during migration ? I tried many options on the conf, but i never fin

[GENERAL] Log all queries before migration ?

2016-07-29 Thread ben.play
Hi guys, We have some storage problem on our dedicated server and we have to migrate. Do you know how can I log all queries in order to have no downtime during migration ? I tried many options on the conf, but i never find the good one. I didn't find out to generate .sql file with all queries.

Re: [GENERAL] Log archiving failing. Seems to be wrong timeline

2016-07-01 Thread Chris Lewis
Hi Jeff, Done as you advised and now things are working again. Many thanks Chris On 30/06/16 20:19, Jeff Janes wrote: On Thu, Jun 30, 2016 at 3:53 AM, Chris Lewis wrote: Hello, We have 2 postgresql servers (v 9.4.2) master and slave in streaming replication. The overall cluster is control

Re: [GENERAL] Log archiving failing. Seems to be wrong timeline

2016-06-30 Thread Jeff Janes
On Thu, Jun 30, 2016 at 3:53 AM, Chris Lewis wrote: > Hello, > > We have 2 postgresql servers (v 9.4.2) master and slave in streaming > replication. The overall cluster is controlled using pacemaker & corosync > and the pgsql cluster agent which handles failover to, and promotion of, the > slave.

[GENERAL] Log archiving failing. Seems to be wrong timeline

2016-06-30 Thread Chris Lewis
Hello, We have 2 postgresql servers (v 9.4.2) master and slave in streaming replication. The overall cluster is controlled using pacemaker & corosync and the pgsql cluster agent which handles failover to, and promotion of, the slave. Recently a failover occured and I noticed that log archiv

Re: [GENERAL] Log Shipping

2016-05-31 Thread Joseph Kregloh
On Tue, May 31, 2016 at 4:12 PM, Alvaro Herrera wrote: > Joseph Kregloh wrote: > > It is my understanding that if PostgeSQL has log shipping enabled, if for > > whatever reason it cannot ship the file the master server will hold it. > But > > for how long? > > Forever (which means it dies because

Re: [GENERAL] Log Shipping

2016-05-31 Thread Alvaro Herrera
Joseph Kregloh wrote: > It is my understanding that if PostgeSQL has log shipping enabled, if for > whatever reason it cannot ship the file the master server will hold it. But > for how long? Forever (which means it dies because of running out of space in the partition containing pg_xlog). > Seco

[GENERAL] Log Shipping

2016-05-31 Thread Joseph Kregloh
It is my understanding that if PostgeSQL has log shipping enabled, if for whatever reason it cannot ship the file the master server will hold it. But for how long? Secondly, I have 2 servers I ship log files to using the following script: #!/usr/local/bin/bash # Slave 1 rsync -a $1 pgi@192.168.1

Re: [GENERAL] log temp files are created twice in PL/pgSQL function

2016-03-19 Thread Adrian Klaver
On 03/16/2016 07:58 AM, Dang Minh Huong wrote: Hi, Why does log temp files are created twice when query is executed in PL/pgSQL function? Would you please explain it to me? As below test result. Log temp files are created twice when SELECT statement is put into a PL/pgSQL function. It led

Re: [GENERAL] log temp files are created twice in PL/pgSQL function

2016-03-19 Thread Pavel Stehule
Hi 2016-03-16 15:58 GMT+01:00 Dang Minh Huong : > Hi, > > Why does log temp files are created twice when query is executed in > PL/pgSQL function? > Would you please explain it to me? > PL/pgSQL materialize result internally. Usually PostgreSQL operations are executed row by row. But some SRF f

[GENERAL] log temp files are created twice in PL/pgSQL function

2016-03-19 Thread Dang Minh Huong
Hi, Why does log temp files are created twice when query is executed in PL/pgSQL function? Would you please explain it to me? As below test result. Log temp files are created twice when SELECT statement is put into a PL/pgSQL function. It led a little of performance degradation. Is there any

Re: [GENERAL] log temp files are created twice in PL/pgSQL function

2016-03-19 Thread Dang Minh Huong
Hi, Thanks for your prompt response. Appreciate your help. Thanks and best regards, bocap > Hi > > 2016-03-16 15:58 GMT+01:00 Dang Minh Huong : >> Hi, >> >> Why does log temp files are created twice when query is executed in PL/pgSQL >> function? >> Would you please explain it to me? > > PL/

Re: [GENERAL] Log Monitoring with PG Admin

2016-02-17 Thread David G. Johnston
On Wed, Feb 17, 2016 at 9:05 AM, Alex Magnum wrote: > Hi, > i am running and RDS instance on AWS but monitoring logs is a bit > cumbersome. > Is there a way to do the through pgadmin3 ? Like tailing the logfile? > ​Only if you are capable of doing something like this on RDS:​ ​http://www.postgr

[GENERAL] Log Monitoring with PG Admin

2016-02-17 Thread Alex Magnum
Hi, i am running and RDS instance on AWS but monitoring logs is a bit cumbersome. Is there a way to do the through pgadmin3 ? Like tailing the logfile? Does anyone know when 9.5 will be available on aws? Thanks Alex

Re: [GENERAL] Log-shipping replication in one machine

2014-10-28 Thread Adrian Klaver
On 10/26/2014 09:46 PM, nurul [via PostgreSQL] wrote: Thank you for your response. May i know what is the difference between log shipping and streaming replication actually? I'm sorry i am very new in postgreSQL and still confused with these two For an overview see: http://www.postgresql.org/

Re: [GENERAL] Log-shipping replication in one machine

2014-10-28 Thread nurul [via PostgreSQL]
Thank you for your response. May i know what is the difference between log shipping and streaming replication actually? I'm sorry i am very new in postgreSQL and still confused with these two __ If you reply to this email, your message will be added to the

Re: [GENERAL] Log-shipping replication in one machine

2014-10-24 Thread Michael Paquier
On Tue, Oct 21, 2014 at 4:02 PM, nurul [via PostgreSQL] wrote: > > We have install PostgreSQL 9.3 in Ubuntu. We want to ask it is possible to do > log shipping replication in one machine with different port such as port 5435 > as a master while 5436 as a slave? We also tried that process in one

Re: [GENERAL] Log-shipping replication in one machine

2014-10-24 Thread John R Pierce
On 10/21/2014 12:02 AM, nurul [via PostgreSQL] wrote: We do the log shipping replication process based on http://www.themagicnumber.es/replication-in-postgresql-i?lang=en thats a rather old blog entry, appears to be talking about postgres 8.3 -- john r pierce

[GENERAL] Log-shipping replication in one machine

2014-10-24 Thread nurul [via PostgreSQL]
We have install PostgreSQL 9.3 in Ubuntu. We want to ask it is posibble to do log shipping replication in one machine with different port such as port 5435 as a master while 5436 as a slave? We also tried that process in one machine but still get an error in slave such as warning: connection to th

Re: [GENERAL] Log Data Analytics : Confused about the choice of Database

2014-05-14 Thread Dorian Hoxha
On Wed, May 14, 2014 at 6:48 AM, Peeyush Agarwal < peeyushagarwal1...@gmail.com> wrote: > Hi, > > Thanks for the reply :) > > Yes, Storing timestamp as timestamp [ (p) ] would be better. I simplified > the session in question. It may contain alphabets as well. So, I will > probably need to store i

Re: [GENERAL] Log Data Analytics : Confused about the choice of Database

2014-05-13 Thread Peeyush Agarwal
Hi, Thanks for the reply :) Yes, Storing timestamp as timestamp [ (p) ] would be better. I simplified the session in question. It may contain alphabets as well. So, I will probably need to store it as a string only. The problem with types of events is that it is not fixed and will keep increasin

[GENERAL] Log Data Analytics : Confused about the choice of Database

2014-05-13 Thread Peeyush Agarwal
Hi, I have log data of the following format: SessionTimestampEventParameters1 1 Started Session 1 2Logged In Username:"user1"2 3Started Session1 3 Started Challengetitle:"Challenge 1

Re: [GENERAL] Log Data Analytics : Confused about the choice of Database

2014-05-13 Thread Dorian Hoxha
Why not store session as integer? And timestamp as timesamp(z?) ? If you know the types of events, also store them as integer , and save a map of them in the app or on another table ? And save the parameters as a json column, so you have more data-types? Hstore only has strings. Be carefull wit

[GENERAL] Log Data Analytics : Confused about the choice of Database

2014-05-12 Thread Peeyush Agarwal
Hi, I have log data of the following format: SessionTimestampEventParameters1 1 Started Session 1 2Logged In Username:"user1"2 3Started Session1 3 Started Challengetitle:"Challenge 1

Re: [GENERAL] Log file monitoring and event notification

2014-04-07 Thread Steve Crawford
On 04/05/2014 08:47 AM, Andy Colson wrote: Hi All. I've started using replication, and I'd like to monitor my logs for any errors or problems. I don't want to do it manually, and I'm not interested in stats (a la PgBadger). What I'd like, is the instant PG logs: "FATAL: wal segment already

Re: [GENERAL] Log file monitoring and event notification

2014-04-05 Thread bricklen
On Sat, Apr 5, 2014 at 8:47 AM, Andy Colson wrote: > Hi All. > > I've started using replication, and I'd like to monitor my logs for any > errors or problems. I don't want to do it manually, and I'm not interested > in stats (a la PgBadger). > > What I'd like, is the instant PG logs: "FATAL: wal

Re: [GENERAL] Log file monitoring and event notification

2014-04-05 Thread Antman, Jason (CMG-Atlanta)
General thought: It's entirely possible my current Postgres environment is missing something (I'm an automation engineer, not a DBA - most of my postgres knowledge has been learned on the job or from Google), but we actively monitor the receive and replay lag (i.e. comparing pg_current_xlog_lo

[GENERAL] Log file monitoring and event notification

2014-04-05 Thread Andy Colson
Hi All. I've started using replication, and I'd like to monitor my logs for any errors or problems. I don't want to do it manually, and I'm not interested in stats (a la PgBadger). What I'd like, is the instant PG logs: "FATAL: wal segment already removed" (or some such bad thing), I'd like

Re: [GENERAL] Log messages regarding automatic vacuum and exclusive locks

2013-04-24 Thread jonesd
I'll give using TRUNCATE to clear the tables a try and see what happens. Dominic Jones Quoting Sergey Konoplev : On Tue, Apr 23, 2013 at 8:50 AM, wrote: Good morning. I'm seeing several of the following log messages each morning (for example, there were five this morning, spaced approxim

Re: [GENERAL] Log messages regarding automatic vacuum and exclusive locks

2013-04-23 Thread Sergey Konoplev
On Tue, Apr 23, 2013 at 8:50 AM, wrote: > Good morning. I'm seeing several of the following log messages each morning > (for example, there were five this morning, spaced approximately one minute > apart, with the closest interval between messages being 44 seconds). > They're occurring during a

Re: [GENERAL] Log messages regarding automatic vacuum and exclusive locks

2013-04-23 Thread Kevin Grittner
"jon...@xmission.com" wrote: > I'm seeing several of the following log messages each > morning (for example, there were five this morning, spaced > approximately one minute apart, with the closest interval between > messages being 44 seconds).  They're occurring during a daily cycle of > dele

[GENERAL] Log messages regarding automatic vacuum and exclusive locks

2013-04-23 Thread jonesd
Good morning. I'm seeing several of the following log messages each morning (for example, there were five this morning, spaced approximately one minute apart, with the closest interval between messages being 44 seconds). They're occurring during a daily cycle of deleting all rows from a t

Re: [GENERAL] Log rolling

2012-12-20 Thread Adrian Klaver
On 12/20/2012 02:08 PM, Alan Nilsson wrote: Is there any sql/psql command to force a log roll (i.e. start a new log file)? http://www.postgresql.org/docs/9.2/interactive/functions-admin.html pg_rotate_logfile() alan -- Adrian Klaver adrian.kla...@gmail.com -- Sent via pgsql-general

[GENERAL] Log rolling

2012-12-20 Thread Alan Nilsson
Is there any sql/psql command to force a log roll (i.e. start a new log file)? alan -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-general

Re: [GENERAL] log select access

2012-07-26 Thread Merlin Moncure
On Thu, Jul 26, 2012 at 4:32 PM, Little, Douglas wrote: > > Hello everybody, > > > > For PCI compliance I need to log user access to my PCI columns in a table > and retain for 2 years. > > I know I can grep the log, but with 1m log rows/day and infrequent PCI > access, I’m thinking this isn’t th

[GENERAL] log select access

2012-07-26 Thread Little, Douglas
Hello everybody, For PCI compliance I need to log user access to my PCI columns in a table and retain for 2 years. I know I can grep the log, but with 1m log rows/day and infrequent PCI access, I'm thinking this isn't the most efficient method. I've been thinking about a SELECT rule, for the

[GENERAL] Log: Untranslatable character: no DETAIL

2012-05-18 Thread Mike Blackwell
When logging long queries we get a DETAIL record with the query parameters, like this: 2012-05-17 08:33:33 CDT [15053]: [1-1] user@database x.x.x.x(35559)LOG: 0: duration: 587.476 ms execute dbdpg_p15947_158: select a_bunch_of_stuff from table_b where id = $1 and 2012-05-17 08:33:33

Re: [GENERAL] Log to syslog in one line

2012-04-22 Thread Roberto De Oliveira
> > If I understand it correct you need a statement playback solution. > Here is some info about it > http://wiki.postgresql.org/wiki/Statement_Playback. > Thanks Sergey, I was reading and a I realized that maybe this can be useful. -- Saludos, Roberto De Oliveira -- Sent via pgsql-general ma

Re: [GENERAL] Log to syslog in one line

2012-04-18 Thread Sergey Konoplev
On Wed, Apr 18, 2012 at 8:55 AM, Roberto De Oliveira wrote: > postgres database. For this purpose , I configured postgres to send > all query to syslog-ng to parser them and store them in another > database, but I have a issue: all query splits into diferents lines > making imposible work with the

[GENERAL] Log to syslog in one line

2012-04-18 Thread Roberto De Oliveira
Hi everybody, I am interesting on do some kind of audit on a web system through his postgres database. For this purpose , I configured postgres to send all query to syslog-ng to parser them and store them in another database, but I have a issue: all query splits into diferents lines making imposibl

Re: [GENERAL] Log statement with timestamp

2012-02-23 Thread Raghavendra
Need to enable log_line_prefix with timestamp "%t". http://www.postgresql.org/docs/9.1/static/runtime-config-logging.html --- Regards, Raghavendra EnterpriseDB Corporation Blog: http://raghavt.blogspot.com/ On Thu, Feb 23, 2012 at 2:15 PM, Madhu S R wrote: > Hi, > > I need to log DDL or M

[GENERAL] Log statement with timestamp

2012-02-23 Thread Madhu S R
Hi,     I need to log DDL or MOD statements with current time when the statement executed. Please let me know if this is possible. Regards, Madhu

[GENERAL] Log or notice values or rows that cause a constraint violation

2011-10-18 Thread Craig Ringer
Hi all I'm once again trying to figure out which row of a 5000-record insert is violating a constraint, and can't help thinking how nice it'd be if Pg would report the contents of the row violating the constraint, or at least the values that were tested by the constraint check. It's really,

[GENERAL] Log-Info Replication

2011-10-01 Thread Andreas
Hello. I tried to use replication with version 9.1. But i get always the Error FATAL: falsche Pr?fsumme in Kontrolldatei I think, it would be good get a hint on a server with 'hot_standby = on' "if you try to replicate: it is not possible to replicate between 32 and 64bit" thanks andreas --

Re: [GENERAL] Log message " last_statrequest ... is later than collector's time" - what does it mean?

2011-09-16 Thread Tom Lane
Stephan Vollmer writes: >>> 2011-09-16 13:48:54 CEST: LOG:  last_statrequest 2011-09-16 >>> 13:48:55.890743+02 is later than collector's time 2011-09-16 >>> 13:48:54.614476+02 > is there a way to filter these log messages other than setting > "log_min_messages" to "fatal" which is not really desi

Re: [GENERAL] Log message " last_statrequest ... is later than collector's time" - what does it mean?

2011-09-16 Thread Stephan Vollmer
On Fri, Sep 16, 2011 at 19:16, Tom Lane wrote: > > Stephan Vollmer writes: > > I upgraded our test database from PostgreSQL 8.4.8 to 9.0.4 via pg_dumpall. > > The database seems to work fine, but now the logfile of the new database is > > flooded with log messages like these: > > > 2011-09-16 13:

Re: [GENERAL] Log duration and statement for slow queries + limiting the number of log files generated

2011-09-16 Thread Alec Swan
Thanks, I got duration logging to work the way I wanted. I will look into logrotate next. On Fri, Sep 16, 2011 at 11:22 AM, Thom Brown wrote: > On 16 September 2011 18:16, Alec Swan wrote: >> Hello, >> >> I am trying to get postgres 8.4.4 to log the duration and statement of >> queries that take

Re: [GENERAL] Log duration and statement for slow queries + limiting the number of log files generated

2011-09-16 Thread Thom Brown
On 16 September 2011 18:16, Alec Swan wrote: > Hello, > > I am trying to get postgres 8.4.4 to log the duration and statement of > queries that take longer than 200 ms. I played with the log settings > in postgresql.conf but I still see logs of durations of very fast > statements. Here is my curre

Re: [GENERAL] Log duration and statement for slow queries + limiting the number of log files generated

2011-09-16 Thread Guillaume Lelarge
On Fri, 2011-09-16 at 11:16 -0600, Alec Swan wrote: > Hello, > > I am trying to get postgres 8.4.4 to log the duration and statement of > queries that take longer than 200 ms. I played with the log settings > in postgresql.conf but I still see logs of durations of very fast > statements. Here is m

Re: [GENERAL] Log message " last_statrequest ... is later than collector's time" - what does it mean?

2011-09-16 Thread Tom Lane
Stephan Vollmer writes: > I upgraded our test database from PostgreSQL 8.4.8 to 9.0.4 via pg_dumpall. > The database seems to work fine, but now the logfile of the new database is > flooded with log messages like these: > 2011-09-16 13:48:54 CEST: LOG: last_statrequest 2011-09-16 > 13:48:55.8907

[GENERAL] Log duration and statement for slow queries + limiting the number of log files generated

2011-09-16 Thread Alec Swan
Hello, I am trying to get postgres 8.4.4 to log the duration and statement of queries that take longer than 200 ms. I played with the log settings in postgresql.conf but I still see logs of durations of very fast statements. Here is my current configuration: log_min_duration_statement = 200 log_d

Re: [GENERAL] Log Apply Delay

2011-09-16 Thread Merlin Moncure
On Fri, Sep 16, 2011 at 10:53 AM, Thom Brown wrote: > On 16 September 2011 16:41, Ian Harding wrote: >> On Fri, Sep 16, 2011 at 8:35 AM, hubert depesz lubaczewski >> wrote: >>> On Fri, Sep 16, 2011 at 08:02:31AM -0700, Ian Harding wrote: Oracle has a configuration option for its version of

Re: [GENERAL] Log Apply Delay

2011-09-16 Thread Thom Brown
On 16 September 2011 16:41, Ian Harding wrote: > On Fri, Sep 16, 2011 at 8:35 AM, hubert depesz lubaczewski > wrote: >> On Fri, Sep 16, 2011 at 08:02:31AM -0700, Ian Harding wrote: >>> Oracle has a configuration option for its version of hot standby >>> (DataGuard) that lets you specify a time ba

Re: [GENERAL] Log Apply Delay

2011-09-16 Thread Ian Harding
On Fri, Sep 16, 2011 at 8:35 AM, hubert depesz lubaczewski wrote: > On Fri, Sep 16, 2011 at 08:02:31AM -0700, Ian Harding wrote: >> Oracle has a configuration option for its version of hot standby >> (DataGuard) that lets you specify a time based delay in applying logs. >>  They get transferred ri

Re: [GENERAL] Log Apply Delay

2011-09-16 Thread hubert depesz lubaczewski
On Fri, Sep 16, 2011 at 08:02:31AM -0700, Ian Harding wrote: > Oracle has a configuration option for its version of hot standby > (DataGuard) that lets you specify a time based delay in applying logs. > They get transferred right away, but changes in them are only applied > as they reach a certain

Re: [GENERAL] Log Apply Delay

2011-09-16 Thread Guillaume Lelarge
On Fri, 2011-09-16 at 08:02 -0700, Ian Harding wrote: > Oracle has a configuration option for its version of hot standby > (DataGuard) that lets you specify a time based delay in applying logs. > They get transferred right away, but changes in them are only applied > as they reach a certain age.

[GENERAL] Log Apply Delay

2011-09-16 Thread Ian Harding
Oracle has a configuration option for its version of hot standby (DataGuard) that lets you specify a time based delay in applying logs. They get transferred right away, but changes in them are only applied as they reach a certain age. The idea is that if something horrible happens on the master,

[GENERAL] Log message " last_statrequest ... is later than collector's time" - what does it mean?

2011-09-16 Thread Stephan Vollmer
Hi, I upgraded our test database from PostgreSQL 8.4.8 to 9.0.4 via pg_dumpall. The database seems to work fine, but now the logfile of the new database is flooded with log messages like these: 2011-09-16 13:48:32 CEST: LOG: database system was shut down at 2011-09-16 13:48:29 CEST 2011-09-16 13

Re: [GENERAL] LOG: unexpected EOF on client connection

2010-10-11 Thread Brar Piening
On Fri, 8 Oct 2010 09:57:31 -0400, akp geek wrote: LOG: unexpected EOF on client connection Npgsql is a very potent EOF-Generator if the client app doesn't properly close (or dispose explicitly) connections. see http://fxjr.blogspot.com/2010/04/npgsql-connection-pool-explained.html Rega

Re: [GENERAL] LOG: unexpected EOF on client connection

2010-10-09 Thread Scott Marlowe
On Sat, Oct 9, 2010 at 5:45 PM, Ben Carbery wrote: > >> Or that OP has a networking issue.  Some firewalls are known for >> dropping what they think are idle connections when they aren't. >> > > I don't think so.. EOF is an explicit termination, not a timeout as would > caused by a firewall droppi

Re: [GENERAL] LOG: unexpected EOF on client connection

2010-10-09 Thread Ben Carbery
> Or that OP has a networking issue. Some firewalls are known for > dropping what they think are idle connections when they aren't. > > I don't think so.. EOF is an explicit termination, not a timeout as would caused by a firewall dropping traffic. It's more like what happens when the remote proce

Re: [GENERAL] LOG: unexpected EOF on client connection

2010-10-08 Thread Scott Marlowe
On Fri, Oct 8, 2010 at 8:11 AM, wrote: >> Hi all - >> >>               I am seeing lot of these records in the log file. Not able >> to >> find why I get this in log file. Is there a way to find out info about >> this >> ? Thanks for your help >> >> >> LOG:  unexpected EOF on client connection >>

Re: [GENERAL] LOG: unexpected EOF on client connection

2010-10-08 Thread tv
> Hi all - > > I am seeing lot of these records in the log file. Not able > to > find why I get this in log file. Is there a way to find out info about > this > ? Thanks for your help > > > LOG: unexpected EOF on client connection > LOG: unexpected EOF on client connection This mea

[GENERAL] LOG: unexpected EOF on client connection

2010-10-08 Thread akp geek
Hi all - I am seeing lot of these records in the log file. Not able to find why I get this in log file. Is there a way to find out info about this ? Thanks for your help LOG: unexpected EOF on client connection LOG: unexpected EOF on client connection LOG: unexpected EOF on clie

Re: [GENERAL] log database in which error occurs

2010-05-11 Thread Alexander Pyhalov
A. Kretschmer wrote: May 11 01:36:46 pgsql pgsql[73794]: [7-3] STATEMENT: select count(*) from forum_msg where grp_id=6709 (200 ok) ACCEPTED and is_blocked='f' Are there any ways to log database, to which invalid query was issued ? Sure: log_line_prefix = '%t ' # special v

Re: [GENERAL] log database in which error occurs

2010-05-11 Thread A. Kretschmer
In response to Alexander Pyhalov : > May 11 01:36:46 pgsql pgsql[73794]: [7-3] STATEMENT: select count(*) > from forum_msg where grp_id=6709 (200 ok) ACCEPTED and is_blocked='f' > > Are there any ways to log database, to which invalid query was issued ? Sure: log_line_prefix = '%t '

[GENERAL] log database in which error occurs

2010-05-11 Thread Alexander Pyhalov
Hello. We have postgresql 8.4 server with about 40 databases. Postgres logs error statements, but it's quite difficult to determine which users on which databases caused error. For example, this is a piece of log: May 11 01:36:46 pgsql pgsql[73794]: [7-1] ERROR: 42601: syntax error at or nea

Re: [GENERAL] Log full of: statement_timeout out of the valid range.

2010-01-25 Thread Craig Ringer
On 26/01/2010 12:31 AM, Joshua D. Drake wrote: On Mon, 2010-01-25 at 09:09 -0600, Abraham, Danny wrote: Our setup is a real valid one. Looks like it has to do more with remote connections. A remote connection trying to set statement_timeout on its own and sending an invalid value. Joshua D.

  1   2   >