Re: Reg: Help to understand the source code

2020-04-23 Thread Preethi S
Thank you Paul! This certainly helps. On Thu, Apr 23, 2020 at 12:26 PM Paul Jungwirth wrote: > On 4/23/20 8:44 AM, Preethi S wrote: > > I am fairly new to postgres and I am trying to understand how the data > > is processed during the insert from buffer to the disk. Can someone help > > me with

Re: Fw: Re: Re: Could Not Connect To Server

2020-04-23 Thread Rob Sargent
On 4/23/20 4:46 PM, Dummy Account wrote: I cd'd into log and listed the files.  There are many. How should I get the log that you want. I tried: open -a TextEdit postgresql-2020-04-22_171300.log I got: LSOpenURLsWithRole() failed for the application /Applications/TextEdit.app with error

Fw: Re: Re: Could Not Connect To Server

2020-04-23 Thread Dummy Account
  I cd'd into log and listed the files.  There are many. How should I get the log that you want.     I tried: open -a TextEdit postgresql-2020-04-22_171300.log I got: LSOpenURLsWithRole() failed for the application /Applications/TextEdit.app with error -610 for the file

Re: Ned to understand why all the idle connections

2020-04-23 Thread Si Chen
Thanks for answering my questions. Sorry I didn't mean to "top post" I thought that my other email got lost because I had sent it to lists.postgresql.org - Si Chen Open Source Strategies, Inc. Our Mission: https://www.youtube.com/watch?v=Uc7lmvnuJHY On Thu, Apr 23, 2020 at 2:31 PM David

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread Rob Sargent
On 4/23/20 4:19 PM, Dummy Account wrote: This worked. sudo su - postgres I was now able to log into data and ls Inside data is "log". How do I open log? and got: LSOpenURLsWithRole() failed with erro -610 I then tried: log show and I got: log: Could not be open local log store: The log archive

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread Dummy Account
This worked. sudo su - postgres I was now able to log into data and ls Inside data is "log".   How do I open log? and got: LSOpenURLsWithRole() failed with erro -610   I then tried: log show and I got: log: Could not be open local log store: The log archive format is corrupt and

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread David G. Johnston
On Thu, Apr 23, 2020 at 3:03 PM Dummy Account wrote: > > I cannot login into a postgres role via command line. > > Issue #1. > I have never done it before via command line, so maybe I am doing it > incorrectly. > I have been attempting to go: > > su - postgres > > The "sudo" in Rob's answer

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread Adrian Klaver
On 4/23/20 3:03 PM, Dummy Account wrote: I cannot login into a postgres role via command line. Issue #1. I have never done it before via command line, so maybe I am doing it incorrectly. I have been attempting to go: su - postgres Then it prompts me for the password. Unfortunately, none of the

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread Dummy Account
  I cannot login into a postgres role via command line.     Issue #1. I have never done it before via command line, so maybe I am doing it incorrectly. I have been attempting to go:    su - postgres   Then it prompts me for the password. Unfortunately, none of the passwords that I think

Re: walreceiver termination

2020-04-23 Thread Justin King
On Thu, Apr 23, 2020 at 3:06 PM Tom Lane wrote: > > Justin King writes: > > I assume it would be related to the following: > > LOG: incorrect resource manager data checksum in record at 2D6/C259AB90 > > since the walreceiver terminates just after this - but I'm unclear > > what precisely this

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread Rob Sargent
On 4/23/20 3:26 PM, Adrian Klaver wrote: On 4/23/20 1:37 PM, Dummy Account wrote:  From the Finder, I think that is the same a File Explorer on Widows.   From Finder, it's locked; I cannot access it.  Could I use the command line and change ownership or access privledges?  And then get

RE: how to slow down parts of Pg

2020-04-23 Thread Kevin Brannen
>From: Ron > >What you need is async replication instead of synchronous replication. The only way I can think of to do that in our present situation would be to buy DRBD-Proxy, which becomes a single-point-of-failure and goes against the idea of HA (it seems like a good product for disaster

Re: Ned to understand why all the idle connections

2020-04-23 Thread David G. Johnston
Please don't top-post; and this is a fairly rude hijack posting given that you already have a thread going, from today no less, where you've basically asked this very same question. On Thu, Apr 23, 2020 at 2:18 PM Si Chen wrote: > Hello David & David, > > I have a similar problem -- a lot of

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread Adrian Klaver
On 4/23/20 1:37 PM, Dummy Account wrote: From the Finder, I think that is the same a File Explorer on Widows.  From Finder, it's locked; I cannot access it.  Could I use the command line and change ownership or access privledges?  And then get access? Why not from the command line use a

RE: how to slow down parts of Pg

2020-04-23 Thread Kevin Brannen
>From: Peter J. Holzer >On 2020-04-21 21:16:57 +, Kevin Brannen wrote: >> From: Michael Loftis >> > drbdsetup allows you to control the sync rates. >> >> I was hoping not to have to do that, but the more I think about this >> I'm realizing that it won't hurt because the network cap is >>

Re: Ned to understand why all the idle connections

2020-04-23 Thread Si Chen
Hello David & David, I have a similar problem -- a lot of idle transactions. I'm using the PostgreSQL JDBC driver. The connections look like this: pid | wait_event | state_change | backend_start | xact_start | query_start | ?column? | query

Re: Question on banner display after PG initial connection

2020-04-23 Thread David G. Johnston
On Thu, Apr 23, 2020 at 2:07 PM Lu, Dan wrote: > Hello PostgreSQL Admin, > > > > I am fairly new to PostgreSQL. I am a curious question regarding the > banner message displayed after connecting to version 12.1 of PostgreSQL. > > > > Is there a way to get rid of this line “*SSL connection

Question on banner display after PG initial connection

2020-04-23 Thread Lu, Dan
Hello PostgreSQL Admin, I am fairly new to PostgreSQL. I am a curious question regarding the banner message displayed after connecting to version 12.1 of PostgreSQL. Is there a way to get rid of this line "SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, bits: 256,

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread Dummy Account
  From the Finder, I think that is the same a File Explorer on Widows.  From Finder, it's locked; I cannot access it.  Could I use the command line and change ownership or access privledges?  And then get access?   Thanks   Sent: Thursday, April 23, 2020 at 3:27 PM From: "Adrian Klaver" To: 

Re: Fw: Re: Could Not Connect To Server

2020-04-23 Thread Adrian Klaver
On 4/23/20 1:24 PM, Dummy Account wrote: Hey Everyone, I did find loggin.h and insallation_summary.log.  Neither of which look to include the info you may want. Please advise as to what log file you want to see. Did you look under: /Library/PostgreSQL/12/data directory for sub-directory?:

Fw: Re: Could Not Connect To Server

2020-04-23 Thread Dummy Account
  Hey Everyone,   I did find loggin.h and insallation_summary.log.  Neither of which look to include the info you may want.   Please advise as to what log file you want to see.   Thanks   Sent: Thursday, April 23, 2020 at 2:32 PM From: "Dummy Account" To: "Adrian Klaver" Cc: "David G.

Re: walreceiver termination

2020-04-23 Thread Tom Lane
Justin King writes: > I assume it would be related to the following: > LOG: incorrect resource manager data checksum in record at 2D6/C259AB90 > since the walreceiver terminates just after this - but I'm unclear > what precisely this means. What it indicates is corrupt data in the WAL stream.

Re: walreceiver termination

2020-04-23 Thread Justin King
I assume it would be related to the following: LOG: incorrect resource manager data checksum in record at 2D6/C259AB90 since the walreceiver terminates just after this - but I'm unclear what precisely this means. Without digging into the code, I would guess that it's unable to verify the

Re: walreceiver termination

2020-04-23 Thread Justin King
On Thu, Apr 23, 2020 at 12:47 PM Tom Lane wrote: > > Justin King writes: > > We've seen unexpected termination of the WAL receiver process. This > > stops streaming replication, but the replica stays available -- > > restarting the server resumes streaming replication where it left off. > >

Re: Could Not Connect To Server

2020-04-23 Thread Adrian Klaver
On 4/23/20 12:36 PM, Rob Sargent wrote: On 4/23/20 1:32 PM, Dummy Account wrote: Hey David, Can you tell me the exact name of the log file?  Then I can search for it. Thanks Apparently in your case it is called "log".  I don't have version 12, To be clear 'log' is the name of the

Re: Could Not Connect To Server

2020-04-23 Thread Rob Sargent
On 4/23/20 1:32 PM, Dummy Account wrote: Hey David, Can you tell me the exact name of the log file?  Then I can search for it. Thanks Apparently in your case it is called "log".  I don't have version 12, but  version 9's start up name "pg_log" and that is the name of the file in the data

Re: Could Not Connect To Server

2020-04-23 Thread Dummy Account
  Hey David,   Can you tell me the exact name of the log file?  Then I can search for it.   Thanks   Sent: Thursday, April 23, 2020 at 2:25 PM From: "Adrian Klaver" To: "Dummy Account" , "David G. Johnston" Cc: "pgsql-general" Subject: Re: Could Not Connect To Server On 4/23/20 12:12

Re: Could Not Connect To Server

2020-04-23 Thread Adrian Klaver
On 4/23/20 12:12 PM, Dummy Account wrote: Hi David, When I backed-up, I don't know if the server was offline?  I can say that I was not running pgAdmin.  For instance, I backed up the Operating System and all of its applications.  If I go run other application, including other servers, they

Re: Could Not Connect To Server

2020-04-23 Thread Rob Sargent
On 4/23/20 1:12 PM, Dummy Account wrote: Hi David, When I backed-up, I don't know if the server was offline?  I can say that I was not running pgAdmin.  For instance, I backed up the Operating System and all of its applications.  If I go run other application, including other servers, they

Re: Could Not Connect To Server

2020-04-23 Thread Dummy Account
  Hi David,   When I backed-up, I don't know if the server was offline?  I can say that I was not running pgAdmin.  For instance, I backed up the Operating System and all of its applications.  If I go run other application, including other servers, they work.  As a matter of fact, if I boot

Re: Ned to understand why all the idle connections

2020-04-23 Thread David G. Johnston
On Thu, Apr 23, 2020 at 11:52 AM David Gauthier wrote: > Thanks! > And an example of connection pooling is pgBouncer ? > >> >> It does describe itself as being a "Lightweight connection pooler for PostgreSQL" ... https://www.pgbouncer.org/ David J.

Re: Could Not Connect To Server

2020-04-23 Thread David G. Johnston
On Thu, Apr 23, 2020 at 11:48 AM Dummy Account wrote: > But here they are: > > waiting for server to start2020-04-22 15:57:51.766 CDT [5255] LOG: > starting PostgreSQL 12.2 on x86_64-apple-darwin, compiled by Apple LLVM > version 6.0 (clang-600.0.54) (based on LLVM 3.5svn), 64-bit >

Re: Ned to understand why all the idle connections

2020-04-23 Thread David Gauthier
Thanks! And an example of connection pooling is pgBouncer ? On Thu, Apr 23, 2020 at 2:41 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Thursday, April 23, 2020, David Gauthier > wrote: > >> Hi: >> >> psql (9.6.7, server 11.3) on linux >> >> I have what appear to be a log of

Re: Could Not Connect To Server

2020-04-23 Thread Dummy Account
  The logs are listed on the stack overlow link. But here they are:   waiting for server to start2020-04-22 15:57:51.766 CDT [5255] LOG:  starting PostgreSQL 12.2 on x86_64-apple-darwin, compiled by Apple LLVM version 6.0 (clang-600.0.54) (based on LLVM 3.5svn), 64-bit 2020-04-22

Re: Ned to understand why all the idle connections

2020-04-23 Thread David G. Johnston
On Thursday, April 23, 2020, David Gauthier wrote: > Hi: > > psql (9.6.7, server 11.3) on linux > > I have what appear to be a log of idle connections to my DB. Query of > pg_stat_activity indicates well over half (127/206) are like this... > > > dvdb=# select

Ned to understand why all the idle connections

2020-04-23 Thread David Gauthier
Hi: psql (9.6.7, server 11.3) on linux I have what appear to be a log of idle connections to my DB. Query of pg_stat_activity indicates well over half (127/206) are like this... dvdb=# select state_change,wait_event_type,wait_event,state,backend_type from pg_stat_activity where query = '';

Re: Could Not Connect To Server

2020-04-23 Thread David G. Johnston
On Thu, Apr 23, 2020 at 11:18 AM Dummy Account wrote: > > Trying to start the server, I navigate too: /Library/PostgreSQL/12/bin > > from bin, I ran: sudo -u postgres ./pg_ctl start -D > /Library/PostgreSQL/12/data > > pg_ctl: could not start server > > > What do the logs say... When I ran

Could Not Connect To Server

2020-04-23 Thread Dummy Account
Hello,   I am running the following: MacOS High Sierra 10.13.6 PostgreSQL 12.2 pgAdmin 4   To see my qustion on Stack Overflow with picture: https://stackoverflow.com/questions/61353989/pgadmin-could-not-connect-to-server-connection-refused   I am trying to view databases with pgAdmin.

Re: walreceiver termination

2020-04-23 Thread Tom Lane
Justin King writes: > We've seen unexpected termination of the WAL receiver process. This > stops streaming replication, but the replica stays available -- > restarting the server resumes streaming replication where it left off. > We've seen this across nearly every recent version of PG, (9.4,

Re: Reg: Help to understand the source code

2020-04-23 Thread Paul Jungwirth
On 4/23/20 8:44 AM, Preethi S wrote: I am fairly new to postgres and I am trying to understand how the data is processed during the insert from buffer to the disk. Can someone help me with that? Also, I would like to see source code workflow. Can someone help me with finding the source code

Re: relationship of backend_start, query_start, state_change

2020-04-23 Thread David G. Johnston
On Thu, Apr 23, 2020 at 9:58 AM Olivier Gautherot wrote: > Hi David, > > On Thu, Apr 23, 2020 at 6:55 PM David G. Johnston < > david.g.johns...@gmail.com> wrote: > >> On Thu, Apr 23, 2020 at 9:37 AM Si Chen >> wrote: >> >>> Hello, >>> >>> I'm looking at my pg_stat_activity and trying to figure

walreceiver termination

2020-04-23 Thread Justin King
We've seen unexpected termination of the WAL receiver process. This stops streaming replication, but the replica stays available -- restarting the server resumes streaming replication where it left off. We've seen this across nearly every recent version of PG, (9.4, 9.5, 11.x, 12.x) -- anything

Re: relationship of backend_start, query_start, state_change

2020-04-23 Thread David G. Johnston
On Thu, Apr 23, 2020 at 9:55 AM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Thu, Apr 23, 2020 at 9:37 AM Si Chen > wrote: > >> Hello, >> >> I'm looking at my pg_stat_activity and trying to figure out what is >> causing some of these processes. I'm using this query: >> >> SELECT

Re: relationship of backend_start, query_start, state_change

2020-04-23 Thread Olivier Gautherot
Hi David, On Thu, Apr 23, 2020 at 6:55 PM David G. Johnston < david.g.johns...@gmail.com> wrote: > On Thu, Apr 23, 2020 at 9:37 AM Si Chen > wrote: > >> Hello, >> >> I'm looking at my pg_stat_activity and trying to figure out what is >> causing some of these processes. I'm using this query: >>

Re: relationship of backend_start, query_start, state_change

2020-04-23 Thread David G. Johnston
On Thu, Apr 23, 2020 at 9:37 AM Si Chen wrote: > Hello, > > I'm looking at my pg_stat_activity and trying to figure out what is > causing some of these processes. I'm using this query: > > SELECT pid, wait_event, state_change, backend_start, xact_start, > query_start, state_change -

Re: relationship of backend_start, query_start, state_change

2020-04-23 Thread Olivier Gautherot
On Thu, Apr 23, 2020 at 6:37 PM Si Chen wrote: > Hello, > > I'm looking at my pg_stat_activity and trying to figure out what is > causing some of these processes. I'm using this query: > > SELECT pid, wait_event, state_change, backend_start, xact_start, > query_start, state_change -

Re: relationship of backend_start, query_start, state_change

2020-04-23 Thread Michael Lewis
If you use a connection pooler, this would likely be expected behavior since the connection is getting reused many times. Else, some app is connected and not closing their connection between queries. At least they aren't idle in transaction though.

Re: Reg: Help to understand the source code

2020-04-23 Thread Preethi S
I am using doxygen On Thu, Apr 23, 2020 at 11:31 AM Rob Sargent wrote: > > > On 4/23/20 10:28 AM, Preethi S wrote: > > Hello Adrian, > > > > Thank you for the quick reply. This link is indeed helpful. This link > > explains how is a query processed. I am aware of how the query > > processing

relationship of backend_start, query_start, state_change

2020-04-23 Thread Si Chen
Hello, I'm looking at my pg_stat_activity and trying to figure out what is causing some of these processes. I'm using this query: SELECT pid, wait_event, state_change, backend_start, xact_start, query_start, state_change - query_start, query from pg_stat_activity where datname= 'my_database'

Re: Reg: Help to understand the source code

2020-04-23 Thread Rob Sargent
On 4/23/20 10:28 AM, Preethi S wrote: Hello Adrian, Thank you for the quick reply. This link is indeed helpful. This link explains how is a query processed. I am aware of how the query processing happens. In addition, I am looking for how the data processed, when data is

Re: Reg: Help to understand the source code

2020-04-23 Thread Preethi S
Hello Adrian, Thank you for the quick reply. This link is indeed helpful. This link explains how is a query processed. I am aware of how the query processing happens. In addition, I am looking for how the data processed, when data is inserted/modified, does the new data gets written to shared

Re: Reg: Help to understand the source code

2020-04-23 Thread Adrian Klaver
On 4/23/20 8:44 AM, Preethi S wrote: Hello, I am fairly new to postgres and I am trying to understand how the data is processed during the insert from buffer to the disk. Can someone help me with that? Also, I would like to see source code workflow. Can someone help me with finding the

Reg: Help to understand the source code

2020-04-23 Thread Preethi S
Hello, I am fairly new to postgres and I am trying to understand how the data is processed during the insert from buffer to the disk. Can someone help me with that? Also, I would like to see source code workflow. Can someone help me with finding the source code for the data insertion/modification

Re: parameter limit

2020-04-23 Thread Scott Ribe
> On Apr 23, 2020, at 8:48 AM, Adrian Klaver wrote: > > No. Ah, thanks a lot for that info. It's not that I really normally want to have bazillions of params. But we have some moderately large inserts, that are showing a sudden non-linear dropoff when scaling up, and a suspicion that the

[no subject]

2020-04-23 Thread servetturan
subscribeend  

Re: parameter limit

2020-04-23 Thread David G. Johnston
On Thursday, April 23, 2020, Adrian Klaver wrote: > On 4/23/20 7:33 AM, Scott Ribe wrote: > >> In libpq, PQexecParams has nParams as type int. So on any reasonable >> platform, that's at least 4 bytes. My question then is: when I see >> documented limits of 65535 params in various drivers and

Re: parameter limit

2020-04-23 Thread Adrian Klaver
On 4/23/20 7:33 AM, Scott Ribe wrote: In libpq, PQexecParams has nParams as type int. So on any reasonable platform, that's at least 4 bytes. My question then is: when I see documented limits of 65535 params in various drivers and libraries, that is NOT a restriction of libpq nor of the

parameter limit

2020-04-23 Thread Scott Ribe
In libpq, PQexecParams has nParams as type int. So on any reasonable platform, that's at least 4 bytes. My question then is: when I see documented limits of 65535 params in various drivers and libraries, that is NOT a restriction of libpq nor of the protocol, but rather an arbitrary limit of

Re: pg11: Query using index, but only for half the partitions

2020-04-23 Thread Michael Lewis
What do the statistics look like for an example table that the index I used vs not? Is ((instance_id)::text = 'test01'::text) rare for the tables where an index scan is happening and common for the tables where a sequential scan is chosen? How many rows in these tables generally?

Re: pg11: Query using index, but only for half the partitions

2020-04-23 Thread Andreas Kretschmer
Am 23.04.20 um 12:30 schrieb Stefan Knecht: There's no question that this is more expensive than just reading the 95 rows from the index directly and returning them not sure, you can play with enable_seqscan = off and compare the costs. What is the setting for random_page_cost ?

Re: pg11: Query using index, but only for half the partitions

2020-04-23 Thread Stefan Knecht
Thanks Andreas, But I don't think that that's what's happening. Take this example line: -> Seq Scan on snap_20200328 s_23 (cost=0.00..51.73 rows=95 width=12) (actual time=0.007..0.225 rows=95 loops=1) Filter: ((instance_id)::text = 'test01'::text)

Re: pg11: Query using index, but only for half the partitions

2020-04-23 Thread Andreas Kretschmer
Am 23.04.20 um 10:13 schrieb Stefan Knecht: Seq Scan on snap_20200225 s  (cost=0.00..1.19 rows=1 width=12) the partition is very small, so it's cheaper to scan only the table (one block) than index + table (1 + 1 block). Regards, Andreas -- 2ndQuadrant - The PostgreSQL Support Company.

pg11: Query using index, but only for half the partitions

2020-04-23 Thread Stefan Knecht
Hello all Does anyone have an explanation for this? Query uses only three columns of the table "snap" and all three are in an index. The planner seems to think that some partitions are better scanned in full. Yet for the other half of them it's using the index just fine. Can someone enlighten

Re: Can I tell libpq to connect to the primary?

2020-04-23 Thread Christian Ramseyer
On 22.04.20 21:10, Christian Ramseyer wrote: > > I see that in pgjdbc there are additional options for targetServerType = > any, primary, secondary, preferSlave and preferSecondary[2]. However > this seems to be java-specific and not implemented in libpq? Is there a > way to get this behaviour in