Re: ERROR: found xmin from before relfrozenxid

2019-01-29 Thread Mariel Cherkassky
It seems that the version of the db is 9.6.10 : psql -U db -d db -c "select version()"; Password for user db: version --- PostgreSQL 9.6.10 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.4.7 20

Re: Will higher shared_buffers improve tpcb-like benchmarks?

2019-01-29 Thread Saurabh Nanda
I did one final test of increasing the shared_buffers=32GB. It seems to be having no impact on TPS (in fact, if I look closely there is a 10-15% **negative** impact on the TPS compared to shared_buffers=2G) I can confirm that **almost** the entire DB has been cached in the shared_buffers: relname

Re: dsa_allocate() faliure

2019-01-29 Thread Thomas Munro
On Tue, Jan 29, 2019 at 10:32 PM Fabio Isabettini wrote: > we are facing a similar issue on a Production system using a Postgresql 10.6: > > org.postgresql.util.PSQLException: ERROR: EXCEPTION on getstatistics ; ID: > EXCEPTION on getstatistics_media ; ID: uidatareader. > run_query_media(2): [a1

Re: How can sort performance be so different

2019-01-29 Thread Tom Lane
Bob Jolliffe writes: > I wonder what can cause such a massive discrepancy in the sort time. Are you using the same locale (LC_COLLATE) setting on both machines? Some locales sort way slower than C locale does. That's not enough to explain a 1000X discrepancy --- I concur with the other opinions

Re: Interpreting shared_buffers setting

2019-01-29 Thread Jerry Sievers
Jerry Sievers writes: > Bob Jolliffe writes: > >> Excuse me if this is a silly question. I am trying to fiddle with >> shared_buffers setting on postgresql 10.6 on ubuntu 18.04 server. >> >> I have this at bottom of my config file: >> shared_buffers = 1GB >> >> Yet when I check the setting from

Re: How can sort performance be so different

2019-01-29 Thread Saurabh Nanda
Run https://github.com/n-st/nench and benchmark the underlying vps first. On Tue 29 Jan, 2019, 11:59 PM Bob Jolliffe The following is output from analyzing a simple query on a table of > 13436 rows on postgresql 10, ubuntu 18.04. > > explain analyze select * from chart order by name; >

Re: How can sort performance be so different

2019-01-29 Thread Pavel Stehule
út 29. 1. 2019 v 19:29 odesílatel Bob Jolliffe napsal: > The following is output from analyzing a simple query on a table of > 13436 rows on postgresql 10, ubuntu 18.04. > > explain analyze select * from chart order by name; >QUERY PLAN > > ---

Re: Interpreting shared_buffers setting

2019-01-29 Thread Jerry Sievers
Bob Jolliffe writes: > Excuse me if this is a silly question. I am trying to fiddle with > shared_buffers setting on postgresql 10.6 on ubuntu 18.04 server. > > I have this at bottom of my config file: > shared_buffers = 1GB > > Yet when I check the setting from pg_setting I see something quite

How can sort performance be so different

2019-01-29 Thread Bob Jolliffe
The following is output from analyzing a simple query on a table of 13436 rows on postgresql 10, ubuntu 18.04. explain analyze select * from chart order by name; QUERY PLAN -

Re: Will higher shared_buffers improve tpcb-like benchmarks?

2019-01-29 Thread Saurabh Nanda
> > That is likely correct, but the data will likely be stored in the OS file > cache, so reading it from there will still be pretty fast. > Right -- but increasing shared_buffers won't increase my TPS, right? Btw, I just realised that irrespective of shared_buffers, my entire DB is already in mem

Re: Will higher shared_buffers improve tpcb-like benchmarks?

2019-01-29 Thread Jeff Janes
On Tue, Jan 29, 2019 at 6:39 AM Saurabh Nanda wrote: > Hi, > > I'm going crazy trying to optimise my Postgres config for a production > setting [1] Once I realised random changes weren't getting my anywhere, I > finally purchased PostgreSQL 10 - Higher Performance [2] and understood the > impact

Re: pg_locks - what is a virtualxid locktype

2019-01-29 Thread Alvaro Herrera
On 2019-Jan-29, Shreeyansh Dba wrote: > The virtualxid lock is special. It’s a exclusive lock on the transaction’s > own virtual transaction ID that every transaction always holds. No other > transaction can ever acquire it while the transaction is running. > The purpose of this is to allow one tr

Re: Interpreting shared_buffers setting

2019-01-29 Thread Bob Jolliffe
Thank you Andrew and Thomas. All is now clear :-) On Tue, 29 Jan 2019 at 13:07, Andrew Gierth wrote: > > > "Bob" == Bob Jolliffe writes: > > Bob> Excuse me if this is a silly question. I am trying to fiddle with > Bob> shared_buffers setting on postgresql 10.6 on ubuntu 18.04 server. > >

Re: Benchmarking: How to identify bottleneck (limiting factor) and achieve "linear scalability"?

2019-01-29 Thread Jeff Janes
On Mon, Jan 28, 2019 at 12:03 AM Saurabh Nanda wrote: > All this benchmarking has led me to a philosophical question, why does PG > need shared_buffers in the first place? > PostgreSQL cannot let the OS get its hands on a dirty shared buffer until the WAL record "protecting" that buffer has been

Re: Interpreting shared_buffers setting

2019-01-29 Thread Andrew Gierth
> "Bob" == Bob Jolliffe writes: Bob> Excuse me if this is a silly question. I am trying to fiddle with Bob> shared_buffers setting on postgresql 10.6 on ubuntu 18.04 server. Bob> I have this at bottom of my config file: Bob> shared_buffers = 1GB Bob> Yet when I check the setting from p

Re: Interpreting shared_buffers setting

2019-01-29 Thread Thomas Markus
Hi, check for blocksize (8k) as factor. 8k*131072=1G regards Thomas Am 29.01.19 um 13:32 schrieb Bob Jolliffe: Excuse me if this is a silly question. I am trying to fiddle with shared_buffers setting on postgresql 10.6 on ubuntu 18.04 server. I have this at bottom of my config file: shared_

Interpreting shared_buffers setting

2019-01-29 Thread Bob Jolliffe
Excuse me if this is a silly question. I am trying to fiddle with shared_buffers setting on postgresql 10.6 on ubuntu 18.04 server. I have this at bottom of my config file: shared_buffers = 1GB Yet when I check the setting from pg_setting I see something quite different: postgres=# SELECT name,

Re: Will higher shared_buffers improve tpcb-like benchmarks?

2019-01-29 Thread Joe Mirabal
Please remove me from this list Serv. I do not use this db anymore and fills my alerts daily. On Tue, Jan 29, 2019 at 06:39 Saurabh Nanda wrote: > Hi, > > I'm going crazy trying to optimise my Postgres config for a production > setting [1] Once I realised random changes weren't getting my anyw

Re: dsa_allocate() faliure

2019-01-29 Thread Fabio Isabettini
Hello, we are facing a similar issue on a Production system using a Postgresql 10.6: org.postgresql.util.PSQLException: ERROR: EXCEPTION on getstatistics ; ID: EXCEPTION on getstatistics_media ; ID: uidatareader. run_query_media(2): [a1] REMOTE FATAL: dsa_allocate could not find 7 free pages T

Will higher shared_buffers improve tpcb-like benchmarks?

2019-01-29 Thread Saurabh Nanda
Hi, I'm going crazy trying to optimise my Postgres config for a production setting [1] Once I realised random changes weren't getting my anywhere, I finally purchased PostgreSQL 10 - Higher Performance [2] and understood the impact of shared_buffers. IIUC, shared_buffers won't have any significan

Re: pg_locks - what is a virtualxid locktype

2019-01-29 Thread Shreeyansh Dba
The virtualxid lock is special. It’s a exclusive lock on the transaction’s own virtual transaction ID that every transaction always holds. No other transaction can ever acquire it while the transaction is running. The purpose of this is to allow one transaction to wait until another transaction com

Re: Benchmarking: How to identify bottleneck (limiting factor) and achieve "linear scalability"?

2019-01-29 Thread Nicolas Charles
Le 29/01/2019 à 07:15, Saurabh Nanda a écrit : c) I tried another cloud hosting provider (E2E Networks) and just the raw performance numbers (with default configuration) are blowing Hetzner out of the water. I noticed that on E2E, the root filesystem is mounted with the following

pg_locks - what is a virtualxid locktype

2019-01-29 Thread Mariel Cherkassky
Hey, I noticed that pg_locks has an addition row for every transaction that is created with a locktype "virtualxid". Tried to search it online but I didnt find an explanation for this behavior. Does anyone can explain why it happens ?