[GENERAL] Logcal replication in large scale

Hallo all I am thrilled about logical replication in PostgreSQL 10. My head have started spinning about use cases. Would it be possible to use logical replication as a distribution method of data? I think about map data from national mapping authorities. The problem is to get the updates of thei

[GENERAL] Logical decoding client has the power to crash the server

>From what I understand about logical decoding, there is no limit to how many log entries will be retained by the server if nobody reads them from the logical slot. This means that a client that fails to read from the slot has the power to bring down the master database because the server's disk wi

R: [GENERAL] Insert large number of records

Dear Alban, thank you for your precious reply, first of all. >> On 19 Sep 2017, at 15:47, Job wrote: >> >> Hi guys, >> >> we need to insert from a table to another (Postgresql 9.6.1) a large amount >> of data (about 10/20 millions of rows) without locking destination table. >> Pg_bulkload is th

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

On 19/09/2017 20:09, Tom Lane wrote: chiru r writes: We are looking for User profiles in ope source PostgreSQL. For example, If a user password failed n+ times while login ,the user access has to be blocked few seconds. Please let us know, is there any plan to implement user profiles in featu

Re: [GENERAL] libpq confusion

2017-09-20 5:36 GMT+02:00 Igor Korot : > Hi, ALL, > > draft=# SELECT 1 FROM abcattbl WHERE abt_tnam = 'leagues'; > ?column? > -- > (0 rows) > > > However running it thru the PQexecParam() I am getting "PGRES_TUPLES_OK" > which means that the such record exist. > That means so this comman

[GENERAL] libpq confusion

Hi, ALL, draft=# SELECT 1 FROM abcattbl WHERE abt_tnam = 'leagues'; ?column? -- (0 rows) However running it thru the PQexecParam() I am getting "PGRES_TUPLES_OK" which means that the such record exist. How do I properly check if the record exists from libpq? -- Sent via pgsql-genera

[GENERAL] problem

Hi I setup a sync rep cluster 9.6 unfortunately I have made a bit of an issue for myself. my backup site is full on the data partition, which i believe has lead for my write site to be full on the pg_xlog partition - i believe that it is holding logs to replicate and can't any more. so now neit

Re: [GENERAL] Up to date conventional wisdom re max shared_buffer size?

On 09/19/2017 05:00 PM, Jerry Sievers wrote: [snip] The DB is 10TB total size with OLTP plus some occasional heavy batching which frequently correlates with degradation that requires intervention. Unrelated server problem forced us to relocate from a Debian/Wheezy 3.x kernel 1T 144 CPU to the ev

Re: [GENERAL] pgcrypto encrypt

On 2017-09-19 15:42, Jeff Janes wrote: > On Tue, Sep 19, 2017 at 12:20 PM, Bruce Momjian > wrote: > > On Wed, Sep  6, 2017 at 04:19:52PM -0400, Stephen Cook wrote: > > Hello! > > > > Is there a way to decrypt data encrypted with the pgcrypto "encrypt" >

[GENERAL] random row from a subset

I'm attempting to get a random, based on a range that spans 1 to the maximum number of rows that for a subset. I run the query in Oracle sucessfully and get a different number each time and only a single number, which is what I am expecting, but when I run the same query, albeit the random functio

Re: [GENERAL] Up to date conventional wisdom re max shared_buffer size?

Hi, On 2017-09-19 17:00:05 -0500, Jerry Sievers wrote: > Briefly, just curious if legacy max values for shared_buffers have > scaled up since 8G was like 25% of RAM? It's very workload dependent. I've successfully used PG with roughly 1TB of shared buffers, where that performed better than lower

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

On 9/19/2017 3:32 PM, chiru r wrote: How those application accounts get recognized in database? Let say  App_user1 authenticated through application ,after that how the App_user1 get access to DB? can you please provide more information ,how the app users are accessing database ? the user

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

Hi John, How those application accounts get recognized in database? Let say App_user1 authenticated through application ,after that how the App_user1 get access to DB? can you please provide more information ,how the app users are accessing database ? Thanks, Chiranjeevi On Tue, Sep 19, 2017

[GENERAL] Up to date conventional wisdom re max shared_buffer size?

Briefly, just curious if legacy max values for shared_buffers have scaled up since 8G was like 25% of RAM? Pg 9.3 on monster 2T/192 CPU Xenial thrashing Upgrade pending but we recently started having $interesting performance issues at times looking like I/O slowness and other times apparently cau

Re: [GENERAL] pgcrypto encrypt

On Tue, Sep 19, 2017 at 12:42:40PM -0700, Jeff Janes wrote: > On Tue, Sep 19, 2017 at 12:20 PM, Bruce Momjian wrote: > > On Wed, Sep  6, 2017 at 04:19:52PM -0400, Stephen Cook wrote: > > Hello! > > > > Is there a way to decrypt data encrypted with the pgcrypto "encrypt" > > fu

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

On 9/19/2017 12:33 PM, chiru r wrote: Yes, LDAP will do. However we need to sync the user accounts and  groups between AD and PG servers.and then AD profiles will apply to PG user accounts for authentication. if you're using LDAP from the AD servers to authenticate, whats to sync? my datab

Re: [GENERAL] pgcrypto encrypt

On Tue, Sep 19, 2017 at 12:20 PM, Bruce Momjian wrote: > On Wed, Sep 6, 2017 at 04:19:52PM -0400, Stephen Cook wrote: > > Hello! > > > > Is there a way to decrypt data encrypted with the pgcrypto "encrypt" > > function, outside the database? Assuming that I know the key etc... > > Yes, I think s

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

Yes, LDAP will do. However we need to sync the user accounts and groups between AD and PG servers.and then AD profiles will apply to PG user accounts for authentication. It is good if we have user profiles in core PostgreSQL database system. So it will add more security. Thanks, Chiranjeevi On

Re: [GENERAL] SAP Application deployment on PostgreSQL

On Fri, Sep 8, 2017 at 12:44:18PM -0700, John R Pierce wrote: > On 9/8/2017 12:34 PM, chiru r wrote: > > > >We have multiple SAP applications running on Oracle as backend and looking > >for an opportunity to migrate from Oracle to PostgreSQL. Has anyone ever > >deployed SAP on PostgreSQL community

Re: [GENERAL] pgcrypto encrypt

On Wed, Sep 6, 2017 at 04:19:52PM -0400, Stephen Cook wrote: > Hello! > > Is there a way to decrypt data encrypted with the pgcrypto "encrypt" > function, outside the database? Assuming that I know the key etc... Yes, I think so. pgcrypto uses openssl and gpg internally, so using those tools sh

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

On Tue, Sep 19, 2017 at 01:28:11PM -0400, Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > chiru r writes: > > > We are looking for User profiles in ope source PostgreSQL. > > > For example, If a user password failed n+ times while login ,the user > > > access has to b

Re: [GENERAL] [HACKERS] pgjdbc logical replication client throwing exception

Hi Andres, I also checked server log. Nothing unusual is recorded there. Do you have any other suggestion. Thank you. Best regards, Dipesh Dangol On Fri, Sep 15, 2017 at 11:32 PM, Dipesh Dangol wrote: > Hi Vladimir, > Ya, initially I was trying with withStatusInterval(20, TimeUnit.SECONDS), >

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

On Tue, Sep 19, 2017 at 1:28 PM, Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > chiru r writes: > > > We are looking for User profiles in ope source PostgreSQL. > > > For example, If a user password failed n+ times while login ,the user > > > access has to be blocke

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > chiru r writes: > > We are looking for User profiles in ope source PostgreSQL. > > For example, If a user password failed n+ times while login ,the user > > access has to be blocked few seconds. > > Please let us know, is there any plan to implement

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

chiru r writes: > We are looking for User profiles in ope source PostgreSQL. > For example, If a user password failed n+ times while login ,the user > access has to be blocked few seconds. > Please let us know, is there any plan to implement user profiles in feature > releases?. Not particularl

[GENERAL] USER Profiles for PostgreSQL

Hi All, Good Morning. We are looking for User profiles in ope source PostgreSQL. For example, If a user password failed n+ times while login ,the user access has to be blocked few seconds. Please let us know, is there any plan to implement user profiles in feature releases?. Thanks, Chiranj

Re: [GENERAL] Insert large number of records

> On 19 Sep 2017, at 15:47, Job wrote: > > Hi guys, > > we need to insert from a table to another (Postgresql 9.6.1) a large amount > of data (about 10/20 millions of rows) without locking destination table. > Pg_bulkload is the fastest way but it locks the table. > > Are there other ways? >

Re: [GENERAL] JDBC: logical replication and LSN feedback

Should we read "In the event that replication has been restarted, it's will start from last successfully processed LSN that was sent via feedback to database." that this last succesfully event will be included (again) after a restart of the replication, or that the next event starting from the this

Re: [GENERAL] JDBC: logical replication and LSN feedback

On 19/09/2017 16:37, Yason TR wrote: Hi all, I am developing an application which connects to a logical replication slot, to consume the WAL events. These WAL events are then forwarded to a MQ broker. The heart of the code can be seen as: while (true) { Connection connection = null;

[GENERAL] Insert large number of records

Hi guys, we need to insert from a table to another (Postgresql 9.6.1) a large amount of data (about 10/20 millions of rows) without locking destination table. Pg_bulkload is the fastest way but it locks the table. Are there other ways? Classic "COPY" from? Thank you! F -- Sent via pgsql-gene

[GENERAL] JDBC: logical replication and LSN feedback

Hi all, I am developing an application which connects to a logical replication slot, to consume the WAL events. These WAL events are then forwarded to a MQ broker. The heart of the code can be seen as: while (true) { Connection connection = null; PGReplicationStream stream = null;

[GENERAL]

Hi all, I am developing an application which connects to a logical replication slot, to consume the WAL events. These WAL events are then forwarded to a MQ broker. The heart of the code can be seen as: while (true) { Connection connection = null; PGRepli

Re: [GENERAL] Writing on replicas?

Ivan Voras wrote: > The reporting code is *almost* read-only, with the major exception being > that it creates temp tables for intermediate results. The main tables are > not written to, just the temp tables. > > Some time ago when I've asked what to do in this situation, the answer was > that ma

[GENERAL] Writing on replicas?

Hello, I have a possibly unusual case, I've asked about it before on this list. There is a bunch of reporting being done regularly on some large databases, which interfere with daily operations performance-wise. So one option is to have hot async replication in place to a secondary server where t

Re: [GENERAL] Postgres 9.6 Logical and Fisical replication

But what is your problem or error ? github's project doesn't support docker images that enable and configure logical replication. 2017-09-19 10:33 GMT+02:00 Murtuza Zabuawala < murtuza.zabuaw...@enterprisedb.com>: > ++ Forwarding query to appropriate mail group. > > > On Tue, Sep 19, 2017 at 3:00

[GENERAL] Postgres 9.6 Logical and Fisical replication

++ Forwarding query to appropriate mail group. On Tue, Sep 19, 2017 at 3:00 AM, guedim wrote: > Hi guys > > I am working with Postgres9.6 with a Master/Slave cluster replication using > Streaming replication. > I would like to add a new Slave server database but this database with > logical rep

Re: [GENERAL] Multicolumn Index on OR conditions

On 2017-09-19 10:08, Job wrote: Hi, within a query with two or more conditions with "OR", example: "where a = 2 or b < 3" could be useful to speed up the query a multi-column index (a,b) even though the two conditions are in "OR" and not in "AND"? Thank you! F Having any kind of index usual

[GENERAL] Multicolumn Index on OR conditions

Hi, within a query with two or more conditions with "OR", example: "where a = 2 or b < 3" could be useful to speed up the query a multi-column index (a,b) even though the two conditions are in "OR" and not in "AND"? Thank you! F -- Sent via pgsql-general mailing list (pgsql-general@postgre