R: [GENERAL] Insert large number of records

2017-09-19 Thread Job
Dear Alban, thank you for your precious reply, first of all. >> On 19 Sep 2017, at 15:47, Job wrote: >> >> Hi guys, >> >> we need to insert from a table to another (Postgresql 9.6.1) a large amount >> of data (about 10/20 millions of rows) without locking destination

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread Achilleas Mantzios
On 19/09/2017 20:09, Tom Lane wrote: chiru r writes: We are looking for User profiles in ope source PostgreSQL. For example, If a user password failed n+ times while login ,the user access has to be blocked few seconds. Please let us know, is there any plan to implement

Re: [GENERAL] libpq confusion

2017-09-19 Thread Pavel Stehule
2017-09-20 5:36 GMT+02:00 Igor Korot : > Hi, ALL, > > draft=# SELECT 1 FROM abcattbl WHERE abt_tnam = 'leagues'; > ?column? > -- > (0 rows) > > > However running it thru the PQexecParam() I am getting "PGRES_TUPLES_OK" > which means that the such record exist. > That

[GENERAL] libpq confusion

2017-09-19 Thread Igor Korot
Hi, ALL, draft=# SELECT 1 FROM abcattbl WHERE abt_tnam = 'leagues'; ?column? -- (0 rows) However running it thru the PQexecParam() I am getting "PGRES_TUPLES_OK" which means that the such record exist. How do I properly check if the record exists from libpq? -- Sent via

[GENERAL] problem

2017-09-19 Thread Alex Samad
Hi I setup a sync rep cluster 9.6 unfortunately I have made a bit of an issue for myself. my backup site is full on the data partition, which i believe has lead for my write site to be full on the pg_xlog partition - i believe that it is holding logs to replicate and can't any more. so now

Re: [GENERAL] Up to date conventional wisdom re max shared_buffer size?

2017-09-19 Thread Ron Johnson
On 09/19/2017 05:00 PM, Jerry Sievers wrote: [snip] The DB is 10TB total size with OLTP plus some occasional heavy batching which frequently correlates with degradation that requires intervention. Unrelated server problem forced us to relocate from a Debian/Wheezy 3.x kernel 1T 144 CPU to the

Re: [GENERAL] pgcrypto encrypt

2017-09-19 Thread Stephen Cook
On 2017-09-19 15:42, Jeff Janes wrote: > On Tue, Sep 19, 2017 at 12:20 PM, Bruce Momjian > wrote: > > On Wed, Sep  6, 2017 at 04:19:52PM -0400, Stephen Cook wrote: > > Hello! > > > > Is there a way to decrypt data encrypted with the

[GENERAL] random row from a subset

2017-09-19 Thread Peter Koukoulis
I'm attempting to get a random, based on a range that spans 1 to the maximum number of rows that for a subset. I run the query in Oracle sucessfully and get a different number each time and only a single number, which is what I am expecting, but when I run the same query, albeit the random

Re: [GENERAL] Up to date conventional wisdom re max shared_buffer size?

2017-09-19 Thread Andres Freund
Hi, On 2017-09-19 17:00:05 -0500, Jerry Sievers wrote: > Briefly, just curious if legacy max values for shared_buffers have > scaled up since 8G was like 25% of RAM? It's very workload dependent. I've successfully used PG with roughly 1TB of shared buffers, where that performed better than lower

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread John R Pierce
On 9/19/2017 3:32 PM, chiru r wrote: How those application accounts get recognized in database? Let say  App_user1 authenticated through application ,after that how the App_user1 get access to DB? can you please provide more information ,how the app users are accessing database ? the

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread chiru r
Hi John, How those application accounts get recognized in database? Let say App_user1 authenticated through application ,after that how the App_user1 get access to DB? can you please provide more information ,how the app users are accessing database ? Thanks, Chiranjeevi On Tue, Sep 19,

[GENERAL] Up to date conventional wisdom re max shared_buffer size?

2017-09-19 Thread Jerry Sievers
Briefly, just curious if legacy max values for shared_buffers have scaled up since 8G was like 25% of RAM? Pg 9.3 on monster 2T/192 CPU Xenial thrashing Upgrade pending but we recently started having $interesting performance issues at times looking like I/O slowness and other times apparently

Re: [GENERAL] pgcrypto encrypt

2017-09-19 Thread Bruce Momjian
On Tue, Sep 19, 2017 at 12:42:40PM -0700, Jeff Janes wrote: > On Tue, Sep 19, 2017 at 12:20 PM, Bruce Momjian wrote: > > On Wed, Sep  6, 2017 at 04:19:52PM -0400, Stephen Cook wrote: > > Hello! > > > > Is there a way to decrypt data encrypted with the pgcrypto

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread John R Pierce
On 9/19/2017 12:33 PM, chiru r wrote: Yes, LDAP will do. However we need to sync the user accounts and  groups between AD and PG servers.and then AD profiles will apply to PG user accounts for authentication. if you're using LDAP from the AD servers to authenticate, whats to sync? my

Re: [GENERAL] pgcrypto encrypt

2017-09-19 Thread Jeff Janes
On Tue, Sep 19, 2017 at 12:20 PM, Bruce Momjian wrote: > On Wed, Sep 6, 2017 at 04:19:52PM -0400, Stephen Cook wrote: > > Hello! > > > > Is there a way to decrypt data encrypted with the pgcrypto "encrypt" > > function, outside the database? Assuming that I know the key etc...

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread chiru r
Yes, LDAP will do. However we need to sync the user accounts and groups between AD and PG servers.and then AD profiles will apply to PG user accounts for authentication. It is good if we have user profiles in core PostgreSQL database system. So it will add more security. Thanks, Chiranjeevi On

Re: [GENERAL] SAP Application deployment on PostgreSQL

2017-09-19 Thread Bruce Momjian
On Fri, Sep 8, 2017 at 12:44:18PM -0700, John R Pierce wrote: > On 9/8/2017 12:34 PM, chiru r wrote: > > > >We have multiple SAP applications running on Oracle as backend and looking > >for an opportunity to migrate from Oracle to PostgreSQL. Has anyone ever > >deployed SAP on PostgreSQL

Re: [GENERAL] pgcrypto encrypt

2017-09-19 Thread Bruce Momjian
On Wed, Sep 6, 2017 at 04:19:52PM -0400, Stephen Cook wrote: > Hello! > > Is there a way to decrypt data encrypted with the pgcrypto "encrypt" > function, outside the database? Assuming that I know the key etc... Yes, I think so. pgcrypto uses openssl and gpg internally, so using those tools

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread Bruce Momjian
On Tue, Sep 19, 2017 at 01:28:11PM -0400, Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > chiru r writes: > > > We are looking for User profiles in ope source PostgreSQL. > > > For example, If a user password failed n+ times while login ,the user >

Re: [GENERAL] [HACKERS] pgjdbc logical replication client throwing exception

2017-09-19 Thread Dipesh Dangol
Hi Andres, I also checked server log. Nothing unusual is recorded there. Do you have any other suggestion. Thank you. Best regards, Dipesh Dangol On Fri, Sep 15, 2017 at 11:32 PM, Dipesh Dangol wrote: > Hi Vladimir, > Ya, initially I was trying with

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread Melvin Davidson
On Tue, Sep 19, 2017 at 1:28 PM, Stephen Frost wrote: > Tom, > > * Tom Lane (t...@sss.pgh.pa.us) wrote: > > chiru r writes: > > > We are looking for User profiles in ope source PostgreSQL. > > > For example, If a user password failed n+ times while login

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > chiru r writes: > > We are looking for User profiles in ope source PostgreSQL. > > For example, If a user password failed n+ times while login ,the user > > access has to be blocked few seconds. > > Please let us know, is there

Re: [GENERAL] [HACKERS] USER Profiles for PostgreSQL

2017-09-19 Thread Tom Lane
chiru r writes: > We are looking for User profiles in ope source PostgreSQL. > For example, If a user password failed n+ times while login ,the user > access has to be blocked few seconds. > Please let us know, is there any plan to implement user profiles in feature >

[GENERAL] USER Profiles for PostgreSQL

2017-09-19 Thread chiru r
Hi All, Good Morning. We are looking for User profiles in ope source PostgreSQL. For example, If a user password failed n+ times while login ,the user access has to be blocked few seconds. Please let us know, is there any plan to implement user profiles in feature releases?. Thanks,

Re: [GENERAL] Insert large number of records

2017-09-19 Thread Alban Hertroys
> On 19 Sep 2017, at 15:47, Job wrote: > > Hi guys, > > we need to insert from a table to another (Postgresql 9.6.1) a large amount > of data (about 10/20 millions of rows) without locking destination table. > Pg_bulkload is the fastest way but it locks the table. >

Re: [GENERAL] JDBC: logical replication and LSN feedback

2017-09-19 Thread Yason TR
Should we read "In the event that replication has been restarted, it's will start from last successfully processed LSN that was sent via feedback to database." that this last succesfully event will be included (again) after a restart of the replication, or that the next event starting from the

Re: [GENERAL] JDBC: logical replication and LSN feedback

2017-09-19 Thread Achilleas Mantzios
On 19/09/2017 16:37, Yason TR wrote: Hi all, I am developing an application which connects to a logical replication slot, to consume the WAL events. These WAL events are then forwarded to a MQ broker. The heart of the code can be seen as: while (true) { Connection connection = null;

[GENERAL] Insert large number of records

2017-09-19 Thread Job
Hi guys, we need to insert from a table to another (Postgresql 9.6.1) a large amount of data (about 10/20 millions of rows) without locking destination table. Pg_bulkload is the fastest way but it locks the table. Are there other ways? Classic "COPY" from? Thank you! F -- Sent via

[GENERAL] JDBC: logical replication and LSN feedback

2017-09-19 Thread Yason TR
Hi all, I am developing an application which connects to a logical replication slot, to consume the WAL events. These WAL events are then forwarded to a MQ broker. The heart of the code can be seen as: while (true) { Connection connection = null; PGReplicationStream stream = null;

[GENERAL]

2017-09-19 Thread Yason TR
Hi all, I am developing an application which connects to a logical replication slot, to consume the WAL events. These WAL events are then forwarded to a MQ broker. The heart of the code can be seen as: while (true) { Connection connection = null;

Re: [GENERAL] Writing on replicas?

2017-09-19 Thread Alvaro Herrera
Ivan Voras wrote: > The reporting code is *almost* read-only, with the major exception being > that it creates temp tables for intermediate results. The main tables are > not written to, just the temp tables. > > Some time ago when I've asked what to do in this situation, the answer was > that

[GENERAL] Writing on replicas?

2017-09-19 Thread Ivan Voras
Hello, I have a possibly unusual case, I've asked about it before on this list. There is a bunch of reporting being done regularly on some large databases, which interfere with daily operations performance-wise. So one option is to have hot async replication in place to a secondary server where

Re: [GENERAL] Postgres 9.6 Logical and Fisical replication

2017-09-19 Thread Jaime Soler
But what is your problem or error ? github's project doesn't support docker images that enable and configure logical replication. 2017-09-19 10:33 GMT+02:00 Murtuza Zabuawala < murtuza.zabuaw...@enterprisedb.com>: > ++ Forwarding query to appropriate mail group. > > > On Tue, Sep 19, 2017 at

[GENERAL] Postgres 9.6 Logical and Fisical replication

2017-09-19 Thread Murtuza Zabuawala
++ Forwarding query to appropriate mail group. On Tue, Sep 19, 2017 at 3:00 AM, guedim wrote: > Hi guys > > I am working with Postgres9.6 with a Master/Slave cluster replication using > Streaming replication. > I would like to add a new Slave server database but this database

Re: [GENERAL] Multicolumn Index on OR conditions

2017-09-19 Thread vinny
On 2017-09-19 10:08, Job wrote: Hi, within a query with two or more conditions with "OR", example: "where a = 2 or b < 3" could be useful to speed up the query a multi-column index (a,b) even though the two conditions are in "OR" and not in "AND"? Thank you! F Having any kind of index

[GENERAL] Multicolumn Index on OR conditions

2017-09-19 Thread Job
Hi, within a query with two or more conditions with "OR", example: "where a = 2 or b < 3" could be useful to speed up the query a multi-column index (a,b) even though the two conditions are in "OR" and not in "AND"? Thank you! F -- Sent via pgsql-general mailing list