Re: [PERFORM] Insert Concurrency

2017-04-18 Thread ROBERT PRICE
to speed on postgres partitioning for a future need. From: Scott Marlowe Sent: Tuesday, April 18, 2017 3:41 PM To: ROBERT PRICE Cc: pgsql-performance@postgresql.org Subject: Re: [PERFORM] Insert Concurrency On Mon, Apr 17, 2017 at 8:55 PM, ROBERT PRICE wrote

Re: [PERFORM] Insert Concurrency

2017-04-18 Thread David McKelvie
>> To understand recursion, one must first understand recursion. This makes no sense unless you also provide the base case. David -- Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-perfor

Re: [PERFORM] Insert Concurrency

2017-04-18 Thread Scott Marlowe
On Mon, Apr 17, 2017 at 8:55 PM, ROBERT PRICE wrote: > I come from an Oracle background and am porting an application to postgres. > App has a table that will contain 100 million rows and has to be loaded by a > process that reads messages off a SQS queue and makes web service calls to > insert re

Re: [PERFORM] Insert Concurrency

2017-04-18 Thread Claudio Freire
On Tue, Apr 18, 2017 at 2:45 AM, Daniel Blanch Bataller wrote: > > But if you are going to move a record at a time you are going to be limited > by the fastest transaction rate you can achieve, which is going to be a few > hundred per second, and limited at the end by the disk hardware you have, .

Re: [PERFORM] Insert Concurrency

2017-04-17 Thread Daniel Blanch Bataller
Yes, postgres has partitions: https://www.postgresql.org/docs/9.6/static/ddl-partitioning.html But this is not going to help much in the scenario you have. Postgres can ingest data very very fast, 100M records in seconds - minu

Re: [PERFORM] Insert Concurrency

2017-04-17 Thread David Rowley
On 18 April 2017 at 14:55, ROBERT PRICE wrote: > I come from an Oracle background and am porting an application to postgres. > App has a table that will contain 100 million rows and has to be loaded by a > process that reads messages off a SQS queue and makes web service calls to > insert records

[PERFORM] Insert Concurrency

2017-04-17 Thread ROBERT PRICE
I come from an Oracle background and am porting an application to postgres. App has a table that will contain 100 million rows and has to be loaded by a process that reads messages off a SQS queue and makes web service calls to insert records one row at a time in a postgres RDS instance. I know