Re: best way to write large data-streams quickly?

2018-04-10 Thread Jerry Sievers
Mark Moellering writes: > > How long can you run COPY?  I have been looking at it more closely.  > In some ways, it would be simple just to take data from stdin and > send it to postgres but can I do that literally 24/7?  I am > monitoring data feeds that will

Re: best way to write large data-streams quickly?

2018-04-10 Thread Mark Moellering
On Mon, Apr 9, 2018 at 12:01 PM, Steve Atkins wrote: > > > On Apr 9, 2018, at 8:49 AM, Mark Moellering com> wrote: > > > > Everyone, > > > > We are trying to architect a new system, which will have to take several > large datastreams (total of

Re: best way to write large data-streams quickly?

2018-04-09 Thread Steve Atkins
> On Apr 9, 2018, at 8:49 AM, Mark Moellering > wrote: > > Everyone, > > We are trying to architect a new system, which will have to take several > large datastreams (total of ~200,000 parsed files per second) and place them > in a database. I am trying to

best way to write large data-streams quickly?

2018-04-09 Thread Mark Moellering
Everyone, We are trying to architect a new system, which will have to take several large datastreams (total of ~200,000 parsed files per second) and place them in a database. I am trying to figure out the best way to import that sort of data into Postgres. I keep thinking i can't be the first