Re: [HACKERS] Logical replication ApplyContext bloat

2017-05-09 Thread Stas Kelvich
> On 9 May 2017, at 21:53, Peter Eisentraut > wrote: > > On 5/8/17 11:26, Peter Eisentraut wrote: >> I think it would make more sense to reset ApplyMessageContext in >> apply_dispatch(), so that it is done consistently after every message >> (as the name

Re: [HACKERS] Logical replication ApplyContext bloat

2017-05-09 Thread Peter Eisentraut
On 5/8/17 11:26, Peter Eisentraut wrote: > I think it would make more sense to reset ApplyMessageContext in > apply_dispatch(), so that it is done consistently after every message > (as the name implies), not only some messages. Committed that. > Also, perhaps ApplyMessageContext should be a

Re: [HACKERS] Logical replication ApplyContext bloat

2017-05-08 Thread Peter Eisentraut
On 5/5/17 03:09, Noah Misch wrote: > [Action required within three days. This is a generic notification.] > > The above-described topic is currently a PostgreSQL 10 open item. Peter, > since you committed the patch believed to have created it, you own this open > item. If some other commit is

Re: [HACKERS] Logical replication ApplyContext bloat

2017-05-08 Thread Peter Eisentraut
I'm not sure about some of the details. I think it would make more sense to reset ApplyMessageContext in apply_dispatch(), so that it is done consistently after every message (as the name implies), not only some messages. Also, perhaps ApplyMessageContext should be a child of

Re: [HACKERS] Logical replication ApplyContext bloat

2017-05-05 Thread Noah Misch
On Wed, May 03, 2017 at 12:02:41PM +0300, Stas Kelvich wrote: > > On 20 Apr 2017, at 17:01, Dilip Kumar wrote: > > On Thu, Apr 20, 2017 at 7:04 PM, Stas Kelvich > > wrote: > >> Thanks for noting. > >> > >> Added short description of ApplyContext

Re: [HACKERS] Logical replication ApplyContext bloat

2017-05-03 Thread Stas Kelvich
> On 20 Apr 2017, at 17:01, Dilip Kumar wrote: > > On Thu, Apr 20, 2017 at 7:04 PM, Stas Kelvich > wrote: >> Thanks for noting. >> >> Added short description of ApplyContext and ApplyMessageContext to README. > > Typo > > /analysys/analysis

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-20 Thread Dilip Kumar
On Thu, Apr 20, 2017 at 7:04 PM, Stas Kelvich wrote: > Thanks for noting. > > Added short description of ApplyContext and ApplyMessageContext to README. +ApplyContext --- permanent during whole lifetime of apply worker. It is +possible to use TopMemoryContext here as

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-20 Thread Stas Kelvich
> On 19 Apr 2017, at 16:07, Alvaro Herrera wrote: > > Stas Kelvich wrote: > >> With patch MemoryContextStats() shows following hierarchy during slot >> operations in >> apply worker: >> >> TopMemoryContext: 83824 total in 5 blocks; 9224 free (8 chunks); 74600 used

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Alvaro Herrera
Stas Kelvich wrote: > With patch MemoryContextStats() shows following hierarchy during slot > operations in > apply worker: > > TopMemoryContext: 83824 total in 5 blocks; 9224 free (8 chunks); 74600 used > ApplyContext: 8192 total in 1 blocks; 6520 free (4 chunks); 1672 used >

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Stas Kelvich
> On 19 Apr 2017, at 14:30, Petr Jelinek wrote: > > On 19/04/17 12:46, Stas Kelvich wrote: >> >> Right now ApplyContext cleaned after each transaction and by this patch I >> basically >> suggest to clean it after each command counter increment. >> >> So in

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Petr Jelinek
On 19/04/17 12:46, Stas Kelvich wrote: > >> On 19 Apr 2017, at 13:34, Simon Riggs wrote: >> >> On 19 April 2017 at 11:24, Petr Jelinek wrote: >> >>> I'd still like you to add comment to the ApplyContext saying that it's >>> cleaned after

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Stas Kelvich
> On 19 Apr 2017, at 13:34, Simon Riggs wrote: > > On 19 April 2017 at 11:24, Petr Jelinek wrote: > >> I'd still like you to add comment to the ApplyContext saying that it's >> cleaned after handling each message so nothing that needs to

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Simon Riggs
On 19 April 2017 at 11:24, Petr Jelinek wrote: > I'd still like you to add comment to the ApplyContext saying that it's > cleaned after handling each message so nothing that needs to survive for > example whole transaction can be allocated in it as that's not very

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Petr Jelinek
On 19/04/17 11:55, Stas Kelvich wrote: > >> On 19 Apr 2017, at 12:37, Petr Jelinek wrote: >> >> On 18/04/17 13:45, Stas Kelvich wrote: >>> Hi, >>> >>> currently logical replication worker uses ApplyContext to decode received >>> data >>> and that context is never

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Stas Kelvich
> On 19 Apr 2017, at 12:37, Petr Jelinek wrote: > > On 18/04/17 13:45, Stas Kelvich wrote: >> Hi, >> >> currently logical replication worker uses ApplyContext to decode received >> data >> and that context is never freed during transaction processing. Hence if

Re: [HACKERS] Logical replication ApplyContext bloat

2017-04-19 Thread Petr Jelinek
On 18/04/17 13:45, Stas Kelvich wrote: > Hi, > > currently logical replication worker uses ApplyContext to decode received data > and that context is never freed during transaction processing. Hence if > publication > side is performing something like 10M row inserts in single transaction, then