Re: [HACKERS] Proposal: generic WAL compression

2018-03-13 Thread Oleg Ivanov
On 02/07/2018 09:02 PM, Markus Nullmeier wrote: One general comment I can already make is that enabling compression should be made optional, which appears to be a small and easy addition to the generic WAL API. The new version of the patch is attached. In order to control generic WAL compressi

Re: [HACKERS] Proposal: generic WAL compression

2018-02-17 Thread Oleg Ivanov
Hello everyone! Unfortunately, I faced the use case with RUM index, in which my patch produced enormously large time overhead (queries execution time is about 2 or 3 times slower). Only now I finally managed to limit this overhead by 20% or even make it statistically insignificant on my HDD. N

Re: [HACKERS] Proposal: generic WAL compression

2018-02-08 Thread Markus Nullmeier
On 07/02/18 19:42, Stephen Frost wrote: >> really useful for my "OUZO" project (a fork of the RUM access method). > > Glad to hear that you're continuing to work on it. Yes, it will be available on Github eventually. >> One general comment I can already make is that enabling compression >> shoul

Re: [HACKERS] Proposal: generic WAL compression

2018-02-07 Thread Stephen Frost
Greetings, * Markus Nullmeier (dq...@uni-heidelberg.de) wrote: > On 07/02/18 18:31, David Steele wrote: > > > This proposal is still in need of review and hasn't really gotten it in > > the last few CFs. > > Agreed. > > > Since the feature does not seem like a good fit for the last CF of PG11 >

Re: [HACKERS] Proposal: generic WAL compression

2018-02-07 Thread Markus Nullmeier
Hi David, On 07/02/18 18:31, David Steele wrote: > This proposal is still in need of review and hasn't really gotten it in > the last few CFs. Agreed. > Since the feature does not seem like a good fit for the last CF of PG11 > I am marking it Returned with Feedback. Is there any chance that it

Re: Re: [HACKERS] Proposal: generic WAL compression

2018-02-07 Thread David Steele
Hi Oleg, On 1/22/18 4:37 PM, Stephen Frost wrote: > Oleg, > > I'm not really sure why this is still in Needs Review as a review was > posted and I don't see any follow-up. I've changed this to be Waiting > for Author. > > * Antonin Houska (a...@cybertec.at) wrote: >> Oleg Ivanov wrote: >> >>>

Re: [HACKERS] Proposal: generic WAL compression

2018-01-22 Thread Stephen Frost
Oleg, I'm not really sure why this is still in Needs Review as a review was posted and I don't see any follow-up. I've changed this to be Waiting for Author. * Antonin Houska (a...@cybertec.at) wrote: > Oleg Ivanov wrote: > > > In order to overcome that issue, I would like to propose the patch

Re: [HACKERS] Proposal: generic WAL compression

2017-11-29 Thread Michael Paquier
On Mon, Nov 20, 2017 at 4:49 PM, Antonin Houska wrote: > One more idea: > > I think the metadata (ALIGN_GAP) should be stored separate from the actual > data so that you can use memcpy() instead of this loop: > > while (i < j) > { > charc = targetRegionAligned[i

Re: [HACKERS] Proposal: generic WAL compression

2017-11-19 Thread Antonin Houska
Antonin Houska wrote: > Oleg Ivanov wrote: > > > In order to overcome that issue, I would like to propose the patch, which > > provides possibility to use another approach of the WAL record > > construction. > > After having spent several hours reviewing this patch I dare to send the > followi

Re: [HACKERS] Proposal: generic WAL compression

2017-11-17 Thread Arthur Silva
I wonder if the algorithm could be changed to operate with units of 2 or 4 bytes (instead of 1). Since the algorithm speed is essentially doubled/quadrupled it could yield a better runtime/compression trade-off. Does that make sense? -- Arthur Silva On Wed, Nov 1, 2017 at 12:43 AM, Oleg Ivanov

Re: [HACKERS] Proposal: generic WAL compression

2017-11-16 Thread Antonin Houska
Oleg Ivanov wrote: > In order to overcome that issue, I would like to propose the patch, which > provides possibility to use another approach of the WAL record > construction. After having spent several hours reviewing this patch I dare to send the following comments: * writeToPtr() and readFr