On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund and...@2ndquadrant.com wrote:
On 2014-09-11 12:55:21 -0400, Robert Haas wrote:
I advise supporting pglz only for the initial patch, and adding
support for the others later if it seems worthwhile. The approach
seems to work well enough with pglz
On 2014-09-11 13:04:43 -0400, Robert Haas wrote:
On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund and...@2ndquadrant.com
wrote:
On 2014-09-11 12:55:21 -0400, Robert Haas wrote:
I advise supporting pglz only for the initial patch, and adding
support for the others later if it seems
On Thu, Sep 11, 2014 at 1:17 PM, Andres Freund and...@2ndquadrant.com wrote:
On 2014-09-11 13:04:43 -0400, Robert Haas wrote:
On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund and...@2ndquadrant.com
wrote:
On 2014-09-11 12:55:21 -0400, Robert Haas wrote:
I advise supporting pglz only for the
On Thu, Sep 11, 2014 at 06:58:06PM +0200, Andres Freund wrote:
On 2014-09-11 12:55:21 -0400, Robert Haas wrote:
I advise supporting pglz only for the initial patch, and adding
support for the others later if it seems worthwhile. The approach
seems to work well enough with pglz that it's
On Thu, Sep 11, 2014 at 07:17:42PM +0200, Andres Freund wrote:
On 2014-09-11 13:04:43 -0400, Robert Haas wrote:
On Thu, Sep 11, 2014 at 12:58 PM, Andres Freund and...@2ndquadrant.com
wrote:
On 2014-09-11 12:55:21 -0400, Robert Haas wrote:
I advise supporting pglz only for the initial
I will repeat the above tests with high load on CPU and using the benchmark
given by Fujii-san and post the results.
Average % of CPU usage at user level for each of the compression algorithm
are as follows.
CompressionMultipleSingle
Off81.1338
On Wed, Aug 27, 2014 at 11:52 PM, Robert Haas robertmh...@gmail.com wrote:
On Tue, Aug 26, 2014 at 8:14 AM, Fujii Masao masao.fu...@gmail.com wrote:
On Tue, Aug 19, 2014 at 6:37 PM, Rahila Syed rahilasye...@gmail.com wrote:
Hello,
Thank you for comments.
Could you tell me where the patch for
On Thu, Aug 28, 2014 at 12:46 AM, Arthur Silva arthur...@gmail.com wrote:
Em 26/08/2014 09:16, Fujii Masao masao.fu...@gmail.com escreveu:
On Tue, Aug 19, 2014 at 6:37 PM, Rahila Syed rahilasye...@gmail.com
wrote:
Hello,
Thank you for comments.
Could you tell me where the patch for
Hello,
It'd be interesting to check avg cpu usage as well
I have collected average CPU utilization numbers by collecting sar output
at interval of 10 seconds for following benchmark:
Server specifications:
Processors:Intel® Xeon ® Processor E5-2650 (2 GHz, 8C/16T, 20 MB) * 2 nos
RAM: 32GB
Disk
On Tue, Sep 2, 2014 at 9:11 AM, Rahila Syed rahilasye...@gmail.com wrote:
Hello,
It'd be interesting to check avg cpu usage as well
I have collected average CPU utilization numbers by collecting sar output
at interval of 10 seconds for following benchmark:
Server specifications:
On Tue, Sep 02, 2014 at 10:30:11AM -0300, Arthur Silva wrote:
On Tue, Sep 2, 2014 at 9:11 AM, Rahila Syed rahilasye...@gmail.com wrote:
Hello,
It'd be interesting to check avg cpu usage as well
I have collected average CPU utilization numbers by collecting sar output
at interval of
On 2014-09-02 08:37:42 -0500, k...@rice.edu wrote:
I agree completely. For day-to-day use we should use LZ4-default. For
read-only
tables, it might be nice to archive them with LZ4-HC for the higher
compression
would increase read speed and reduce storage space needs. I believe that
On Tue, Aug 26, 2014 at 8:14 AM, Fujii Masao masao.fu...@gmail.com wrote:
On Tue, Aug 19, 2014 at 6:37 PM, Rahila Syed rahilasye...@gmail.com wrote:
Hello,
Thank you for comments.
Could you tell me where the patch for single block in one run is?
Please find attached patch for single block
Em 26/08/2014 09:16, Fujii Masao masao.fu...@gmail.com escreveu:
On Tue, Aug 19, 2014 at 6:37 PM, Rahila Syed rahilasye...@gmail.com
wrote:
Hello,
Thank you for comments.
Could you tell me where the patch for single block in one run is?
Please find attached patch for single block
On Tue, Aug 19, 2014 at 6:37 PM, Rahila Syed rahilasye...@gmail.com wrote:
Hello,
Thank you for comments.
Could you tell me where the patch for single block in one run is?
Please find attached patch for single block compression in one run.
Thanks! I ran the benchmark using pgbench and
On Tue, Aug 19, 2014 at 2:08 AM, Andres Freund and...@2ndquadrant.com wrote:
On 2014-08-18 13:06:15 -0400, Robert Haas wrote:
On Mon, Aug 18, 2014 at 7:19 AM, Rahila Syed rahilasye...@gmail.com wrote:
According to the measurement result, the amount of WAL generated in
Multiple Blocks in one
Hello,
Thank you for comments.
Could you tell me where the patch for single block in one run is?
Please find attached patch for single block compression in one run.
Thank you,
On Tue, Aug 19, 2014 at 1:17 PM, Fujii Masao masao.fu...@gmail.com wrote:
On Tue, Aug 19, 2014 at 2:08 AM, Andres
So, it seems like you're basically using malloc to work around the
fact that a palloc failure is an error, and we can't throw an error in
a critical section. I don't think that's good; we want all of our
allocations, as far as possible, to be tracked via palloc. It might
be a good idea to add a
On Mon, Aug 18, 2014 at 7:19 AM, Rahila Syed rahilasye...@gmail.com wrote:
According to the measurement result, the amount of WAL generated in
Multiple Blocks in one run than that in Single Block in one run.
So ISTM that compression of multiple blocks at one run can improve
the compression ratio.
On 2014-08-18 13:06:15 -0400, Robert Haas wrote:
On Mon, Aug 18, 2014 at 7:19 AM, Rahila Syed rahilasye...@gmail.com wrote:
According to the measurement result, the amount of WAL generated in
Multiple Blocks in one run than that in Single Block in one run.
So ISTM that compression of multiple
On Thu, Jul 3, 2014 at 3:58 PM, Rahila Syed rahilasye...@gmail.com wrote:
Updated version of patches are attached.
Changes are as follows
1. Improved readability of the code as per the review comments.
2. Addition of block_compression field in BkpBlock structure to store
information about
On Sat, Aug 16, 2014 at 6:51 PM, Rahila Syed rahilasye...@gmail.com wrote:
So, you're compressing backup blocks one by one. I wonder if that's the
right idea and if we shouldn't instead compress all of them in one run to
increase the compression ratio
Please find attached patch for compression
On Tue, Aug 5, 2014 at 6:25 PM, Fujii Masao masao.fu...@gmail.com wrote:
This gradual approach looks good to me. And, if the additional compression
algorithm like lz4 is always better than pglz for every scenarios, we can
just
change the code so that the additional algorithm is always used.
On Wed, Jul 23, 2014 at 5:21 PM, Pavan Deolasee
pavan.deola...@gmail.com wrote:
1. Need for compressing full page backups:
There are good number of benchmarks done by various people on this list
which clearly shows the need of the feature. Many people have already voiced
their agreement on
I'm trying to understand what would it take to have this patch in an
acceptable form before the next commitfest. Both Abhijit and Andres has
done some extensive review of the patch and have given many useful
suggestions to Rahila. While she has incorporated most of them, I feel we
are still some
On 2014-07-04 19:27:10 +0530, Rahila Syed wrote:
+ /* Allocates memory for compressed backup blocks according to the
compression
+ * algorithm used.Once per session at the time of insertion of first XLOG
+ * record.
+ * This memory stays till the end of session. OOM is
Thank you for review.
So, you're compressing backup blocks one by one. I wonder if that's the
right idea and if we shouldn't instead compress all of them in one run to
increase the compression ratio.
The idea behind compressing blocks one by one was to keep the code as much
similar to the
But though the code looks better locally than before, the larger problem
is that this is still unsafe. As Pavan pointed out, XLogInsert is called
from inside critical sections, so we can't allocate memory here.
Could you look into his suggestions of other places to do the
allocation, please?
If I
Thank you for review comments.
There are still numerous formatting changes required, e.g. spaces around
= and correct formatting of comments. And git diff --check still has
a few whitespace problems. I won't point these out one by one, but maybe
you should run pgindent
I will do this.
Could you
At 2014-07-04 14:38:27 +0900, masao.fu...@gmail.com wrote:
But 0002-CompressBackupBlock_snappy_lz4_pglz-2.patch doesn't seem to
be able to apply to HEAD cleanly.
Yes, and it needs quite some reformatting beyond fixing whitespace
damage too (long lines, comment formatting, consistent spacing
At 2014-07-04 19:27:10 +0530, rahilasye...@gmail.com wrote:
Please find attached patches with no whitespace error and improved
formatting.
Thanks.
There are still numerous formatting changes required, e.g. spaces around
= and correct formatting of comments. And git diff --check still has
a
At 2014-07-04 21:02:33 +0530, a...@2ndquadrant.com wrote:
+/*
+ */
+static const struct config_enum_entry backup_block_compression_options[] =
{
Oh, I forgot to mention that the configuration setting changes are also
pending. I think we had a working consensus to use
On Fri, Jul 4, 2014 at 4:58 AM, Rahila Syed rahilasye...@gmail.com wrote:
Hello,
Updated version of patches are attached.
Changes are as follows
1. Improved readability of the code as per the review comments.
2. Addition of block_compression field in BkpBlock structure to store
information
Hello ,
I have a few preliminary comments about your patch
Thank you for review comments.
the patch creates src/common/lz4/.travis.yml, which it shouldn't.
Agree. I will remove it.
Shouldn't this use palloc?
palloc() is disallowed in critical sections and we are already in CS while
executing
On 2014-06-18 18:10:34 +0530, Rahila Syed wrote:
Hello ,
I have a few preliminary comments about your patch
Thank you for review comments.
the patch creates src/common/lz4/.travis.yml, which it shouldn't.
Agree. I will remove it.
Shouldn't this use palloc?
palloc() is disallowed in
At 2014-06-18 18:10:34 +0530, rahilasye...@gmail.com wrote:
palloc() is disallowed in critical sections and we are already in CS
while executing this code. So we use malloc().
Are these allocations actually inside a critical section? It seems to me
that the critical section starts further
At 2014-06-18 18:25:34 +0530, a...@2ndquadrant.com wrote:
Are these allocations actually inside a critical section? It seems to me
that the critical section starts further down, but perhaps I am missing
something.
OK, I was missing that XLogInsert() itself can be called from inside a
critical
On Wed, Jun 18, 2014 at 6:25 PM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
At 2014-06-18 18:10:34 +0530, rahilasye...@gmail.com wrote:
palloc() is disallowed in critical sections and we are already in CS
while executing this code. So we use malloc().
Are these allocations actually
On Tue, Jun 17, 2014 at 8:47 AM, Abhijit Menon-Sen a...@2ndquadrant.com wrote:
if (compress_backup_block = BACKUP_BLOCK_COMPRESSION_SNAPPY)
You mean == right?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
At 2014-06-17 15:31:33 -0300, klaussfre...@gmail.com wrote:
On Tue, Jun 17, 2014 at 8:47 AM, Abhijit Menon-Sen a...@2ndquadrant.com
wrote:
if (compress_backup_block = BACKUP_BLOCK_COMPRESSION_SNAPPY)
You mean == right?
Of course. Thanks.
-- Abhijit
--
Sent via pgsql-hackers
201 - 240 of 240 matches
Mail list logo
