On Mon, Jul 6, 2020 at 5:24 PM Daniel Gustafsson wrote:
> > On 12 Jun 2020, at 19:28, Robert Haas wrote:
>
> > I am sure that nobody is arguing that the patch has to be beneficial
> > in all cases in order to justify applying it. However, there are
> > several good arguments against proceding
> On 12 Jun 2020, at 19:28, Robert Haas wrote:
> I am sure that nobody is arguing that the patch has to be beneficial
> in all cases in order to justify applying it. However, there are
> several good arguments against proceding with this patch:
This thread has stalled with no resolution to the
On Thu, Jun 11, 2020 at 1:41 PM Hamid Akhtar wrote:
> As far I understand, parallel backup is not a mandatory performance feature,
> rather, one at user's discretion. This IMHO indicates that it will benefit
> some users and it may not others.
>
> IMHO, parallel backup has obvious performance
As far I understand, parallel backup is not a mandatory performance
feature, rather, one at user's discretion. This IMHO indicates that it will
benefit some users and it may not others.
Taking a backup is an I/O intensive workload. So by parallelizing it
through multiple worker threads/processes,
On Thu, May 21, 2020 at 2:06 AM Rushabh Lathia wrote:
> Yes. My colleague Suraj tried this and here are the pg_stat_activity output
> files.
>
> Captured wait events after every 3 seconds during the backup for -
> 1: parallel backup for 100GB data with 4 workers
>
On Thu, May 21, 2020 at 11:36 AM Rushabh Lathia
wrote:
>
> On Thu, May 21, 2020 at 10:47 AM Ahsan Hadi wrote:
>>
During an offlist discussion with Robert, he pointed out that current
basebackup's code doesn't account for the wait event for the reading
of files which can
On Thu, Apr 30, 2020 at 4:15 PM Amit Kapila wrote:
> On Wed, Apr 29, 2020 at 6:11 PM Suraj Kharage
> wrote:
> >
> > Hi,
> >
> > We at EnterpriseDB did some performance testing around this parallel
> backup to check how this is beneficial and below are the results. In this
> testing, we run the
On Thu, Apr 30, 2020 at 4:15 PM Amit Kapila wrote:
>
> On Wed, Apr 29, 2020 at 6:11 PM Suraj Kharage
> wrote:
> >
> > Hi,
> >
> > We at EnterpriseDB did some performance testing around this parallel backup
> > to check how this is beneficial and below are the results. In this testing,
> > we
On Wed, Apr 29, 2020 at 6:11 PM Suraj Kharage
wrote:
>
> Hi,
>
> We at EnterpriseDB did some performance testing around this parallel backup
> to check how this is beneficial and below are the results. In this testing,
> we run the backup -
> 1) Without Asif’s patch
> 2) With Asif’s patch and
Hi,
Would it be possible to put in the absolute numbers of the perf
so that it is easier to understand the amount of improvement with
and without the patch and different loads and workers.
I am also unsure why the swapper is taking such a huge percentage of the
absolute time
in the base run of
On Mon, Apr 27, 2020 at 10:23 PM David Zhang wrote:
>
> Hi,
>
> Here is the parallel backup performance test results with and without
> the patch "parallel_backup_v15" on AWS cloud environment. Two
> "t2.xlarge" machines were used: one for Postgres server and the other
> one for pg_basebackup
Hi,
Here is the parallel backup performance test results with and without
the patch "parallel_backup_v15" on AWS cloud environment. Two
"t2.xlarge" machines were used: one for Postgres server and the other
one for pg_basebackup with the same machine configuration showing below.
Machine
On Thu, Apr 23, 2020 at 1:47 PM Asif Rehman wrote:
>
>
> On Thu, Apr 23, 2020 at 11:43 AM Rajkumar Raghuwanshi <
> rajkumar.raghuwan...@enterprisedb.com> wrote:
>
>>
>>
>> On Wed, Apr 22, 2020 at 7:48 PM Asif Rehman
>> wrote:
>>
>>>
>>> Hi Dipesh,
>>>
>>> The rebased and updated patch is
On Thu, Apr 23, 2020 at 11:43 AM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterprisedb.com> wrote:
>
>
> On Wed, Apr 22, 2020 at 7:48 PM Asif Rehman
> wrote:
>
>>
>> Hi Dipesh,
>>
>> The rebased and updated patch is attached. Its rebased to (9f2c4ede).
>>
>
> Make is failing for v15 patch.
>
On Wed, Apr 22, 2020 at 7:48 PM Asif Rehman wrote:
>
> Hi Dipesh,
>
> The rebased and updated patch is attached. Its rebased to (9f2c4ede).
>
Make is failing for v15 patch.
gcc -std=gnu99 -Wall -Wmissing-prototypes -Wpointer-arith
-Wdeclaration-after-statement -Werror=vla -Wendif-labels
On Wed, Apr 22, 2020 at 10:18 AM Asif Rehman wrote:
> I don't foresee memory to be a challenge here. Assuming a database containing
> 10240
> relation files (that max reach to 10 TB of size), the list will occupy
> approximately 102MB
> of space in memory. This obviously can be reduced, but it
Hi Dipesh,
The rebased and updated patch is attached. Its rebased to (9f2c4ede).
> +typedef struct
> +{
> ...
> +} BackupFile;
> +
> +typedef struct
> +{
> ...
> +} BackupState;
>
> These structures need comments.
>
Done.
>
> +list_wal_files_opt_list:
> + SCONST SCONST
>
Hi Asif,
I am reviewing your recent patch and found the patch is not applicable on
latest master.
Could you please resolve the conflicts and update a new patch?
Thanks,
Dipesh
EnterpriseDB: http://www.enterprisedb.com
On Tue, Apr 21, 2020 at 5:26 PM Ahsan Hadi wrote:
>
> On Tue, Apr 21, 2020 at 4:50 PM Amit Kapila wrote:
>>
>> On Tue, Apr 21, 2020 at 5:18 PM Amit Kapila wrote:
>> >
>> > On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman wrote:
>> > >
>> > > I did some tests a while back, and here are the results.
On Tue, Apr 21, 2020 at 4:50 PM Amit Kapila wrote:
> On Tue, Apr 21, 2020 at 5:18 PM Amit Kapila
> wrote:
> >
> > On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman
> wrote:
> > >
> > > I did some tests a while back, and here are the results. The tests
> were done to simulate
> > > a live database
On Tue, Apr 21, 2020 at 5:18 PM Amit Kapila wrote:
>
> On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman wrote:
> >
> > I did some tests a while back, and here are the results. The tests were
> > done to simulate
> > a live database environment using pgbench.
> >
> > machine configuration used for
On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman wrote:
>
> I did some tests a while back, and here are the results. The tests were done
> to simulate
> a live database environment using pgbench.
>
> machine configuration used for this test:
> Instance Type:t2.xlarge
> Volume Type :io1
>
On Tue, 21 Apr 2020 at 2:36 PM, Jeevan Ladhe
wrote:
> Hi Asif,
>
> On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman
> wrote:
>
>> Hi,
>>
>> I did some tests a while back, and here are the results. The tests were
>> done to simulate
>> a live database environment using pgbench.
>>
>> machine
Hi Asif,
On Tue, Apr 21, 2020 at 1:00 PM Asif Rehman wrote:
> Hi,
>
> I did some tests a while back, and here are the results. The tests were
> done to simulate
> a live database environment using pgbench.
>
> machine configuration used for this test:
> Instance Type:t2.xlarge
> Volume Type
Hi,
I did some tests a while back, and here are the results. The tests were
done to simulate
a live database environment using pgbench.
machine configuration used for this test:
Instance Type:t2.xlarge
Volume Type :io1
Memory (MiB) :16384
vCPU # :4
Architecture:
On Tue, Apr 14, 2020 at 8:07 PM Asif Rehman wrote:
>
> I forgot to make a check for no-manifest. Fixed. Attached is the updated
> patch.
>
>
Have we done any performance testing with this patch to see the benefits?
If so, can you point me to the results? If not, then can we perform some
tests on
On Tue, Apr 14, 2020 at 5:33 PM Asif Rehman wrote:
>
>
> On Wed, Apr 8, 2020 at 6:53 PM Kashif Zeeshan <
> kashif.zees...@enterprisedb.com> wrote:
>
>>
>>
>> On Tue, Apr 7, 2020 at 9:44 PM Asif Rehman
>> wrote:
>>
>>> Hi,
>>>
>>> Thanks, Kashif and Rajkumar. I have fixed the reported issues.
On Tue, Apr 14, 2020 at 7:37 PM Asif Rehman wrote:
>
>
> On Tue, Apr 14, 2020 at 6:32 PM Kashif Zeeshan <
> kashif.zees...@enterprisedb.com> wrote:
>
>> Hi Asif
>>
>> Getting the following error on Parallel backup when --no-manifest option
>> is used.
>>
>> [edb@localhost bin]$
>> [edb@localhost
On Wed, Apr 15, 2020 at 4:49 AM Ahsan Hadi wrote:
> Fair enough. Some of this is also due to backup related features i.e backup
> manifest, progress reporting that got committed to master towards the tail
> end of PG-13. Rushing to get parallel backup feature compatible with these
> features
Hi Asif,
In below scenarios backup verification failed for tablespace, when backup
taken with parallel option.
without parallel for the same scenario pg_verifybackup is passed without
any error.
[edb@localhost bin]$ mkdir /tmp/test_bkp/tblsp1
[edb@localhost bin]$ ./psql postgres -p 5432 -c
On Wed, 15 Apr 2020 at 1:49 AM, Robert Haas wrote:
> On Tue, Apr 14, 2020 at 10:37 AM Asif Rehman
> wrote:
> > I forgot to make a check for no-manifest. Fixed. Attached is the updated
> patch.
>
> +typedef struct
> +{
> ...
> +} BackupFile;
> +
> +typedef struct
> +{
> ...
> +} BackupState;
>
>
On Tue, Apr 14, 2020 at 10:37 AM Asif Rehman wrote:
> I forgot to make a check for no-manifest. Fixed. Attached is the updated
> patch.
+typedef struct
+{
...
+} BackupFile;
+
+typedef struct
+{
...
+} BackupState;
These structures need comments.
+list_wal_files_opt_list:
+
Hi Asif
Getting the following error on Parallel backup when --no-manifest option is
used.
[edb@localhost bin]$
[edb@localhost bin]$
[edb@localhost bin]$ ./pg_basebackup -v -j 5 -D
/home/edb/Desktop/backup/ --no-manifest
pg_basebackup: initiating base backup, waiting for checkpoint to complete
On Tue, Apr 7, 2020 at 9:44 PM Asif Rehman wrote:
> Hi,
>
> Thanks, Kashif and Rajkumar. I have fixed the reported issues.
>
> I have added the shared state as previously described. The new grammar
> changes
> are as follows:
>
> START_BACKUP [LABEL ''] [FAST] [MAX_RATE %d]
> - This will
rebased and updated to current master (d025cf88ba). v12 is attahced.
Also, changed the grammar for LIST_WAL_FILES and SEND_FILE to:
- LIST_WAL_FILES 'startptr' 'endptr'
- SEND_FILE 'FILE' [NOVERIFY_CHECKSUMS]
On Wed, Apr 8, 2020 at 10:48 AM Rajkumar Raghuwanshi <
Hi Asif,
Thanks for new patches.
Patches need to be rebased on head. Getting a failure while applying the
0003 patch.
edb@localhost postgresql]$ git apply
v11/0003-Parallel-Backup-Backend-Replication-commands.patch
error: patch failed: src/backend/storage/ipc/ipci.c:147
error:
On Tue, Apr 7, 2020 at 1:25 PM Asif Rehman wrote:
> I will, however parallel backup is already quite a large patch. So I think we
> should first
> agree on the current work before adding a backup manifest and
> progress-reporting support.
It's going to be needed for commit, but it may make
On Fri, Apr 3, 2020 at 4:46 AM Asif Rehman wrote:
> Non-parallel backup already does the early error checking. I only intended
> to make parallel behave the same as non-parallel here. So, I agree with
> you that the behavior of parallel backup should be consistent with the
> non-parallel one.
On Tue, Apr 7, 2020 at 10:03 PM Jeevan Chalke <
jeevan.cha...@enterprisedb.com> wrote:
>
>
> On Tue, Apr 7, 2020 at 10:14 PM Asif Rehman
> wrote:
>
>> Hi,
>>
>> Thanks, Kashif and Rajkumar. I have fixed the reported issues.
>>
>> I have added the shared state as previously described. The new
On Tue, Apr 7, 2020 at 10:14 PM Asif Rehman wrote:
> Hi,
>
> Thanks, Kashif and Rajkumar. I have fixed the reported issues.
>
> I have added the shared state as previously described. The new grammar
> changes
> are as follows:
>
> START_BACKUP [LABEL ''] [FAST] [MAX_RATE %d]
> - This will
Hi,
Thanks, Kashif and Rajkumar. I have fixed the reported issues.
I have added the shared state as previously described. The new grammar
changes
are as follows:
START_BACKUP [LABEL ''] [FAST] [MAX_RATE %d]
- This will generate a unique backupid using pg_strong_random(16) and
hex-encoded
On Fri, Apr 3, 2020 at 3:01 PM Kashif Zeeshan <
kashif.zees...@enterprisedb.com> wrote:
> Hi Asif
>
> When a non-existent slot is used with tablespace then correct error is
> displayed but then the backup folder is not cleaned and leaves a corrupt
> backup.
>
> Steps
> ===
>
> edb@localhost
Asif,
After recent backup manifest addition, patches needed to rebase and
reconsideration of a few things like making sure that parallel backup
creates
a manifest file correctly or not etc.
--
Jeevan Chalke
Associate Database Architect & Team Lead, Product Development
EnterpriseDB Corporation
Hi Asif
When a non-existent slot is used with tablespace then correct error is
displayed but then the backup folder is not cleaned and leaves a corrupt
backup.
Steps
===
edb@localhost bin]$
[edb@localhost bin]$ mkdir /home/edb/tbl1
[edb@localhost bin]$ mkdir /home/edb/tbl_res
[edb@localhost
On Thu, Apr 2, 2020 at 8:45 PM Robert Haas wrote:
> On Thu, Apr 2, 2020 at 11:17 AM Asif Rehman
> wrote:
> >> Why would you need to do that? As long as the process where
> >> STOP_BACKUP can do the check, that seems good enough.
> >
> > Yes, but the user will get the error only after the
On Thu, Apr 2, 2020 at 11:17 AM Asif Rehman wrote:
>> Why would you need to do that? As long as the process where
>> STOP_BACKUP can do the check, that seems good enough.
>
> Yes, but the user will get the error only after the STOP_BACKUP, not while
> the backup is
> in progress. So if the
On Thu, Apr 2, 2020 at 4:47 PM Robert Haas wrote:
> On Fri, Mar 27, 2020 at 1:34 PM Asif Rehman
> wrote:
> > Yes, we are fetching a single file. However, SEND_FILES is still capable
> of fetching multiple files in one
> > go, that's why the name.
>
> I don't see why it should work that way. If
On Thu, Apr 2, 2020 at 9:46 AM Kashif Zeeshan <
kashif.zees...@enterprisedb.com> wrote:
> Does it fail to clean up the backup folder in all cases where the
>> backup failed, or just in this case?
>>
> The cleanup is done in the cases I have seen so far with base
> pg_basebackup functionality (not
On Thu, Apr 2, 2020 at 6:23 PM Robert Haas wrote:
> On Thu, Apr 2, 2020 at 7:55 AM Kashif Zeeshan
> wrote:
> > Thanks alot Robert,
> > In this case the backup folder was not being emptied as the backup was
> failed, the cleanup should be done in this case too.
>
> Does it fail to clean up the
On Thu, Apr 2, 2020 at 7:55 AM Kashif Zeeshan
wrote:
> Thanks alot Robert,
> In this case the backup folder was not being emptied as the backup was
> failed, the cleanup should be done in this case too.
Does it fail to clean up the backup folder in all cases where the
backup failed, or just in
On Thu, Apr 2, 2020 at 4:48 PM Robert Haas wrote:
> On Thu, Apr 2, 2020 at 7:30 AM Kashif Zeeshan <
> kashif.zees...@enterprisedb.com> wrote:
>
>> The backup failed with errors "error: could not connect to server: could
>> not look up local user ID 1000: Too many open files" when the
>>
On Thu, Apr 2, 2020 at 7:30 AM Kashif Zeeshan <
kashif.zees...@enterprisedb.com> wrote:
> The backup failed with errors "error: could not connect to server: could
> not look up local user ID 1000: Too many open files" when the
> max_wal_senders was set to 2000.
> The errors generated for the
On Fri, Mar 27, 2020 at 1:34 PM Asif Rehman wrote:
> Yes, we are fetching a single file. However, SEND_FILES is still capable of
> fetching multiple files in one
> go, that's why the name.
I don't see why it should work that way. If we're fetching individual
files, why have an unused capability
Hi Asif
The backup failed with errors "error: could not connect to server: could
not look up local user ID 1000: Too many open files" when the
max_wal_senders was set to 2000.
The errors generated for the workers starting from backup worke=1017.
Please note that the backup directory was also not
Hi Asif,
My colleague Kashif Zeeshan reported an issue off-list, posting here,
please take a look.
When executing two backups at the same time, getting FATAL error due to
max_wal_senders and instead of exit Backup got completed
And when tried to start the server from the backup cluster, getting
On Mon, Mar 30, 2020 at 3:44 PM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterprisedb.com> wrote:
> Thanks Asif,
>
> I have re-verified reported issue. expect standby backup, others are fixed.
>
Yes As Asif mentioned he is working on the standby issue and adding
bandwidth throttling
Thanks Asif,
I have re-verified reported issue. expect standby backup, others are fixed.
Thanks & Regards,
Rajkumar Raghuwanshi
On Fri, Mar 27, 2020 at 11:04 PM Asif Rehman wrote:
>
>
> On Wed, Mar 25, 2020 at 12:22 PM Rajkumar Raghuwanshi <
> rajkumar.raghuwan...@enterprisedb.com> wrote:
>
On Wed, Mar 25, 2020 at 12:22 PM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterprisedb.com> wrote:
> Hi Asif,
>
> While testing further I observed parallel backup is not able to take
> backup of standby server.
>
> mkdir /tmp/archive_dir
> echo "archive_mode='on'">> data/postgresql.conf
> echo
Hi Asif,
While testing further I observed parallel backup is not able to take backup
of standby server.
mkdir /tmp/archive_dir
echo "archive_mode='on'">> data/postgresql.conf
echo "archive_command='cp %p /tmp/archive_dir/%f'">> data/postgresql.conf
./pg_ctl -D data -l logs start
./pg_basebackup
Hi Asif,
In another scenarios, bkp data is corrupted for tablespace. again this is
not reproducible everytime,
but If I am running the same set of commands I am getting the same error.
[edb@localhost bin]$ ./pg_ctl -D data -l logfile start
waiting for server to start done
server started
Hi Asif,
On testing further, I found when taking backup with -R, pg_basebackup
crashed
this crash is not consistently reproducible.
[edb@localhost bin]$ ./psql postgres -p 5432 -c "create table test (a
text);"
CREATE TABLE
[edb@localhost bin]$ ./psql postgres -p 5432 -c "insert into test values
Hi Asif,
> Thanks Rajkumar. I have fixed the above issues and have rebased the patch
> to the latest master (b7f64c64).
> (V9 of the patches are attached).
>
I had a further review of the patches and here are my few observations:
1.
+/*
+ * stop_backup() - ends an online backup
+ *
+ * The
On Mon, Mar 16, 2020 at 11:52 AM Asif Rehman wrote:
>
>
> On Mon, Mar 16, 2020 at 11:08 AM Rajkumar Raghuwanshi <
> rajkumar.raghuwan...@enterprisedb.com> wrote:
>
>> Thanks for the patches.
>>
>> I have verified reported issues with new patches, issues are fixed now.
>>
>> I got another
On Mon, Mar 16, 2020 at 11:08 AM Rajkumar Raghuwanshi <
rajkumar.raghuwan...@enterprisedb.com> wrote:
> Thanks for the patches.
>
> I have verified reported issues with new patches, issues are fixed now.
>
> I got another observation where If a new slot name given without -C
> option, it leads to
Thanks for the patches.
I have verified reported issues with new patches, issues are fixed now.
I got another observation where If a new slot name given without -C option,
it leads to server crash error.
[edb@localhost bin]$ ./pg_basebackup -p 5432 -j 4 -D /tmp/bkp --slot
test_bkp_slot
Hi Asif
I have started testing this feature. I have applied v6 patch on commit
a069218163704c44a8996e7e98e765c56e2b9c8e (30 Jan).
I got few observations, please take a look.
*--if backup failed, backup directory is not getting removed.*
[edb@localhost bin]$ ./pg_basebackup -p 5432 --jobs=9 -D
Hi,
I have created a commitfest entry.
https://commitfest.postgresql.org/27/2472/
On Mon, Feb 17, 2020 at 1:39 PM Asif Rehman wrote:
> Thanks Jeevan. Here is the documentation patch.
>
> On Mon, Feb 10, 2020 at 6:49 PM Jeevan Chalke <
> jeevan.cha...@enterprisedb.com> wrote:
>
>> Hi Asif,
>>
Thanks Jeevan. Here is the documentation patch.
On Mon, Feb 10, 2020 at 6:49 PM Jeevan Chalke <
jeevan.cha...@enterprisedb.com> wrote:
> Hi Asif,
>
> On Thu, Jan 30, 2020 at 7:10 PM Asif Rehman
> wrote:
>
>>
>> Here are the the updated patches, taking care of the issues pointed
>> earlier. This
Hi Asif,
On Thu, Jan 30, 2020 at 7:10 PM Asif Rehman wrote:
>
> Here are the the updated patches, taking care of the issues pointed
> earlier. This patch adds the following commands (with specified option):
>
> START_BACKUP [LABEL ''] [FAST]
> STOP_BACKUP [NOWAIT]
> LIST_TABLESPACES [PROGRESS]
On Thu, Dec 19, 2019 at 10:47 PM Robert Haas wrote:
> On Thu, Dec 12, 2019 at 10:20 AM Asif Rehman
> wrote:
> > I have updated the patches (v7 attached) and have taken care of all
> issues pointed by Jeevan, additionally
> > ran the pgindent on each patch. Furthermore, Command names have been
>
On Thu, Dec 12, 2019 at 10:20 AM Asif Rehman wrote:
> I have updated the patches (v7 attached) and have taken care of all issues
> pointed by Jeevan, additionally
> ran the pgindent on each patch. Furthermore, Command names have been renamed
> as suggested and I
> have simplified the SendFiles
On Thu, Nov 28, 2019 at 12:57 AM Robert Haas wrote:
> On Wed, Nov 27, 2019 at 3:38 AM Jeevan Chalke
> wrote:
> > I am still not sure why we need SEND_BACKUP_FILELIST as a separate
> command.
> > Can't we return the file list with START_BACKUP itself?
>
> I had the same thought, but I think it's
On Wed, Nov 27, 2019 at 1:38 PM Jeevan Chalke <
jeevan.cha...@enterprisedb.com> wrote:
>
>
> On Wed, Nov 13, 2019 at 7:04 PM Asif Rehman
> wrote:
>
>>
>> Sorry, I sent the wrong patches. Please see the correct version of the
>> patches (_v6).
>>
>
> Review comments on these patches:
>
> 1.
> +
On Wed, Nov 27, 2019 at 3:38 AM Jeevan Chalke
wrote:
> I am still not sure why we need SEND_BACKUP_FILELIST as a separate command.
> Can't we return the file list with START_BACKUP itself?
I had the same thought, but I think it's better to keep them separate.
Somebody might want to use the
On Wed, Nov 13, 2019 at 7:04 PM Asif Rehman wrote:
>
> Sorry, I sent the wrong patches. Please see the correct version of the
> patches (_v6).
>
Review comments on these patches:
1.
+XLogRecPtrwal_location;
Looking at the other field names in basebackup_options structure, let's use
On Fri, Nov 1, 2019 at 8:53 PM Robert Haas wrote:
> On Wed, Oct 30, 2019 at 10:16 AM Asif Rehman
> wrote:
> > 'startptr' is used by sendFile() during checksum verification. Since
> > SendBackupFiles() is using sendFIle we have to set a valid WAL location.
>
> Ugh, global variables.
>
> Why are
On Wed, Oct 30, 2019 at 10:16 AM Asif Rehman wrote:
> 'startptr' is used by sendFile() during checksum verification. Since
> SendBackupFiles() is using sendFIle we have to set a valid WAL location.
Ugh, global variables.
Why are START_BACKUP, SEND_BACKUP_FILELIST, SEND_BACKUP_FILES, and
On Mon, Oct 28, 2019 at 8:29 PM Robert Haas wrote:
> On Mon, Oct 28, 2019 at 10:03 AM Asif Rehman
> wrote:
> > I have updated the patch to include the changes suggested by Jeevan.
> This patch also implements the thread workers instead of
> > processes and fetches a single file at a time. The
On Mon, Oct 28, 2019 at 10:03 AM Asif Rehman wrote:
> I have updated the patch to include the changes suggested by Jeevan. This
> patch also implements the thread workers instead of
> processes and fetches a single file at a time. The tar format has been
> disabled for first version of parallel
On Thu, Oct 24, 2019 at 3:21 PM Ibrar Ahmed wrote:
>
>
> On Fri, Oct 18, 2019 at 4:12 PM Jeevan Chalke <
> jeevan.cha...@enterprisedb.com> wrote:
>
>>
>>
>> On Thu, Oct 17, 2019 at 10:51 AM Asif Rehman
>> wrote:
>>
>>>
>>> Attached are the updated patches.
>>>
>>
>> I had a quick look over
On Fri, Oct 18, 2019 at 4:12 PM Jeevan Chalke <
jeevan.cha...@enterprisedb.com> wrote:
>
>
> On Thu, Oct 17, 2019 at 10:51 AM Asif Rehman
> wrote:
>
>>
>> Attached are the updated patches.
>>
>
> I had a quick look over these changes and they look good overall.
> However, here are my few review
On Thu, Oct 17, 2019 at 10:51 AM Asif Rehman wrote:
>
> Attached are the updated patches.
>
I had a quick look over these changes and they look good overall.
However, here are my few review comments I caught while glancing the patches
0002 and 0003.
--- 0002 patch
1.
Can lsn option be
I quickly tried to have a look at your 0001-refactor patch.
Here are some comments:
1. The patch fails to compile.
Sorry if I am missing something, but am not able to understand why in new
function collectTablespaces() you have added an extra parameter NULL while
calling sendTablespace(), it
On Mon, Oct 7, 2019 at 6:35 PM Asif Rehman wrote:
>
>
> On Mon, Oct 7, 2019 at 6:05 PM Robert Haas wrote:
>
>> On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman
>> wrote:
>> > Sure. Though the backup manifest patch calculates and includes the
>> checksum of backup files and is done
>> > while the
On Mon, Oct 7, 2019 at 9:43 AM Ibrar Ahmed wrote:
> What about have an API to get the single file or list of files? We will use a
> single file in
> our application and other tools can get the benefit of list of files.
That sounds a bit speculative to me. Who is to say that anyone will
find
On Mon, Oct 7, 2019 at 6:06 PM Robert Haas wrote:
> On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman wrote:
> > Sure. Though the backup manifest patch calculates and includes the
> checksum of backup files and is done
> > while the file is being transferred to the frontend-end. The manifest
> file
On Mon, Oct 7, 2019 at 6:05 PM Robert Haas wrote:
> On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman wrote:
> > Sure. Though the backup manifest patch calculates and includes the
> checksum of backup files and is done
> > while the file is being transferred to the frontend-end. The manifest
> file
On Mon, Oct 7, 2019 at 8:48 AM Asif Rehman wrote:
> Sure. Though the backup manifest patch calculates and includes the checksum
> of backup files and is done
> while the file is being transferred to the frontend-end. The manifest file
> itself is copied at the
> very end of the backup. In
On Mon, Oct 7, 2019 at 1:52 PM Rushabh Lathia
wrote:
> Thanks Asif for the patch. I am opting this for a review. Patch is
> bit big, so here are very initial comments to make the review process
> easier.
>
Thanks Rushabh for reviewing the patch.
> 1) Patch seems doing lot of code shuffling,
Thanks Asif for the patch. I am opting this for a review. Patch is
bit big, so here are very initial comments to make the review process
easier.
1) Patch seems doing lot of code shuffling, I think it would be easy
to review if you can break the clean up patch separately.
Example:
a:
On Fri, Oct 4, 2019 at 7:02 AM Asif Rehman wrote:
> Based on my understanding your main concern is that the files won't be
> distributed fairly i.e one worker might get a big file and take more time
> while others get done early with smaller files? In this approach I have
> created a list of
On Thu, Oct 3, 2019 at 6:40 PM Robert Haas wrote:
> On Fri, Sep 27, 2019 at 12:00 PM Asif Rehman
> wrote:
> >> > - SEND_FILES_CONTENTS (file1, file2,...) - returns the files in given
> list.
> >> > pg_basebackup will then send back a list of filenames in this
> command. This commands will be
On Fri, Sep 27, 2019 at 12:00 PM Asif Rehman wrote:
>> > - SEND_FILES_CONTENTS (file1, file2,...) - returns the files in given list.
>> > pg_basebackup will then send back a list of filenames in this command.
>> > This commands will be send by each worker and that worker will be getting
>> >
Hi Asif,
I was looking at the patch and tried comipling it. However, got few errors
and warnings.
Fixed those in the attached patch.
On Fri, Sep 27, 2019 at 9:30 PM Asif Rehman wrote:
> Hi Robert,
>
> Thanks for the feedback. Please see the comments below:
>
> On Tue, Sep 24, 2019 at 10:53
Hi Robert,
Thanks for the feedback. Please see the comments below:
On Tue, Sep 24, 2019 at 10:53 PM Robert Haas wrote:
> On Wed, Aug 21, 2019 at 9:53 AM Asif Rehman
> wrote:
> > - BASE_BACKUP [PARALLEL] - returns a list of files in PGDATA
> > If the parallel option is there, then it will only
On Wed, Aug 21, 2019 at 9:53 AM Asif Rehman wrote:
> - BASE_BACKUP [PARALLEL] - returns a list of files in PGDATA
> If the parallel option is there, then it will only do pg_start_backup, scans
> PGDATA and sends a list of file names.
So IIUC, this would mean that BASE_BACKUP without PARALLEL
Greetings,
* Ahsan Hadi (ahsan.h...@gmail.com) wrote:
> On Fri, 23 Aug 2019 at 10:26 PM, Stephen Frost wrote:
> > I would expect you to quickly want to support compression on the server
> > side, before the data is sent across the network, and possibly
> > encryption, and so it'd likely make
On Fri, 23 Aug 2019 at 10:26 PM, Stephen Frost wrote:
> Greetings,
>
> * Asif Rehman (asifr.reh...@gmail.com) wrote:
> > On Fri, Aug 23, 2019 at 3:18 PM Asim R P wrote:
> > > Interesting proposal. Bulk of the work in a backup is transferring
> files
> > > from source data directory to
On Fri, Aug 23, 2019 at 10:26 PM Stephen Frost wrote:
> Greetings,
>
> * Asif Rehman (asifr.reh...@gmail.com) wrote:
> > On Fri, Aug 23, 2019 at 3:18 PM Asim R P wrote:
> > > Interesting proposal. Bulk of the work in a backup is transferring
> files
> > > from source data directory to
Greetings,
* Asif Rehman (asifr.reh...@gmail.com) wrote:
> On Fri, Aug 23, 2019 at 3:18 PM Asim R P wrote:
> > Interesting proposal. Bulk of the work in a backup is transferring files
> > from source data directory to destination. Your patch is breaking this
> > task down in multiple sets of
1 - 100 of 104 matches
Mail list logo
