Re: Assistance with doveadm backup...

2019-03-11 Thread Natu via dovecot 
Well for the purpose of upgrading your server, you could do the following:

 1. Rsync your maildir directories to a backup directory
 2. repeat rsync again so most of your changed files have been synced
 3. shut down your dovecot and smtp servers
 4. rsync one more time - should be relatively fast
 5. restart your dovecot and smtp servers

Otherwise, just schedule downtime and use whatever backup method you
prefer. You will probably want to backup and then leave dovecot and smtp
server down until you've completed the dovecot upgrade anyway, so you
may as well schedule downtime for the entire process and do the dovecot
upgrade between steps 4 and 5 above.

Natu




On 3/10/19 10:07 PM, Aki Tuomi via dovecot wrote:
> I'd say the correct term is not broken but missing. backup was
> implemented in later version. 
>
> With maildir keeping a copy of the content is sufficient backup. 
>
> Aki
>> On 11 March 2019 03:39 SH Development via dovecot
>>  wrote:
>>
>>
>> Ok, so for lack of any other information, am I to conclude that the
>> doveadm backup command is broken in 2.0.9?  I have been through the
>> wiki for it back and forth and cannot make it work.
>>
>> Yes, I need to upgrade, there is no question, but I won’t do that
>> without a maildir backup.  I would prefer to use the built-in dovecot
>> mechanisms for that than something else.
>>
>> Jeff
>>
>>> On Mar 10, 2019, at 12:09 PM, SH Development via dovecot <
>>> dovecot@dovecot.org > wrote:
>>> Okay then.  Is there a document on how to do that in place?  That
>>> was one of the reasons I was trying to do a backup first.
>>>
>>> Jeff
>>>
>>>
 On Mar 10, 2019, at 4:12 AM, Aki Tuomi via dovecot <
 dovecot@dovecot.org > wrote:
 You should upgrade 2.2 first.

 Aki
> On 10 March 2019 10:45 SH Development via dovecot <
> dovecot@dovecot.org > wrote:
>
>
> V 2.0.9
>
> Jeff
>
>> On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot <
>> dovecot@dovecot.org > wrote:
>> What version are you using?
>>
>> Aki
>>> On 10 March 2019 00:40 SH Development via dovecot <
>>> dovecot@dovecot.org > wrote:
>>>
>>>
>>> Okay, apparently I’m just a complete idiot. Why doesn’t this work?
>>>
>>> doveadm -Dv backup -u user
>>> Maildir:/mnt/maelstrombackups/vmailbackup
>>>
>>> With user being the actual user name.
>>>
>>> No backup is created, it just shows me the usage text.
>>>
>>> Jeff
>>>
 On Feb 20, 2019, at 10:11 PM, SH Development <
 listacco...@starionline.com
 > wrote:

 I am having trouble locating examples of how to use doveadm
 backup. All the examples I see are for sync. I simply want to
 create a backup to a network volume of the email server's vmail
 folders. The goal here is to have a reasonably current backup
 should the main drive on the email server go south.

 We currently authenticate our users from a mysql database.
 User’s mailboxes are stored as domainname/username/Maildir

 I assume what I will wind up on the network volume is a
 duplicate directory structure as the vmail folder on the email
 server?

 Can someone help get me started here?

 Jeff
>>
>> ---
>> Aki Tuomi
>

 ---
 Aki Tuomi
>>>
>>
>
> ---
> Aki Tuomi

Re: Assistance with doveadm backup...

2019-03-10 Thread Aki Tuomi via dovecot 


 
 
  
   I'd say the correct term is not broken but missing. backup was implemented in later version. 
  
  
   
  
  
   With maildir keeping a copy of the content is sufficient backup. 
  
  
   
  
  
   Aki
  
  
   
On 11 March 2019 03:39 SH Development via dovecot  wrote:
   
   

   
   

   Ok, so for lack of any other information, am I to conclude that the doveadm backup command is broken in 2.0.9?  I have been through the wiki for it back and forth and cannot make it work.
   

   
   
Yes, I need to upgrade, there is no question, but I won’t do that without a maildir backup.  I would prefer to use the built-in dovecot mechanisms for that than something else.
   
   

   
   
Jeff


 
 
  
   On Mar 10, 2019, at 12:09 PM, SH Development via dovecot <
   dovecot@dovecot.org> wrote:
  
  
   
Okay then.  Is there a document on how to do that in place?  That was one of the reasons I was trying to do a backup first.

 


 Jeff


 
 
  
  
   
On Mar 10, 2019, at 4:12 AM, Aki Tuomi via dovecot <
dovecot@dovecot.org> wrote:
   
   

 
  You should upgrade 2.2 first.
 
 
  
 
 
  Aki
 
 
  
   On 10 March 2019 10:45 SH Development via dovecot <
   dovecot@dovecot.org> wrote:
  
  
   
  
  
   
  V 2.0.9
  
   
  
  
   Jeff 
   
   


 
  On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot < 
  dovecot@dovecot.org> wrote:
 
 
  
   
What version are you using?
   
   

   
   
Aki
   
   

 On 10 March 2019 00:40 SH Development via dovecot < 
 dovecot@dovecot.org> wrote:


 


 


 Okay, apparently I’m just a complete idiot. Why doesn’t this work?


 


 doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup


 


 With user being the actual user name.


 


 No backup is created, it just shows me the usage text.


 


 Jeff


 


 
  On Feb 20, 2019, at 10:11 PM, SH Development < 
  listacco...@starionline.com> wrote:
 
 
  
 
 
  I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.
 
 
  
 
 
  We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir
 
 
  
 
 
  I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?
 
 
  
 
 
  Can someone help get me started here?
 
 
  
 
 
  Jeff

Re: Assistance with doveadm backup...

2019-03-10 Thread SH Development via dovecot 
Ok, so for lack of any other information, am I to conclude that the doveadm 
backup command is broken in 2.0.9?  I have been through the wiki for it back 
and forth and cannot make it work.

Yes, I need to upgrade, there is no question, but I won’t do that without a 
maildir backup.  I would prefer to use the built-in dovecot mechanisms for that 
than something else.

Jeff

> On Mar 10, 2019, at 12:09 PM, SH Development via dovecot  > wrote:
> 
> Okay then.  Is there a document on how to do that in place?  That was one of 
> the reasons I was trying to do a backup first.
> 
> Jeff
> 
> 
>> On Mar 10, 2019, at 4:12 AM, Aki Tuomi via dovecot > > wrote:
>> 
>> You should upgrade 2.2 first.
>> 
>> Aki
>>> On 10 March 2019 10:45 SH Development via dovecot >> > wrote:
>>> 
>>> 
>>> V 2.0.9
>>> 
>>> Jeff 
>>> 
 On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot < dovecot@dovecot.org 
 > wrote:
 What version are you using?
 
 Aki
> On 10 March 2019 00:40 SH Development via dovecot < dovecot@dovecot.org 
> > wrote:
> 
> 
> Okay, apparently I’m just a complete idiot. Why doesn’t this work?
> 
> doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup
> 
> With user being the actual user name.
> 
> No backup is created, it just shows me the usage text.
> 
> Jeff
> 
>> On Feb 20, 2019, at 10:11 PM, SH Development < 
>> listacco...@starionline.com > wrote:
>> 
>> I am having trouble locating examples of how to use doveadm backup. All 
>> the examples I see are for sync. I simply want to create a backup to a 
>> network volume of the email server's vmail folders. The goal here is to 
>> have a reasonably current backup should the main drive on the email 
>> server go south.
>> 
>> We currently authenticate our users from a mysql database. User’s 
>> mailboxes are stored as domainname/username/Maildir
>> 
>> I assume what I will wind up on the network volume is a duplicate 
>> directory structure as the vmail folder on the email server?
>> 
>> Can someone help get me started here?
>> 
>> Jeff
 
 ---
 Aki Tuomi
>>> 
>> 
>> ---
>> Aki Tuomi
>

Re: Assistance with doveadm backup...

2019-03-10 Thread SH Development via dovecot 
Okay then.  Is there a document on how to do that in place?  That was one of 
the reasons I was trying to do a backup first.

Jeff


> On Mar 10, 2019, at 4:12 AM, Aki Tuomi via dovecot  > wrote:
> 
> You should upgrade 2.2 first.
> 
> Aki
>> On 10 March 2019 10:45 SH Development via dovecot > > wrote:
>> 
>> 
>> V 2.0.9
>> 
>> Jeff 
>> 
>>> On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot < dovecot@dovecot.org 
>>> > wrote:
>>> What version are you using?
>>> 
>>> Aki
 On 10 March 2019 00:40 SH Development via dovecot < dovecot@dovecot.org 
 > wrote:
 
 
 Okay, apparently I’m just a complete idiot. Why doesn’t this work?
 
 doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup
 
 With user being the actual user name.
 
 No backup is created, it just shows me the usage text.
 
 Jeff
 
> On Feb 20, 2019, at 10:11 PM, SH Development < 
> listacco...@starionline.com > wrote:
> 
> I am having trouble locating examples of how to use doveadm backup. All 
> the examples I see are for sync. I simply want to create a backup to a 
> network volume of the email server's vmail folders. The goal here is to 
> have a reasonably current backup should the main drive on the email 
> server go south.
> 
> We currently authenticate our users from a mysql database. User’s 
> mailboxes are stored as domainname/username/Maildir
> 
> I assume what I will wind up on the network volume is a duplicate 
> directory structure as the vmail folder on the email server?
> 
> Can someone help get me started here?
> 
> Jeff
>>> 
>>> ---
>>> Aki Tuomi
>> 
> 
> ---
> Aki Tuomi

Re: Assistance with doveadm backup...

2019-03-10 Thread Aki Tuomi via dovecot 


 
 
  
   You should upgrade 2.2 first.
  
  
   
  
  
   Aki
  
  
   
On 10 March 2019 10:45 SH Development via dovecot  wrote:
   
   

   
   

   V 2.0.9
   

   
   
Jeff


 
 
  
   On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot <
   dovecot@dovecot.org> wrote:
  
  
   

 What version are you using?


 


 Aki


 
  On 10 March 2019 00:40 SH Development via dovecot < 
  dovecot@dovecot.org> wrote:
 
 
  
 
 
  
 
 
  Okay, apparently I’m just a complete idiot. Why doesn’t this work?
 
 
  
 
 
  doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup
 
 
  
 
 
  With user being the actual user name.
 
 
  
 
 
  No backup is created, it just shows me the usage text.
 
 
  
 
 
  Jeff
 
 
  
 
 
  
   On Feb 20, 2019, at 10:11 PM, SH Development < 
   listacco...@starionline.com> wrote:
  
  
   
  
  
   I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.
  
  
   
  
  
   We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir
  
  
   
  
  
   I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?
  
  
   
  
  
   Can someone help get me started here?
  
  
   
  
  
   Jeff
  
 


 


 ---
Aki Tuomi

   
  
 


   
  
  
   
  
  
   ---
Aki Tuomi

Re: Assistance with doveadm backup...

2019-03-10 Thread SH Development via dovecot 
V 2.0.9

Jeff

> On Mar 10, 2019, at 12:07 AM, Aki Tuomi via dovecot  > wrote:
> 
> What version are you using?
> 
> Aki
>> On 10 March 2019 00:40 SH Development via dovecot < dovecot@dovecot.org 
>> > wrote:
>> 
>> 
>> Okay, apparently I’m just a complete idiot. Why doesn’t this work?
>> 
>> doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup
>> 
>> With user being the actual user name.
>> 
>> No backup is created, it just shows me the usage text.
>> 
>> Jeff
>> 
>>> On Feb 20, 2019, at 10:11 PM, SH Development < listacco...@starionline.com 
>>> > wrote:
>>> 
>>> I am having trouble locating examples of how to use doveadm backup. All the 
>>> examples I see are for sync. I simply want to create a backup to a network 
>>> volume of the email server's vmail folders. The goal here is to have a 
>>> reasonably current backup should the main drive on the email server go 
>>> south.
>>> 
>>> We currently authenticate our users from a mysql database. User’s mailboxes 
>>> are stored as domainname/username/Maildir
>>> 
>>> I assume what I will wind up on the network volume is a duplicate directory 
>>> structure as the vmail folder on the email server?
>>> 
>>> Can someone help get me started here?
>>> 
>>> Jeff
> 
> ---
> Aki Tuomi

Re: Assistance with doveadm backup...

2019-03-09 Thread Aki Tuomi via dovecot 


 
 
  
   What version are you using?
  
  
   
  
  
   Aki
  
  
   
On 10 March 2019 00:40 SH Development via dovecot <
dovecot@dovecot.org> wrote:
   
   

   
   

   
   
Okay, apparently I’m just a complete idiot. Why doesn’t this work?
   
   

   
   
doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup
   
   

   
   
With user being the actual user name.
   
   

   
   
No backup is created, it just shows me the usage text.
   
   

   
   
Jeff
   
   

   
   

 On Feb 20, 2019, at 10:11 PM, SH Development <
 listacco...@starionline.com> wrote:


 


 I am having trouble locating examples of how to use doveadm backup. All the examples I see are for sync. I simply want to create a backup to a network volume of the email server's vmail folders. The goal here is to have a reasonably current backup should the main drive on the email server go south.


 


 We currently authenticate our users from a mysql database. User’s mailboxes are stored as domainname/username/Maildir


 


 I assume what I will wind up on the network volume is a duplicate directory structure as the vmail folder on the email server?


 


 Can someone help get me started here?


 


 Jeff

   
  
  
   
  
  
   ---
Aki Tuomi

Re: Assistance with doveadm backup...

2019-03-09 Thread SH Development via dovecot 
Okay, apparently I’m just a complete idiot.  Why doesn’t this work?

doveadm -Dv backup -u user Maildir:/mnt/maelstrombackups/vmailbackup

With user being the actual user name.

No backup is created, it just shows me the usage text.

Jeff

> On Feb 20, 2019, at 10:11 PM, SH Development  
> wrote:
> 
> I am having trouble locating examples of how to use doveadm backup.  All the 
> examples I see are for sync.  I simply want to create a backup to a network 
> volume of the email server's vmail folders.  The goal here is to have a 
> reasonably current backup should the main drive on the email server go south.
> 
> We currently authenticate our users from a mysql database.  User’s mailboxes 
> are stored as domainname/username/Maildir
> 
> I assume what I will wind up on the network volume is a duplicate directory 
> structure as the vmail folder on the email server?
> 
> Can someone help get me started here?
> 
> Jeff

Re: Assistance with doveadm backup...

2019-02-22 Thread Odhiambo Washington via dovecot 
On Thu, 21 Feb 2019 at 07:11, SH Development via dovecot <
dovecot@dovecot.org> wrote:

> I am having trouble locating examples of how to use doveadm backup.  All
> the examples I see are for sync.  I simply want to create a backup to a
> network volume of the email server's vmail folders.  The goal here is to
> have a reasonably current backup should the main drive on the email server
> go south.
>
> We currently authenticate our users from a mysql database.  User’s
> mailboxes are stored as domainname/username/Maildir
>
> I assume what I will wind up on the network volume is a duplicate
> directory structure as the vmail folder on the email server?
>
> Can someone help get me started here?
>
> Jeff


In my previous life as a SysAdmin, I solved this kind of problem easily
using the MTA.
Exim is my MTA of choice and has a facility to do concurrent delivery to
two storage locations using shadow_transport.
I am guessing your MTA is Postfix (or maybe Sendmail). Try and see if they
have such a capability or switch to Exim.



-- 
Best regards,
Odhiambo WASHINGTON,
Nairobi,KE
+254 7 3200 0004/+254 7 2274 3223
"Oh, the cruft.", grep ^[^#] :-)

Re: Assistance with doveadm backup...

2019-02-21 Thread Joseph Tam via dovecot 

On Wed, 20 Feb 2019, SH Development wrote:


To: Joan Moreau via dovecot 


Jean did take over the list for a while when developing his FTS backend,
so you can be forgiven that he actually runs this list.


2.  It was also suggested to rsync the directory, but the question was
brought up, and not answered, about whether it was advisable to copy
live mail, thus the need for doveadm sync/backup.


You're right that consistency is a problem if you rsync a large amount of
changing data.  (If I remember correctly, mdbox is especially sensitive to
content/index mismatches.)

One way this can be done is via filesystem magic i.e. filesystems with
copy-on-write that allow you to take a moment in time snapshot of your
entire filesystem e.g. LVM, ZFS and others.  You can then rsync the
snapshot, but some of these filesystem also support methods to export
snapshots to a remote filesystem.

Joseph Tam

Re: Assistance with doveadm backup...

2019-02-20 Thread SH Development via dovecot 
Here is my command:

[root@mail maelstrombackups] #mkdir -p vmailbackup
[root@mail vmail]# chown: vmail:vmail /mnt/maelstrombackups/vmailbackup
[root@mail vmail]# doveadm backup -u jeff 
maildir:/mnt/maelstrombackups/vmailbackup

Only gives me the usage list, doesn’t work.

Jeff

> On Feb 21, 2019, at 1:29 AM, Sami Ketola  wrote:
> 
> 
> 
>> On 21 Feb 2019, at 7.04, SH Development via dovecot  
>> wrote:
>> 
>> Not really.  The suggestions in the post seemed to be:
>> 
>> 1. Tar the whole directory, which I have tried in the past, but a 130G 
>> directory takes over 18 hours to tar.
>> 2. It was also suggested to rsync the directory, but the question was 
>> brought up, and not answered, about whether it was advisable to copy live 
>> mail, thus the need for doveadm sync/backup.
>> 
>> Based on what I’ve read about doveadm backup, this is what I need, I am just 
>> having a hard time formulating the syntax for the command.
>> 
> 
> root@ketola ~]# mkdir -p /backup/sami
> root@ketola ~]# chown vmail:vmail /backup/sami
> root@ketola ~]# doveadm backup -u sami Maildir:/backup/sami
> 
> root@ketola ~]# du -hs /backup/sami
> 211M /backup/sami
> 
> does this help?
> 
> Sami

Re: Assistance with doveadm backup...

2019-02-20 Thread Sami Ketola via dovecot 



> On 21 Feb 2019, at 7.04, SH Development via dovecot  
> wrote:
> 
> Not really.  The suggestions in the post seemed to be:
> 
> 1. Tar the whole directory, which I have tried in the past, but a 130G 
> directory takes over 18 hours to tar.
> 2. It was also suggested to rsync the directory, but the question was brought 
> up, and not answered, about whether it was advisable to copy live mail, thus 
> the need for doveadm sync/backup.
> 
> Based on what I’ve read about doveadm backup, this is what I need, I am just 
> having a hard time formulating the syntax for the command.


root@ketola ~]# mkdir -p /backup/sami
root@ketola ~]# chown vmail:vmail /backup/sami
root@ketola ~]# doveadm backup -u sami Maildir:/backup/sami

root@ketola ~]# du -hs /backup/sami
211M /backup/sami

does this help?

Sami

Re: Assistance with doveadm backup...

2019-02-20 Thread SH Development via dovecot 
Sorry for being such a newbie when it comes to scripting.  Created a directory 
for the backups on the network mounted volume.  Changed the appropriate path 
and am getting this:

dsync(root): Error: user nfsnobody: Initialization failed: Namespace 'INBOX.': 
mkdir(/mnt/maelstrombackups/vmailbackup/nfsnobody) failed: Permission denied 
(euid=5000(vmail) egid=5000(vmail) missing +w perm: 
/mnt/maelstrombackups/vmailbackup, euid is not dir owner)
dsync(root): Fatal: User init failed
Backup done for user: nfsnobody ...
dsync(root): Error: user vmail: Initialization failed: Namespace 'INBOX.': 
mkdir(/mnt/maelstrombackups/vmailbackup/vmail) failed: Permission denied 
(euid=5000(vmail) egid=5000(vmail) missing +w perm: 
/mnt/maelstrombackups/vmailbackup, euid is not dir owner)
dsync(root): Fatal: User init failed
Backup done for user: vmail …

After running the script, I suddenly have two new directories in my vmail 
folder, “nfsnobody” and “vmail”.  What are these users?

Jeff


> On Feb 20, 2019, at 10:32 PM, Kenren Taisho  wrote:
> 
> No worries, that simple script should get you to a start though. Just replace 
> the path with a mounted NFS share if you want it to reside in a network 
> volume.
> 
> On Thu, Feb 21, 2019 at 12:10 PM SH Development  
> wrote:
> My apologies, you are absolutely right.  Will post this in the correct forum.
> 
> Spent the last couple of hours poring through the postfix list and just 
> forgot where I was at.
> 
> Jeff
> 
> 
> 
>> On Feb 20, 2019, at 8:55 PM, Kenren Taisho  wrote:
>> 
>> Hi Jeff,
>> 
>> This should be better off to dovecot's mailing list. 
>> But just to get you started, this is the first thing that came to mind:
>> 
>> example:
>> 
>> for users in `doveadm user "*"`; do
>> LOCALPART=${users%%@*}
>> dsync -v -f -u $users backup maildir:/tmp/backup/$LOCALPART
>> echo "Backup done for user: $users ..." 
>> done
>> 
>> HTH
>> 
>> 
>> On Thu, Feb 21, 2019 at 9:55 AM SH Development  
>> wrote:
>> I am having trouble locating examples of how to use doveadm backup.  All the 
>> examples I see are for sync.  I simply want to create a backup to a network 
>> volume of the email server's vmail folders.  The goal here is to have a 
>> reasonably current backup should the main drive on the email server go south.
>> 
>> We currently authenticate our users from a mysql database.  User’s mailboxes 
>> are stored as domainname/username/Maildir
>> 
>> I assume what I will wind up on the network volume is a duplicate directory 
>> structure as the vmail folder on the email server?
>> 
>> Can someone help get me started here?
>> 
>> Jeff
>

Re: Assistance with doveadm backup...

2019-02-20 Thread SH Development via dovecot 
Not really.  The suggestions in the post seemed to be:

1. Tar the whole directory, which I have tried in the past, but a 130G 
directory takes over 18 hours to tar.
2. It was also suggested to rsync the directory, but the question was brought 
up, and not answered, about whether it was advisable to copy live mail, thus 
the need for doveadm sync/backup.

Based on what I’ve read about doveadm backup, this is what I need, I am just 
having a hard time formulating the syntax for the command.

Jeff

> On Feb 20, 2019, at 10:45 PM, @lbutlr via dovecot  wrote:
> 
> On Feb 20, 2019, at 21:11, SH Development via dovecot  > wrote:
>> Can someone help get me started here?
> 
> Does this help?
> 
> https://unix.stackexchange.com/questions/489331/what-is-the-correct-way-to-use-doveadm-to-backup-emails
>  
> 
> 
> -- 
> This is my signature. There are many like it, but this one is mine.
>

Re: Assistance with doveadm backup...

2019-02-20 Thread @lbutlr via dovecot 
On Feb 20, 2019, at 21:11, SH Development via dovecot  
wrote:
> Can someone help get me started here?

Does this help?

https://unix.stackexchange.com/questions/489331/what-is-the-correct-way-to-use-doveadm-to-backup-emails

-- 
This is my signature. There are many like it, but this one is mine.