Re: [CentOS] Backup KVM Guest VM in OVA or VMDK format
On Tue, Sep 20, 2022 at 3:38 PM Ondřej Budai wrote:
> Hi Kaushal,
>
> st 14. 9. 2022 v 16:07 odesílatel Kaushal Shriyan <
> kaushalshri...@gmail.com>
> napsal:
>
> > On Fri, Sep 2, 2022 at 5:41 PM Fabian Arrotin wrote:
> >
> > > On 01/09/2022 18:14, Kaushal Shriyan wrote:
> > > > Hi,
> > > >
> > > > Is there a way to backup KVM Guest VM running CentOS Linux release
> > > 7.9.2009
> > > > (Core) OS in kvmguestosimage.ova or kvmguestosimage.vmdk format as I
> am
> > > > trying to restore it in AWS by referring to
> > > > https://aws.amazon.com/ec2/vm-import/ article as per the below
> > supported
> > > > file format.
> > > >
> > > > [1] Open Virtualization Archive (OVA)
> > > > [2] Virtual Machine Disk (VMDK)
> > > > [3] Virtual Hard Disk (VHD/VHDX)
> > > > [4] raw
> > > >
> > > > Also any method to take full and incremental backup of KVM Guest VM.
> > > >
> > > > Any help will be highly appreciated. I look forward to hearing from
> > you.
> > > > Thanks in Advance.
> > > >
> > > > Best Regards,
> > > >
> > > > Kaushal
> > >
> > > Stop the vm
> > > qemu-img convert -f raw origin.qcow2 dest.raw
> > >
> > > You can then import but while we use this to create official centos
> > > image, don't forget to ensure that you node is ready to be imported, so
> > > cloud-init, etc, etc
> > >
> > > It's usually easier/better/faster to have automation in place to
> > > configure an application and so replay it on a new node, and then
> > > replicate data
> > >
> > > I guess only option why you'd want to not do this is that it's a
> running
> > > machine that was configured "by hands" by someone who left the company
> > > (and so without automation in place)
> > >
> > > --
> > > Fabian Arrotin
> > > The CentOS Project | https://www.centos.org
> > > gpg key: 17F3B7A1 | twitter: @arrfab
> > > ___
> > > CentOS mailing list
> > > CentOS@centos.org
> > > https://lists.centos.org/mailman/listinfo/centos
> >
> >
> > Thanks Fabian for the detailed email. I followed the below steps by
> > referring to
> >
> >
> https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html
> > .
> >
> > # qemu-img -h | grep Supported
> > Supported formats: blkdebug blklogwrites blkverify compress
> > copy-before-write copy-on-read file ftp ftps gluster host_cdrom
> host_device
> > http https iscsi iser luks nbd null-aio null-co nvme preallocate qcow2
> > quorum raw rbd ssh throttle vhdx vmdk vpc
> >
> > # qemu-img --version
> > qemu-img version 6.2.0 (qemu-kvm-6.2.0-12.module_el8.7.0+1140+ff0772f9)
> > Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
> > #
> >
> > *Step No. 1*
> > #qemu-img convert -O vmdk openapibox.img openapibox.vmdk -p
> >
>
> I'm not 100% sure but I think that AWS only accepts the stream-optimized
> subformat, the command is:
>
> $ qemu-img convert -O vmdk -o subformat=streamOptimized openapibox.img
> openapibox.vmdk
>
>
> >
> > *Step No. 2*
> > #aws ec2 import-image --disk-containers
> > Format=vmdk,UserBucket="{S3Bucket=daclabservers,S3Key=openapidbox.vmdk}"
> > {
> > "ImportTaskId": "import-ami-0232f452194f6efe0",
> > "Progress": "1",
> > "SnapshotDetails": [
> > {
> > "DiskImageSize": 0.0,
> > "Format": "VMDK",
> > "UserBucket": {
> > "S3Bucket": "daclabservers",
> > "S3Key": "openapibox.vmdk"
> > }
> > }
> > ],
> > "Status": "active",
> > "StatusMessage": "pending"
> > }
> >
>
> Our project (https://www.osbuild.org/) uses the raw format for disks,
> uploads it to S3, calls import-snapshot to import it as an EBS snapshot and
> finally calls register-image to create a new AMI. Basically:
>
> $ qemu-img convert -O raw openapibox.img openapibox.raw
> # upload into S3
> $ aws ec2 import-snapshot ...
> # wait for the snapshot to be imported
> $ aws ec2 register-image ...
>
> Docs:
> - https://docs.aws.amazon.com/cli/latest/reference/ec2/register-image.html
> -
> https://docs.aws.amazon.com/cli/latest/reference/ec2/import-snapshot.html
> -
>
> https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-import-snapshot.html
>
> If you want to see this in practice, we have some Go code. As awscli is
> just a thin wrapper over the API, it should be pretty easy to translate our
> code into awscli calls:
>
> https://github.com/osbuild/osbuild-composer/blob/bfd90cf191eece5c1331dcb43a85bcca02d8d7d4/internal/cloud/awscloud/awscloud.go#L211
>
> Hope that helps,
>
> Ondřej
>
Thanks Ondřej and appreciate it. I will try it out and keep you posted.
Thanks in advance.
Best Regards,
Kaushal
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup KVM Guest VM in OVA or VMDK format
Hi Kaushal,
st 14. 9. 2022 v 16:07 odesílatel Kaushal Shriyan
napsal:
> On Fri, Sep 2, 2022 at 5:41 PM Fabian Arrotin wrote:
>
> > On 01/09/2022 18:14, Kaushal Shriyan wrote:
> > > Hi,
> > >
> > > Is there a way to backup KVM Guest VM running CentOS Linux release
> > 7.9.2009
> > > (Core) OS in kvmguestosimage.ova or kvmguestosimage.vmdk format as I am
> > > trying to restore it in AWS by referring to
> > > https://aws.amazon.com/ec2/vm-import/ article as per the below
> supported
> > > file format.
> > >
> > > [1] Open Virtualization Archive (OVA)
> > > [2] Virtual Machine Disk (VMDK)
> > > [3] Virtual Hard Disk (VHD/VHDX)
> > > [4] raw
> > >
> > > Also any method to take full and incremental backup of KVM Guest VM.
> > >
> > > Any help will be highly appreciated. I look forward to hearing from
> you.
> > > Thanks in Advance.
> > >
> > > Best Regards,
> > >
> > > Kaushal
> >
> > Stop the vm
> > qemu-img convert -f raw origin.qcow2 dest.raw
> >
> > You can then import but while we use this to create official centos
> > image, don't forget to ensure that you node is ready to be imported, so
> > cloud-init, etc, etc
> >
> > It's usually easier/better/faster to have automation in place to
> > configure an application and so replay it on a new node, and then
> > replicate data
> >
> > I guess only option why you'd want to not do this is that it's a running
> > machine that was configured "by hands" by someone who left the company
> > (and so without automation in place)
> >
> > --
> > Fabian Arrotin
> > The CentOS Project | https://www.centos.org
> > gpg key: 17F3B7A1 | twitter: @arrfab
> > ___
> > CentOS mailing list
> > CentOS@centos.org
> > https://lists.centos.org/mailman/listinfo/centos
>
>
> Thanks Fabian for the detailed email. I followed the below steps by
> referring to
>
> https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html
> .
>
> # qemu-img -h | grep Supported
> Supported formats: blkdebug blklogwrites blkverify compress
> copy-before-write copy-on-read file ftp ftps gluster host_cdrom host_device
> http https iscsi iser luks nbd null-aio null-co nvme preallocate qcow2
> quorum raw rbd ssh throttle vhdx vmdk vpc
>
> # qemu-img --version
> qemu-img version 6.2.0 (qemu-kvm-6.2.0-12.module_el8.7.0+1140+ff0772f9)
> Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
> #
>
> *Step No. 1*
> #qemu-img convert -O vmdk openapibox.img openapibox.vmdk -p
>
I'm not 100% sure but I think that AWS only accepts the stream-optimized
subformat, the command is:
$ qemu-img convert -O vmdk -o subformat=streamOptimized openapibox.img
openapibox.vmdk
>
> *Step No. 2*
> #aws ec2 import-image --disk-containers
> Format=vmdk,UserBucket="{S3Bucket=daclabservers,S3Key=openapidbox.vmdk}"
> {
> "ImportTaskId": "import-ami-0232f452194f6efe0",
> "Progress": "1",
> "SnapshotDetails": [
> {
> "DiskImageSize": 0.0,
> "Format": "VMDK",
> "UserBucket": {
> "S3Bucket": "daclabservers",
> "S3Key": "openapibox.vmdk"
> }
> }
> ],
> "Status": "active",
> "StatusMessage": "pending"
> }
>
Our project (https://www.osbuild.org/) uses the raw format for disks,
uploads it to S3, calls import-snapshot to import it as an EBS snapshot and
finally calls register-image to create a new AMI. Basically:
$ qemu-img convert -O raw openapibox.img openapibox.raw
# upload into S3
$ aws ec2 import-snapshot ...
# wait for the snapshot to be imported
$ aws ec2 register-image ...
Docs:
- https://docs.aws.amazon.com/cli/latest/reference/ec2/register-image.html
- https://docs.aws.amazon.com/cli/latest/reference/ec2/import-snapshot.html
-
https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-import-snapshot.html
If you want to see this in practice, we have some Go code. As awscli is
just a thin wrapper over the API, it should be pretty easy to translate our
code into awscli calls:
https://github.com/osbuild/osbuild-composer/blob/bfd90cf191eece5c1331dcb43a85bcca02d8d7d4/internal/cloud/awscloud/awscloud.go#L211
Hope that helps,
Ondřej
>
> *Step No. 3*
> #aws ec2 describe-import-image-tasks --import-task-ids
> import-ami-0232f452194f6efe0
> {
> "ImportImageTasks": [
> {
> "ImportTaskId": "import-ami-0232f452194f6efe0",
> "SnapshotDetails": [
> {
> "DiskImageSize": 0.0,
> "Status": "completed"
> }
> ],
> "Status": "deleted",
> "StatusMessage": "ClientError: Disk validation failed
> [Unsupported VMDK File Format]",
> "Tags": []
> }
> ]
> }
>
> Please guide me. Am I missing anything? Thanks in advance.
>
> Best Regards,
>
> Kaushal
> ___
> CentOS mailing list
> CentOS@centos.org
>
Re: [CentOS] Backup KVM Guest VM in OVA or VMDK format
On Wed, Sep 14, 2022 at 7:37 PM Kaushal Shriyan
wrote:
>
> On Fri, Sep 2, 2022 at 5:41 PM Fabian Arrotin wrote:
>
>> On 01/09/2022 18:14, Kaushal Shriyan wrote:
>> > Hi,
>> >
>> > Is there a way to backup KVM Guest VM running CentOS Linux release
>> 7.9.2009
>> > (Core) OS in kvmguestosimage.ova or kvmguestosimage.vmdk format as I am
>> > trying to restore it in AWS by referring to
>> > https://aws.amazon.com/ec2/vm-import/ article as per the below
>> supported
>> > file format.
>> >
>> > [1] Open Virtualization Archive (OVA)
>> > [2] Virtual Machine Disk (VMDK)
>> > [3] Virtual Hard Disk (VHD/VHDX)
>> > [4] raw
>> >
>> > Also any method to take full and incremental backup of KVM Guest VM.
>> >
>> > Any help will be highly appreciated. I look forward to hearing from you.
>> > Thanks in Advance.
>> >
>> > Best Regards,
>> >
>> > Kaushal
>>
>> Stop the vm
>> qemu-img convert -f raw origin.qcow2 dest.raw
>>
>> You can then import but while we use this to create official centos
>> image, don't forget to ensure that you node is ready to be imported, so
>> cloud-init, etc, etc
>>
>> It's usually easier/better/faster to have automation in place to
>> configure an application and so replay it on a new node, and then
>> replicate data
>>
>> I guess only option why you'd want to not do this is that it's a running
>> machine that was configured "by hands" by someone who left the company
>> (and so without automation in place)
>>
>> --
>> Fabian Arrotin
>> The CentOS Project | https://www.centos.org
>> gpg key: 17F3B7A1 | twitter: @arrfab
>> ___
>> CentOS mailing list
>> CentOS@centos.org
>> https://lists.centos.org/mailman/listinfo/centos
>
>
> Thanks Fabian for the detailed email. I followed the below steps by
> referring to
> https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html
> .
>
> # qemu-img -h | grep Supported
> Supported formats: blkdebug blklogwrites blkverify compress
> copy-before-write copy-on-read file ftp ftps gluster host_cdrom host_device
> http https iscsi iser luks nbd null-aio null-co nvme preallocate qcow2
> quorum raw rbd ssh throttle vhdx vmdk vpc
>
> # qemu-img --version
> qemu-img version 6.2.0 (qemu-kvm-6.2.0-12.module_el8.7.0+1140+ff0772f9)
> Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
> #
>
> *Step No. 1*
> #qemu-img convert -O vmdk openapibox.img openapibox.vmdk -p
>
> *Step No. 2*
> #aws ec2 import-image --disk-containers
> Format=vmdk,UserBucket="{S3Bucket=daclabservers,S3Key=openapidbox.vmdk}"
> {
> "ImportTaskId": "import-ami-0232f452194f6efe0",
> "Progress": "1",
> "SnapshotDetails": [
> {
> "DiskImageSize": 0.0,
> "Format": "VMDK",
> "UserBucket": {
> "S3Bucket": "daclabservers",
> "S3Key": "openapibox.vmdk"
> }
> }
> ],
> "Status": "active",
> "StatusMessage": "pending"
> }
>
> *Step No. 3*
> #aws ec2 describe-import-image-tasks --import-task-ids
> import-ami-0232f452194f6efe0
> {
> "ImportImageTasks": [
> {
> "ImportTaskId": "import-ami-0232f452194f6efe0",
> "SnapshotDetails": [
> {
> "DiskImageSize": 0.0,
> "Status": "completed"
> }
> ],
> "Status": "deleted",
> "StatusMessage": "ClientError: Disk validation failed
> [Unsupported VMDK File Format]",
> "Tags": []
> }
> ]
> }
>
> Please guide me. Am I missing anything? Thanks in advance.
>
> Best Regards,
>
> Kaushal
>
Hi,
I will appreciate it if someone can pitch in for my earlier post to this
mailing list and need guidance in this regard. I look forward to hearing
from you. Thanks in advance.
Best Regards,
Kaushal
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup KVM Guest VM in OVA or VMDK format
On Fri, Sep 2, 2022 at 5:41 PM Fabian Arrotin wrote:
> On 01/09/2022 18:14, Kaushal Shriyan wrote:
> > Hi,
> >
> > Is there a way to backup KVM Guest VM running CentOS Linux release
> 7.9.2009
> > (Core) OS in kvmguestosimage.ova or kvmguestosimage.vmdk format as I am
> > trying to restore it in AWS by referring to
> > https://aws.amazon.com/ec2/vm-import/ article as per the below supported
> > file format.
> >
> > [1] Open Virtualization Archive (OVA)
> > [2] Virtual Machine Disk (VMDK)
> > [3] Virtual Hard Disk (VHD/VHDX)
> > [4] raw
> >
> > Also any method to take full and incremental backup of KVM Guest VM.
> >
> > Any help will be highly appreciated. I look forward to hearing from you.
> > Thanks in Advance.
> >
> > Best Regards,
> >
> > Kaushal
>
> Stop the vm
> qemu-img convert -f raw origin.qcow2 dest.raw
>
> You can then import but while we use this to create official centos
> image, don't forget to ensure that you node is ready to be imported, so
> cloud-init, etc, etc
>
> It's usually easier/better/faster to have automation in place to
> configure an application and so replay it on a new node, and then
> replicate data
>
> I guess only option why you'd want to not do this is that it's a running
> machine that was configured "by hands" by someone who left the company
> (and so without automation in place)
>
> --
> Fabian Arrotin
> The CentOS Project | https://www.centos.org
> gpg key: 17F3B7A1 | twitter: @arrfab
> ___
> CentOS mailing list
> CentOS@centos.org
> https://lists.centos.org/mailman/listinfo/centos
Thanks Fabian for the detailed email. I followed the below steps by
referring to
https://docs.aws.amazon.com/vm-import/latest/userguide/vmimport-image-import.html
.
# qemu-img -h | grep Supported
Supported formats: blkdebug blklogwrites blkverify compress
copy-before-write copy-on-read file ftp ftps gluster host_cdrom host_device
http https iscsi iser luks nbd null-aio null-co nvme preallocate qcow2
quorum raw rbd ssh throttle vhdx vmdk vpc
# qemu-img --version
qemu-img version 6.2.0 (qemu-kvm-6.2.0-12.module_el8.7.0+1140+ff0772f9)
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers
#
*Step No. 1*
#qemu-img convert -O vmdk openapibox.img openapibox.vmdk -p
*Step No. 2*
#aws ec2 import-image --disk-containers
Format=vmdk,UserBucket="{S3Bucket=daclabservers,S3Key=openapidbox.vmdk}"
{
"ImportTaskId": "import-ami-0232f452194f6efe0",
"Progress": "1",
"SnapshotDetails": [
{
"DiskImageSize": 0.0,
"Format": "VMDK",
"UserBucket": {
"S3Bucket": "daclabservers",
"S3Key": "openapibox.vmdk"
}
}
],
"Status": "active",
"StatusMessage": "pending"
}
*Step No. 3*
#aws ec2 describe-import-image-tasks --import-task-ids
import-ami-0232f452194f6efe0
{
"ImportImageTasks": [
{
"ImportTaskId": "import-ami-0232f452194f6efe0",
"SnapshotDetails": [
{
"DiskImageSize": 0.0,
"Status": "completed"
}
],
"Status": "deleted",
"StatusMessage": "ClientError: Disk validation failed
[Unsupported VMDK File Format]",
"Tags": []
}
]
}
Please guide me. Am I missing anything? Thanks in advance.
Best Regards,
Kaushal
___
CentOS mailing list
CentOS@centos.org
https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup KVM Guest VM in OVA or VMDK format
On 01/09/2022 18:14, Kaushal Shriyan wrote: Hi, Is there a way to backup KVM Guest VM running CentOS Linux release 7.9.2009 (Core) OS in kvmguestosimage.ova or kvmguestosimage.vmdk format as I am trying to restore it in AWS by referring to https://aws.amazon.com/ec2/vm-import/ article as per the below supported file format. [1] Open Virtualization Archive (OVA) [2] Virtual Machine Disk (VMDK) [3] Virtual Hard Disk (VHD/VHDX) [4] raw Also any method to take full and incremental backup of KVM Guest VM. Any help will be highly appreciated. I look forward to hearing from you. Thanks in Advance. Best Regards, Kaushal Stop the vm qemu-img convert -f raw origin.qcow2 dest.raw You can then import but while we use this to create official centos image, don't forget to ensure that you node is ready to be imported, so cloud-init, etc, etc It's usually easier/better/faster to have automation in place to configure an application and so replay it on a new node, and then replicate data I guess only option why you'd want to not do this is that it's a running machine that was configured "by hands" by someone who left the company (and so without automation in place) -- Fabian Arrotin The CentOS Project | https://www.centos.org gpg key: 17F3B7A1 | twitter: @arrfab OpenPGP_signature Description: OpenPGP digital signature ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
[CentOS] Backup KVM Guest VM in OVA or VMDK format
Hi, Is there a way to backup KVM Guest VM running CentOS Linux release 7.9.2009 (Core) OS in kvmguestosimage.ova or kvmguestosimage.vmdk format as I am trying to restore it in AWS by referring to https://aws.amazon.com/ec2/vm-import/ article as per the below supported file format. [1] Open Virtualization Archive (OVA) [2] Virtual Machine Disk (VMDK) [3] Virtual Hard Disk (VHD/VHDX) [4] raw Also any method to take full and incremental backup of KVM Guest VM. Any help will be highly appreciated. I look forward to hearing from you. Thanks in Advance. Best Regards, Kaushal ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup Suggestion on C7
Hi Martin, this is the off-site backup. Each server is backupped also in farm. Il 13/10/2016 22:24, J Martin Rushton ha scritto: I'm not a bacula expert, but have had 30+ years in the industry doing backups. I'm a little concerned about what you are planning. As I understand it you are going to be keeping just one copy of each machine on a disk attached to the server. This will help if you loose the running disks (though it is hardly backup in depth), but what happens if you loose the server due to fire, flood, electrical problems, theft or even plain old dropping it? In general you should aim for multiple backup copies; are you willing to bet the company's future on one untried copy? You should ensure that the backup copies are held preferably off site, failing that in a separate building, or else in a secure fireproof strongbox. Start by assuming you come into work one day to find the building burnt out and collapsed. Now work out how to rebuild your system on new kit on another site and you'll find that you define your backup needs. Regards, Martin On 12/10/16 12:54, Alessandro Baggi wrote: Hi list, I'm building a backup server for 3 hosts (1 workstation, 2 server). I will use bacula to perform backups. The backup is performed on disks (2 x 3TB on mdraid mirror) and for each hosts I've created a logical volume with related size. This 3 hosts have different data size with different disk change rate. Each host must have a limited sized resource and a reserved space. If a server needs more space to perform backup, It must be enabled and provisioned. My first solution was put each host pools on different logical volumes, like: host1 -> lv1 host2 -> lv2 host3 -> lv3 and store pools/volumes on specified storage daemon that uses a specified device for each different hosts. host1 -> storage1 -> device_lv1 host2 -> storage2 -> device_lv2 host3 -> storage3 -> device_lv3 Unfortunately, I can't define on bacula-sd.conf multiple storage definition but only multiple devices. To use different storage I must run 3 bacula-sd on same host (I can?), run a bacula-sd on a vm/host. Ah, I must use only one physical server. With one single machine and the current state I can't use multiple storage daemons. There are other ways to store host volumes on different devices? My second solution was, use only one storage daemon (on the same host) with a single device LVM over mdraid, create pool for each hosts and limit the size for each volumes on related pool. Suggestions? Thanks in advance. ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup Suggestion on C7
Mark has asked me to forward this to the list: -%<- Subject: Re: [CentOS] Backup Suggestion on C7 Date: Thu, 13 Oct 2016 17:09:41 -0400 From: m.r...@5-cent.us To: J Martin Rushton <martinrushto...@btinternet.com> Please forward to the CentOS list - my hosting provider claims they been working with SORBS, but it's now been blocking me from posting for two weeks J Martin Rushton wrote: > I'm not a bacula expert, but have had 30+ years in the industry doing > backups. I'm a little concerned about what you are planning. As I > understand it you are going to be keeping just one copy of each machine > on a disk attached to the server. This will help if you loose the > running disks (though it is hardly backup in depth), but what happens if > you loose the server due to fire, flood, electrical problems, theft or > even plain old dropping it? In general you should aim for multiple > backup copies; are you willing to bet the company's future on one > untried copy? You should ensure that the backup copies are held > preferably off site, failing that in a separate building, or else in a > secure fireproof strongbox. To start, is the OP doing disaster recovery backups, or archive backups? The difference is the former you only keep for a limited amount of time, and the later forever. If the former: first, you should not be backing up one server to its own disks. The backup should reside on a different server. Secondly, consider an offline backup. Here at work, we use a home-grown rsync solution (with hard links), and typically save those for five weeks. We also back up the backups to either offline disks (mounted in hot swap bays, and they reside the rest of the time in a fire safe; for things that one fit on one disk, they're backed up to a server in another building with a large RAID. For the latter, which I have not been involved with, you should probably have x weeks, then save the weeklies for x months, then the montlies for x years. And *ALL* of that should be offsite. mark, hoping this gets through --%< signature.asc Description: OpenPGP digital signature ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup Suggestion on C7
I'm not a bacula expert, but have had 30+ years in the industry doing backups. I'm a little concerned about what you are planning. As I understand it you are going to be keeping just one copy of each machine on a disk attached to the server. This will help if you loose the running disks (though it is hardly backup in depth), but what happens if you loose the server due to fire, flood, electrical problems, theft or even plain old dropping it? In general you should aim for multiple backup copies; are you willing to bet the company's future on one untried copy? You should ensure that the backup copies are held preferably off site, failing that in a separate building, or else in a secure fireproof strongbox. Start by assuming you come into work one day to find the building burnt out and collapsed. Now work out how to rebuild your system on new kit on another site and you'll find that you define your backup needs. Regards, Martin On 12/10/16 12:54, Alessandro Baggi wrote: > Hi list, > I'm building a backup server for 3 hosts (1 workstation, 2 server). I > will use bacula to perform backups. The backup is performed on disks (2 > x 3TB on mdraid mirror) and for each hosts I've created a logical volume > with related size. > > This 3 hosts have different data size with different disk change rate. > Each host must have a limited sized resource and a reserved space. If a > server needs more space to perform backup, It must be enabled and > provisioned. > > My first solution was put each host pools on different logical volumes, > like: > > host1 -> lv1 > host2 -> lv2 > host3 -> lv3 > > and store pools/volumes on specified storage daemon that uses a > specified device for each different hosts. > > host1 -> storage1 -> device_lv1 > host2 -> storage2 -> device_lv2 > host3 -> storage3 -> device_lv3 > > > Unfortunately, I can't define on bacula-sd.conf multiple storage > definition but only multiple devices. To use different storage I must > run 3 bacula-sd on same host (I can?), run a bacula-sd on a vm/host. > Ah, I must use only one physical server. > > With one single machine and the current state I can't use multiple > storage daemons. > > There are other ways to store host volumes on different devices? > > My second solution was, use only one storage daemon (on the same host) > with a single device LVM over mdraid, create pool for each hosts and > limit the size for each volumes on related pool. > > Suggestions? > > Thanks in advance. > > > > > > ___ > CentOS mailing list > CentOS@centos.org > https://lists.centos.org/mailman/listinfo/centos signature.asc Description: OpenPGP digital signature ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
[CentOS] Backup Suggestion on C7
Hi list, I'm building a backup server for 3 hosts (1 workstation, 2 server). I will use bacula to perform backups. The backup is performed on disks (2 x 3TB on mdraid mirror) and for each hosts I've created a logical volume with related size. This 3 hosts have different data size with different disk change rate. Each host must have a limited sized resource and a reserved space. If a server needs more space to perform backup, It must be enabled and provisioned. My first solution was put each host pools on different logical volumes, like: host1 -> lv1 host2 -> lv2 host3 -> lv3 and store pools/volumes on specified storage daemon that uses a specified device for each different hosts. host1 -> storage1 -> device_lv1 host2 -> storage2 -> device_lv2 host3 -> storage3 -> device_lv3 Unfortunately, I can't define on bacula-sd.conf multiple storage definition but only multiple devices. To use different storage I must run 3 bacula-sd on same host (I can?), run a bacula-sd on a vm/host. Ah, I must use only one physical server. With one single machine and the current state I can't use multiple storage daemons. There are other ways to store host volumes on different devices? My second solution was, use only one storage daemon (on the same host) with a single device LVM over mdraid, create pool for each hosts and limit the size for each volumes on related pool. Suggestions? Thanks in advance. ___ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Sorin Srbu Sent: den 11 maj 2015 07:49 To: CentOS mailing list Subject: Re: [CentOS] Backup PC or other solution Main Config: Schedule: FullPeriod: 27.9 FullKeepCnt: 24 FullKeepCntMin: 8 FullAgeMax: 360 IncrPeriod: 0.97 IncrKeepCnt: 30 IncrKeepMin: 1 IncrAgeMax: 30 IncrLevels: 1 How did you get away with using 27,9 on Fullperiod? 8-) I'm seeing Error: No save due to errors and Error: FullPeriod must be a real-valued number, unless I change the value to e.g. 27. This is on BPC v3.2.1. -- //Sorin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of John R Pierce Sent: den 11 maj 2015 08:19 To: centos@centos.org Subject: Re: [CentOS] Backup PC or other solution On 5/10/2015 11:03 PM, Sorin Srbu wrote: How did you get away with using 27,9 on Fullperiod?8-) I'm seeing Error: No save due to errors and Error: FullPeriod must be a real-valued number, unless I change the value to e.g. 27. This is on BPC v3.2.1. 27.9 not 27,9 (point, not comma). expletive. Why can't everybody follow the standards and use a comma when writing decimals. ;-) Thanks for the heads-up! -- //Sorin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/10/2015 11:57 PM, Sorin Srbu wrote: expletive. Why can't everybody follow the standards and use a comma when writing decimals. our standard is a . comma is a 1000s seperator. thats the best part about standards, there are so many to choose from!! -- john r pierce, recycling bits in santa cruz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of John R Pierce Sent: den 11 maj 2015 09:25 To: centos@centos.org Subject: Re: [CentOS] Backup PC or other solution On 5/10/2015 11:57 PM, Sorin Srbu wrote: expletive. Why can't everybody follow the standards and use a comma when writing decimals. our standard is a . comma is a 1000s seperator. thats the best part about standards, there are so many to choose from!! Spot on. 8-D Thanks. -- //Sorin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/10/2015 11:03 PM, Sorin Srbu wrote: How did you get away with using 27,9 on Fullperiod?8-) I'm seeing Error: No save due to errors and Error: FullPeriod must be a real-valued number, unless I change the value to e.g. 27. This is on BPC v3.2.1. 27.9 not 27,9 (point, not comma). -- john r pierce, recycling bits in santa cruz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 7/5/2015 5:01 μμ, Robert Nichols wrote: I use rdiff-backup, but I hesitate to recommend a tool that has been unsupported for over 6 years and does have quite a few bugs. I have had good experience with mondrescue (mondoarchive, mondorestore) for years. It's a free, active project. See: http://www.mondorescue.org/ We are backing-up about 20 production servers (using cron jobs) weekly. Bare-metal recovery has been successful as well as cloning. Their mailing list is helpful and polite. I has saved my neck many times during the last 5 years. Although I have no experience with mondorescue on Centos 7, I recommend it at least for the other versions. Nick ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of John R Pierce Sent: den 8 maj 2015 17:12 To: centos@centos.org Subject: Re: [CentOS] Backup PC or other solution On 5/7/2015 11:44 PM, Sorin Srbu wrote: May I ask what your settings are to achieve that retention rate? there's a lot of settings... but these are probably applicable... Main Config: Schedule: FullPeriod: 27.9 FullKeepCnt: 24 FullKeepCntMin: 8 FullAgeMax: 360 IncrPeriod: 0.97 IncrKeepCnt: 30 IncrKeepMin: 1 IncrAgeMax: 30 IncrLevels: 1 on a few hosts where dailies are not appropriate due to how long they take, I override to do weekly incrementals instead Schedule: FullPeriod: 89.6 FullKeepCnt: 2 FullKeepCntMin: 2 IncrPeriod: 6.97 IncrKeepCnt: 15 IncrAgeMax: 100 many of those are probably defaults, but I didn't keep track which ones I modified another thing, many of my servers are SQL database servers (mostly postgresql and oracle). I do NOT backup the sql data file systems directly with backuppc, instead, I have the SQL do archiving or scheduled dumps, and I backup those archive and dump destinations... Thanks, very much appreciated! I'll play around with the settings a bit more, but yours is a good starter. Thanks again! -- //Sorin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of John R Pierce Sent: den 7 maj 2015 19:09 To: centos@centos.org Subject: Re: [CentOS] Backup PC or other solution my year of monthlies and month of dailies of 25 servers has been more or less constant size for a year or two now as it deletes the oldest backups. May I ask what your settings are to achieve that retention rate? Thanks. -- //Sorin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
-Original Message- From: centos-boun...@centos.org [mailto:centos-boun...@centos.org] On Behalf Of Timothy Murphy Sent: den 7 maj 2015 23:21 To: centos@centos.org Subject: Re: [CentOS] Backup PC or other solution The worst thing about BackupPC is the insane error message Unable to read 4 bytes, which comes up if anything is wrong. Possibly the worst error message anywhere? thats an rsync protocol message, and yeah, debugging connection/authentication issues is a bit ugly. I'm sure you are right. But I use rsync several times a day, and I have never received this message. It's not ugly, it is inexcusable. Yeah, well, but it's free. I'm not sure you can complain too much in that case. 8-) -- //Sorin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/7/2015 11:44 PM, Sorin Srbu wrote: May I ask what your settings are to achieve that retention rate? there's a lot of settings... but these are probably applicable... Main Config: Schedule: FullPeriod: 27.9 FullKeepCnt: 24 FullKeepCntMin: 8 FullAgeMax: 360 IncrPeriod: 0.97 IncrKeepCnt: 30 IncrKeepMin: 1 IncrAgeMax: 30 IncrLevels: 1 on a few hosts where dailies are not appropriate due to how long they take, I override to do weekly incrementals instead Schedule: FullPeriod: 89.6 FullKeepCnt: 2 FullKeepCntMin: 2 IncrPeriod: 6.97 IncrKeepCnt: 15 IncrAgeMax: 100 many of those are probably defaults, but I didn't keep track which ones I modified another thing, many of my servers are SQL database servers (mostly postgresql and oracle). I do NOT backup the sql data file systems directly with backuppc, instead, I have the SQL do archiving or scheduled dumps, and I backup those archive and dump destinations... -- john r pierce, recycling bits in santa cruz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On May 8, 2015, at 10:24 AM, James B. Byrne byrn...@harte-lyne.ca wrote: If a project is backed/picked up by a corporation, say Redhat or Oracle, or a foundation, say Apache or LibreOffice, then it may have a future more or less independent of any single individual or group. Commercial software and company-backed F/OSS software gets abandoned all the time. - OpenOffice may well die due to brain drain from LibreOffice. They’ve both got big corporate backers. - The MySQL mailing list is getting a tiny fraction of the traffic it once enjoyed before the Oracle takeover; MySQL won’t go away any time soon for reasons of inertia, but MariaDB and NoSQL are surely taking large bites out of its user base. - Remember ESD and aRTS? They’ve all but been killed off by PulseAudio. They were the “standard” of their time, backed by major Linux distributors. - How many “standard” window managers has GNOME had over the years? - How many desktop managers and GUI toolkits preceded GNOME/Gtk? NeWS, NeXTSTEP, CDE/Motif, Tk, all with big-name support in their day. - Adobe’s killed off dozens of products over the years. FrameMaker, Director, Flash Builder, PageMaker, Contribute, Fireworks… - Got a smartphone? How many apps have you bought that never went anywhere after they got your money? There’s more than one in my case, at least. At least with F/OSS, you have the option of taking over maintainership of an abandoned code base. My company has done that a few times now, as it was easier to do that than switch to the abandoned package’s replacement. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On Fri, May 8, 2015 07:59, Timothy Murphy wrote: Sorin Srbu wrote: Yeah, well, but it's free. I'm not sure you can complain too much in that case. 8-) I find this comment, often made, completely unacceptable. The implication is that inferior code is OK if the developer is not being paid. (Actually, the premise is probably nonsense, as most Linux developers _are_ paid, even if formally their pay is not specifically for Linux development. But presumably the company that pays them believes that it is of value to the company to have a Linux developer on board.) But is Linux code in fact inferior to code produced by Microsoft, say? I don't think so. And I don't think Linux developers are less keen to improve their code. Just the opposite. The difference is that a large portion of the FOSS corpus, if not a preponderant majority, is ultimately dependent upon the interest of the people responsible for its existance and not the people using it. Once a project's core team either loses enthusiasm for something, or have otherwise moved on in life, their project oft-times is left without any meaningful support. If a project is backed/picked up by a corporation, say Redhat or Oracle, or a foundation, say Apache or LibreOffice, then it may have a future more or less independent of any single individual or group. Otherwise it does not. -- *** e-Mail is NOT a SECURE channel *** Do NOT transmit sensitive data via e-Mail James B. Byrnemailto:byrn...@harte-lyne.ca Harte Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/8/2015 10:40 AM, Warren Young wrote: - Adobe’s killed off dozens of products over the years. FrameMaker ... Frame isn't dead, my wife is a technical writer in the EDA (electronic design automation) business, and thats about all they use. -- john r pierce, recycling bits in santa cruz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On May 8, 2015, at 12:02 PM, John R Pierce pie...@hogranch.com wrote: On 5/8/2015 10:40 AM, Warren Young wrote: - Adobe’s killed off dozens of products over the years. FrameMaker ... Frame isn't dead When I think of FrameMaker, I think of the program that started out on Solaris, then moved to other big iron Unices and OS X. Wikipedia informs me that it’s been Windows-only for about a decade, which must be how it dropped off my radar. Still, it’s good to know the old thing is still shambling along in some form. I was impressed with it when I used it way back when. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On Fri, May 08, 2015 at 01:59:12PM +0200, Timothy Murphy wrote: Sorin Srbu wrote: The worst thing about BackupPC is the insane error message Unable to read 4 bytes, which comes up if anything is wrong. Possibly the worst error message anywhere? thats an rsync protocol message, and yeah, debugging connection/authentication issues is a bit ugly. I'm sure you are right. But I use rsync several times a day, and I have never received this message. It's not ugly, it is inexcusable. It may not be rsync fault. I vaguely recall that BackupPC uses an ancient (~2006) Perl rsync library, which, for instance, does not support compressed transfers. Maybe that terse message is all that BackupPC gets from the library. :-) Mihai ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Sorin Srbu wrote: The worst thing about BackupPC is the insane error message Unable to read 4 bytes, which comes up if anything is wrong. Possibly the worst error message anywhere? thats an rsync protocol message, and yeah, debugging connection/authentication issues is a bit ugly. I'm sure you are right. But I use rsync several times a day, and I have never received this message. It's not ugly, it is inexcusable. Yeah, well, but it's free. I'm not sure you can complain too much in that case. 8-) I find this comment, often made, completely unacceptable. The implication is that inferior code is OK if the developer is not being paid. (Actually, the premise is probably nonsense, as most Linux developers _are_ paid, even if formally their pay is not specifically for Linux development. But presumably the company that pays them believes that it is of value to the company to have a Linux developer on board.) But is Linux code in fact inferior to code produced by Microsoft, say? I don't think so. And I don't think Linux developers are less keen to improve their code. Just the opposite. -- Timothy Murphy gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/8/2015 12:47 PM, Warren Young wrote: On May 8, 2015, at 12:02 PM, John R Pierce pie...@hogranch.com wrote: On 5/8/2015 10:40 AM, Warren Young wrote: - Adobe’s killed off dozens of products over the years. FrameMaker ... Frame isn't dead When I think of FrameMaker, I think of the program that started out on Solaris, then moved to other big iron Unices and OS X. Wikipedia informs me that it’s been Windows-only for about a decade, which must be how it dropped off my radar. Still, it’s good to know the old thing is still shambling along in some form. I was impressed with it when I used it way back when. That does bring back memories of Solaris and Framemaker from the mid 90's. We had folks using Frame as a word processor, absolutely insane, especially since they had Applixware (originally Aster*x) installed on the same machines. Fun times! -- //steve ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Am 07.05.2015 um 08:35 schrieb Michael Schumacher michael.schumac...@pamas.de: Everybody has its favorite backup program, but why rely on only one system? I have to backup 8 servers and use three backup systems in parallel. -- BackupPC. Easy to use, nice user interface with graphical recovery of individual files. A pain to set up, basically, all errors in setup give the same error message. Reduces used space by hardlinks. Data structure is not transparent, so no recovery by browsing the storage directories. -- storeBackup. easy to use, easy to set up, but no nice user interface. Reduces used space nicely by using hardlinks. Used as second line of defence. Stores 1:1 copies of original filesystem, so easy browsing. -- tar. Used for disaster recovery. Produces large dumps. Only use it for system data, not for user data. just another one (rsyns/hardlink based): rsnapshot its used here extensively and just works (storage browsable). -- LF ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Il 07/05/2015 11:24, Marcin Trendota ha scritto: W dniu 06.05.2015 o 21:21, Alessandro Baggi pisze: What do you mean about Backup PC? Any experiences? What solution do you use? BackupPC is good, howewer it's a pity you can't search for a file in GUI. But it works well, i'm backing up 32 hosts (servers, desktops). Can somebody tell me why it's not available for CentOS7? I don't know why and don't know if in previous CentOS releases was included. BackupPC is available for C7 from nux repo, but this is an external repo. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
I wonder why nobody has yet mentioned rdiff-backup. It combines browsable directories with multiple versions - the version data is stored in a separate rdiff-backup-data subdirectory (one per backup task). One downside is that rdiff-backup causes a lot of network traffic. For that reason I currently use rsync to copy over network, and then I use rdiff-backup locally to create a repository with multiple versions. Another system that we use is rdiffweb. It uses rdiff-backup over network and adds a web interface for clients to browse and restore files or directories. I did not personally set it up, but it seems to work fine. - Jussi ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Il 07/05/2015 00:47, John R Pierce ha scritto: On 5/6/2015 1:34 PM, Valeri Galtsev wrote: My assistant liked backuppc. It is OK and will do decent job for really small number of machines (thinking 3-4 IMHO). I run bacula which has close to a hundred of clients; all is stored in files on RAID units, no tapes. Once you configure it it is nice. But to make a configuration work for the first time is really challenging (says one who still managed to configure it I've been using BackupPC to backup about 25-30 servers and VMs for a couple years now. My backup server has a 20TB raid dedicated to BackupPC, using XFS on LVM, on CentOS 6.latest... That backup raid is mirrored to an identical server in a seperate building via drbd for disaster recovery. I keep 12+ months of monthly full backups, and 30+ days of daily incrementals. The deduplicated and compressed backups of all this take all of 4800GB, containing 9.1 million files and 4369 directories. The full backups WOULD have taken 68TB and the incrementals 25TB without dedup. I'm very happy with it. its a 'pull' based backup, no agents are required for the clients... it can use a variety of methods, I mostly use rsync-over-ssh, all you need to configure is a ssh key so the backup server's backuppc user can connect to the target via ssh as a user with sufficient privs to backup the desired file systems. for my couple windows servers, I install a cygwin based rsync.BackupPC also can use nfs, smb, and tar-over-ssh as backup methods. adding a new host to the backup service takes me about 5 minutes. it would probably take even less time if I bothered to document and/or automate the process :) users can be given access to their own backups via the web interface, and they can either download single files, a tar or zip of a directory tree, or tell the server to push a restore onto the original target. you can download or restore ANY version of any file thats in the hive. the major downside is that ALL the backups have to be stored on one monolithic file system, and it uses tons of hard links. If you use XFS, this is not a problem.maintaining a backup of your backups can be done a couple ways, I am using drbd to a mirror server, but there's also a provision I haven't explored for generating archives. Hi John, when disk is filled, on bacula we can recycle disk volumes. What's for BackupPC? There is automatic backup deletion over retention time? Thanks in advance. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Hello Alessandro, Wednesday, May 6, 2015, 9:21:10 PM, you wrote: I'm new with backup ops and I'm searching a good system to accomplish this work. Everybody has its favorite backup program, but why rely on only one system? I have to backup 8 servers and use three backup systems in parallel. -- BackupPC. Easy to use, nice user interface with graphical recovery of individual files. A pain to set up, basically, all errors in setup give the same error message. Reduces used space by hardlinks. Data structure is not transparent, so no recovery by browsing the storage directories. -- storeBackup. easy to use, easy to set up, but no nice user interface. Reduces used space nicely by using hardlinks. Used as second line of defence. Stores 1:1 copies of original filesystem, so easy browsing. -- tar. Used for disaster recovery. Produces large dumps. Only use it for system data, not for user data. Span all three systems on two independent backup machines. Put these backup servers into independent locations and sleep better :-) best regards --- Michael Schumacher ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
W dniu 06.05.2015 o 21:21, Alessandro Baggi pisze: What do you mean about Backup PC? Any experiences? What solution do you use? BackupPC is good, howewer it's a pity you can't search for a file in GUI. But it works well, i'm backing up 32 hosts (servers, desktops). Can somebody tell me why it's not available for CentOS7? -- Over And Out MoonWolf ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
W dniu 07.05.2015 o 11:46, Alessandro Baggi pisze: I don't know why and don't know if in previous CentOS releases was included. It is in EPEL. BackupPC is available for C7 from nux repo, but this is an external repo. Good enough, thanks for info. -- Over And Out MoonWolf ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/7/2015 2:21 PM, Timothy Murphy wrote: John R Pierce wrote: On 5/7/2015 4:56 AM, Timothy Murphy wrote: The worst thing about BackupPC is the insane error message Unable to read 4 bytes, which comes up if anything is wrong. Possibly the worst error message anywhere? thats an rsync protocol message, and yeah, debugging connection/authentication issues is a bit ugly. I'm sure you are right. But I use rsync several times a day, and I have never received this message. It's not ugly, it is inexcusable. I just tried a command line rsync to a host that wasn't listening to ssh at all, and got.. ssh: connect to host castillc2-PC port 22: Connection timed out rsync: connection unexpectedly closed (0 bytes received so far) [receiver] rsync error: unexplained error (code 255) at io.c(600) [receiver=3.0.6] not exactly the pinnacle of clarity. -- john r pierce, recycling bits in santa cruz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On May 7, 2015 6:05 AM, Jussi Hirvi greens...@greenspot.fi wrote: I wonder why nobody has yet mentioned rdiff-backup. It combines browsable directories with multiple versions - the version data is stored in a separate rdiff-backup-data subdirectory (one per backup task). One downside is that rdiff-backup causes a lot of network traffic. For that reason I currently use rsync to copy over network, and then I use rdiff-backup locally to create a repository with multiple versions. Another system that we use is rdiffweb. It uses rdiff-backup over network and adds a web interface for clients to browse and restore files or directories. I did not personally set it up, but it seems to work fine. I am one of the people who use rsync with hardlinks. Reason is very simple and even humble: I built my home backup server around a OpenWrt - Seagate dockstar if you want to date that - box and an external backup drive. So I wanted something low resources that did not require me to install any packages. That script grew a bit (or a lot) and became my old job's backup code. But, I admit one think it does miss is having a convenient way to look for a file, specially if you physically rotate drives. If rdiff-backup will tell when was the last time a file has been backed up/touched even if drive with said file is not mounted, I will need to get to learn more about it. - Jussi ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 05/07/2015 05:04 AM, Jussi Hirvi wrote: I wonder why nobody has yet mentioned rdiff-backup. It combines browsable directories with multiple versions - the version data is stored in a separate rdiff-backup-data subdirectory (one per backup task). I use rdiff-backup, but I hesitate to recommend a tool that has been unsupported for over 6 years and does have quite a few bugs. -- Bob Nichols NOSPAM is really part of my email address. Do NOT delete it. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
John R Pierce wrote: On 5/7/2015 4:56 AM, Timothy Murphy wrote: The worst thing about BackupPC is the insane error message Unable to read 4 bytes, which comes up if anything is wrong. Possibly the worst error message anywhere? thats an rsync protocol message, and yeah, debugging connection/authentication issues is a bit ugly. I'm sure you are right. But I use rsync several times a day, and I have never received this message. It's not ugly, it is inexcusable. -- Timothy Murphy gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 7.5.2015 14.24, Mauricio Tavares wrote: I admit one think it does miss is having a convenient way to look for a file, specially if you physically rotate drives. If rdiff-backup will tell when was the last time a file has been backed up/touched even if drive with said file is not mounted, I will need to get to learn more about it. I don't think rdiff-backup would work with rotated drives - not really. You could make it work to some extent with some cumbersome gimmicks, but not perfectly. I found this thread on the subject: http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/rdiff-backup-23/rdiff-backup-and-rotated-external-drives-122523/ But why rotate drives? Big drives are not very expensive nowadays. Regards, Jussi ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
--On Thursday, May 07, 2015 06:41:03 PM +0300 Jussi Hirvi greens...@greenspot.fi wrote: But why rotate drives? Big drives are not very expensive nowadays. 1. Redundant copies. 2. Sometimes your filesystems are larger than the largest drives. For example, I'm currently seting up backups for a 24TB filesystem where a network-based DR is not feasible (the average rate of churn exceeds the available network bandwidth). Good luck trying to find drives that big. I had a sense of deja vu the other day; I was taken back to the time when I first ran into a filesystem that was larger than the size of a backup tape and the software I was using at the time (Amanda) had the assumption that a single filesystem was smaller than a single tape. (I understand they fixed that assumption shortly thereafter, but I had already moved on to another product.) For the record, my favourite product is Bacula. Devin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 2015-05-06, Valeri Galtsev galt...@kicp.uchicago.edu wrote: This sounds like Apple borrowed your idea for their time machine (I bet you are doing it for much-much linger than Apple time machine exists)! rsnapshot has been using rsync with hard links for ages. http://rsnapshot.org/ --keith -- kkel...@wombat.san-francisco.ca.us ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Il 07/05/2015 11:55, Marcin Trendota ha scritto: W dniu 07.05.2015 o 11:46, Alessandro Baggi pisze: I don't know why and don't know if in previous CentOS releases was included. It is in EPEL. BackupPC is available for C7 from nux repo, but this is an external repo. Good enough, thanks for info. Then, I'm trying BackupPc 3.3.1, Installed, configured, CGI configured. I've some questions: 1) There is a systemd start file? 2) Apache on C7 seems not have mod_perl support. There is a way to accomplish this? Thanks in advance. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/7/2015 4:56 AM, Timothy Murphy wrote: The worst thing about BackupPC is the insane error message Unable to read 4 bytes, which comes up if anything is wrong. Possibly the worst error message anywhere? thats an rsync protocol message, and yeah, debugging connection/authentication issues is a bit ugly. -- john r pierce, recycling bits in santa cruz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/6/2015 11:23 PM, Alessandro Baggi wrote: when disk is filled, on bacula we can recycle disk volumes. What's for BackupPC? There is automatic backup deletion over retention time? my year of monthlies and month of dailies of 25 servers has been more or less constant size for a year or two now as it deletes the oldest backups. I don't think there's an option to delete based on volume free space, its age based, so you adjust the retention age to suit. the compression and dedup works so well it amazes me, that I have about 100TB worth of incremental backups stored on 6TB of actual disk. My backup servers actually have 32TB after raid 6+0, but only 20TB is currently allocated to the backuppc data volume, so I can grow the /data volume if needed. -- john r pierce, recycling bits in santa cruz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Geenhuizen wrote: I’ve been using BackupPC for several years for my 10 hosts, and works extremely well, however it can take a lot of disk space, so I’d recommend a dedicated drive for the backups. I've been running BackupPC on two home servers (in different places) running CentOS for many years, and am very happy with it. I actually backup to a different disk on the same server, and then archive that on an external disk every couple of months. The worst thing about BackupPC is the insane error message Unable to read 4 bytes, which comes up if anything is wrong. Possibly the worst error message anywhere? -- Timothy Murphy gayleard /at/ eircom.net School of Mathematics, Trinity College, Dublin ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 5/6/2015 1:34 PM, Valeri Galtsev wrote: My assistant liked backuppc. It is OK and will do decent job for really small number of machines (thinking 3-4 IMHO). I run bacula which has close to a hundred of clients; all is stored in files on RAID units, no tapes. Once you configure it it is nice. But to make a configuration work for the first time is really challenging (says one who still managed to configure it I've been using BackupPC to backup about 25-30 servers and VMs for a couple years now. My backup server has a 20TB raid dedicated to BackupPC, using XFS on LVM, on CentOS 6.latest... That backup raid is mirrored to an identical server in a seperate building via drbd for disaster recovery. I keep 12+ months of monthly full backups, and 30+ days of daily incrementals. The deduplicated and compressed backups of all this take all of 4800GB, containing 9.1 million files and 4369 directories. The full backups WOULD have taken 68TB and the incrementals 25TB without dedup. I'm very happy with it. its a 'pull' based backup, no agents are required for the clients... it can use a variety of methods, I mostly use rsync-over-ssh, all you need to configure is a ssh key so the backup server's backuppc user can connect to the target via ssh as a user with sufficient privs to backup the desired file systems. for my couple windows servers, I install a cygwin based rsync.BackupPC also can use nfs, smb, and tar-over-ssh as backup methods. adding a new host to the backup service takes me about 5 minutes. it would probably take even less time if I bothered to document and/or automate the process :) users can be given access to their own backups via the web interface, and they can either download single files, a tar or zip of a directory tree, or tell the server to push a restore onto the original target. you can download or restore ANY version of any file thats in the hive. the major downside is that ALL the backups have to be stored on one monolithic file system, and it uses tons of hard links. If you use XFS, this is not a problem.maintaining a backup of your backups can be done a couple ways, I am using drbd to a mirror server, but there's also a provision I haven't explored for generating archives. -- john r pierce, recycling bits in santa cruz ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On 6 May 2015 at 22:49, J Martin Rushton martinrushto...@btinternet.com wrote: Don't dismiss Amanda it works well in a disk based setup. I don't bother with the spooling disk though. I back up to virtual tape slots on an external disk and rotate three external disks; two are in the firesafe at work, one is on top of my PC. I can say the same about Bacula, just spooling to virtual tape slots on external disks work just fine here, it has worked more than a decade w/o a hitch and I'm not changing it for the sake of having a change any time soon (originally was backing to an external SCSI tape using DDS2 media virtually using the same config files but rotating multi-TB external disks is cheaper easier). ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
m.r...@5-cent.us wrote: Alessandro Baggi wrote: I list, I'm new with backup ops and I'm searching a good system to accomplish this work. I know that on centos there are bacula and amanda but they are too tape oriented. Another is that they are very powerfull but more complex. I need a solution for small office for disk storage and I found Backup PC. Many people say that it is great for small stuff and for great number of data. What do you mean about Backup PC? Any experiences? What solution do you use? Les, who I'm sure will hop in, likes it. We have a home-grown system that automates rsync. Oh, let me add that we use rsync with hard links, which saves a *lot* of space, and time. mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On Wed, May 6, 2015 3:27 pm, m.r...@5-cent.us wrote: m.r...@5-cent.us wrote: Alessandro Baggi wrote: I list, I'm new with backup ops and I'm searching a good system to accomplish this work. I know that on centos there are bacula and amanda but they are too tape oriented. Another is that they are very powerfull but more complex. I need a solution for small office for disk storage and I found Backup PC. Many people say that it is great for small stuff and for great number of data. What do you mean about Backup PC? Any experiences? What solution do you use? Les, who I'm sure will hop in, likes it. We have a home-grown system that automates rsync. Oh, let me add that we use rsync with hard links, which saves a *lot* of space, and time. This sounds like Apple borrowed your idea for their time machine (I bet you are doing it for much-much linger than Apple time machine exists)! Valeri Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Don't dismiss Amanda it works well in a disk based setup. I don't bother with the spooling disk though. I back up to virtual tape slots on an external disk and rotate three external disks; two are in the firesafe at work, one is on top of my PC. On 06/05/15 20:21, Alessandro Baggi wrote: I list, I'm new with backup ops and I'm searching a good system to accomplish this work. I know that on centos there are bacula and amanda but they are too tape oriented. Another is that they are very powerfull but more complex. I need a solution for small office for disk storage and I found Backup PC. Many people say that it is great for small stuff and for great number of data. What do you mean about Backup PC? Any experiences? What solution do you use? Thanks in advances. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJVSoxqAAoJEAF3yXsqtyBl7D0QAIcxxZPq8/CqQzP1apo+fKGa BimbScb4zf7fx9OH3C/8fmFUtxSiIXSCXqYT2AwStPaLnhqaeXchjOmLnT4Ev4Sb 37u64ykIi5Umu8ukJwtwpcD3eoW61Xy5rErf7g5oxKa4GOYtWrCOi7QEop9g/oh1 yQSEMVj2L0Y7wA5uzkKQJ58sicONUC58ZFGHNlpAMykAxD5p0vUn1bccCOWyGT/2 jduwLusLr4rbI46pTnD4hoVri4w3AjIzdi/wZySlq2ZSapOIc+PHP5CdKttKh4dd OFZvk3Y572S/6/T/wWgL7R6ve1OHeEh3kyTV51tiLpT28gmKD17kSWjXjBjiAoWt j4mfsVQdKC7nVNBRw+VJz7+LhHNaINc/23DquB5QjGzdsdOPfZbIFiqfT1YS0O2V rNrswvlTRFDXGauhE77GeFj8wfr9YjGbF/mmpl8sFeyzcVk1r+UdMg3qYZNVdm/p GRKaiC31ilsKZVGH6TSPkHyo3N22wM0A5gPF2wcpXs5cVAN9EcKSLxMVSMCLKT4h gImmnV21zrCVyDO59H2GiqO49vtrakDULI1zdo9nB58Rw4Tfp2HIgiRij1O46MFB TJpsJsK4v320HKb07sPTxPH2ysgu7dKF8zbp8I+y8Kc6MKAygU2vU2ZKWH0pKlej 9/vElQtUUseesx6gH++c =tD6K -END PGP SIGNATURE- ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On May 6, 2015, at 9:46 PM, m.r...@5-cent.us wrote: Alessandro Baggi wrote: I list, I'm new with backup ops and I'm searching a good system to accomplish this work. I know that on centos there are bacula and amanda but they are too tape oriented. Another is that they are very powerfull but more complex. I need a solution for small office for disk storage and I found Backup PC. Many people say that it is great for small stuff and for great number of data. What do you mean about Backup PC? Any experiences? What solution do you use? Les, who I'm sure will hop in, likes it. We have a home-grown system that automates rsync. mark I’ve been using BackupPC for several years for my 10 hosts, and works extremely well, however it can take a lot of disk space, so I’d recommend a dedicated drive for the backups. I’ve restored many files over the years but haven’t as yet needed to do a bare metal restore. One further recommendation I have is that you might also consider a second host that backs up the primary backup host in the event that it fails, which is what I’m doing, and because I have it I backup all my hosts to 2 different servers. The BackupPC list is very active at times and can provide you with lots of tips and help. Pete ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
On Wed, May 6, 2015 2:46 pm, m.r...@5-cent.us wrote: Alessandro Baggi wrote: I list, I'm new with backup ops and I'm searching a good system to accomplish this work. I know that on centos there are bacula and amanda but they are too tape oriented. Another is that they are very powerfull but more complex. I need a solution for small office for disk storage and I found Backup PC. Many people say that it is great for small stuff and for great number of data. What do you mean about Backup PC? Any experiences? What solution do you use? Les, who I'm sure will hop in, likes it. We have a home-grown system that automates rsync. My assistant liked backuppc. It is OK and will do decent job for really small number of machines (thinking 3-4 IMHO). I run bacula which has close to a hundred of clients; all is stored in files on RAID units, no tapes. Once you configure it it is nice. But to make a configuration work for the first time is really challenging (says one who still managed to configure it ;-) Good luck! Valeri Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Alessandro Baggi wrote: I list, I'm new with backup ops and I'm searching a good system to accomplish this work. I know that on centos there are bacula and amanda but they are too tape oriented. Another is that they are very powerfull but more complex. I need a solution for small office for disk storage and I found Backup PC. Many people say that it is great for small stuff and for great number of data. What do you mean about Backup PC? Any experiences? What solution do you use? Les, who I'm sure will hop in, likes it. We have a home-grown system that automates rsync. mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup PC or other solution
Valeri Galtsev wrote: On Wed, May 6, 2015 3:27 pm, m.r...@5-cent.us wrote: m.r...@5-cent.us wrote: Alessandro Baggi wrote: I list, I'm new with backup ops and I'm searching a good system to accomplish this work. I know that on centos there are bacula and amanda but they are too tape oriented. Another is that they are very powerfull but more complex.I need a solution for small office for disk storage and I found Backup PC.Many people say that it is great for small stuff and for great number of data. What do you mean about Backup PC? Any experiences? What solution do you use? Les, who I'm sure will hop in, likes it. We have a home-grown system that automates rsync. Oh, let me add that we use rsync with hard links, which saves a *lot* of space, and time. This sounds like Apple borrowed your idea for their time machine (I bet you are doing it for much-much linger than Apple time machine exists)! We try to keep five weeks. Except for the giant RAID boxen, which are more, ahhh, challenging to backup (or not, as the case may be). mark ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Backup PC or other solution
I list, I'm new with backup ops and I'm searching a good system to accomplish this work. I know that on centos there are bacula and amanda but they are too tape oriented. Another is that they are very powerfull but more complex. I need a solution for small office for disk storage and I found Backup PC. Many people say that it is great for small stuff and for great number of data. What do you mean about Backup PC? Any experiences? What solution do you use? Thanks in advances. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] backup incrementals on mysql
On Mon, Nov 10, 2014 at 8:25 PM, Rodrigo Pichiñual Norin rodrigo.pichin...@gmail.com wrote: Hi all. I usally make backups of databases mysql. I make buckups of all datbase for example: mysqldump -u user -ppassword name_db backups.sql also I make backups just its schema for example mysqldump -u user -ppassword name_db --no-data backups.sql Hola, If size's a concern, just export your DBs gziped : mysqldump -u user database | gzip backup_database.sql Also if you are concerned about time to compress, you can enable multithreaded parallel gzip compress with pigz (available in EPEL) : mysqldump -u user database | pigz backup_database.sql TBH, 500 MB databases aren't big enough to seek a more complex approach (unless you have more ambitious RTO/RPO requirements) HTH ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] backup incrementals on mysql
On Tue, November 11, 2014 9:10 am, Fran Garcia wrote: On Mon, Nov 10, 2014 at 8:25 PM, Rodrigo Pichiñual Norin rodrigo.pichin...@gmail.com wrote: Hi all. I usally make backups of databases mysql. I make buckups of all datbase for example: mysqldump -u user -ppassword name_db backups.sql also I make backups just its schema for example mysqldump -u user -ppassword name_db --no-data backups.sql Hola, If size's a concern, just export your DBs gziped : mysqldump -u user database | gzip backup_database.sql I would stay away from compression. Compression results in binary file. Even though compressed result is smaller, when you will try to keep several versions back, you will have multiple compressed file versions hitting your backup storage. The original poster's intent is better: to keep diff of ASCII dump files. If one commits dumps into some version control system, even though its files may be treated as binary, you will need only one latest version on backup (as it contains all versions of database). Just my $0.02. Valeri Also if you are concerned about time to compress, you can enable multithreaded parallel gzip compress with pigz (available in EPEL) : mysqldump -u user database | pigz backup_database.sql TBH, 500 MB databases aren't big enough to seek a more complex approach (unless you have more ambitious RTO/RPO requirements) Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] backup incrementals on mysql
On Nov 10, 2014 9:25 PM, Rodrigo Pichiñual Norin rodrigo.pichin...@gmail.com wrote: Hi all. I usally make backups of databases mysql. I make buckups of all datbase for example: mysqldump -u user -ppassword name_db backups.sql also I make backups just its schema for example mysqldump -u user -ppassword name_db --no-data backups.sql but now I need a backups incrementals, because the size of DB is very big (500 mb) How to make this? Rodrigo, If your storage engine is InnoDB I would advise to use Percona Innobackupex tool. Take a look here: [1] http://www.percona.com/doc/percona-xtrabackup/2.1/innobackupex/incremental_backups_innobackupex.html [2] http://www.slideshare.net/mobile/kastauyra/bitmap-33283809 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] backup incrementals on mysql
Hi all. I usally make backups of databases mysql. I make buckups of all datbase for example: mysqldump -u user -ppassword name_db backups.sql also I make backups just its schema for example mysqldump -u user -ppassword name_db --no-data backups.sql but now I need a backups incrementals, because the size of DB is very big (500 mb) How to make this? regards -- *Atte. Rodrigo Pichiñual N.* *Ingeniero Administrador de Sistemas Linux* *rodrigo.pichin...@gmail.com rodrigo.pichin...@gmail.com* *+56 9 87272971* *@Roodrigo0461* *http://cl.linkedin.com/in/rodrigopichinual http://cl.linkedin.com/in/rodrigopichinual* ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] backup incrementals on mysql
Check out MySQL-zrm. Handles all types of MySQL backups locally or remotely. http://www.zmanda.com/download-zrm.php Chris On Mon, Nov 10, 2014 at 12:25 PM, Rodrigo Pichiñual Norin rodrigo.pichin...@gmail.com wrote: Hi all. I usally make backups of databases mysql. I make buckups of all datbase for example: mysqldump -u user -ppassword name_db backups.sql also I make backups just its schema for example mysqldump -u user -ppassword name_db --no-data backups.sql but now I need a backups incrementals, because the size of DB is very big (500 mb) How to make this? regards -- *Atte. Rodrigo Pichiñual N.* *Ingeniero Administrador de Sistemas Linux* *rodrigo.pichin...@gmail.com rodrigo.pichin...@gmail.com* *+56 9 87272971* *@Roodrigo0461* *http://cl.linkedin.com/in/rodrigopichinual http://cl.linkedin.com/in/rodrigopichinual* ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- Chris Stone AxisInternet, Inc. www.axint.net ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] backup incrementals on mysql
On Mon, November 10, 2014 1:25 pm, Rodrigo Pichiñual Norin wrote: Hi all. I usally make backups of databases mysql. I make buckups of all datbase for example: mysqldump -u user -ppassword name_db backups.sql also I make backups just its schema for example mysqldump -u user -ppassword name_db --no-data backups.sql but now I need a backups incrementals, because the size of DB is very big (500 mb) How to make this? I've seen somewhere script that does the following. It dumps all databases, then commits them into some version control system. CVS and subversion come to my mind. As the last do diff of text files 9and this way keep changes from version to version), and database dump is ASCII text, this will fulfill pretty well what you need. I must confess, I never came to the point of setting it that way myself: with database sizes we have, and the space we can devote we can handle a week worth of daily full dumps, and a couple of Months of backups of these... but one day I'll do it this way. The following may not be what I originally saw, but seems to be doing exactly this: http://rocketmodule.com/blog/database-backup-dump-and-svn-commit-script-drupal-workflow Valeri Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
From: Fred Smith fre...@fcshome.stoneham.ma.us Looking for suggestions on backup software I can use. You could also have a look at bacula... JD ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On Fri, May 16, 2014 at 06:12:17PM +, Richer, Mark (CIV) wrote: We're using rsnapshot. My colleague set it up, but I will be taking over administration soon. Mark Having looked more deeply into rsnapshot, I think I'll try it, as it appears it'll do what I need. My thanks to ALL OF YOU who have replied with suggestions. Ain't Community Great? :) Fred -- Fred Smith -- fre...@fcshome.stoneham.ma.us - And he will be called Wonderful Counselor, Mighty God, Everlasting Father, Prince of Peace. Of the increase of his government there will be no end. He will reign on David's throne and over his kingdom, establishing and upholding it with justice and righteousness from that time on and forever. --- Isaiah 9:7 (niv) -- ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On Fri, May 16, 2014 at 8:38 PM, Derrik Walker v2.0 dwal...@doomd.net wrote: I've been using BackupPC for years. I currently have it running on a small CentOS system that mainly does backups. I like it because it's agentless ( it uses ssh/rsync ). The Pooling and Data-deduping is also nice, and saves on space. +1 to backuppc. A word of caution - database backups should be done with their respective native tools. A colleague, was backing up /var/lib/mysql/ thinking he could restore the db from the backup! -- Arun Khan ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On 05/16/2014 04:58 PM, Fred Smith wrote: I know there's rsync, which may be a good solution. Another solution could be rsync with zfs or btrfs. -- Gruß, Christian ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Centos backup tools
Hi all! I'm building a raid box to use for backups, connectivity will be either USB3 or esata. Looking for suggestions on backup software I can use. I know there's rsync, which may be a good solution. I also find backupPC at epel, backintime also at epel, kbackup. DejaDup looks interesting, but none of the repos I'm set up to use shows it being available. some small details: I plan to use this to keep backups of my centos desktop, which has two 320GB drives in linux RAID-1. The backup box will have two 1TB drives, also in RAID-1. It will be a two drive enclosure with PS and cooling, with USB3 or esata, but not networking. I was thinking that it would be nice to have a full backup followed by a set of incrementals, and software that allows access to the state of the system for any specific date (similar to a source control system), but it may be that nothing free and/or uncomplicated will offer such a feature. But, as I always say, suggestions welcomed! -- Fred Smith -- fre...@fcshome.stoneham.ma.us - Not everyone who says to me, 'Lord, Lord,' will enter the kingdom of heaven, but only he who does the will of my Father who is in heaven. -- Matthew 7:21 (niv) - ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On 05/16/2014 10:58 AM, Fred Smith wrote: Hi all! I'm building a raid box to use for backups, connectivity will be either USB3 or esata. Looking for suggestions on backup software I can use. I know there's rsync, which may be a good solution. I also find backupPC at epel, backintime also at epel, kbackup. I've been using BackupPC for years. I currently have it running on a small CentOS system that mainly does backups. I like it because it's agentless ( it uses ssh/rsync ). The Pooling and Data-deduping is also nice, and saves on space. - Derrik ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On Fri, May 16, 2014 at 9:58 AM, Fred Smith fre...@fcshome.stoneham.ma.us wrote: Hi all! I'm building a raid box to use for backups, connectivity will be either USB3 or esata. Looking for suggestions on backup software I can use. I know there's rsync, which may be a good solution. I also find backupPC at epel, backintime also at epel, kbackup. DejaDup looks interesting, but none of the repos I'm set up to use shows it being available. some small details: I plan to use this to keep backups of my centos desktop, which has two 320GB drives in linux RAID-1. The backup box will have two 1TB drives, also in RAID-1. It will be a two drive enclosure with PS and cooling, with USB3 or esata, but not networking. I was thinking that it would be nice to have a full backup followed by a set of incrementals, and software that allows access to the state of the system for any specific date (similar to a source control system), but it may be that nothing free and/or uncomplicated will offer such a feature. But, as I always say, suggestions welcomed! Try backuppc first. And don't overthink the full/incremental numbers until you understand how it compresses and pools the data. You will be able to keep much more history than you expect on line. Backuppc has its own mail list - you can ask there if you have any problems but the EPEL package should 'just work'. Just look at /etc/httpd/conf.d/BackupPC.conf for an example of how to set the web user passwords. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
Hi Fred, I know there's rsync, which may be a good solution. I also find backupPC at epel, backintime also at epel, kbackup. I am using backuppc here to backup 5 servers to a dedicated backup-machine. In parallel, I am using http://storebackup.org/ . Storebackup uses hard links to reduce space requirements. I have to say that backuppc has the nicer restore tool. best regards --- Michael ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
Hi Fred, BackupPC is really good solution for Small Office/Home office enviroment. If you don't need anything complex but want to manage 2-3 workstation's backup than you can use BackupPC without any issues. 16.05.2014 19:33, Michael Schumacher ?: Hi Fred, I know there's rsync, which may be a good solution. I also find backupPC at epel, backintime also at epel, kbackup. I am using backuppc here to backup 5 servers to a dedicated backup-machine. In parallel, I am using http://storebackup.org/ . Storebackup uses hard links to reduce space requirements. I have to say that backuppc has the nicer restore tool. best regards --- Michael ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- Best Regards, *Alexander Danilov* ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On 5/16/2014 10:58 AM, Fred Smith wrote: Hi all! I'm building a raid box to use for backups, connectivity will be either USB3 or esata. Looking for suggestions on backup software I can use. I know there's rsync, which may be a good solution. I also find backupPC at epel, backintime also at epel, kbackup. DejaDup looks interesting, but none of the repos I'm set up to use shows it being available. some small details: I plan to use this to keep backups of my centos desktop, which has two 320GB drives in linux RAID-1. The backup box will have two 1TB drives, also in RAID-1. It will be a two drive enclosure with PS and cooling, with USB3 or esata, but not networking. I was thinking that it would be nice to have a full backup followed by a set of incrementals, and software that allows access to the state of the system for any specific date (similar to a source control system), but it may be that nothing free and/or uncomplicated will offer such a feature. BackupPC works great. I'm using it to back up about 20 servers. The pooling allows you to keep many more backups online than you expect. A couple of things to watch for: 1) The data directory must be on a filesystem that supports hardlinks as that is how the pooling is done. 2) Due to the massive number of hardlinks used in the pool, it can be very difficult to backup or copy the backup server itself depending on the number of files in the pool. If you want an offsite copy, I would suggest breaking the mirrored pair, sending one of those disks offsite, and then rebuilding to a new drive. I actually have 3 drives in my raid1 setup so that there is still redundancy while it is rebuilding. -- Bowie ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On 5/16/2014 8:58 AM, Fred Smith wrote: Hi all! I'm building a raid box to use for backups, connectivity will be either USB3 or esata. Looking for suggestions on backup software I can use. snip There are a number of good ready made choices (backuppc seems to be the most suggested so far) but you can always grow your own, it just depends on how sophisticated you want to get. In the office I built a backup server and a private subnet (using a second gigE interface on each server) to create a backup network. Then it was just a little scripting using nfs to connect and tar for the backups. It's basic, it's simple and it works. What ever you chose be sure that you can do recoveries without having to install the entire application again. If the application stores data in a proprietary format you can be screwed when it comes time to recover. If you don't already have it, a good read on the subject is: Backup Recovery by W. Curtis Preston http://shop.oreilly.com/product/9780596102463.do Good luck! -- Steve ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
We're using rsnapshot. My colleague set it up, but I will be taking over administration soon. Mark From: centos-boun...@centos.org [centos-boun...@centos.org] on behalf of Fred Smith [fre...@fcshome.stoneham.ma.us] Sent: Friday, May 16, 2014 10:58 To: centos@centos.org Subject: [CentOS] Centos backup tools Hi all! I'm building a raid box to use for backups, connectivity will be either USB3 or esata. Looking for suggestions on backup software I can use. I know there's rsync, which may be a good solution. I also find backupPC at epel, backintime also at epel, kbackup. DejaDup looks interesting, but none of the repos I'm set up to use shows it being available. some small details: I plan to use this to keep backups of my centos desktop, which has two 320GB drives in linux RAID-1. The backup box will have two 1TB drives, also in RAID-1. It will be a two drive enclosure with PS and cooling, with USB3 or esata, but not networking. I was thinking that it would be nice to have a full backup followed by a set of incrementals, and software that allows access to the state of the system for any specific date (similar to a source control system), but it may be that nothing free and/or uncomplicated will offer such a feature. But, as I always say, suggestions welcomed! -- Fred Smith -- fre...@fcshome.stoneham.ma.us - Not everyone who says to me, 'Lord, Lord,' will enter the kingdom of heaven, but only he who does the will of my Father who is in heaven. -- Matthew 7:21 (niv) - ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On Fri, May 16, 2014 at 12:42 PM, Steve Lindemann st...@marmot.org wrote: On 5/16/2014 8:58 AM, Fred Smith wrote: Hi all! I'm building a raid box to use for backups, connectivity will be either USB3 or esata. Looking for suggestions on backup software I can use. snip There are a number of good ready made choices (backuppc seems to be the most suggested so far) but you can always grow your own, it just depends on how sophisticated you want to get. In the office I built a backup server and a private subnet (using a second gigE interface on each server) to create a backup network. Then it was just a little scripting using nfs to connect and tar for the backups. It's basic, it's simple and it works. What ever you chose be sure that you can do recoveries without having to install the entire application again. If the application stores data in a proprietary format you can be screwed when it comes time to recover. That is kinda of what I do at home: I have a 5W openwrt(!) device which does the backup using rsync and hard links for the incremental crap. It backs up my fileserver and then certain hosts (and specific directories since users are NFS mounted). During daytime it also backups the, well, backup drive to an external drive. Not that fancy but does the job. There are a lot of people who like Amanda, but it might be a bit more time consuming to setup properly If you don't already have it, a good read on the subject is: Backup Recovery by W. Curtis Preston http://shop.oreilly.com/product/9780596102463.do Good luck! -- Steve ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On Fri, May 16, 2014 at 1:43 PM, Mauricio Tavares raubvo...@gmail.com wrote: There are a lot of people who like Amanda, but it might be a bit more time consuming to setup properly Amanda is sort-of tuned to work with tapes with the unique feature of being able to pre-estimate the sizes of full and incremental runs and pick the right mix across a set of hosts each day to fill a tape. If you don't need that feature it probably won't be your first choice. The most unique thing about backuppc is that it has its own implementation of rsync that can work with the compressed archive files on the server and a stock remote rsync version accessing the target files. Or, it can use tar or samba to transfer the files, with all duplicate files pooled regardless of the location or transfer method. And it has a nice web interface for configuration/browsing/restores. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On 2014-05-16, Les Mikesell lesmikes...@gmail.com wrote: The most unique thing about backuppc is that it has its own implementation of rsync that can work with the compressed archive files on the server and a stock remote rsync version accessing the target files. Or, it can use tar or samba to transfer the files, with all duplicate files pooled regardless of the location or transfer method. And it has a nice web interface for configuration/browsing/restores. One thing I really like about rsnapshot is that you can take the latest snapshot and almost literally drop it in to replace the original. This is appealing to me if, for example, the mobo on the main fileserver dies; I can simply change IP addresses, run the right daemons, and my users are back up without too much data loss and without having to wait a long time for a restore process. Is this possible with backuppc? I don't know enough about how the backend data store is organized to know if this is a reasonable use case. --keith -- kkel...@wombat.san-francisco.ca.us ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On Fri, May 16, 2014 at 3:14 PM, Keith Keller kkel...@wombat.san-francisco.ca.us wrote: On 2014-05-16, Les Mikesell lesmikes...@gmail.com wrote: The most unique thing about backuppc is that it has its own implementation of rsync that can work with the compressed archive files on the server and a stock remote rsync version accessing the target files. Or, it can use tar or samba to transfer the files, with all duplicate files pooled regardless of the location or transfer method. And it has a nice web interface for configuration/browsing/restores. One thing I really like about rsnapshot is that you can take the latest snapshot and almost literally drop it in to replace the original. This is appealing to me if, for example, the mobo on the main fileserver dies; I can simply change IP addresses, run the right daemons, and my users are back up without too much data loss and without having to wait a long time for a restore process. Is this possible with backuppc? I don't know enough about how the backend data store is organized to know if this is a reasonable use case. The internal storage format on the backuppc server is compressed files with slightly munged filenames so you can't quite use them 'as is' or use the usual tools to copy them back out.However, there is a web browser view of the backups where you simply select the backup/directory/file(s) you want and can either restore them back where they came from (or to some other configured target) or download to the browser (single file or tar/zip archive). And there are command line tools to generate tar/zip images if you prefer or want to use an ssh pipeline. So, you do have to wait for a restore to get usable files but the process is convenient and the tradeoff is that the same disk space can hold a much longer history (or more targets) due to the compression and pooling of duplicate content matched in different locations.If you anticipate needing an instant replacement you might want a separate disk kept current with rsync and use backuppc for those cases where something is accidentally deleted and you don't notice for weeks. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On Fri, May 16, 2014 at 02:43:55PM -0400, Mauricio Tavares wrote: On Fri, May 16, 2014 at 12:42 PM, Steve Lindemann st...@marmot.org wrote: On 5/16/2014 8:58 AM, Fred Smith wrote: Hi all! I'm building a raid box to use for backups, connectivity will be either USB3 or esata. Looking for suggestions on backup software I can use. snip There are a number of good ready made choices (backuppc seems to be the most suggested so far) but you can always grow your own, it just depends on how sophisticated you want to get. In the office I built a backup server and a private subnet (using a second gigE interface on each server) to create a backup network. Then it was just a little scripting using nfs to connect and tar for the backups. It's basic, it's simple and it works. BackupPC looks like a good tool, but... 1) it requires I configure Apache, and this is just my personal/home workstation where I don't really have any other use for Apache, and I don't really feel like having to learn Apache just to do backups. 2) backuppc config, itself, looks (potentially) complicated, especially since all I want to backup is my own PC, to a raid-1 drive set that's locally connected via USB3 or esata. What ever you chose be sure that you can do recoveries without having to install the entire application again. If the application stores data in a proprietary format you can be screwed when it comes time to recover. and backuppc seems (in my advanced state of ignorance) like one of those tools you can't use to recover without first making your system, once again, bootable, then reinstalling and reconfiguring BackupPC. That is kinda of what I do at home: I have a 5W openwrt(!) device which does the backup using rsync and hard links for the incremental crap. It backs up my fileserver and then certain hosts (and specific directories since users are NFS mounted). During daytime it also backups the, well, backup drive to an external drive. Not that fancy but does the job. that's an interesting way of doing it! I have a router with USB capability,... but I don't think I want my internet gateway to also be responsible for storing important data--it's just a thing I have... but if I had an extra one I could certainly do that, just turn off the wireless. The device I plan to use (if Fedex ever gets it here--can you sense some impatience? :) ) this: http://www.newegg.com/Product/Product.aspx?Item=N82E16817332028 There are a lot of people who like Amanda, but it might be a bit more time consuming to setup properly If you don't already have it, a good read on the subject is: Backup Recovery by W. Curtis Preston http://shop.oreilly.com/product/9780596102463.do Good luck! -- Steve -- Fred Smith -- fre...@fcshome.stoneham.ma.us - For the word of God is living and active. Sharper than any double-edged sword, it penetrates even to dividing soul and spirit, joints and marrow; it judges the thoughts and attitudes of the heart. Hebrews 4:12 (niv) -- ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
On Fri, May 16, 2014 at 3:37 PM, Fred Smith fre...@fcshome.stoneham.ma.us wrote: BackupPC looks like a good tool, but... 1) it requires I configure Apache, and this is just my personal/home workstation where I don't really have any other use for Apache, and I don't really feel like having to learn Apache just to do backups. 2) backuppc config, itself, looks (potentially) complicated, especially since all I want to backup is my own PC, to a raid-1 drive set that's locally connected via USB3 or esata. Remember that we are talking about rpm packages here, so that complicated install process will be 'yum install backuppc and the config is mostly done through the web forms. I don't think you need to know anything about apache itself other than running htpasswd to make a web login/password. But if you intend to run it on the same machine as the as the stuff you want to back up it probably is the wrong tool. Or the wrong idea in the first place. and backuppc seems (in my advanced state of ignorance) like one of those tools you can't use to recover without first making your system, once again, bootable, then reinstalling and reconfiguring BackupPC. Yes, you really want it on a different system from the one with the valuable data. However, it does have an option to generate tar archive snapshots compressed/split so you take them offsite for archival storage and you can restore from those with just standard tools. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Centos backup tools
If you think that you'll expand out to more machines, you may also want to consider Bacula. It's a very stable and capable solution with enterprise grade features. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On Mon, May 27, 2013 at 9:41 PM, Les Mikesell lesmikes...@gmail.com wrote: On Mon, May 27, 2013 at 7:05 PM, Mike Watson mi...@crucis.net wrote: That's my problem. I hitting a file size limit with dump and tar. Reformatting is the obvious solution so you can use one of the rsync-based backups - but if you really had to keep the FAT format you could use tar |split -b with some reasonable size for output. +1 for reformatting and using a file system that supports large(r) files Although not recommended... ... you could create a file with dd (that will take some time) on the NTFS drive, and then loopback mount it and format it ext3 (or whatever you choose). Then mount that and go on your merry way with your rsync backups. *Disclaimer*: This solution is ugly and I'm almost certainly I will get tomatoes thrown at me for suggesting it. ;) Which includes converting FAT32 to NTFS if you don't already have NTFS on the drive. There are more gotchas to what I suggested than just reformatting the drive ext3, etc and then sharing it out via Samba (or NFS) to your Windows clients (if you need to). -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos -- ---~~.~~--- Mike // SilverTip257 // ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
Or, you can use http://sourceforge.net/projects/ext2read/ (or similar) CopyPaste: Ext2Read is an explorer like utility to explore ext2/ext3/ext4 files. It now supports LVM2 and EXT4 extents. It can be used to view and copy files and folders. It can recursively copy entire folders. It can also be used to view and copy disk and file -- Diego - Yo no soy paranoico! (pero que me siguen, me siguen) ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
How well does it run under cron? mw -- Lose not thy airspeed, lest the ground rises up and smites thee. -- William Kershner http://crucis-court.com http://www.crucis.net/1632search On 05/28/2013 11:54 AM, Diego Sanchez wrote: Or, you can use http://sourceforge.net/projects/ext2read/ (or similar) CopyPaste: Ext2Read is an explorer like utility to explore ext2/ext3/ext4 files. It now supports LVM2 and EXT4 extents. It can be used to view and copy files and folders. It can recursively copy entire folders. It can also be used to view and copy disk and file ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On 05/27/2013 11:06 PM, Mark LaPierre wrote: On 05/27/2013 02:13 PM, Mike Watson wrote: I've a small system that I use to support a number of churches. I provide web and email for them. My current server is running CentOS 6.3 with paired 1TB drives in a RAID1 configuration. It works well. One filesystem is very large, 500GB, and contains numerous large files: SQL, docs, church libraries in ebook and digital form, plus stored videos of church services. My problem is that I've found no means of backing up that file system. Dump and tar both error out as exceeding the maximum size. Neither will backup just the video directory (the largest) even with compression. Backup will be to an external (USB) removable HD. Can any suggest a prog or a method to back up this filesystem? mw I use rsync to make a daily back up of my data on a second drive that is not normally mounted except when the backup is running. The drive is inside the same box so this backup is still subject to loss if the box is stolen or destroyed. I really should be backing up to an off site location. If you have a fire, flood, or other general disaster your local backup on an external drive isn't going to buy you anything unless you store the external drive off site. You can place that HDD in a rack (eSATA?) and add another that is kept off-site, and replace/swap them every week? Full off-site backup would require another box off-site with internet/wireless link for backups. Since you use rsync, data transfer should be fairly minimal. -- Ljubomir Ljubojevic (Love is in the Air) PL Computers Serbia, Europe StarOS, Mikrotik and CentOS/RHEL/Linux consultant ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On 5/27/2013 11:13 AM, Mike Watson wrote: One filesystem is very large, 500GB, and contains numerous large files: SQL, docs, church libraries in ebook and digital form, plus stored videos of church services. note that SQL database files generally can't be backed up safely while the SQL database server is active. either take a 'dump' or whatever backup of the database, and backup that dump rather than the raw SQL files, or stop the SQL service before making the backup, then restart it afterwards. Details vary per SQL server, of course. what would work really well for your general backup requirements, ignoring the above issue, is BackupPC. build an onsite backupPC server with a file system sufficiently large to hold all backups you want kept online (I generally do 2 weeks of incrementals), and setup BackupPC archiving to move copies of completed backups to an offsite repository for disaster recovery. -- john r pierce 37N 122W somewhere on the middle of the left coast ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
[CentOS] Backup of large filesystems 500GB
I've a small system that I use to support a number of churches. I provide web and email for them. My current server is running CentOS 6.3 with paired 1TB drives in a RAID1 configuration. It works well. One filesystem is very large, 500GB, and contains numerous large files: SQL, docs, church libraries in ebook and digital form, plus stored videos of church services. My problem is that I've found no means of backing up that file system. Dump and tar both error out as exceeding the maximum size. Neither will backup just the video directory (the largest) even with compression. Backup will be to an external (USB) removable HD. Can any suggest a prog or a method to back up this filesystem? mw -- -- Lose not thy airspeed, lest the ground rises up and smites thee. -- William Kershner http://crucis-court.com http://www.crucis.net/1632search ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On Mon, 27 May 2013 13:13:30 -0500 Mike Watson wrote: Can any suggest a prog or a method to back up this filesystem? rsync -- MELVILLE THEATRE ~ Real D 3D Digital Cinema ~ www.melvilletheatre.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On 27.05.2013 19:13, Mike Watson wrote: I've a small system that I use to support a number of churches. I provide web and email for them. My current server is running CentOS 6.3 with paired 1TB drives in a RAID1 configuration. It works well. One filesystem is very large, 500GB, and contains numerous large files: SQL, docs, church libraries in ebook and digital form, plus stored videos of church services. My problem is that I've found no means of backing up that file system. Dump and tar both error out as exceeding the maximum size. Neither will backup just the video directory (the largest) even with compression. Backup will be to an external (USB) removable HD. Can any suggest a prog or a method to back up this filesystem? Have you tried good old rsync? Or, if you want incremental backups, check rdiff-backup. I'm sure our list colleagues will come up with even more solutions. -- Sent from the Delta quadrant using Borg technology! Nux! www.nux.ro ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On 05/27/2013 01:13 PM, Mike Watson wrote: Backup will be to an external (USB) removable HD. What file system is on that external HD? FAT32 has a 4GB limit for file size. -- Bob Nichols NOSPAM is really part of my email address. Do NOT delete it. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On 2013-05-27, Mike Watson mi...@crucis.net wrote: I've a small system that I use to support a number of churches. I provide web and email for them. My current server is running CentOS 6.3 with paired 1TB drives in a RAID1 configuration. It works well. One filesystem is very large, 500GB, and contains numerous large files: SQL, docs, church libraries in ebook and digital form, plus stored videos of church services. Backup will be to an external (USB) removable HD. Can any suggest a prog or a method to back up this filesystem? People have already suggested rsync and rdiff-backup; there's also rsnapshot which is built on top of rsync. Another option could be mdadm if your RAID1 is already an mdadm array. You can add your USB drive to the array, wait for it to rebuild, then remove it from the array. I'd be wary of backing up an SQL database in that way, but I'd be wary of using rsync, dump, or tar too, so be sure to take a real backup (e.g., mysqldump, pg_dump) of your database first. --keith -- kkel...@wombat.san-francisco.ca.us ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On Mon, May 27, 2013 at 2:55 PM, Keith Keller kkel...@wombat.san-francisco.ca.us wrote: On 2013-05-27, Mike Watson mi...@crucis.net wrote: I've a small system that I use to support a number of churches. I provide web and email for them. My current server is running CentOS 6.3 with paired 1TB drives in a RAID1 configuration. It works well. One filesystem is very large, 500GB, and contains numerous large files: SQL, docs, church libraries in ebook and digital form, plus stored videos of church services. Backup will be to an external (USB) removable HD. Can any suggest a prog or a method to back up this filesystem? People have already suggested rsync and rdiff-backup; there's also rsnapshot which is built on top of rsync. Another option could be mdadm if your RAID1 is already an mdadm array. You can add your USB drive to the array, wait for it to rebuild, then remove it from the array. I'd be wary of backing up an SQL database in that way, but I'd be wary of using rsync, dump, or tar too, so be sure to take a real backup (e.g., mysqldump, pg_dump) of your database first. --keith -- kkel...@wombat.san-francisco.ca.us ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos I'd take the extra step of reformatting the USB drive into an ext3 filesystem, then just use rsync in a nightly cron job. We do it with about a dozen or so workstations here with user data either on internal disks, or other external USB drives. Works great. -- Matt Phelps System Administrator, Computation Facility Harvard - Smithsonian Center for Astrophysics mphe...@cfa.harvard.edu, http://www.cfa.harvard.edu ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
I've used rsync for remote file xfer of directory trees. It's been awhile, I'd forgotten about it. mw -- Lose not thy airspeed, lest the ground rises up and smites thee. -- William Kershner http://crucis-court.com http://www.crucis.net/1632search On 05/27/2013 01:24 PM, Nux! wrote: On 27.05.2013 19:13, Mike Watson wrote: I've a small system that I use to support a number of churches. I provide web and email for them. My current server is running CentOS 6.3 with paired 1TB drives in a RAID1 configuration. It works well. One filesystem is very large, 500GB, and contains numerous large files: SQL, docs, church libraries in ebook and digital form, plus stored videos of church services. My problem is that I've found no means of backing up that file system. Dump and tar both error out as exceeding the maximum size. Neither will backup just the video directory (the largest) even with compression. Backup will be to an external (USB) removable HD. Can any suggest a prog or a method to back up this filesystem? Have you tried good old rsync? Or, if you want incremental backups, check rdiff-backup. I'm sure our list colleagues will come up with even more solutions. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
I'll check again, maybe NTSF. It's a singlepartition 1TB HD so it can't be FAT32. mw -- Lose not thy airspeed, lest the ground rises up and smites thee. -- William Kershner http://crucis-court.com http://www.crucis.net/1632search On 05/27/2013 01:43 PM, Robert Nichols wrote: On 05/27/2013 01:13 PM, Mike Watson wrote: Backup will be to an external (USB) removable HD. What file system is on that external HD? FAT32 has a 4GB limit for file size. ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On 05/27/2013 02:13 PM, Mike Watson wrote: I've a small system that I use to support a number of churches. I provide web and email for them. My current server is running CentOS 6.3 with paired 1TB drives in a RAID1 configuration. It works well. One filesystem is very large, 500GB, and contains numerous large files: SQL, docs, church libraries in ebook and digital form, plus stored videos of church services. My problem is that I've found no means of backing up that file system. Dump and tar both error out as exceeding the maximum size. Neither will backup just the video directory (the largest) even with compression. Backup will be to an external (USB) removable HD. Can any suggest a prog or a method to back up this filesystem? mw I use rsync to make a daily back up of my data on a second drive that is not normally mounted except when the backup is running. The drive is inside the same box so this backup is still subject to loss if the box is stolen or destroyed. I really should be backing up to an off site location. If you have a fire, flood, or other general disaster your local backup on an external drive isn't going to buy you anything unless you store the external drive off site. -- _ °v° /(_)\ ^ ^ Mark LaPierre Registered Linux user No #267004 https://linuxcounter.net/ ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On Mon, May 27, 2013 at 4:05 PM, Mike Watson mi...@crucis.net wrote: I'll check again, maybe NTSF. It's a singlepartition 1TB HD so it can't be FAT32. FAT32 can go to 2TB (you just can't format one that size in windows), but has the 4GB file size limit. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
That's my problem. I hitting a file size limit with dump and tar. mw -- Lose not thy airspeed, lest the ground rises up and smites thee. -- William Kershner http://crucis-court.com http://www.crucis.net/1632search On 05/27/2013 06:51 PM, Les Mikesell wrote: On Mon, May 27, 2013 at 4:05 PM, Mike Watson mi...@crucis.net wrote: I'll check again, maybe NTSF. It's a singlepartition 1TB HD so it can't be FAT32. FAT32 can go to 2TB (you just can't format one that size in windows), but has the 4GB file size limit. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On 2013-05-27, Mike Watson mi...@crucis.net wrote: I'll check again, maybe NTSF. It's a singlepartition 1TB HD so it can't be FAT32. Unless you have a desperate need for this disk to be readable by Windows, I would definitely make an ext3, ext4, or xfs filesystem on that USB drive. It is absolutely not worth the headache you'll have trying to restore properly from NTFS or FAT??. --keith -- kkel...@wombat.san-francisco.ca.us ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup of large filesystems 500GB
On Mon, May 27, 2013 at 7:05 PM, Mike Watson mi...@crucis.net wrote: That's my problem. I hitting a file size limit with dump and tar. Reformatting is the obvious solution so you can use one of the rsync-based backups - but if you really had to keep the FAT format you could use tar |split -b with some reasonable size for output. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup Redux
So going back to Amanda and Bacula ... I seem to recall that Amanda uses standard tools on the back end like gtar and/or dump, is that right? What does Bacula use? Does it use one of the standard tools? Or does it have its own proprietary format that it uses? thanks, -Alan -- “Don't eat anything you've ever seen advertised on TV” - Michael Pollan, author of In Defense of Food ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Re: [CentOS] Backup Redux
On Mon, Dec 19, 2011 at 9:49 AM, Alan McKay alan.mc...@gmail.com wrote: So going back to Amanda and Bacula ... I seem to recall that Amanda uses standard tools on the back end like gtar and/or dump, is that right? Amanda needs its own client installation on the target, but uses the local gnutar or dump, then stores it with a header pre-pended so you can skip over it if you needed to restore with only standard tools. What does Bacula use? Does it use one of the standard tools? Or does it have its own proprietary format that it uses? Not sure about that - it does also have its own agent to install, though. -- Les Mikesell lesmikes...@gmail.com ___ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
