[BackupPC-users] Efficiently browsing backups
Hi all, What is the best way to find a file or directory within a backup? The problem is that I know the sub-path, but not some of the parent paths. I also do not know which host the backup got created on... Example: /var/lib/origin/openshift.local.volumes/pods/5eded17a-73f8-11ea-b0e7-001a4a160109/volumes/kubernetes.io~glusterfs/pvc-02bd67a0-0493-11e9-8c1a-001a4a160120/volume/mydata I know everything that is belowpvc-XXX/volume/mydata, but I do not know5eded17a-73f8-11ea-b0e7-001a4a160109. There are hundreds to a couple thousand directories directly underpods/, and that is the case for every host. How would you guys go about this? I started by doing /usr/local/BackupPC/bin/BackupPC_ls -R -h myhost -n 497 -s / /var/lib/origin/openshift.local.volumes/pods | grep "pvc-02bd67a0-0493-11e9-8c1a-001a4a160120" but that seems like an inefficient approach... Thanks! ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Recommended settings for BackupPC v4 on ZFS
Thanks for the tip! On 9/12/19 3:03 PM, Robert Trevellyan wrote: > Your TL;DR: for ashift refers to ZFS record size, but ashift > determines block size (smallest I/O, with 12 meaning 4096 bytes to > match common modern hard drive sector size), not record size. > > Robert Trevellyan > > > On Thu, Sep 12, 2019 at 8:22 AM Alwyn Kik <mailto:al...@proteon.com>> wrote: > > I did some research for this a while ago, and currently use the > following setup: > https://gist.github.com/Alveel/c8e80aeef208a7e27c9bd50d0023420c Of > course this is still far from perfect, and I welcome more ideas or > criticism, but perhaps it can help you out :) > > On 9/11/19 10:03 PM, Carl Soderstrom wrote: > >> On 09/11 09:40 , Alexander Moisseev via BackupPC-users wrote: >>> On 11.09.2019 18:19, Robert Trevellyan wrote: >>>> I'm letting ZFS do the compression (using the default of LZ4) with >>>> BackupPC handling deduplication. I think you'll find a reasonable >>>> consensus that ZFS compression is always a win for storage space (it will >>>> store un-compressible data unmodified), whereas ZFS deduplication is best >>>> avoided in most cases, mostly due to its high memory usage. It's possible >>>> that BackupPC compression would be tighter than LZ4, >>> Actually, on ZFS you are not limited to LZ4, but in ZFS each file block >>> is compressed independently, that is why in most cases BackupPC compression >>> is higher, though it depends on data. >>> >>> We moved from 77.96G cpool to pool on compressed filesystem recently. >>> Now it consumes 81.2G, so there is not much difference. >>> >>> # zfs get compression,compressratio,recordsize,referenced zroot/bpc/pool >>> NAMEPROPERTY VALUE SOURCE >>> zroot/bpc/pool compressiongzip-3local >>> zroot/bpc/pool compressratio 3.87x - >>> zroot/bpc/pool recordsize 128K default >>> zroot/bpc/pool referenced 81,2G - >> Thanks Alexander, those details are really helpful. >> > -- > Alwyn > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > <mailto:BackupPC-users@lists.sourceforge.net> > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki: http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > > > > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ -- A. Kik ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Recommended settings for BackupPC v4 on ZFS
I did some research for this a while ago, and currently use the following setup: https://gist.github.com/Alveel/c8e80aeef208a7e27c9bd50d0023420c Of course this is still far from perfect, and I welcome more ideas or criticism, but perhaps it can help you out :) On 9/11/19 10:03 PM, Carl Soderstrom wrote: > On 09/11 09:40 , Alexander Moisseev via BackupPC-users wrote: >> On 11.09.2019 18:19, Robert Trevellyan wrote: >>> I'm letting ZFS do the compression (using the default of LZ4) with BackupPC >>> handling deduplication. I think you'll find a reasonable consensus that ZFS >>> compression is always a win for storage space (it will store >>> un-compressible data unmodified), whereas ZFS deduplication is best avoided >>> in most cases, mostly due to its high memory usage. It's possible that >>> BackupPC compression would be tighter than LZ4, >> Actually, on ZFS you are not limited to LZ4, but in ZFS each file block is >> compressed independently, that is why in most cases BackupPC compression is >> higher, though it depends on data. >> >> We moved from 77.96G cpool to pool on compressed filesystem recently. Now it >> consumes 81.2G, so there is not much difference. >> >> # zfs get compression,compressratio,recordsize,referenced zroot/bpc/pool >> NAMEPROPERTY VALUE SOURCE >> zroot/bpc/pool compressiongzip-3local >> zroot/bpc/pool compressratio 3.87x - >> zroot/bpc/pool recordsize 128K default >> zroot/bpc/pool referenced 81,2G - > Thanks Alexander, those details are really helpful. > > -- Alwyn ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Steps to get V4.3.0 running ASAP
Glad you found your solution. I would argue the easiest/fastest right now would be to use a docker image. We've been using adferrand/backuppc for about a month now and it works well. Besides learning docker if you haven't already and setting up a reverse proxy, I don't think it should take longer than 30 minutes to set up. Cheers! -BEGIN PGP MESSAGE- hQEMA0e9wLtV1fbpAQf+ICvT3ABerTSXYOG8+vB21KL2M8IWzn0dUvMHUQ+sIfdo OlF07MjltDNPU4GukhBqVNNx0sGRwS/H8tOMeTXnY8jXQMc3CSTTXpTkp4INaWk9 JUcOwcocaVkxgZogTmtl9eMU5AMAsLIa+K6LkRijD2HIPHQqL1v4k84fGWIZxHvh MDk5Eg0tMIQuzGeJ/huZb/8pCN7e00XWbnnudapNBvM2gYFQuykDKqg2zTw6aAHM Sa75YNFihY5c1KuO6DT3BpSk2v4okZqEWj+9ypSRYhbKhkVVlpw6q8GYPL5Zsm4o 792gPIU6RJi2/CI2DIvUmIRjlu3oHEafkX/dANCrCNLrAdh180hQ5/OBp7hfd+wE IxXKhCxJgYhHlXXVxyJwgZH808UxQlEr6iGNNofLcNrl/Iuyx9pKTHMnenUoxBvH hXhjFEc3bJTlNf9Xh4IhglYVXmRPN0GVaArznDy6N7ir6SQWk496be3Z83BrtdQM JZJxzksk10RR41szSz0m2pnXx76PyznQferrNj9zRFyU4IL2MSn77yU72DJLf+3o BYZIciUSiPkSHuCmKq8TPBbiMkZXg18k7LAjrIIY1kH0uAxBGECkTfYHGU9kT9Wu cI7UaL/R7SpqQHdsnRkFXt9tscoWNeHXZjjlCG9tnoRSYfv7WglBgshkTA5mQ6hi o/mjEsPvkoLcyrnPlnPXjUjq1DOQWUmHEqV59g6b5gPjJPwKixhf7/8HryK9NwkQ r85o9DqbnBzkWofGS3fQwcgA7ggsRyw2dNcgWssdHRT84qHMsfhi6f4tM+tYWLe4 VXjbMtcfgqNHRtVPa4MFP5PDJXJycVCyWE5i8qfHApY8kbaBg5GobizGz2E/IHii wFa05OsmxW2+nr14pQE8UDhAGuX5msEucP8mjR2gplq+IN+Oboi9ZqWpcGpLD2Mh +y48NRfBajR/LmpOlDz/rw370xFl+XjWf3KW3XfUMwfQaHhkpTQS5uN+6vKsBcWq DkpBuWXn3xvg7/RBbP4BDSXxlcIYrCJjDsyp7PKEd/nVhy1NfstyM2FAUSt48UaP VIIElHFOHymfxpZAhpaswTAjXlSzvcqUvlu/M0EyMHetKgY4o45d2euu/OStP+pZ 85ShlAOe3f0HHOCF7O1MMIw+u1iF5Mv4BKZyp1JOSoMEV/qEwO8xVus1l9FMx/ji zNa0kXyihXNqK1kv7DAmt5EH0CM9624xCzIRUXqMBpAD1l5hl+eJyvUsrO0P/zmX auWPklxWB/v+S7HAlznk4l1I9N/5I3X+loYsRbDggYCj5lQIXBdiUMJjE/Lf1weJ s7m8y6pfOhCbh6KY53rqgpU5d3AzDA7YaxAUmae2YC4yuiiChbXH4lJ9gQtVnF79 QQlpGH3fQW9uzm+uh8TIW9n+b17O611rkjHQEWTZ91pl0jKWdIUQ0wRpddU8Bt6u mewJacs4noiCYubUCLhk9ncfkYRuMn0QiMdX5C+OcA+so/c8G3KcP4i9CiJ6iese 92jWaUdXlLRww/UnLyfIK2HiaTzOnB2Eqk3ET1CyH1C6zTzedAlbT+8uEwFWwRel E1nJwQwcsdFLEeYB0KeRd5E9h1UFN7XY7Sra7FALCqdmqBKtwsF7u3+YWc6RklRe Y2vInax8rHOm0TeCDdnkAsm9jZiEa3ys5lDfeqZKXTxJB0FcqSNXhIbWHD1Fdrct WDUVlf/7Fxj23LDu0eOrNfHqDOlZ9tFLbl1jDSmVh7Jq7dNLGAIkpCVABwgka1R+ jjmji3Z/8FWgHlL6JnIrGEQd2d3oPTgKa6LJvqPeekKUvQ08Yv5RSkF3R53k5kSQ Z5QXL+vPoWBeD7jPGSIQr64PxIa2H7RGfd1KWPrPFPkNNyRr/VrWHrq1MM5X5MD8 iWwt+JQ4LagMgTBbO/1mIJ6oi/aKV0MIZ4c5MicmLNXS2cuxR7NL+Jogh8HUMXLR zUju8kuxddBtCNG+AM2MXzKMG5nxat523ZyrWKR49tsnmFwtoeWz+hSkuu3Ne9NB Get6uzQ4mxAk/QtcSKMuN71S2bRe9o50C9lgOtQWdxvphkSo/b7hFB4W09qsFs3U jw84Jil0m1pobNekIliJbkGeQQJOfaRoErnCofdK2Qu/vcJgVQCmVsiJBfSCWk32 Wyk9r/+Ehhg470xOU3sqyqTkaz3JjLgPOAP9oFTi1qig01i0+8jpRkn14mX6JYLd rpKTCZ5DFnJB3Fvvy9fAshg84QUD2WdF0YBCOiymr6+8/P6C0em0K5F9Ws17/bT1 OUQNGtlOWXi1lAuBF45LtnniNhAjc6Zv6ek+V19B0jWIa3LaV3P4wmT8l5/57ttG 1YgvBWl5Coif2N4/ndU6dcqxcXb3NMTbJ4DkqTJm2TnQF2Y2BdXdEFfaa/O96a18 /Y84zEd0P4/XiljXDtqrAALAEb8yIy1XA3LR4mPd61Xo8NDGuYC8XvB6YrOx0PO/ D12mYmce7XLY35zLZnBNZaWxUFIr2xaw0KaNNodj3/FV6NZwpewE0ACXt/fq3YGz UoonpCSXG3xlF1AkUUj6TNwZFGoo3EtapGujOOl/fIIn4JeS4Eow8XORMRW8sXa1 1UwNch7aP7ml4af7DhNTu/pV1S5+VflBifpXGintK/ncZPZlfigFG1k6AmpxAuNl tGJluGuOmPJYLBeQgMWA+G9eNnkFNy4MlzH5RV2oCyhquedAPPtTOzHQzv5gFp09 pE4m3lwQamnD0c+DcSkLdITptYJZqVnXa6Du90K6NbhVW29slcYQv1PQf/lSjlk2 AsqaTumBUCZ5M08vUK37yN+4inHpTsywp2kUhiUzEFNf1yRw8A2X8q1nbDzmKn9e VlPp61IaL83JbB92n+9YhYTiX3HRnY70Nc+YBeqyhPtNo3ZrA4Aohv6f1DRagAv4 nXaOCze60TROzF4hc7X2QC0sVKO5Jdsm+OxGWU2tAGtw7gAJys9G38ZWkHuCcskX j1O5C3sgRMH48P0m2ug4mHB6pbrc/nrkIexBoJzxsqn83Nwa6ohlhgxsAtwnd0gz FGe3aPfsALPYql1oqlSC5rrpW3dobggYgV8oAcAzuhW2/UMEUXyNlX7qaDEiCyZS O+KXlFbNq+2Eci/jN3keYn6+aTHBhvBYd88gYtRQd/Y36l2SZpm1N/aw+AndwTJ6 u3bjsBm6K8u6c8nzqQZBWYRqO0SpvAi7THx7SEms2V5BZdDn/UrY6EH3KqrBthuP fKSxheMdvEiTNUyONNt8nsimipuSjSz5TeJi/laWO4JdcsrfGmZBBYDM0yzCdLOm g73Lo/j4TSyuKPmrn9qiMHwFmslVyc57ukL8jYncnQADS+iZkQKl0yZlmxwjkJFh HxIx+xsFKqtCfVLW/CAOyMky61lMhAYEgHJQv8EkiGVrpiqWmlCzDSD6S914bfXf 3rXzK+sXmwVI9lOoFY3Ta3wye2vi3FUoS6uZ4p0RuWVI9PMTCcUvajzUG6VP8kVZ Mkst1Fs2mtKITE4gimaONstOz39K/r11GpRI1o1RhQI9GUDjN9U9Ktjroujc6em4 /9/PXDZa87p/eXocfwpJ0whOMtstNh8YAOTDRqTMVjdVmqsansTDjrgCcD0ucz00 Ev+LkOElQB8x3JjPB4GzSntHWQM4zCyl2bLqmQF090ZJu4LbWXiDSJWt2M4pu2I1 Epke4giRG/E4sCCPNBREDFX6ajK+uoCU9BifvzbwC9o55BPx63cZlM41tf5SZniQ 3H2CBBVv7PG5R/Fy4C8tDKHpSTBEYdhv+DUBu3u0yEiogpuZPYo6gB8TkWYSZszT D1BstvZEjJR6ExPcIeI7rQHrrJj3CdY5bbTS2usuNfTi0njGofbYIY+si32nGCdx PATb3VLwoWHhkjJp5ENBGoEfJyMeeR1YZ0iisRBrUGMhrBZZbw== =fT0t -END PGP MESSAGE- noname Description: application/pgp-encrypted ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Pool subdirectories default permissions
Craig,
The permissions are as follows:
- pc 0755
- pool 0755
- pc subdirectories 0750, so those look fine
- pool subdirectories are 0777, and files are 0444
I manually changed the permissions of these folders for now, however
new directories appear to be 0777 again.
Using rsyncd on one instance, and rsync on another. The specific
instance I started this mail for is rsync.
BackupPC version 4.3.0, rsync_bpc version 3.0.9.12.
Thanks,
Alwyn
On Wed, Dec 5, 2018 at 6:19 PM Craig Barratt via BackupPC-users
wrote:
>
> Alwyn,
>
> 23 decimal = 027 (octal), which should mask off other rwx permissions.
>
> What are the permissions on the pool and cpool directories? If you look
> inside the pc directories, what are the permissions there?
>
> What Xfer method are you using? What is your version of BackupPC and
> rsync_bpc?
>
> Craig
>
>
>
> On Wed, Dec 5, 2018 at 12:47 AM Alwyn Kik wrote:
>>
>> ZFS, where the ZFS filesystem is in a zpool using disks directly.
>>
>> On Tue, Dec 4, 2018 at 3:52 PM Tony Schreiner
>> wrote:
>> >
>> > What type of filesystem is your pool on?
>> >
>> > On Tue, Dec 4, 2018 at 6:44 AM Alwyn Kik wrote:
>> >>
>> >> Apparently it was set to '23'.
>> >>
>> >> That said, on our other instances it is set to '027', and the directories
>> >> in pool are still 777.
>> >>
>> >> Thanks,
>> >>
>> >> Alwyn
>> >>
>> >> On 12/4/18 5:51 AM, Craig Barratt via BackupPC-users wrote:
>> >>
>> >> What is $Conf{UmaskMode} set to?
>> >>
>> >> Craig
>> >>
>> >> On Mon, Dec 3, 2018 at 9:43 AM Alwyn Kik wrote:
>> >>>
>> >>> I noticed every subdirectory under $TopDir/pool has permissions 777 by
>> >>> default.
>> >>>
>> >>> Can anyone elaborate on why that might be? Is it possible to change that?
>> >>>
>> >>> Thanks!
>> >>>
>> >>> ___
>> >>> BackupPC-users mailing list
>> >>> BackupPC-users@lists.sourceforge.net
>> >>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> >>> Wiki:http://backuppc.wiki.sourceforge.net
>> >>> Project: http://backuppc.sourceforge.net/
>> >>
>> >>
>> >>
>> >> ___
>> >> BackupPC-users mailing list
>> >> BackupPC-users@lists.sourceforge.net
>> >> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> >> Wiki:http://backuppc.wiki.sourceforge.net
>> >> Project: http://backuppc.sourceforge.net/
>> >>
>> >> ___
>> >> BackupPC-users mailing list
>> >> BackupPC-users@lists.sourceforge.net
>> >> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> >> Wiki:http://backuppc.wiki.sourceforge.net
>> >> Project: http://backuppc.sourceforge.net/
>> >
>> > ___
>> > BackupPC-users mailing list
>> > BackupPC-users@lists.sourceforge.net
>> > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> > Wiki:http://backuppc.wiki.sourceforge.net
>> > Project: http://backuppc.sourceforge.net/
>>
>>
>> ___
>> BackupPC-users mailing list
>> BackupPC-users@lists.sourceforge.net
>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> Wiki:http://backuppc.wiki.sourceforge.net
>> Project: http://backuppc.sourceforge.net/
>
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Pool subdirectories default permissions
ZFS, where the ZFS filesystem is in a zpool using disks directly.
On Tue, Dec 4, 2018 at 3:52 PM Tony Schreiner wrote:
>
> What type of filesystem is your pool on?
>
> On Tue, Dec 4, 2018 at 6:44 AM Alwyn Kik wrote:
>>
>> Apparently it was set to '23'.
>>
>> That said, on our other instances it is set to '027', and the directories in
>> pool are still 777.
>>
>> Thanks,
>>
>> Alwyn
>>
>> On 12/4/18 5:51 AM, Craig Barratt via BackupPC-users wrote:
>>
>> What is $Conf{UmaskMode} set to?
>>
>> Craig
>>
>> On Mon, Dec 3, 2018 at 9:43 AM Alwyn Kik wrote:
>>>
>>> I noticed every subdirectory under $TopDir/pool has permissions 777 by
>>> default.
>>>
>>> Can anyone elaborate on why that might be? Is it possible to change that?
>>>
>>> Thanks!
>>>
>>> ___
>>> BackupPC-users mailing list
>>> BackupPC-users@lists.sourceforge.net
>>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>>> Wiki:http://backuppc.wiki.sourceforge.net
>>> Project: http://backuppc.sourceforge.net/
>>
>>
>>
>> ___
>> BackupPC-users mailing list
>> BackupPC-users@lists.sourceforge.net
>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> Wiki:http://backuppc.wiki.sourceforge.net
>> Project: http://backuppc.sourceforge.net/
>>
>> ___
>> BackupPC-users mailing list
>> BackupPC-users@lists.sourceforge.net
>> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
>> Wiki:http://backuppc.wiki.sourceforge.net
>> Project: http://backuppc.sourceforge.net/
>
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] Pool subdirectories default permissions
Apparently it was set to '23'.
That said, on our other instances it is set to '027', and the
directories in pool are still 777.
Thanks,
Alwyn
On 12/4/18 5:51 AM, Craig Barratt via BackupPC-users wrote:
> What is $Conf{UmaskMode} set to?
>
> Craig
>
> On Mon, Dec 3, 2018 at 9:43 AM Alwyn Kik <mailto:al...@proteon.com>> wrote:
>
> I noticed every subdirectory under $TopDir/pool has permissions
> 777 by default.
>
> Can anyone elaborate on why that might be? Is it possible to
> change that?
>
> Thanks!
>
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> <mailto:BackupPC-users@lists.sourceforge.net>
> List: https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki: http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
>
>
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
[BackupPC-users] Pool subdirectories default permissions
I noticed every subdirectory under $TopDir/pool has permissions 777 by default. Can anyone elaborate on why that might be? Is it possible to change that? Thanks! ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
