Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Op 20120104 om 09:17 schreef Marco van Wieringen: On 01/ 4/12 12:03 AM, Wolfgang Denk wrote: snip/ I understand the problem is located, and no further testing on my side is needed? Correct I see what is causing this problem. I only need to code a workaround and test it so that will take some time to do it right. As I expected the current extended attribute code is sufficient perform the backup and restore no need to implement yet an other interface. I installed a Fedora 16 VM and can now test on that. As I read http://bugs.bacula.org/view.php?id=1807 is bug #1807 fixed. HtH -- Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Geert Stappers Geert.Stappers at vanadgroup.com writes: Op 20120104 om 09:17 schreef Marco van Wieringen: On 01/ 4/12 12:03 AM, Wolfgang Denk wrote: snip/ I understand the problem is located, and no further testing on my side is needed? Correct I see what is causing this problem. I only need to code a workaround and test it so that will take some time to do it right. As I expected the current extended attribute code is sufficient perform the backup and restore no need to implement yet an other interface. I installed a Fedora 16 VM and can now test on that. As I read http://bugs.bacula.org/view.php?id=1807 is bug #1807 fixed. Correct I tested this on fedora16 with backing up the ping binary and restoring it. After the restore now the extended attribute is put back and the posix capability is visible in both the attr -l and ls output. Using the restored binary also works as a normal user so that is enough proof for me that it works. So you need to enable xattrsupport=yes and then posix file capabilities are saved. On restore the posix file capabilities are restored on the files being restored. This is done using a so called delayed restore of both the ACL and XATTR streams by the bacula filed on restore. This new implementation also fixes the acl restore on AIX (which also got clobbered by the chmod and chown done by the filed after it extracted all file data.) And Solaris without the proper acl setting on for instance zfs would clobber its acls on restore. As the bug report shows its put back for the 5.2.4 release. So when that is released the patches will be part of that. I fixed quite some other problems with xattr and acls on both Solaris and AIX this week which were triggered by a more extended regression test which showed problems on the Solaris platform. And some AIX problems which were reported here on the user list after working with the reporter on it today we found the problem and fixed it for the future. All bugs are traceable via the bugs site and have gotten separate ids and have been closed as of today. Marco -- Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
On 01/ 4/12 12:03 AM, Wolfgang Denk wrote: Dear Marco, In message4f030530.7050...@planets.elm.net you wrote: But I think I have found the problem. First of all we have a snarfu due to fixing bug #1610 see http://bugs.bacula.org/view.php?id=1806 ... http://bugs.bacula.org/view.php?id=1807 I need to think how we are going to solve that. I understand the problem is located, and no further testing on my side is needed? Correct I see what is causing this problem. I only need to code a workaround and test it so that will take some time to do it right. As I expected the current extended attribute code is sufficient perform the backup and restore no need to implement yet an other interface. I installed a Fedora 16 VM and can now test on that. Marco -- Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Hello, So I would say give the xattr=yes a go on your install and see if it works for these attributes. You could create a test fileset with a known file with a posix file capability and run the bacula-fd with a debug level of 100 and watch for xattr save messages. I can confirm the version of Bacula in Fedora 16 you're mentioning is compiled with the xattr support. If you want to try a later build with the same version you can download the latest update which is still in updates-testing. To install you should do (as root): yum -y --enablerepo=updates-testing update bacula-\* Here are the update requests with the changes: https://admin.fedoraproject.org/updates/bacula-5.0.3-17.fc16 https://admin.fedoraproject.org/updates/bacula-5.0.3-17.fc15 The line that states POSIX.1e capabilities in the changelog is only relevant to the enablement of bacula-fd regarding the ReadAll capability (-k). Otherwise you can try the latest 5.2.3 backported from rawhide: http://repos.fedorapeople.org/repos/slaanesh/bacula/ http://repos.fedorapeople.org/repos/slaanesh/bacula/README.txt Regards, --Simone -- You cannot discover new oceans unless you have the courage to lose sight of the shore (R. W. Emerson). -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
On 01/ 3/12 11:10 AM, Simone Caronni wrote: Hello, So I would say give the xattr=yes a go on your install and see if it works for these attributes. You could create a test fileset with a known file with a posix file capability and run the bacula-fd with a debug level of 100 and watch for xattr save messages. I can confirm the version of Bacula in Fedora 16 you're mentioning is compiled with the xattr support. If you want to try a later build with the same version you can download the latest update which is still in updates-testing. To install you should do (as root): yum -y --enablerepo=updates-testing update bacula-\* Here are the update requests with the changes: https://admin.fedoraproject.org/updates/bacula-5.0.3-17.fc16 https://admin.fedoraproject.org/updates/bacula-5.0.3-17.fc15 The line that states POSIX.1e capabilities in the changelog is only relevant to the enablement of bacula-fd regarding the ReadAll capability (-k). Ok interesting I'm currently looking at this code as I'm not so sure it wise to use that if you want to save acls and xattr successfully. I still have to see how this priviledge stuff works on Linux but we might need some more to be able to save acl and xattr data. Otherwise you can try the latest 5.2.3 backported from rawhide: http://repos.fedorapeople.org/repos/slaanesh/bacula/ http://repos.fedorapeople.org/repos/slaanesh/bacula/README.txt For now I just compiled the code myself so that I can run a debugger on it. But I think I have found the problem. First of all we have a snarfu due to fixing bug #1610 see http://bugs.bacula.org/view.php?id=1806 for those interested. We currently save on Linux and OSX one and only one xattr for any file. The fix for that was obvious and is commited. I also created a second bug for the posix file capability restore issue. Because when I fix bug 1806 the backup of the posix file capability works and the restore also (when I strace the process I see the restore of the data taking place.) But we run into an other problem as Bacula restores the mode and owner later on and does a chown and chmod on the restored file and that clears the posix file capability. For those interested: http://bugs.bacula.org/view.php?id=1807 I need to think how we are going to solve that. Marco -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Dear Marco, In message 4f030530.7050...@planets.elm.net you wrote: But I think I have found the problem. First of all we have a snarfu due to fixing bug #1610 see http://bugs.bacula.org/view.php?id=1806 ... http://bugs.bacula.org/view.php?id=1807 I need to think how we are going to solve that. I understand the problem is located, and no further testing on my side is needed? Thanks. Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de ...one of the main causes of the fall of the Roman Empire was that, lacking zero, they had no way to indicate successful termination of their C programs. - Robert Firth -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Wolfgang Denk wd at denx.de writes: A number of tools in recent Linux distributions (say, Fedora 16) rely on file capabilities for correct operation. For example, rlogin will only work for regular uses when the cap_net_bind_service capability is set: - getcap -v /usr/bin/rlogin /usr/bin/rlogin = cap_net_bind_service+ep Without this capability, non-root users will only get: - rlogin name rcmd: socket: Permission denied It appears that bacula does not save, and thus cannot restore, such file capabilities. Thats not really true. I did some searching on google to find out how these so called POSIX file capabilities are implemented. Its also quite new code it went into Linux 2.6.24 in may last year or so. There is quite some info on the new option at http://www.friedhoff.org/posixfilecaps.html As it seems there is a new interface which mimics the acl subsytem. But the low level implementation is based on extended attributes. So probably if you enable xattr = yes and save the extended attributes you are set and things backup fine and restore fine. The result is that any restore of a root file system will have a (usually unknown) number of files that don't work correctly any more. I searched the mailing list archives and the documentation, but could not find any reference to dealing with file capabilities. Am I missing something? Nope they are so new and no mainstream distro seems to have implemented them already. (Fedora is probably one of the first to do so.) Is there a way to perform correct backups under Linux, i. e. to backup and be able to restore things like ACLs and especially file capabilities? Yup add acl = yes and xattr = yes to your fileset and you should be set to backup most of the future options. Bacula is one of the few Open Source backup products (probably the only) which has very broad support for all these kind of exotic acl's, extended attributes and extensible attributes. I had to write everything from scratch as no other projects address all know interfaces. So we are quite good in doing the exotic stuff. You want to do xattrs anyhow as selinux also uses it a lot. If not, are there any plans to add such a feature? I don't plan on adding the additional interface for capabilities as the generic xattr interface should be sufficient. If its not we may look at cloning the acl code and interface to the posix file capabilities API. Its quite the same as acl, but as acl's on Linux are also stored as extended attributes we already have enough overhead in supporting both the ACL and XATTR interfaces on Linux so I would prefer not to add an other interface if the generic extended attribute code works. We already found out that Novell uses extended attributes for storing additional access control lists on there NSS filesystem. And those also backup and restore fine with the generic xattr code. See http://www.bacula-konferenz.de/historie/2011/sicherung-von-nss-filesystemen-mit-bacula/at_download/file Note that this is probably a bigger problem - it appears that neither cpio nor tar nor rsync etc. can deal with file capabilities. At the moment I don't know how to create a 100% correct backup of a plain vanilla Linux root filesystem... If you look at the linked webpage you will see that rsync and cpio have support for extended attributes and that is used to copy these posix file capabilities. So I would say give the xattr=yes a go on your install and see if it works for these attributes. You could create a test fileset with a known file with a posix file capability and run the bacula-fd with a debug level of 100 and watch for xattr save messages. Marco -- Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Dear Marco,
In message loom.20120102t163316-...@post.gmane.org you wrote:
It appears that bacula does not save, and thus cannot restore, such
file capabilities.
Thats not really true. I did some searching on google to find out
how these so called POSIX file capabilities are implemented.
Its also quite new code it went into Linux 2.6.24 in may last year or so.
Hm... v2.6.24 is four years old... Maybe you mean v2.6.34,
and May 2010?
Yup add acl = yes and xattr = yes to your fileset and you should
be set to backup most of the future options. Bacula is one of
Hm... I have these settings in the FileSet definition:
Include {
Options {
signature = MD5
xattrsupport = yes
aclsupport = yes
}
File = /usr/bin
}
When restoring, the file attributes were lost anyway.
Is there any other place I need to give extra options? When
restoring?
the few Open Source backup products (probably the only)
which has very broad support for all these kind of exotic
acl's, extended attributes and extensible attributes. I had to
write everything from scratch as no other projects address all
know interfaces. So we are quite good in doing the exotic stuff.
Guess why I've been using bacula for so long...
And btw: thanks :-)
We already found out that Novell uses extended attributes for
storing additional access control lists on there NSS filesystem.
And those also backup and restore fine with the generic xattr code.
I'm just a user of bacula, no developer of it, so I don't care much
about the implementation or the interface. As long as the
functionality is present and working I'm fine with it.
Note that this is probably a bigger problem - it appears that
neither cpio nor tar nor rsync etc. can deal with file capabilities.
At the moment I don't know how to create a 100% correct backup of a
plain vanilla Linux root filesystem...
If you look at the linked webpage you will see that rsync and cpio
have support for extended attributes and that is used to copy these
posix file capabilities.
In the linked PDF file I cannot find a reference to cpio or rsync.
But rsync does indeed work as needed when using -X. Sorry, I missed that.
The cpio in Fedora 16 does not appear to support this.
So I would say give the xattr=yes a go on your install and see
if it works for these attributes. You could create a test fileset
with a known file with a posix file capability and run the bacula-fd
with a debug level of 100 and watch for xattr save messages.
Done that, but I could not see any.
This is with bacula as distributed with Fedora 16, most recent
updates installed:
bacula-client-5.0.3-13.fc16.x86_64
bacula-common-5.0.3-13.fc16.x86_64
bacula-console-5.0.3-13.fc16.x86_64
bacula-console-bat-5.0.3-13.fc16.x86_64
bacula-director-common-5.0.3-13.fc16.x86_64
bacula-director-mysql-5.0.3-13.fc16.x86_64
bacula-docs-5.0.3-13.fc16.x86_64
bacula-storage-common-5.0.3-13.fc16.x86_64
bacula-storage-mysql-5.0.3-13.fc16.x86_64
Anything ales I could look for?
Thanks in advance.
Best regards,
Wolfgang Denk
--
DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de
Punishment becomes ineffective after a certain point. Men become in-
sensitive.
-- Eneg, Patterns of Force, stardate 2534.7
--
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution, easily deploy virtual
desktops for less than the cost of PCs and save 60% on VDI infrastructure
costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox
___
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Dear Marco, I wrote: So I would say give the xattr=yes a go on your install and see if it works for these attributes. You could create a test fileset with a known file with a posix file capability and run the bacula-fd with a debug level of 100 and watch for xattr save messages. Done that, but I could not see any. The log file is available here: ftp://ftp.denx.de/pub/tmp/log.gz Best regards, Wolfgang Denk -- DENX Software Engineering GmbH, MD: Wolfgang Denk Detlev Zundel HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: w...@denx.de They that can give up essential liberty to obtain a little temporary saftey deserve neither liberty not saftey. - Benjamin Franklin, 1759 -- Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Wolfgang Denk wd at denx.de writes:
Dear Marco,
In message loom.20120102T163316-867 at post.gmane.org you wrote:
It appears that bacula does not save, and thus cannot restore, such
file capabilities.
Thats not really true. I did some searching on google to find out
how these so called POSIX file capabilities are implemented.
Its also quite new code it went into Linux 2.6.24 in may last year or so.
Hm... v2.6.24 is four years old... Maybe you mean v2.6.34,
and May 2010?
No idea I'm not much into Linux these days the website describing
the info seems to indicate its in 2.6.24 and later if that's 4 years
old then it been there a bit longer only no one used it or nobody
did look at it.
Yup add acl = yes and xattr = yes to your fileset and you should
be set to backup most of the future options. Bacula is one of
Hm... I have these settings in the FileSet definition:
Include {
Options {
signature = MD5
xattrsupport = yes
aclsupport = yes
}
File = /usr/bin
}
When restoring, the file attributes were lost anyway.
Ok interesting.
Is there any other place I need to give extra options? When
restoring?
Nope when its saved it will be restored unless you restore
to a filesystem where no xattr can be restored. So restore
to a real filesystem not to things like ramfs etc because
those are known to be very limited.
the few Open Source backup products (probably the only)
which has very broad support for all these kind of exotic
acl's, extended attributes and extensible attributes. I had to
write everything from scratch as no other projects address all
know interfaces. So we are quite good in doing the exotic stuff.
Guess why I've been using bacula for so long...
No idea I only have been for 3 years+ and the support was
only added in these 3 years before that the support was rather
sparse.
And btw: thanks
We already found out that Novell uses extended attributes for
storing additional access control lists on there NSS filesystem.
And those also backup and restore fine with the generic xattr code.
I'm just a user of bacula, no developer of it, so I don't care much
about the implementation or the interface. As long as the
functionality is present and working I'm fine with it.
But I do because I don't want to code and support code when
it not needed and we can use a generic support layer.
Note that this is probably a bigger problem - it appears that
neither cpio nor tar nor rsync etc. can deal with file capabilities.
At the moment I don't know how to create a 100% correct backup of a
plain vanilla Linux root filesystem...
If you look at the linked webpage you will see that rsync and cpio
have support for extended attributes and that is used to copy these
posix file capabilities.
In the linked PDF file I cannot find a reference to cpio or rsync.
The PDF linked is about the NSS stuff the website linked has quite
some info on how they are implemented and how they are stored and retrieved
using normal tools like attr etc.
But rsync does indeed work as needed when using -X. Sorry, I missed that.
Ok that is at least something as rsync uses the same interface
we use on Linux for extended attributes so either you are restoring
to a filesystem without xattr support or we are for whatever reason
not getting these xattr saved,
The cpio in Fedora 16 does not appear to support this.
So I would say give the xattr=yes a go on your install and see
if it works for these attributes. You could create a test fileset
with a known file with a posix file capability and run the bacula-fd
with a debug level of 100 and watch for xattr save messages.
Done that, but I could not see any.
Ok, so its on Fedora 16 will install a VM then and see what
is going on in that respect. You got me curious on why the
xattr doesn't get saved.
This is with bacula as distributed with Fedora 16, most recent
updates installed:
bacula-client-5.0.3-13.fc16.x86_64
bacula-common-5.0.3-13.fc16.x86_64
bacula-console-5.0.3-13.fc16.x86_64
bacula-console-bat-5.0.3-13.fc16.x86_64
bacula-director-common-5.0.3-13.fc16.x86_64
bacula-director-mysql-5.0.3-13.fc16.x86_64
bacula-docs-5.0.3-13.fc16.x86_64
bacula-storage-common-5.0.3-13.fc16.x86_64
bacula-storage-mysql-5.0.3-13.fc16.x86_64
Anything ales I could look for?
I guess its compiled with xattr support otherwise you
would get fatal errors on backup trying to enable xattrsupport.
I'll see if I can find something obvious, but as rsync works
I'm confident that we should just save them without additional code.
Marco
--
Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex
infrastructure or vast IT resources to deliver seamless, secure access to
virtual desktops. With this all-in-one solution,
Re: [Bacula-users] Linux: backup and restore of file capabilities ?
Wolfgang Denk wd at denx.de writes: Dear Marco, I wrote: So I would say give the xattr=yes a go on your install and see if it works for these attributes. You could create a test fileset with a known file with a posix file capability and run the bacula-fd with a debug level of 100 and watch for xattr save messages. Done that, but I could not see any. The log file is available here: ftp://ftp.denx.de/pub/tmp/log.gz Ok that log is kind of useless. I'm not to interested in the director and sd as they are not really involved and couldn't care less what you save. I would run a bacula-fd -d 100 -f on a test client. e.g. first shutdown the original bacula-fd and redirect that output to a file. When we get that data of the backup we can take it from there first lets make sure bacula-fd sees the xattr. Like I said in the other mail its better to put just one file you know for sure has these capability in a seperate fileset. Marco -- Ridiculously easy VDI. With Citrix VDI-in-a-Box, you don't need a complex infrastructure or vast IT resources to deliver seamless, secure access to virtual desktops. With this all-in-one solution, easily deploy virtual desktops for less than the cost of PCs and save 60% on VDI infrastructure costs. Try it free! http://p.sf.net/sfu/Citrix-VDIinabox ___ Bacula-users mailing list Bacula-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bacula-users
