[gentoo-dev] GSoC proposal: cp --reflink support for zfs.
Hi, I would like to implement cp --reflink support for ZFSOnLinux as my GSoC project. cp --reflink is used to create a COW copy of a file, so the file will not take any disk space if it's not modified. This feature is very useful for cases like storing a lot of almost identical virtual machine images. Also this is a frequently requested feature for ZoL. [1][2][3] Currently only btrfs support this feature, so my goal it to bring it to ZoL as well. I think the only way to do it (without changing too many parts of ZoL) is to use the deduplication feature of zfs. A COW copy could be done by create a new entry in ddt for the old file, and create a new file which points to the ddt entry. Please let me know if this proposal makes sense, and if that's the right way to do it. Thanks. [1]: https://groups.google.com/a/zfsonlinux.org/forum/#!topic/zfs-discuss/mvGB7QEpt3w [2]: https://github.com/zfsonlinux/zfs/issues/405 [3]: https://github.com/zfsonlinux/zfs/issues/1063 -- Regards Yuxuan Shui
Re: [gentoo-dev] GSoC proposal: cp --reflink support for zfs.
On 12/03/14 03:15 AM, Yuxuan Shui wrote: Hi, I would like to implement cp --reflink support for ZFSOnLinux as my GSoC project. cp --reflink is used to create a COW copy of a file, so the file will not take any disk space if it's not modified. This feature is very useful for cases like storing a lot of almost identical virtual machine images. Also this is a frequently requested feature for ZoL. [1][2][3] Currently only btrfs support this feature, so my goal it to bring it to ZoL as well. I think the only way to do it (without changing too many parts of ZoL) is to use the deduplication feature of zfs. A COW copy could be done by create a new entry in ddt for the old file, and create a new file which points to the ddt entry. Please let me know if this proposal makes sense, and if that's the right way to do it. Thanks. [1]: https://groups.google.com/a/zfsonlinux.org/forum/#!topic/zfs-discuss/mvGB7QEpt3w [2]: https://github.com/zfsonlinux/zfs/issues/405 [3]: https://github.com/zfsonlinux/zfs/issues/1063 While I can't comment too much on the technical aspects, they seem to be relatively sound. However, there are some issues with the, er... other aspects, for lack of better terminology. 1. This is possibly out of scope as a Gentoo project, since ZOL is not really part of Gentoo. If it's not, then you're out of luck, because ZOL is not an accepted organization. 2. This is likely too small to be a GSoC project. Perhaps see [0] for a list of example ideas, if only so you can get a grasp on the size of a good project. It does sound like a good idea though, and even if you can't do it as part of GSoC, you should pursue it anyways. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] GSoC proposal: cp --reflink support for zfs.
On 03/12/2014 08:45 AM, Alex Xu wrote: On 12/03/14 03:15 AM, Yuxuan Shui wrote: Hi, I would like to implement cp --reflink support for ZFSOnLinux as my GSoC project. cp --reflink is used to create a COW copy of a file, so the file will not take any disk space if it's not modified. This feature is very useful for cases like storing a lot of almost identical virtual machine images. Also this is a frequently requested feature for ZoL. [1][2][3] Currently only btrfs support this feature, so my goal it to bring it to ZoL as well. I think the only way to do it (without changing too many parts of ZoL) is to use the deduplication feature of zfs. A COW copy could be done by create a new entry in ddt for the old file, and create a new file which points to the ddt entry. Please let me know if this proposal makes sense, and if that's the right way to do it. Thanks. [1]: https://groups.google.com/a/zfsonlinux.org/forum/#!topic/zfs-discuss/mvGB7QEpt3w [2]: https://github.com/zfsonlinux/zfs/issues/405 [3]: https://github.com/zfsonlinux/zfs/issues/1063 While I can't comment too much on the technical aspects, they seem to be relatively sound. However, there are some issues with the, er... other aspects, for lack of better terminology. 1. This is possibly out of scope as a Gentoo project, since ZOL is not really part of Gentoo. If it's not, then you're out of luck, because ZOL is not an accepted organization. Things that provide us with improvements over what we have are definitely worth consideration as GSoC projects. However, what is accepted ultimately depends on not only feedback from a potential mentor, but also a vote of Gentoo developers. 2. This is likely too small to be a GSoC project. Perhaps see [0] for a list of example ideas, if only so you can get a grasp on the size of a good project. It does sound like a good idea though, and even if you can't do it as part of GSoC, you should pursue it anyways. Leaning on my understanding of ZFS internals, I can say that this is large enough. However, I do not think it will accomplish the desired result if implemented in the manner suggested. signature.asc Description: OpenPGP digital signature
Re: [gentoo-dev] GSoC proposal: cp --reflink support for zfs.
Hi, On Wed, Mar 12, 2014 at 9:30 PM, Richard Yao r...@gentoo.org wrote: On 03/12/2014 08:45 AM, Alex Xu wrote: On 12/03/14 03:15 AM, Yuxuan Shui wrote: Hi, I would like to implement cp --reflink support for ZFSOnLinux as my GSoC project. cp --reflink is used to create a COW copy of a file, so the file will not take any disk space if it's not modified. This feature is very useful for cases like storing a lot of almost identical virtual machine images. Also this is a frequently requested feature for ZoL. [1][2][3] Currently only btrfs support this feature, so my goal it to bring it to ZoL as well. I think the only way to do it (without changing too many parts of ZoL) is to use the deduplication feature of zfs. A COW copy could be done by create a new entry in ddt for the old file, and create a new file which points to the ddt entry. Please let me know if this proposal makes sense, and if that's the right way to do it. Thanks. [1]: https://groups.google.com/a/zfsonlinux.org/forum/#!topic/zfs-discuss/mvGB7QEpt3w [2]: https://github.com/zfsonlinux/zfs/issues/405 [3]: https://github.com/zfsonlinux/zfs/issues/1063 While I can't comment too much on the technical aspects, they seem to be relatively sound. However, there are some issues with the, er... other aspects, for lack of better terminology. 1. This is possibly out of scope as a Gentoo project, since ZOL is not really part of Gentoo. If it's not, then you're out of luck, because ZOL is not an accepted organization. Things that provide us with improvements over what we have are definitely worth consideration as GSoC projects. However, what is accepted ultimately depends on not only feedback from a potential mentor, but also a vote of Gentoo developers. Maybe I could propose this project to illumos? Since they seems to accept proposal for OpenZFS. 2. This is likely too small to be a GSoC project. Perhaps see [0] for a list of example ideas, if only so you can get a grasp on the size of a good project. It does sound like a good idea though, and even if you can't do it as part of GSoC, you should pursue it anyways. Leaning on my understanding of ZFS internals, I can say that this is large enough. However, I do not think it will accomplish the desired result if implemented in the manner suggested. Could you elaborate why this won't work? Thanks. -- Regards Yuxuan Shui
Re: [gentoo-dev] GSoC proposal: cp --reflink support for zfs.
On Wed, Mar 12, 2014 at 9:30 AM, Richard Yao r...@gentoo.org wrote: Things that provide us with improvements over what we have are definitely worth consideration as GSoC projects. However, what is accepted ultimately depends on not only feedback from a potential mentor, but also a vote of Gentoo developers. Honestly, this seems like a less-than-ideal fit for Gentoo. We don't really use ZFS at all, other than as yet another package in our tree. Any of the Mozilla/ GSoC ideas would make as much sense to deliver as part of the Gentoo GSoC. Now, if this were about integrating auto-snapshots into the package manager, (like can be done with snapper), etc, then I could see more relevance (though it probably wouldn't rise to a full project). I could also see some kind of project to integrate advanced filesystem features into core elements of our distro across many filesystems (though I don't really see the relevance for reflinks in particular, and they only apply to COW filesystems anyway). This just seems like a ZFS project, and not like a Gentoo project. I'd say the same if this were about adding a mute button to tabs in Chromium, or fixing the offline btrfsck, or whatever. All of those things would be useful to Gentoo, but only insofar as they'd be useful to anybody. Rich
Re: [gentoo-dev] GSoC proposal: cp --reflink support for zfs.
On Wed, Mar 12, 2014 at 01:19:08PM -0400, Rich Freeman wrote: This just seems like a ZFS project, and not like a Gentoo project. I'd say the same if this were about adding a mute button to tabs in Chromium, or fixing the offline btrfsck, or whatever. All of those things would be useful to Gentoo, but only insofar as they'd be useful to anybody. Agreed. Your dear friends in Debian would enjoy such changes as well, for those using zfs with kfreebsd or a home build of the zol package. Perhaps this would be a fit for upstream? Rich Cheers, Paul -- .''`. Paul Tagliamonte paul...@debian.org | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt signature.asc Description: Digital signature
Re: [gentoo-dev] GSoC proposal: cp --reflink support for zfs.
It is a moot point because I do not think this project idea is feasible. On Mar 12, 2014, at 1:19 PM, Rich Freeman ri...@gentoo.org wrote: On Wed, Mar 12, 2014 at 9:30 AM, Richard Yao r...@gentoo.org wrote: Things that provide us with improvements over what we have are definitely worth consideration as GSoC projects. However, what is accepted ultimately depends on not only feedback from a potential mentor, but also a vote of Gentoo developers. Honestly, this seems like a less-than-ideal fit for Gentoo. We don't really use ZFS at all, other than as yet another package in our tree. Any of the Mozilla/ GSoC ideas would make as much sense to deliver as part of the Gentoo GSoC. Now, if this were about integrating auto-snapshots into the package manager, (like can be done with snapper), etc, then I could see more relevance (though it probably wouldn't rise to a full project). I could also see some kind of project to integrate advanced filesystem features into core elements of our distro across many filesystems (though I don't really see the relevance for reflinks in particular, and they only apply to COW filesystems anyway). This just seems like a ZFS project, and not like a Gentoo project. I'd say the same if this were about adding a mute button to tabs in Chromium, or fixing the offline btrfsck, or whatever. All of those things would be useful to Gentoo, but only insofar as they'd be useful to anybody. Rich
Re: [gentoo-dev] GSoC proposal: cp --reflink support for zfs.
I do not think the original project idea is feasible as stated. On Mar 12, 2014, at 1:22 PM, Paul Tagliamonte paul...@debian.org wrote: On Wed, Mar 12, 2014 at 01:19:08PM -0400, Rich Freeman wrote: This just seems like a ZFS project, and not like a Gentoo project. I'd say the same if this were about adding a mute button to tabs in Chromium, or fixing the offline btrfsck, or whatever. All of those things would be useful to Gentoo, but only insofar as they'd be useful to anybody. Agreed. Your dear friends in Debian would enjoy such changes as well, for those using zfs with kfreebsd or a home build of the zol package. Perhaps this would be a fit for upstream? Rich Cheers, Paul -- .''`. Paul Tagliamonte paul...@debian.org | Proud Debian Developer : :' : 4096R / 8F04 9AD8 2C92 066C 7352 D28A 7B58 5B30 807C 2A87 `. `'` http://people.debian.org/~paultag `- http://people.debian.org/~paultag/conduct-statement.txt
