Re: APFS cloning not working across volumes inside same container?
That makes sense to me. Thank you for taking the time to explain. Thomas On Tue, May 7, 2019 at 4:33 PM Dominic Giampaolo wrote: > > > I accept that it's currently not possible - but just to help me > understand this, could you please elaborate: > > > > In theory, would it be possible to share a cloned file between two > volumes? Since they're in the same container, and the space management is > container-wide, could this be made possible? Or are there technical reason > why this couldn't work at all (such as that IDs needed to manage this are > not sharable between volumes)? > > > Space management (i.e. is this block allocated or not) is shared. But > cloning introduces another layer which is a reference count the blocks and > that is not shared between volumes within a container. Further it's > complicated by snapshots and the management of the reference counts across > snapshots and the live view of the file system. > > Consider what would happen if you cloned a file between two volumes, took > a snapshot on both volumes, modified part of the file on one of the volumes > then deleted it on the other. Managing the reference counting would be a > nightmare, especially since you'd have to manage locking some new shared > data structure between two different volumes. > > We'll definitely think about it but as always, a compelling use case is > needed to justify the work. In other words, we can't just do something > because it would be kinda cool - there has to be a good reason to expend > the effort required to do it. > > > --dominic > > ___ Do not post admin requests to the list. They will be ignored. Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: APFS cloning not working across volumes inside same container?
Hi Dominic, Thank you for jumping in. I accept that it's currently not possible - but just to help me understand this, could you please elaborate: In theory, would it be possible to share a cloned file between two volumes? Since they're in the same container, and the space management is container-wide, could this be made possible? Or are there technical reason why this couldn't work at all (such as that IDs needed to manage this are not sharable between volumes)? Thomas ___ Do not post admin requests to the list. They will be ignored. Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: APFS cloning not working across volumes inside same container?
I would expect that the path being taking with the iPad Pro series could lead to multiple volumes, among other things making the iPad more like a “computer”. But, I would not make product plans for near term based on that thought. Jim James R. Cutler james.cut...@consultant.com GPG keys: hkps://hkps.pool.sks-keyservers.net > On May 6, 2019, at 4:33 PM, Thomas Tempelmann wrote: > > The think that most closely resembles the HFS catalog isn't shared amongst > volumes. > > But it is. There is one btree for all volumes in a container (well, > technically, there are two btrees, one for the IDs and one for the actual > file catalog, but both are shared over all vols). > > clonefile(2) has the same restriction as link(2) and rename(2), it has to be > done within the same volume/mounted filesystem. > > So that's an API restriction, but not a technically necessary one on APFS, > IMO. I guess APFS was created with iOS in mind, where we do not have multiple > volumes, so no one gave this a second thought. It's a shame, as it could have > some great savings when sharing volumes in some situations like the one I > described. Still, it's a niche, and so I'm not hopeful this will get > improved. Sigh. > > Thomas > > ___ > Do not post admin requests to the list. They will be ignored. > Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > https://lists.apple.com/mailman/options/filesystem-dev/james.cutler%40consultant.com > > This email sent to james.cut...@consultant.com ___ Do not post admin requests to the list. They will be ignored. Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: APFS cloning not working across volumes inside same container?
> > The think that most closely resembles the HFS catalog isn't shared amongst > volumes. But it is. There is one btree for all volumes in a container (well, technically, there are two btrees, one for the IDs and one for the actual file catalog, but both are shared over all vols). > clonefile(2) has the same restriction as link(2) and rename(2), it has to > be done within the same volume/mounted filesystem. > So that's an API restriction, but not a technically necessary one on APFS, IMO. I guess APFS was created with iOS in mind, where we do not have multiple volumes, so no one gave this a second thought. It's a shame, as it could have some great savings when sharing volumes in some situations like the one I described. Still, it's a niche, and so I'm not hopeful this will get improved. Sigh. Thomas ___ Do not post admin requests to the list. They will be ignored. Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
Re: APFS cloning not working across volumes inside same container?
> On May 6, 2019, at 6:21 AM, Thomas Tempelmann wrote: > > I was just looking into making a Deduplication tool for macOS / APFS. > > One common scenario would be to deduplicate macOS systems on separate > volumes. Like, for us developers who have several macOS versions on their > computers for software testing purposes. > > I thought I could save a lot of space by having files inside /System and > /Library share the same space by relying on APFS's cloning feature. > > However, when I tested copying a file from one volume to another volume, both > being in the same APFS container (partition), Finder would still copy the > data instead of doing the clone thing. > > Now I wonder if that's just a shortcoming of the Finder or a problem with the > macOS API. > > After all, since APFS shares a single catalog between all volumes of its > container, cloning should be possible across volumes, shouldn't it? The think that most closely resembles the HFS catalog isn't shared amongst volumes. clonefile(2) has the same restriction as link(2) and rename(2), it has to be done within the same volume/mounted filesystem. > -- > Thomas Tempelmann, http://apps.tempel.org/ > Follow me on Twitter: https://twitter.com/tempelorg > Read my programming blog: http://blog.tempel.org/ > ___ > Do not post admin requests to the list. They will be ignored. > Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) > Help/Unsubscribe/Update your Subscription: > https://lists.apple.com/mailman/options/filesystem-dev/vivek_verma%40apple.com > > This email sent to vivek_ve...@apple.com ___ Do not post admin requests to the list. They will be ignored. Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com
APFS cloning not working across volumes inside same container?
I was just looking into making a Deduplication tool for macOS / APFS. One common scenario would be to deduplicate macOS systems on separate volumes. Like, for us developers who have several macOS versions on their computers for software testing purposes. I thought I could save a lot of space by having files inside /System and /Library share the same space by relying on APFS's cloning feature. However, when I tested copying a file from one volume to another volume, both being in the same APFS container (partition), Finder would still copy the data instead of doing the clone thing. Now I wonder if that's just a shortcoming of the Finder or a problem with the macOS API. After all, since APFS shares a single catalog between all volumes of its container, cloning should be possible across volumes, shouldn't it? -- Thomas Tempelmann, http://apps.tempel.org/ Follow me on Twitter: https://twitter.com/tempelorg Read my programming blog: http://blog.tempel.org/ ___ Do not post admin requests to the list. They will be ignored. Filesystem-dev mailing list (Filesystem-dev@lists.apple.com) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com This email sent to arch...@mail-archive.com