Re: APFS cloning not working across volumes inside same container?

2019-05-07 Thread Thomas Tempelmann
That makes sense to me.
Thank you for taking the time to explain.

Thomas


On Tue, May 7, 2019 at 4:33 PM Dominic Giampaolo 
wrote:

>
> > I accept that it's currently not possible - but just to help me
> understand this, could you please elaborate:
> >
> > In theory, would it be possible to share a cloned file between two
> volumes? Since they're in the same container, and the space management is
> container-wide, could this be made possible? Or are there technical reason
> why this couldn't work at all (such as that IDs needed to manage this are
> not sharable between volumes)?
> >
> Space management (i.e. is this block allocated or not) is shared.  But
> cloning introduces another layer which is a reference count the blocks and
> that is not shared between volumes within a container.  Further it's
> complicated by snapshots and the management of the reference counts across
> snapshots and the live view of the file system.
>
> Consider what would happen if you cloned a file between two volumes, took
> a snapshot on both volumes, modified part of the file on one of the volumes
> then deleted it on the other.  Managing the reference counting would be a
> nightmare, especially since you'd have to manage locking some new shared
> data structure between two different volumes.
>
> We'll definitely think about it but as always, a compelling use case is
> needed to justify the work.  In other words, we can't just do something
> because it would be kinda cool - there has to be a good reason to expend
> the effort required to do it.
>
>
> --dominic
>
>
 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com


Re: APFS cloning not working across volumes inside same container?

2019-05-07 Thread Thomas Tempelmann
Hi Dominic,
Thank you for jumping in.

I accept that it's currently not possible - but just to help me understand
this, could you please elaborate:

In theory, would it be possible to share a cloned file between two volumes?
Since they're in the same container, and the space management is
container-wide, could this be made possible? Or are there technical reason
why this couldn't work at all (such as that IDs needed to manage this are
not sharable between volumes)?

Thomas
 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com


Re: APFS cloning not working across volumes inside same container?

2019-05-06 Thread James R Cutler
I would expect that the path being taking with the iPad Pro series could lead 
to multiple volumes, among other things making the iPad more like a “computer”.

But, I would not make product plans for near term based on that thought.

Jim

James R. Cutler
james.cut...@consultant.com
GPG keys: hkps://hkps.pool.sks-keyservers.net



> On May 6, 2019, at 4:33 PM, Thomas Tempelmann  wrote:
> 
> The think that most closely resembles the HFS catalog isn't shared amongst 
> volumes.
> 
> But it is. There is one btree for all volumes in a container (well, 
> technically, there are two btrees, one for the IDs and one for the actual 
> file catalog, but both are shared over all vols).
>  
> clonefile(2) has the same restriction as link(2) and rename(2), it has to be 
> done within the same volume/mounted filesystem.
> 
> So that's an API restriction, but not a technically necessary one on APFS, 
> IMO. I guess APFS was created with iOS in mind, where we do not have multiple 
> volumes, so no one gave this a second thought. It's a shame, as it could have 
> some great savings when sharing volumes in some situations like the one I 
> described. Still, it's a niche, and so I'm not hopeful this will get 
> improved. Sigh.
> 
> Thomas
> 
> ___
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/filesystem-dev/james.cutler%40consultant.com
> 
> This email sent to james.cut...@consultant.com

 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com


Re: APFS cloning not working across volumes inside same container?

2019-05-06 Thread Thomas Tempelmann
>
> The think that most closely resembles the HFS catalog isn't shared amongst
> volumes.


But it is. There is one btree for all volumes in a container (well,
technically, there are two btrees, one for the IDs and one for the actual
file catalog, but both are shared over all vols).


> clonefile(2) has the same restriction as link(2) and rename(2), it has to
> be done within the same volume/mounted filesystem.
>

So that's an API restriction, but not a technically necessary one on APFS,
IMO. I guess APFS was created with iOS in mind, where we do not have
multiple volumes, so no one gave this a second thought. It's a shame, as it
could have some great savings when sharing volumes in some situations like
the one I described. Still, it's a niche, and so I'm not hopeful this will
get improved. Sigh.

Thomas
 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com


Re: APFS cloning not working across volumes inside same container?

2019-05-06 Thread Vivek Verma


> On May 6, 2019, at 6:21 AM, Thomas Tempelmann  wrote:
> 
> I was just looking into making a Deduplication tool for macOS / APFS.
> 
> One common scenario would be to deduplicate macOS systems on separate 
> volumes. Like, for us developers who have several macOS versions on their 
> computers for software testing purposes.
> 
> I thought I could save a lot of space by having files inside /System and 
> /Library share the same space by relying on APFS's cloning feature.
> 
> However, when I tested copying a file from one volume to another volume, both 
> being in the same APFS container (partition), Finder would still copy the 
> data instead of doing the clone thing.
> 
> Now I wonder if that's just a shortcoming of the Finder or a problem with the 
> macOS API.
> 
> After all, since APFS shares a single catalog between all volumes of its 
> container, cloning should be possible across volumes, shouldn't it?


The think that most closely resembles the HFS catalog isn't shared amongst 
volumes. clonefile(2) has the same restriction as link(2) and rename(2), it has 
to be done within the same volume/mounted filesystem.


> -- 
> Thomas Tempelmann, http://apps.tempel.org/
> Follow me on Twitter: https://twitter.com/tempelorg
> Read my programming blog: http://blog.tempel.org/
> ___
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/filesystem-dev/vivek_verma%40apple.com
> 
> This email sent to vivek_ve...@apple.com

 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com


APFS cloning not working across volumes inside same container?

2019-05-06 Thread Thomas Tempelmann
I was just looking into making a Deduplication tool for macOS / APFS.

One common scenario would be to deduplicate macOS systems on separate
volumes. Like, for us developers who have several macOS versions on their
computers for software testing purposes.

I thought I could save a lot of space by having files inside /System and
/Library share the same space by relying on APFS's cloning feature.

However, when I tested copying a file from one volume to another volume,
both being in the same APFS container (partition), Finder would still copy
the data instead of doing the clone thing.

Now I wonder if that's just a shortcoming of the Finder or a problem with
the macOS API.

After all, since APFS shares a single catalog between all volumes of its
container, cloning should be possible across volumes, shouldn't it?

-- 
Thomas Tempelmann, http://apps.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my programming blog: http://blog.tempel.org/
 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com