Re: searchfs support on APFS

[Thomas Tempelmann]

After more testing and communications with Apple engineers and other devs
relying on this function I like to post a summary, in case anyone else is
interested in using the CatSearch operation with APFS:

1. searchfs() is implemented for APFS on 10.12.6 and 10.13b3 (and earlier
versions probably, too).

2. searchfs() has one major issue that makes it quite useless when
searching for file names: It currently *compares case-sensitively*, but
should be comparing case-insensitively. Whether this will get fixed before
the 10.13 release is not clear. I've filed a bug report:
https://openradar.appspot.com/33455597 - *if you care for searchfs support,
please file one as well, adding your vote to have this fixed.*

3. *FSCatalogSearch cannot be used on APFS* as it cannot handle the 64 bit
node IDs APFS uses. Currently, the volparms claim CatalogSearch support,
and calling FSCatalogSearch does not return an error either, but there's an
open bug to fix that (https://openradar.appspot.com/33428180).
Unfortunately, some non-beta versions of macOS, e.g. 10.12.6, have this bug
incorrectly claiming CatSearch support, meaning we'll have to
special-handle this case.

4. searchfs() does not return paths but only names and node IDs (which are
64 bit), e.g. through ATTR_CMN_FILEID. However, *there is currently no
documented way to resolve node IDs to actual file paths or URLs*. (
https://openradar.appspot.com/33507188)

5. searchfs() does not handle the ATTR_CMNEXT_LINKID attribute, meaning *one
cannot learn about the separate dir entries of multiple hard links* to the
same file. (https://openradar.appspot.com/33473247)

I've also got a blog post about this here:
http://blog.tempel.org/2017/07/apfs-and-fast-catalog-search.html

-- 
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my blog: http://blog.tempel.org/
 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: searchfs support on APFS

[Jim Luther]

FSCatalogSearch is part of the deprecated Carbon File Manager API. The Carbon 
File Manager API exposes file IDs and directory IDs as 32-bit values and so 
FSCatalogSearch will never be supported on APFS (which use 64-bit file IDs and 
directory IDs).

In 10.13:
• PBCatSearch on APFS volumes will return paramErr.
• FSCatalogSearch on APFS volumes will return errFSOperationNotSupported.
• PBGetVolParms and FSGetVolumeParms on APFS volumes will return with 
vMAttrib.bHasCatSearch and vMExtendedAttributes.bSupportsFSCatalogSearch clear.

(There was a bug that made it look like PBCatSearch and FSCatalogSearch were 
supported on APFS.)

> On Jul 6, 2017, at 7:49 AM, Thomas Tempelmann  wrote:
> 
> Two things:
> 
> 1.
> A long while ago the APFS team at Apple asked for input on what's needed - I 
> then asked for fast catalog search support, in the ways of the searchfs BSD 
> function. Has anything happened in that direction?
> 
> 2.
> And then there's a possible issue around reporting support for this operation 
> (as of 10.13 beta 2):
> 
> My search tool unsuccessfully attempts to use FSCatalogSearch on APFS volumes 
> because it's being told by the API that the APFS volume supports that 
> operation. Which it doesn't do, though, as my app then gets no results from 
> the search. I haven't been able to look deeper into this yet, I get this from 
> customer reports that all have the same problem now after installing High 
> Sierra and upgrading to APFS.
> 
> So, could it be that the volume flags are incorrectly set, indicating that 
> the file system supports CatalogSearch even though it doesn't? (I'm just 
> throwing this in as early as possible, but I'll make sure to file a bug 
> report should my suspicion be confirmed).
> 
> -- 
> Thomas Tempelmann, http://www.tempel.org/ 
> Follow me on Twitter: https://twitter.com/tempelorg 
> 
> Read my programming blog: http://blog.tempel.org/ 
> ___
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/filesystem-dev/luther.j%40apple.com
> 
> This email sent to luthe...@apple.com

 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: searchfs support on APFS

[Eric Tamura]

I believe searchfs(2) was not implemented in last year’s releases, but it 
should have been there starting with the WWDC build this year.

Eric


> On 6 Jul 2017, at 10:42 AM, Thomas Tempelmann  wrote:
> 
> I was kindly informed that 10.12 already supports searchfs (and 
> FSCatalogSearch along with it) on APFS, and after doing some testing myself 
> it turned out that it's all working as intended, and that I just did the 
> testing wrong, on my misguided assumptions, along with using an outdated 
> version of my app for testing.
> 
> So, nothing to see here, all is well - and thanks to the File System team for 
> making the effort to support searchfs on APFS!
> 
> Thomas
> 
> ___
> Do not post admin requests to the list. They will be ignored.
> Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
> Help/Unsubscribe/Update your Subscription:
> https://lists.apple.com/mailman/options/filesystem-dev/etamura%40apple.com
> 
> This email sent to etam...@apple.com

 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Re: searchfs support on APFS

[Thomas Tempelmann]

I was kindly informed that 10.12 already supports searchfs (and
FSCatalogSearch along with it) on APFS, and after doing some testing myself
it turned out that it's all working as intended, and that I just did the
testing wrong, on my misguided assumptions, along with using an outdated
version of my app for testing.

So, nothing to see here, all is well - and thanks to the File System team
for making the effort to support searchfs on APFS!

Thomas
 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

searchfs support on APFS

[Thomas Tempelmann]

Two things:

1.
A long while ago the APFS team at Apple asked for input on what's needed -
I then asked for fast catalog search support, in the ways of the searchfs
BSD function. Has anything happened in that direction?

2.
And then there's a possible issue around reporting support for this
operation (as of 10.13 beta 2):

My search tool unsuccessfully attempts to use FSCatalogSearch on APFS
volumes because it's being told by the API that the APFS volume supports
that operation. Which it doesn't do, though, as my app then gets no results
from the search. I haven't been able to look deeper into this yet, I get
this from customer reports that all have the same problem now after
installing High Sierra and upgrading to APFS.

So, could it be that the volume flags are incorrectly set, indicating that
the file system supports CatalogSearch even though it doesn't? (I'm just
throwing this in as early as possible, but I'll make sure to file a bug
report should my suspicion be confirmed).

-- 
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my programming blog: http://blog.tempel.org/
 ___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list  (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com