I don’t really have time to look at the current fts implementation, but… it has 
several options that effect performance (in particular, the FTS_NOCHDIR, 
FTS_NOSTAT, FTS_NOSTAT_TYPE, and FTS_XDEV options). If you are trying to 
compare fts to CFURLEnumerator (for example), use FTS_NOCHDIR and FTS_XDEV, but 
don’t use FTS_NOSTAT and FTS_NOSTAT_TYPE.

> On Apr 22, 2019, at 9:59 AM, Thomas Tempelmann <tempelm...@gmail.com> wrote:
> 
> Jim,
> thanks for your comments.
> 
> If all you need is filenames and no other attributes, readdir is usually 
> faster than getattrlistbulk because it doesn't have to do as much work. 
> However, if you need additional attributes, getattrlistbulk is usually much 
> faster. Some of that extra work done by getattrlistbulk involves checking to 
> see what attributes were requested and packing the results into the result 
> buffer. 
> 
> What's interesting is that on HFS+, readdir is not faster in my tests, but on 
> a recent and fast Mac (i.e. not on my MacPro 2010), it can be twice as fast 
> as the others when scanning an APFS volume. I wonder why. Is the 
> implementation for getattrlistbulk in the APFS driver inefficient compared to 
> the one in HFS+? The source code for the APFS FS driver has still not be 
> published, or has it?
> 
> You'll find that lstat is slightly faster than getattrlist (when getattrlist 
> is returning the same set of attributes) for the same reason. There's no 
> extra code needed in lstat to see what attributes were requested and packing 
> the results into the result buffer.
> 
> It's also significantly faster than using NSURL's getResourceValue, even if 
> the NSURL has already been created regardless. That's probably due to all the 
> objc overhead.
> 
> By the way, I haven't tested this but I would expect 
> enumeratorAtURL:includingPropertiesForKeys:options:errorHandler: (followed by 
> a "for (NSURL *fileURL in directoryEnumerator)" loop) to be slightly faster 
> than contentsOfDirectoryAtURL:includingPropertiesForKeys:options:error: 
> because the URLs aren't retained in a NSArray. Using CFURLEnumerator may also 
> be slightly faster than NSFileManager's directory enumeration.
> 
> Now, that's something I had not considered, yet. Will try.
>  
> Using POSIX/BSD APIs will be the fastest, but that means you have to deal 
> with the different capabilities between file systems yourself (although 
> getattrlistbulk helps with that a lot).
> 
> Most interesting, though:
> 
> Today someone pointed out fts_read. This does, so far always beat all other 
> methods, especially if I also need extra attributes (e.g. file size).
> 
> Can you give some more information about the fts implementation? Is this 
> user-library-level oder kernel code that's doing this? I had expected that 
> this would only be a convenience userland function that uses readdir or 
> similar BSD functions, but it appears to beat them all, suggesting this is 
> optimized at a lower level.
> 
> 
> I have updated my test project accordingly (with the fts code) in case anyone 
> likes to run their own tests:
> 
>   http://files.tempel.org/Various/DirScanner.zip 
> <http://files.tempel.org/Various/DirScanner.zip>
> 
> Also, I am wondering if using concurrent threads will speed up scanning a dir 
> tree on an SSD as well, by distributing each directory read to one thread (or 
> dispatch queue). Will eventually try, but probably not soon. Gotta get my 
> program out of the door soon, first.
> 
> Thomas
> 

 _______________________________________________
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list      (Filesystem-dev@lists.apple.com)
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/filesystem-dev/archive%40mail-archive.com

This email sent to arch...@mail-archive.com

Reply via email to