Re: readdir vs. getdirentriesattr

2019-04-29 Thread Thomas Tempelmann
> The volume ID is at a higher layer, but the enumeration code attempts to > retrieve the value less than once per URL returned. That said, if the > directory hierarchy has few items per directory, the number of times it is > retrieved will be higher. You can write a bug report and I'll look to see

Re: readdir vs. getdirentriesattr

2019-04-29 Thread Thomas Tempelmann
Quick update: > -[enumeratorAtURL:includingPropertiesForKeys:options:errorHandler:] also >> supports recursive enumeration (which stops at device boundaries -- you'll >> see mount points but not their contents) so you don't have to do that >> yourself. >> > This is indeed faster than most of the

Re: readdir vs. getdirentriesattr

2019-04-29 Thread Jim Luther
> On Apr 29, 2019, at 1:19 PM, Thomas Tempelmann wrote: > > Jim, > > In contentsOfDirectoryAtURL, instead of "includingPropertiesForKeys:nil", use > "includingPropertiesForKeys:@[NSURLVolumeIdentifierKey]" (and add whatever > other property keys you know you'll need). The whole purpose of th

Re: readdir vs. getdirentriesattr

2019-04-29 Thread Thomas Tempelmann
Jim, In contentsOfDirectoryAtURL, instead of "includingPropertiesForKeys:nil", > use "includingPropertiesForKeys:@[NSURLVolumeIdentifierKey]" (and add > whatever other property keys you know you'll need). The whole purpose of > the includingPropertiesForKeys argument is so the enumerator code can

Re: readdir vs. getdirentriesattr

2019-04-29 Thread Jim Luther
In contentsOfDirectoryAtURL, instead of "includingPropertiesForKeys:nil", use "includingPropertiesForKeys:@[NSURLVolumeIdentifierKey]" (and add whatever other property keys you know you'll need). The whole purpose of the includingPropertiesForKeys argument is so the enumerator code can pre-fetch

Re: readdir vs. getdirentriesattr

2019-04-29 Thread Thomas Tempelmann
Doing more performance tests for directory traversal I ran into a performance issue with [NSURL contentsOfDirectoryAtURL:]: See this typical code for scanning a directory: NSArray *contentURLs = [fileMgr contentsOfDirectoryAtURL:parentURL includingPropertiesForKeys:nil options:0 error:nil];

Re: readdir vs. getdirentriesattr

2019-04-22 Thread Jim Luther
I don’t really have time to look at the current fts implementation, but… it has several options that effect performance (in particular, the FTS_NOCHDIR, FTS_NOSTAT, FTS_NOSTAT_TYPE, and FTS_XDEV options). If you are trying to compare fts to CFURLEnumerator (for example), use FTS_NOCHDIR and FTS_

Re: readdir vs. getdirentriesattr

2019-04-22 Thread Wim Lewis
On Apr 22, 2019, at 9:59 AM, Thomas Tempelmann wrote: > Can you give some more information about the fts implementation? Is this > user-library-level oder kernel code that's doing this? I had expected that > this would only be a convenience userland function that uses readdir or > similar BSD f

Re: readdir vs. getdirentriesattr

2019-04-22 Thread Thomas Tempelmann
Jim, thanks for your comments. If all you need is filenames and no other attributes, readdir is usually > faster than getattrlistbulk because it doesn't have to do as much > work. However, if you need additional attributes, getattrlistbulk is > usually much faster. Some of that extra work done > b

Re: readdir vs. getdirentriesattr

2019-04-22 Thread Jim Luther
If all you need is filenames and no other attributes, readdir is usually faster than getattrlistbulk because it doesn't have to do as much work. However, if you need additional attributes, getattrlistbulk is usually much faster. Some of that extra work done by getattrlistbulk involves checking t

Re: readdir vs. getdirentriesattr

2019-04-21 Thread Thomas Tempelmann
I like to add some info on a thread from 2015: I recently worked on my file search tool (FAF) and wanted to make sure that I use the best method to deep-scan directory contents. I had expected that getattrlistbulk() would always be the best choice, but it turns out that opendir/readdir perform

Re: readdir vs. getdirentriesattr

2015-01-13 Thread Thomas Tempelmann
> > For the most part, all of Apple's code has switched from > getdirentriesattr() to getattrlistbulk(). > *applauds* Thomas ___ Do not post admin requests to the list. They will be ignored. Filesystem-dev mailing list (Filesystem-dev@lists.apple.

Re: readdir vs. getdirentriesattr

2015-01-13 Thread Jim Luther
In Yosemite (10.10), Carbon's PBGetCatalogInfoBulk and directory enumerators created with CoreFoundation's CFURLEnumerator (it's in CFURLEnumerator.h, not CFURL.h) both use getattrlistbulk() instead of readdir() or getdirentriesattr(). Foundation API like- [NSFileManager contentsOfDirectoryAtUR

Re: readdir vs. getdirentriesattr

2015-01-13 Thread Thomas Tempelmann
On Tue, Jan 13, 2015 at 7:21 PM, Eric Tamura wrote: > HFS, AFP, and SMB all support getattrlistbulk() natively. > Thanks for the clarification, Eric. Do any of the higher level APIs also make use of this call, or do I have to use this call to take advantage of its functionality? I guess that N

Re: readdir vs. getdirentriesattr

2015-01-13 Thread Eric Tamura
HFS, AFP, and SMB all support getattrlistbulk() natively. Eric > On 13 Jan 2015, at 3:52 AM, Thomas Tempelmann wrote: > > James, > > I have to say, the new getattrlistbulk() function is working very well > here.[...] > And, I can confirm that's it's fast. :) > > Can you or someone else who

Re: readdir vs. getdirentriesattr

2015-01-13 Thread Thomas Tempelmann
James, I have to say, the new getattrlistbulk() function is working very well > here.[...] > And, I can confirm that's it's fast. :) > Can you or someone else who tried this new function share with us where this improves speed and where not? In particular, do any of the network file systems (CIFS

Re: readdir vs. getdirentriesattr

2015-01-12 Thread James Bucanek
Jim, Eric, list, I have to say, the new getattrlistbulk() function is working very well here. Requiring ATTR_CMN_RETURNED_ATTRS made me reorganize my code a bit, but that was minor. Specifically, it's nice to have a single routine to call (instead of getdirentriesattr() and a readdir() fallbac

Re: readdir vs. getdirentriesattr

2015-01-12 Thread Jim Luther
getattrlistbulk() works on all file systems. If the file system supports bulk enumeration natively, great! If it does not, then the kernel code takes care of it. In addition, getattrlistbulk() supports all non-volume attributes (getattrlistbulk only supported a large subset). The API calling co

Re: readdir vs. getdirentriesattr

2015-01-11 Thread James Bucanek
Eric, I would just like to clarify: the new getattrlistbulk() function works on all filesystem. We don't have to check the volume's VOL_CAP_INT_READDIRATTR capability before calling it, correct? James Bucanek Eric Tamura December 10, 2014 at 5:57 PM It should be m

Re: readdir vs. getdirentriesattr

2014-12-10 Thread Sean Farley
Jim Luther writes: > And to clarify... readdir may be faster than getattrlistbulk if all you need > are the names. If you call getattrlist (or lstat) on every item you get back > from readdir, you'll find that getattrlistbulk is faster. That is exactly what we are doing: calling lstat per file

Re: readdir vs. getdirentriesattr

2014-12-10 Thread Sean Farley
Eric Tamura writes: > It should be much faster. > > Also note that as of Yosemite, we have added a new API: getattrlistbulk(2), > which is like getdirentriesattr(), but supported in VFS for all filesystems. > getdirentriesattr() is now deprecated. Aha, that is interesting and a good lead. Th

Re: readdir vs. getdirentriesattr

2014-12-10 Thread Jim Luther
And to clarify... readdir may be faster than getattrlistbulk if all you need are the names. If you call getattrlist (or lstat) on every item you get back from readdir, you'll find that getattrlistbulk is faster. - Jim > On Dec 10, 2014, at 4:57 PM, Eric Tamura wrote: > > It should be much fas

Re: readdir vs. getdirentriesattr

2014-12-10 Thread Eric Tamura
It should be much faster. Also note that as of Yosemite, we have added a new API: getattrlistbulk(2), which is like getdirentriesattr(), but supported in VFS for all filesystems. getdirentriesattr() is now deprecated. The main advantage of the bulk call is that we can return results in most c