Out of curiosity, I just ran the test on my 10.13.6 system:
On HFS+, the tool finishes in about 5 seconds, ending up with a 4 GB file.
However, on a freshly created APFS vol (no encryption, on same SSD as the
HFS+ volume), it runs for about an hour! But no extra wait time around 2 GB.
I guess th
Just a small heads-up when you still have to deal with HFS paths:
Getting the HFS Path from an NSURL using CFURLCopyFileSystemPath (path, 1)
does not work if the path contains a file that uses a "/" (or ":",
respectively) in its name.
Filed as FB7291855, see also
http://www.openradar.me/radar?id=
s perform a separate searchfs() on that extra volume.
Thanks for the quick help (which I got in a private email)!
Thomas
> On 4. Jun 2019, at 21:20, Thomas Tempelmann wrote:
>
> The new system splits up the boot volume into a user's and the system's
> volume, somehow. The pr
how to deal
with this, i.e. how I can search both the system and the user volumes with
searchfs? How do I determine and address them?
--
Thomas Tempelmann, http://apps.tempel.org/
___
Do not post admin requests to the list. They will be ignored
On Thu, May 16, 2019 at 7:43 PM Dominic Giampaolo
wrote:
>
> > Without knowing the particulars about sandboxing, I wonder why you
> originally wanted to use multiple streams anyway. I had thought it's
> obvious that using multiple streams for watching multiple locations would
> be rather ineffici
Without knowing the particulars about sandboxing, I wonder why you
originally wanted to use multiple streams anyway. I had thought it's
obvious that using multiple streams for watching multiple locations would
be rather inefficient compared to having only one watcher for "/", and then
filter the it
That makes sense to me.
Thank you for taking the time to explain.
Thomas
On Tue, May 7, 2019 at 4:33 PM Dominic Giampaolo
wrote:
>
> > I accept that it's currently not possible - but just to help me
> understand this, could you please elaborate:
> >
> > In theory, would it be possible to share
Hi Dominic,
Thank you for jumping in.
I accept that it's currently not possible - but just to help me understand
this, could you please elaborate:
In theory, would it be possible to share a cloned file between two volumes?
Since they're in the same container, and the space management is
container
>
> The think that most closely resembles the HFS catalog isn't shared amongst
> volumes.
But it is. There is one btree for all volumes in a container (well,
technically, there are two btrees, one for the IDs and one for the actual
file catalog, but both are shared over all vols).
> clonefile(2
Now I wonder if that's just a shortcoming of the Finder or a problem with
the macOS API.
After all, since APFS shares a single catalog between all volumes of its
container, cloning should be possible across volumes, shouldn't it?
--
Thomas Tempelmann, http://apps.tempel.org/
Follow me on
> The volume ID is at a higher layer, but the enumeration code attempts to
> retrieve the value less than once per URL returned. That said, if the
> directory hierarchy has few items per directory, the number of times it is
> retrieved will be higher. You can write a bug report and I'll look to see
Quick update:
> -[enumeratorAtURL:includingPropertiesForKeys:options:errorHandler:] also
>> supports recursive enumeration (which stops at device boundaries -- you'll
>> see mount points but not their contents) so you don't have to do that
>> yourself.
>>
>
This is indeed faster than most of the
er ways I've tried.
And it brings along the st_dev value without time penalty, unlike
contentsOfDirectoryAtURL.
Regardless, I'll give that a try.
--
Thomas Tempelmann, http://apps.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my programming blog: http://blog.t
Doing more performance tests for directory traversal I ran into a
performance issue with [NSURL contentsOfDirectoryAtURL:]:
See this typical code for scanning a directory:
NSArray *contentURLs = [fileMgr contentsOfDirectoryAtURL:parentURL
includingPropertiesForKeys:nil options:0 error:nil];
Jim,
thanks for your comments.
If all you need is filenames and no other attributes, readdir is usually
> faster than getattrlistbulk because it doesn't have to do as much
> work. However, if you need additional attributes, getattrlistbulk is
> usually much faster. Some of that extra work done
> b
ous methods.
Any comments, insights, clarifications and bug reports are most welcome.
Enjoy,
Thomas Tempelmann
> On 12. Jan 2015, at 17:33, Jim Luther wrote:
>
> getattrlistbulk() works on all file systems. If the file system supports bulk
> enumeration natively, great! If it do
Eric,
What information are you trying to get out of each scan? You will always
> have a time-of-use vs. time-of-check race condition here .. the filesystem
> is in a perennial state of flux.
>
That's one thing I was surprised about when using searchfs() on APFS vs.
HFS+: On HFS, I'd frequently g
>
> So far I hadn't had much lack in scanning by this order, sparse filesystem
> makes the /.vol// option inefficient.
> As for the searchfs option, I haven't seen in the man page any way to
> control the order of the files.
>
The order is arbitrary, as it walks over the btree nodes in the most
ef
>
> 1. Is there any way to extract the current file-id range (minimum to
> maximum fileid).
>
Well, both HFS+ and APFS know the last used FileID and whenever a new node
(file, dir) is created, the last ID + 1 will be used for it. But you cannot
query that value directly (only indirectly, by creati
Disregard my previous post. I did some testing on my own and believe it was
just a user (programming) error.
My apologies for the unnecessary alertism.
Thomas
___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list
With this comment, I'll cross-post this to the filesystem-dev list, as this
could be a bug in the APFS code, making this a bit urgent with the imminent
release of 10.13
But it could also be that there's a new(?) flag somewhere that you need to
test in order to tell whether the values are in second
e, meaning *one
cannot learn about the separate dir entries of multiple hard links* to the
same file. (https://openradar.appspot.com/33473247)
I've also got a blog post about this here:
http://blog.tempel.org/2017/07/apfs-and-fast-catalog-search.html
--
Thomas Tempelmann, http://www.temp
I was kindly informed that 10.12 already supports searchfs (and
FSCatalogSearch along with it) on APFS, and after doing some testing myself
it turned out that it's all working as intended, and that I just did the
testing wrong, on my misguided assumptions, along with using an outdated
version of my
;t? (I'm just
throwing this in as early as possible, but I'll make sure to file a bug
report should my suspicion be confirmed).
--
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my programming blog: http://blog.tempel.org/
__
On Sun, Apr 30, 2017 at 1:47 AM, Toby Thain
wrote:
> On 2017-04-29 7:42 PM, Thomas Tempelmann wrote:
>
>> On Sun, Apr 30, 2017 at 1:37 AM, Toby Thain > <mailto:t...@telegraphics.com.au>> wrote:
>>
>> Seems you have % instead of & ? And watch out for s
lgorithm. It's
always modulo (%) MAXINT-1. And 0x's gets properly expanded to
unsigned 64 bit, as it should. I double checked using uint64_t variable all
the way, and it didn't change the outcome.
--
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitt
light on this?
I have an Xcode test project here, along with sample block data ("NXSB"):
http://files.tempel.org/tmp/apfs-fletcher-test.zip
--
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my programming blog: http://blog.tempel.or
Just to put a final word out on this topic I had raised two weeks ago:
I cannot reproduce it any more. I had thought for a while that it was
related to unencrypted CoreStorage-managed partitions, but even that
doesn't seem to be the case. Initially, I certainly and repeatedly had very
bad performa
o let the disk layer know about invoking TRIM on
these areas? Or does everyone have to hand-code talking to the disk
interface, issuing TRIM cmds, and hoping they don't get it wrong, i.e. can
every file system make use of the trimforce setting, or does everyone have
to bake their own?
--
On Wed, Apr 5, 2017 at 12:14 AM, Thomas Engelmeier wrote:
> Have you tried fs_usage or iosnoop on both DU versions? It will require
> some extra tampering with csrutil to get to work on Sierra but might bring
> an answer..
>
I am reading entire disks or partitions, from within the Recovery syste
An update:
I've done hours of testing. First, with DU 15.0 from El Cap. With that, I
found that copy perfomance on "Restore" from one to another disk or
partition, as well as "New Disk Image from a partition" is optimal.
But with DU 16.0, which is part of Sierra, performance goes does
significant
On Sat, Apr 1, 2017 at 2:07 AM, Eric Tamura wrote:
> No, as indicated in the document, iOS 10.3 uses the case-sensitive
> variant.
>
Oh, shoot. Of course it is. I confused the two. English is not my first
language, yet this mixup is still embarassing. Thanks for setting me
straight.
-
10.12 when copying
partitions, such as with using Restore or New Image from disk/partition?
Thomas
On Fri, Mar 31, 2017 at 11:51 PM, Thomas Tempelmann
wrote:
> Not relly a file system related thing, but since I'm tired of filing bug
> with Apple's bugreporter and mostly just gettin
On Sat, Apr 1, 2017 at 1:32 AM, Brendan Shanks wrote:
> The Apple File System Guide was updated yesterday with additional info
> about filenames, iOS 10.3, and macOS 10.12.4. Quick summary: no
> normalization, iOS 10.3 is case-sensitive, 10.12.4 now has (beta)
> case-insensitive AFPS.
>
> https:/
rming the same with "dd" and "rdisk", I got to the maximum,
which was about 200 MB/s. That's more than five times slower than it should
be.
I'm going to write a blog article about this over the weekend, too.
--
Thomas Tempelmann, http://www.tempel.org/
Fo
ust admit didn't think Brendan's suggestion would be helpful because I
assumed that fsck can only work on unmounted volumes (I'm sure that was the
case at one point in the past), and that DFA would take its own steps to
flush and freeze the boot volume before running fsck on it.
--
Tho
verify and possibly repair it.
How does DFA do that? Is there some OS function I could use for that
purpose, too?
--
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my programming blog: http://blog.tempel.org/
__
On Sat, Dec 3, 2016 at 8:47 PM, Chris Murphy
wrote:
> HFS Plus 1s
> NTFS 100ns
> ext4, XFS, Btrfs, ZFS, APFS 1ns
>
FAT is 2s, IIRC. (correct me if I'm wrong)
And then there's the network file protocols. I guess AFP also has a 1s
resolution, whereas SMB (and SMB2?) have higher ones, I guess?
Th
Hi Alan,
what makes you think that you get such a high resolution with the file
dates? Most file systems only have a one second resolution (some even have
less), meaning you have to wait for a second to actually see the value on
disk change. So, even if the OS would use a higher resolution in its A
least OSX notify me when the user on this Mac makes changes to
files on the network volume? (Meaning I could detect changes made by the
user without polling, while I'd still need to poll to detect changes made
from other computers?)
Any advice and insights are appreciated.
--
Thomas Tempel
George,
Please write up a bug report. Please give a detail description of the
> functionality you are interested in and the options you think it should
> support. Apple is always looking at improving the file system and will
> take a look at all suggestions.
>
I little guidedance would help. Aft
On Fri, Jun 17, 2016 at 7:39 PM, Thomas Tempelmann
wrote:
> 1. It would be nice if it would support the BSD searchfs (CatalogSearch)
> operation, or something similar on macOS.
>
To clarify - I watched the APFS session where it was briefly mentioned that
searchfs will not be supporte
eturn - I've been getting plenty of help
from their side in the past and just like to give something back if I can.
--
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my programming blog: http://blog.tempel.org/
_
A clarification:
While the first part of my mail described slow access on network volumes,
the last part (see below) is about a local (non-SSD) hard disk. So the
slowdown is not limited to network file systems.
Actually, anohter customer just contacted me who noticed the same slowdown
> at even g
cache to a percentage of
the RAM, whereas old systems used it all when available. Is that so?
Is there a remedy, e.g. a setting to increase the file cache, at least
temporarily?
--
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my programming blog:
(This may be a bit off-topic, as this is a user-level question, but there
are the right people on this list who can answer this, I think)
I'm trying to write bookmark files with content from Alias data generated
by the Alias Manager. But that doesn't work, and I find no options that
would suggest
f there's a
file at both places, and I can alert the user to give him the choice.
But once Apple changes the format as well, I'm screwed, I guess.
--
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tempelorg
Read my progra
What about using the API that fs_usage uses to monitor file changes?
It's a bit harder to manage but will give you much more detailed
information per-file.
Or is that API not production-savvy?
Thomas
___
Do not post admin requests to the list. They will
On Sun, Feb 22, 2015 at 9:11 PM, Thomas Tempelmann
wrote:
> I did some testing and can confirm that what Jim wrote below only tells me
> whether a path has an auto-mount trigger, but not whether it's actually
> reachable at the moment.
>
Ooops, correction! I had tested this
I did some testing and can confirm that what Jim wrote below only tells me
whether a path has an auto-mount trigger, but not whether it's actually
reachable at the moment.
So far, to tell if the volume is actually available I must try to read its
content. Often I would then get an error returned f
On Sun, Feb 22, 2015 at 12:18 AM, Mike Mackovitch wrote:
>
> You can use getfsstat(..., ..., MNT_NOWAIT) to get the current list of
> mounted filesystems.
> (Note: this will include things you may not consider a "filesystem" or
> "volume".
> For example, synthetic autofs directories (/net, /home,
(I realize this isn't really the best place to ask this question as it's
rather a user-level API question, not a filesys designer's question, but I
give it a try anyway because I didn't have luck finding answers elsewhere
so far.)
My question is: How can a user level app determine whether an autom
>
> For the most part, all of Apple's code has switched from
> getdirentriesattr() to getattrlistbulk().
>
*applauds*
Thomas
___
Do not post admin requests to the list. They will be ignored.
Filesystem-dev mailing list (Filesystem-dev@lists.apple.
On Tue, Jan 13, 2015 at 7:21 PM, Eric Tamura wrote:
> HFS, AFP, and SMB all support getattrlistbulk() natively.
>
Thanks for the clarification, Eric.
Do any of the higher level APIs also make use of this call, or do I have to
use this call to take advantage of its functionality?
I guess that N
James,
I have to say, the new getattrlistbulk() function is working very well
> here.[...]
>
And, I can confirm that's it's fast. :)
>
Can you or someone else who tried this new function share with us where
this improves speed and where not? In particular, do any of the network
file systems (CIFS
.apple.com)
> Help/Unsubscribe/Update your Subscription:
>
> https://lists.apple.com/mailman/options/filesystem-dev/tempelmann%40gmail.com
>
> This email sent to tempelm...@gmail.com
>
>
--
Thomas Tempelmann, http://www.tempel.org/
Follow me on Twitter: https://twitter.com/tem
56 matches
Mail list logo
