Re: NSDocument save incremental file package in-place

2014-03-03 Thread Quincey Morris
On Mar 3, 2014, at 14:26 , Trygve Inda wrote: > I did some testing > > All 3750 in one folder: 40 seconds to save. > > Split into two levels: 16 folders -> 16 folders -> image files: 5 sec > > Three levels: 16 folders -> 16 folders -> 16 folders -> image files: 10 sec It seems to me this

Re: NSDocument save incremental file package in-place

2014-03-03 Thread Trygve Inda
> I did some testing about optimal number > of files per folder for our package format and 3750 files per directory are > way too many. The file system is much more efficient if you spread those > files into sub-folders. We got the best performance with a 3-level sparse > folder index (files are

Re: NSDocument save incremental file package in-place

2014-03-03 Thread Mike Abdullah
If you do end up writing your own safe-saving logic, that's achieved by overriding -writeSafelyToURL:… Sent from my iPad On 2 Mar 2014, at 15:23, Trygve Inda wrote: >> 7,500 file operations of any kind are going to take some time, even just >> creating hard links. Does it require 40 seconds? W

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Trygve Inda
> Can't say, probably less than 100 per folder. It's easy to find out, though, > just create different layouts and see which one is getting saved most > quickly. I can remember that we experimented extensively with the number of > index levels (each level having 16 potential slots) and 3 turned o

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Markus Spoettl
On 3/2/14 5:22 PM, Trygve Inda wrote: I did some testing about optimal number of files per folder for our package format and 3750 files per directory are way too many. The file system is much more efficient if you spread those files into sub-folders. We got the best performance with a 3-level

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Trygve Inda
> I did some testing about optimal number of files per folder for our package > format and 3750 files per directory are way too many. The file system is > much more efficient if you spread those files into sub-folders. We got the > best performance with a 3-level sparse folder index (files are st

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Markus Spoettl
On 3/2/14 3:43 PM, Trygve Inda wrote: -Root - plist file (about 20mb) - Folder containing 3750 image files of about 50kb each - Folder containing 3750 image files of about 100kb each I did some testing about optimal number of files per folder for our package format and 3750 files per directory

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Trygve Inda
> 7,500 file operations of any kind are going to take some time, even just > creating hard links. Does it require 40 seconds? Well, I don't know. But I do > seriously doubt that it could be done in 1 second. > > You might try it out yourself, write test code to create a new package, walk > through

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Trygve Inda
>> A follow up... >> >> Adding this to my NSDocument subclass: >> >> - (BOOL)writeToURL:(NSURL *)absoluteURL ofType:(NSString *)typeName >> forSaveOperation:(NSSaveOperationType)saveOperation >> originalContentsURL:(NSURL *)absoluteOriginalContentsURL error:(NSError >> **)outError >> { >>NSE

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Scott Ribe
7,500 file operations of any kind are going to take some time, even just creating hard links. Does it require 40 seconds? Well, I don't know. But I do seriously doubt that it could be done in 1 second. You might try it out yourself, write test code to create a new package, walk through your pac

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Trygve Inda
> You also need to overwrite > - (void)setFileURL:(NSURL *)absoluteURL > > in which you call super and then update your stored wrapper (if you have one > already) using -[NSFileWrapper readFromURL:options:error:]. > > You need to do this because the filewrapper you return in -fileWrapperOfType

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Markus Spoettl
On 3/2/14 12:06 AM, Quincey Morris wrote: > I’ve never used this, but I *have* seen documents use hard linking anyway > (though, on reflection, that might have been UIDocument, not NSDocument — I’m > not sure now). No, it is NSDocument/NSFileWrapper, UIDocument/NSFileWrapper doesn't know this ki

Re: NSDocument save incremental file package in-place

2014-03-02 Thread Mike Abdullah
On 2 Mar 2014, at 00:40, Trygve Inda wrote: >> >> On 1 Mar 2014, at 23:26, Trygve Inda wrote: >> >>> I have a top level FileWrapper which contains the wrappers for all my files. >>> Is there any sample code that would help me figure out the right way to do >>> this? >>> >>> [NSFileWrapper wr

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Trygve Inda
> > On 1 Mar 2014, at 23:26, Trygve Inda wrote: > >> I have a top level FileWrapper which contains the wrappers for all my files. >> Is there any sample code that would help me figure out the right way to do >> this? >> >> [NSFileWrapper writeToURL:options:originalContentsURL:error:] >> >> See

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Trygve Inda
> > On 1 Mar 2014, at 23:26, Trygve Inda wrote: > >> I have a top level FileWrapper which contains the wrappers for all my files. >> Is there any sample code that would help me figure out the right way to do >> this? >> >> [NSFileWrapper writeToURL:options:originalContentsURL:error:] >> >> See

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Mike Abdullah
On 1 Mar 2014, at 23:26, Trygve Inda wrote: > I have a top level FileWrapper which contains the wrappers for all my files. > Is there any sample code that would help me figure out the right way to do > this? > > [NSFileWrapper writeToURL:options:originalContentsURL:error:] > > Seems to call my

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Trygve Inda
> > On 1 Mar 2014, at 19:11, Trygve Inda wrote: > >>> On Mar 1, 2014, at 07:23 , Trygve Inda wrote: >>> The problem is that when there is a very small change (just adding or removing one of the files in the package), the system does not save in place. Rather it reads

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Quincey Morris
On Mar 1, 2014, at 14:53 , Mike Abdullah wrote: > It is -[NSFileWrapper writeToURL:options:originalContentsURL:error:] which > has the feature of writing out hard links for efficiency. But it only does it > if requested, by passing a suitable URL as the original contents URL. It is > my suspic

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Mike Abdullah
On 1 Mar 2014, at 19:11, Trygve Inda wrote: >> On Mar 1, 2014, at 07:23 , Trygve Inda wrote: >> >>> The problem is that when there is a very small change (just adding or >>> removing one of the files in the package), the system does not save in >>> place. >>> >>> Rather it reads the previous

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Trygve Inda
> On Mar 1, 2014, at 11:11 , Trygve Inda wrote: > >> I really need this to be faster. > > I think the point I was trying to reach was that your next step is to > investigate what is taking the time. IIRC there’s a NSURL attribute key you > can use to retrieve a file’s inode number, so you can ch

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Trygve Inda
> On Mar 1, 2014, at 07:23 , Trygve Inda wrote: > >> The problem is that when there is a very small change (just adding or >> removing one of the files in the package), the system does not save in >> place. >> >> Rather it reads the previous package file completely, writes out a copy of >> the p

Re: NSDocument save incremental file package in-place

2014-03-01 Thread Quincey Morris
On Mar 1, 2014, at 07:23 , Trygve Inda wrote: > The problem is that when there is a very small change (just adding or > removing one of the files in the package), the system does not save in > place. > > Rather it reads the previous package file completely, writes out a copy of > the package (40

NSDocument save incremental file package in-place

2014-03-01 Thread Trygve Inda
I have a Document whose file type is a package and potentially contains a few thousand files. My test case is about 7500 files (mostly images). I am using -(NSFileWrapper *)fileWrapperOfType:(NSString *)typeName error:(NSError **)outError The problem is that when there is a very small change (j