On Jan 20, 2008, at 11:01 AM, Christiaan Hofman wrote: > > On 20 Jan 2008, at 7:50 PM, Adam R. Maxwell wrote: > >> >> On Jan 20, 2008, at 10:40 AM, Christiaan Hofman wrote: >> >>> >>> On 20 Jan 2008, at 7:33 PM, Adam R. Maxwell wrote: >>> >>>> >>>> On Jan 20, 2008, at 10:21 AM, Christiaan Hofman wrote: >>>> >>>>> >>>>> On 20 Jan 2008, at 7:08 PM, Adam R. Maxwell wrote: >>>>> >>>>>> >>>>>> On Jan 20, 2008, at 9:52 AM, Christiaan Hofman wrote: >>>>>> >>>>>>> >>>>>>> On 20 Jan 2008, at 5:56 PM, Adam R. Maxwell wrote: >>>>>>> >>>>>>>> >>>>>>>> On Jan 20, 2008, at 8:35 AM, Christiaan Hofman wrote: >>>>>>>> >>>>>>>>> Would it be possible to initialize the SKIndexRef on the >>>>>>>>> thread >>>>>>>>> rather than in init? I find that initializing the cached index >>>>>>>>> blocks >>>>>>>>> the UI for a very noticeable time. I think the index can be >>>>>>>>> accessed >>>>>>>>> on the main thread only in the -index method and in - >>>>>>>>> cancelForDocumentURL:. >>>>>>>> >>>>>>>> Can you check with Shark or Spin Control to see what's slow? >>>>>>>> Initializing the index on the thread would require avoiding >>>>>>>> NSFileManager, unless it's only the SKIndex creation that's >>>>>>>> slow. >>>>>>>> >>>>>>> >>>>>>> It seems to spend a lot of time unarchiving (14.5% in Shark). >>>>>>> About >>>>>>> half comes from +indexCachePathForDocumentURL: and half from >>>>>>> unarchiving the index we're actually using (I have 2 cached >>>>>>> indexes, >>>>>>> the one I use here is significantly larger than the other). >>>>>> >>>>>> That's what I was afraid of (unarchiving). How big is the actual >>>>>> archive in MB? Mine at work has ~500 files indexed, and I didn't >>>>>> see >>>>>> any slowdown or beachball. >>>>>> >>>>> >>>>> It's about 50 MB, indexing about 2/3 of about 1000 items. I did >>>>> some >>>>> dumb timing in the initializer, and getting the cache path and >>>>> unarchiving step are indeed the slowest. >>>>> SKIndexOpenWithMutableData >>>>> was pretty fast. >>>> >>>> Wow! That explains it; my index was < 10 MB. >> >> My index at home is 1.4 MB with ~200 files indexed. This is some odd >> scaling if 5x the number of items leads to 30x disk usage. I wonder >> if the space is due to the SKIndex or the signatures? >> > > Don't forget average file size. What do you guess your average file > size is? I think for me most of my files are about 30-40 pages, but > there are some files with several hundreds of pages.
Average file sizes I have are likely smaller on average, but only the first 2000 terms are indexed in any file. That's why it would be really interesting to see if it's the signatures or the SKIndex that's taking up all that space. >>>> Yet another thing we might try is archiving objects separately >>>> instead >>>> of in an NSDictionary. Then we could use -[NSKeyedUnarchiver >>>> initForReadingWithData:[NSData dataWithContentsOfMappedFile:]] to >>>> unarchive only the document URL instead of the full root object >>>> graph. That assumes NSKeyedUnarchiver doesn't need to read the >>>> entire >>>> file into memory just to archive a single key, of course. >>>> >>>> -- >>>> adam >>> >>> It would still require a full unarchiving of the cached index we >>> want. >> >> True; the unarchiving can be done on the thread, though, after -init >> has returned. It would just cut down on overhead in case of multiple >> huge indexes stored on disk, so you wouldn't have to read all of them >> fully in order to find the right one. > > Of course unarchiving on the thread would mean creating the index on > the thread. And then we need safeguards in -index. (in the cancel > method the lock is probably sufficient). I'd say just creating it as a local variable and then assigning to the ivar when it's done is sufficient. The BDSKSearch now ignores a NULL index (but it won't get any delegate messages until the index is fully created anyway), so -index should be fine as-is. ------------------------------------------------------------------------- This SF.net email is sponsored by: Microsoft Defy all challenges. Microsoft(R) Visual Studio 2008. http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ _______________________________________________ Bibdesk-develop mailing list Bibdesk-develop@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bibdesk-develop
