ok, I don't see anything wrong with the predicate code, but I'm no core data
expert.
I'll make one totally challengable statement. Assuming that core data uses
sqllite in a rational way to store objects (eg not storing everything as blobs
of opaque data) for instance one table per entity where each column of the
table is an attribute and evaluating the predicate does what you would expect
it to do, ie uses SQL to do as much of the heavy lifting on a fetch request as
possible, that column is indexed in the table and sqllite is using the index;
taking multi-minutes to find one row out of 20,000 just doesn't make any sense,
it should take seconds at most.
I believe core data does use table-per-entity. I think that partly because the
documentation hints at it, partly because it makes sense and partly because I
looked at the implementation of one data model that I have.
I can't see the point of making indexes if the predicate code doesn't generate
SQL which doesn't use them, but it's possible. It's possible that core data
goes and loads all the entity rows and inspects their attributes by hand and
filters them in code, but this is apple not microsoft.
So that leaves column isn't indexed as the most likely. But you've checked the
'indexed' box. Here's another wild assed guess, does coredata only create a
store when you have no current store? It certainly checks to see if the store
is compatible with the model but as the indexed property is just a hint anyway,
that store is compatible, just non-optimal .. it's possible if you created the
store with the property defined as not-indexed and have just checked that box
later, without regenerating the whole store, the index was never added. Did you
do that, just check it later? Have you regenerated a complete new store since
or are you using a store you've been populating for a while.
Here's a particularly ugly idea, purists please stop reading now. We can look
at the store and see if it has an index on that property ... first get up a
terminal window and go to the path where your store is. I'm assuming you have
sqlite3 installed like I do .. it came with the OS as far as I know.
Your store should be called something.sqlite, let's say it's Foo. Type
sqlite3 Foo.sqlite
and that should open the store and give you a prompt. First you want to find
the tables in the store, so type
.tables
as far as I can see they are called Z<YOUR ENTITY NAME>, so for you I'd expect
to see one of the tables called ZMCARTICLE. If there is one, you can find out
what indices are on it
.indices ZMCARTICLE
I believe again the indices are called Z<YOUR ENTITY NAME>_Z<YOUR ATTRIBUTE
NAME>_INDEX, so you'd expect to find ZMCARTICLE_ZMESSAGEID_INDEX in that list.
If you don't have it, the store wasn't created with that index. If none of
those tables exist at all, my rudimentary reverse engineering of the whole
coredata thing is flawed (or I'm using some entirely different version from
you).
If the tables and indices exist, including the one on ZMESSAGEID, I'm out of
ideas unless someone knows of a way to put coredata into a form of debug mode
and see the SQL generated to figure out if it's doing anything smart.
If either none of the above works or it does work but you don't have the index,
you have a couple of options. The right one is to delete your whole message
store and run your app and make a brand new one to see if that then adds the
indexed property with an index. Depending on how you've populated the store,
that might be a real pain, perhaps you can force a migration or something. The
other really stupid idea would be to just add the index and hope that doesn't
break everything entirely which is entirely possible at which point you delete
the store and start over. You would do that by running
CREATE INDEX ZMCARTICLE_ZMESSAGEID_INDEX ON ZMCARTICLE (ZMESSAGEID);
Here's another useful thing I just came across, I would certainly run this to
see if the SQL being executed makes sense.
With Mac OS X version 10.4.3 and later, you can use the user default
com.apple.CoreData.SQLDebug to log to stderr the actual SQL sent to SQLite.
(Note that user default names are case sensitive.) For example, you can pass
the following as an argument to the application:
-com.apple.CoreData.SQLDebug 1
Higher levels of debug numbers produce more information, although using higher
numbers is likely to be of diminishing utility.
I'd love to hear about any other ways people have to debug coredata. I sort of
trust apple has done a good job with it and for it to break down performance
wise on looking for a row in 20,000 with a certain attribute doesn't make sense
to me. If you really can't get it to work, I'd write a short project which
inserts 20,000 simple objects into a store and another one which opened the
store and goes looking for one by attribute in the way you have. If it takes
multi-minutes, I'd sent it to apple as a bug.
On 13-Feb-2010, at 6:32 PM, daniele malcom wrote:
>
>
> On Sat, Feb 13, 2010 at 4:06 AM, Roland King <[email protected]> wrote:
> That's not a horrible solution, except for the feeling that core data ought
> to let you do what you want without having to implement your own UUID cache.
> I'm still a bit surprised that a lookup for an object by one attribute is
> taking so long, over just 30,000 objects. You do have the uuid attribute
> marked as indexed right?
>
> I found http://cocoawithlove.com/2008/03/testing-core-data-with-very-big.html
> whilst hunting around for some examples of core data with big data sets. This
> guy was working on sets of 1 million objects and doing fetches with indexed
> properties was taking about 2 seconds, vs non-indexed, 600 seconds. There are
> some comments at the bottom from an apple engineer too.
>
> Hi Roland,
> Uhmmm probability there is something wrong with my code, because with indexed
> messageid property it takes a very long time too (I can't see any visible
> difference, 20k messages took minutes to be completed)
> Here you will found my simple storage data model:
> http://img197.imageshack.us/img197/4388/screenshot20100213at111.jpg
>
> This is my code:
> for (NSString *cMessage in messagedata) {
> NSString *idd = [cMessage
> headerValueFor:ARTICLE_TAG_ID]; // takes the MESSAGE UUID
> if ([thegroup articleForID:idd inCtx:ctx] == nil) { //
> CHECK IF MESSAGE UUID IS ALREADY ON DB
> MCArticle *c = [MCArticle newArticleWithID:idd
> context:ctx]; // WE CAN MAKE THE MESSAGE AND ADD IT TO COREDATA
>
> // WE WANT TO CHECK FOR PARENT
> NSString *parentidms = [[[f
> headerValueFor:ARTICLE_TAG_REFS] reverseOrderedReferences] objectAtIndex:0];
> MCArticle *parent = [thegroup
> articleForID:parentidms inCtx:ctx]; // WE QUERY FOR PARENT UUID
> if (parent != nil) {
> // LINK BOTH PARENT AND CHILD
> }
> }
> }
>
> ArticleForID:inCtx: method follow (inCtx is required becase these functions
> works in a multithreading environment)
> (I've also tried to re-use fetchRequest object but it's not the main
> problem...)
>
> - (MCArticle *) articleForID:(NSString*) _msgid
> inCtx:(NSManagedObjectContext*) ctx {
> if (_msgid == nil) return nil;
> NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
> [fetchRequest setFetchLimit:1];
> [fetchRequest setEntity: [NSEntityDescription
> entityForName:@"MCArticle" inManagedObjectContext:ctx]];
> NSPredicate *predicate = [NSPredicate predicateWithFormat:@"messageid
> == %@",_msgid];
> [fetchRequest setPredicate: predicate];
> NSArray *results = [ctx executeFetchRequest:fetchRequest error:nil];
> [fetchRequest release];
> if ([results count]==0)return nil;
> return [results objectAtIndex:0];
> }
>
> Finally the init method for Message object:
>
> + (MCArticle *) newArticleWithID:(NSString *) _messageID
> context:(NSManagedObjectContext *) _context {
> MCArticle *oj = (MCArticle*)[[NSManagedObject alloc]
> initWithEntity:[NSEntityDescription entityForName:@"MCArticle"
>
>
> inManagedObjectContext:_context]
>
> insertIntoManagedObjectContext:_context];
> [oj setValue:_messageID forKey:@"messageid"];
> return oj;
> }
_______________________________________________
Cocoa-dev mailing list ([email protected])
Please do not post admin requests or moderator comments to the list.
Contact the moderators at cocoa-dev-admins(at)lists.apple.com
Help/Unsubscribe/Update your Subscription:
http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com
This email sent to [email protected]
