my apologies for the word typo, i meant the same but my question still remains - if we have to store 1 million customer records, do we create 100 different "Kinds" ~= Entity Groups ~= db.Model and distribute those 1 million records among those entity groups ?
then what about scalability ? does this mean === more records = mroe entity groups to be created ? isnt this an awkward of doing simple things ? thanks. On Oct 28, 10:51 am, Robert Kluin <[email protected]> wrote: > There are no "tables" on App Engine, only Kinds which are arbitrary > collections of properties. In fact, all applications' entities are > stored in a single bigtable "table." > > Entity groups refers to a logical group of entities that are > physically stored together; this allows that group of entities to be > updated in a transaction. > > If you make entity groups with lots and lots of entities in them your > data can not be sharded across bigtable tablets. That means if you > are try to write 5 entities in that group the writes are done > serially. Fetches will also be less efficient for the same reasons. > > You can read all about these > topics:http://code.google.com/appengine/articles/datastore/overview.html > > Robert > > On Wed, Oct 27, 2010 at 21:40, sodso <[email protected]> wrote: > > Hi Ikan Lan, > > > I have seen this on so many discussions here in the forums - > > "A best practice, however, is to keep entity groups as small as > > possible" > > > As far as I understand it, one Model class should not have too many > > records i.e. one table should not have many records > > > Isn't this strange ? When we talk of GAE, we talk of scalability but > > if the best practice says to limit ther records to only a few in a > > Model class, what use is the scalability for ? If we store millions fo > > records (for eg- customer records) in one db.Model table, are you > > saying it would really impact the performance. > > > Please correct me if I am wrong. I am unclear if my app can survive on > > GAE if its working with millions of records in one db.Model table ?? > > > Thanks, > > Sodso > > > On Oct 27, 11:27 pm, "Ikai Lan (Google)" <[email protected]> > > wrote: > >> Generally speaking: no. Entity groups will guarantee a stronger chance of > >> data locality, but this should not affect index traversal or batch reads. > > >> A best practice, however, is to keep entity groups as small as possible. > >> There aren't many compelling reasons to not use root entities if you don't > >> need transactions, as working with root entities is generally simpler. > > >> -- > >> Ikai Lan > >> Developer Programs Engineer, Google App Engine > >> Blogger:http://googleappengine.blogspot.com > >> Reddit:http://www.reddit.com/r/appengine > >> Twitter:http://twitter.com/app_engine > > >> On Tue, Oct 26, 2010 at 3:26 PM, djidjadji <[email protected]> wrote: > >> > Parent and child objects are stored in the same Bigtable node. This is > >> > done for the transactions. A transaction works on a single > >> > entity-group. > >> > If you perform a query you use an index to find the objects needed. At > >> > this point there is no performance penalty for parent or child objects > >> > that match the query. The query results in a number of keys of objects > >> > that need to be retrieved. The best query responds time is when the > >> > objects > >> > to fetch are stored in as many Bigtable nodes as possible (parallel > >> > fetch). > > >> > Why do you need so many child objects? > >> > Can you implement it with adding a Parent Reference Property to the > >> > child object and thus remove the need to store the entity group all in > >> > one Bigtable node? > > >> > The parent-child objects are needed > >> > 1) if you need transactions on them > >> > or > >> > 2) if you want to extract the parent key given a child key > >> > Perform a keys_only query on child objects and from these keys get > >> > the set of parent keys of objects that you need to fetch complete. > >> > Brett Slatkin uses this technique in a number of Google IO talks, > >> > the child object has a ListProperty that is used in the keys_only > >> > query. > > >> > Most other applications can be implemented without the explicit parent > >> > object. > > >> > 2010/10/26 Charles <[email protected]>: > >> > > Hi all, > > >> > > I'm wondering, since the datastore is hierarchical, does the number of > >> > > children an entity has affect the performance on querying on the > >> > > parents > >> > > themselves? For example, if I have a set of parents, say... > > >> > > Jane > >> > > Margaret > >> > > Graham > >> > > Arthur > > >> > > ...and I have a set of children associated with those parents... > > >> > > Jane > >> > > -Sam > >> > > -Robert > >> > > Margaret > >> > > -Lisa > >> > > Graham > >> > > Arthur > >> > > -Rowen > >> > > -Jerry > > >> > > ...will the number of children for each parent affect the performance > >> > > of > >> > > querying the parents themselves? For instance, if I wanted to select > >> > > all > >> > of > >> > > the parents (SELECT * FROM parents), that would be easy with the data > >> > > above. But, since the datastore is hierarchical, does the performance > >> > get > >> > > hampered if say the parents have many thousands or even millions of > >> > > children? Say, like... > > >> > > Jane > >> > > -Sam > >> > > -Robert > >> > > ...1 million more > >> > > Margaret > >> > > ... > > >> > > If so, I'm just wondering if it would make more sense to make the > >> > children > >> > > root entities too, so as not to affect the performance of querying on > >> > > the > >> > > parents. Anyways, hope I've explained my question well enough. > > >> > > Thanks in advance! > > >> > > Charles > > >> > > -- > >> > > You received this message because you are subscribed to the Google > >> > > Groups > >> > > "Google App Engine" group. > >> > > To post to this group, send email to [email protected]. > >> > > To unsubscribe from this group, send email to > >> > > [email protected]<google-appengine%[email protected]> > >> > . > >> > > For more options, visit this group at > >> > >http://groups.google.com/group/google-appengine?hl=en. > > >> > -- > >> > You received this message because you are subscribed to the Google Groups > >> > "Google App Engine" group. > >> > To post to this group, send email to [email protected]. > >> > To unsubscribe from this group, send email to > >> > [email protected]<google-appengine%[email protected]> > >> > . > >> > For more options, visit this group at > >> >http://groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
