On Thu, Jul 17, 2014 at 4:36 AM, Grant Jason <[email protected]> wrote:
> Can you please give us an example or two, wrt re-designing your schema to > avoid running into the 1MB limit for a List property. > Would love to know you recommend here that works with the Datastore. > Well, the problem is that the right schema is so heavily dependent on what type of app is being written, and what the data being stored looks like. But here's a quick example: Some applications need only a short list of values in a single property name. For example, a simple blogging app could use an Entity similar to the below for storing blog posts: Kind: blogpost ID: *[number allocated by the datastore]* Properties: blogtitle: "Hi!" blogtext: "This is my first post on my new blog." creationdate: 1405643123460 (number of milliseconds since UNIX epoch) tags: "blog" , "google" , "myfirstpost" Obviously, the list of tags on a blog post are going to be very short; realistically, you'll never have thousands of tags on a single blog post. So in cases such as this using a list property is fine. The problem comes when a list could be very, very long. Consider an example app designed to hold a list of the user's favorite bookmarks. *A problematic entity could look like this:* Kind: Bookmarks ID: *[number allocated by the datastore]* Properties: email: "[email protected]" favoritelinks: " http://movies.disney.com/frozen " , " learntogoogleit.com " , " https://www.google.com/finance " The problem is that some people have a lot of bookmarks, and URLs can get very long. It's very, very easy to have over 1 MB of bookmark links. Instead of that, create an entity per bookmark, then map the bookmark as the child of the user entity. For instance, you could do this: Kind: UserBookmarks ID: 1 *[number allocated by the datastore]* Properties: email: "[email protected]" *(1 bookmark entity per link, you can set ancestors like this: Entity bookmark = new Entity("Bookmark", id, user_entity.getKey()); )* Kind: Bookmark Ancestor: User:1 (above entity) ID: [number allocated by the datastore] bookmarklink: " http://movies.disney.com/frozen " bookmarktitle: " Frozen " bookmarkdate: 1405643123460 >From there, it's easy to query by setting the ancestor (the user [email protected]). This is also a superior arrangement than before since now metadata can be added about the link. There are other ways to handle this as well. Some developers don't like using ancestor/child relationships; in that case you could skip setting the ancestor on the Bookmark entity, but add a property listing the user the bookmark belongs to: *user: [email protected] <[email protected]> *(or the ID of the user)*. *Then query by adding an *AND user=useremail/ID* property to the query. ' Another option is to use namespacing. Declare a namespace per user ( https://developers.google.com/appengine/docs/java/multitenancy/multitenancy#Java_Creating_namespaces_on_a_per_user_basis ), then add an entity per bookmark. But the entities don't need to list an ancestor or an owning user because the namespacing service will automatically know which namespace (the username of the user) the entity belongs to. Some lists can be dramatically shortened by using IDs instead of exact names. For example, suppose an entity stored a list of books by name. That list of names could be replaced by instead listing the ISBN <http://en.wikipedia.org/wiki/International_Standard_Book_Number> number of the books. This makes the list much shorter and simpler to debug. With that said, a caveat: in cases such as this, it's always important to reevaluate your storage options. Some people try to force their data to fit into the datastore when purchasing a Cloud SQL instance would be better. Always consider your data and the queries you run: some queries and schema will simply be cheaper with Cloud SQL even though the datastore has a free quota. There is nothing wrong with an application using both the datastore and Cloud SQL services: complex applications often do that. -------------------------------- *Last item: this email turned out longer than I wanted it to be, and I glossed over a lot of important topics and discussion (bolded for a reason!)*. If you're interested in this topic, I suggest getting a copy of Programming App Engine: http://www.amazon.com/Programming-Google-App-Engine-Sanderson/dp/144939826X <http://www.amazon.com/Programming-Google-App-Engine-Sanderson/dp/144939826X> . It covers a lot about the datastore and I believe the author is a Google engineer himself, so you're getting the facts straight from the proverbial mouth :-) I would also suggest 7 Databases in 7 Weeks <http://www.amazon.com/Seven-Databases-Weeks-Modern-Movement/dp/1934356921/> (covers NoSQL, not the App Engine Datastore directly but it's useful) and Database Design For Mere Mortals <http://www.amazon.com/Database-Design-Mere-Mortals-Hands-/dp/0321884493> (covers relational design, still useful). And finally (whew, this is getting out of hand!) to any college students and fresh graduates reading this: database design and management is a hard, but very fun subject. It's also a well paying job, if that's something that interests you..... ----------------- -Vinny P Technology & Media Consultant Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
