The most important number in my mind is the number of line items in an order. If you're seeing 20s+ queries, it must be a pretty large number.
Objectify (and Twig) support queryable collections of embedded objects, so you can put all the line items in a single order object. There are some practical limits to this approach, though - the collection can have at most 5k items and the total entity size cannot exceed 1MB. Also, writing out indexed collections with 5,000 entries is *very* expensive in terms of $, so if you have a high churn rate you'll burn through your budget pretty fast. Obviously serialization is an option, but then you won't be able to query for orders by line item data. If it makes the most sense to keep the line items as separate objects (because of churn or simple volume of line items) you can still use the memcache to your advantage. You'll want a bidirectional relationship, with Keys stored in the Order pointing to the LineItems as well as vice-versa. You can load an Order and use the LineItem keys to perform a batch get, hopefully most of which will come out of the memcache. It's hard to suggest a strategy without knowing the shape of the data, the query expected profile, and the churn rate. Jeff On Tue, Apr 6, 2010 at 2:23 PM, Steve Pritchard <steve...@gmail.com> wrote: > Before proposing a solution, what is your expected active user base. > 1000's or 1,000,000s or something in between. > Steve > > On Apr 6, 3:50 pm, Matt Hall <matt.h...@gmail.com> wrote: >> Thanks for the response, could you point me to somewhere where I can >> read about batched queries by key in JDO? Or do I need to go to the >> low level API for that? >> >> I think the conclusion I'm coming to here is 1) I shouldn't have made >> the detail for each order as a separate object, (although this has >> huge problems for querying later as in how many people have bought >> item x), and 2) JDO managed relationships get slow with small numbers >> of child objects, are hard to optimize, and probably not worth the >> trouble overall. Is that pretty much what I should be concluding? >> >> Thanks again, I'm really trying to understand how to translate what's >> a basic structure in RDMS land into something that will work in app >> engine beyond toy implementations. >> >> On Apr 6, 12:54 pm, "Ikai L (Google)" <ika...@google.com> wrote: >> >> > For speed, you'll get the most gains if you denormalize when possible - >> > yes, >> > the relational purists are going to riot, but you don't have many of the >> > benefits of normalization anyway such as foreign key constraints or native >> > joins. >> >> > If that isn't an option - and I really do recommend looking into it where >> > it >> > makes sense - if you're able to retrieve keys, we may also be able to make >> > use of batch queries by key. >> >> > On Mon, Apr 5, 2010 at 2:59 PM, Matt Hall <matt.h...@gmail.com> wrote: >> > > Hi Everyone, >> >> > > I've been trying for the past while to get our relatively simple JDO >> > > based site to run at a reasonable speed, but I'm failing and I'd love >> > > any suggestions. I'll describe it as succinctly as possible to >> > > hopefully keep your interest :) >> >> > > * The basic structure is a User -> Order -> Line items type >> > > structure. >> > > * The main query performed is "Orders for a user." >> > > * A user usually has around 10 orders, each order has under 100 line >> > > items. >> > > * The Order -> Items relationship is done as a One-to-many owned >> > > relationship. >> >> > > My problem is that the basic request the site handles: "Get all orders >> > > for a user and do something based on the line items" is very slow. In >> > > real response time, my average is now 1.5 seconds, and some of the >> > > users are seeing 20 second response times. Clearly not ok. >> >> > > What appears to be slow, after profiling with the new app stats tool >> > > is that each line item is getting loaded as it's used (lazy loading), >> > > which works great in some cases but terrible in this case since each >> > > item takes 20-30ms to load. >> >> > > I have tried: >> >> > > * Moving items into the default fetch group for orders, seems to have >> > > no effect (may not even be possible due to limitations of joins) >> > > * Level 2 caching with memcache - not much effect either since only >> > > the individual line items are cached, not the whole List, so we still >> > > pay a price in round trip to the cache on each line item. >> > > * A sort of wacky cache that I wrote on top of the JDO managed >> > > relationship to cache the whole collection. This went poorly and >> > > resulted in odd data inconsistencies, probably because I did something >> > > wrong but the whole thing felt wrong. >> >> > > So my question is - what am I doing wrong? What's the best/standard >> > > way to represent a Master -> Detail type relationship, where the total >> > > items for a user will be relatively small but requests need to work >> > > with a fair number of them. >> >> > > Thanks for any suggestions, I'd be happy to answer any questions if >> > > what I've written isn't clear. >> >> > > Matt >> >> > > -- >> > > You received this message because you are subscribed to the Google Groups >> > > "Google App Engine for Java" group. >> > > To post to this group, send email to >> > > google-appengine-j...@googlegroups.com. >> > > To unsubscribe from this group, send email to >> > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2B >> > > unsubscr...@googlegroups.com> >> > > . >> > > For more options, visit this group at >> > >http://groups.google.com/group/google-appengine-java?hl=en. >> >> > -- >> > Ikai Lan >> > Developer Programs Engineer, Google App >> > Enginehttp://googleappengine.blogspot.com|http://twitter.com/app_engine > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
