But also donot forget the response size limitation: 10 MB; From my experience, it is not enough for 20000 entities except you just pass few info of an entity.
2010/6/22 Alfred Fuller <[email protected]<arfuller%[email protected]> > > Ultimately you should do performance testing, but in my experience setting > a really large limit on a single query is significantly faster than anything > else. > > > On Sat, Jun 19, 2010 at 11:19 AM, coltsith <[email protected]> wrote: > >> Brandon, thanks for responding, >> >> Will the request take significantly longer as I add more queries? Or >> are 2, 3, 4 queries not much slower than just 1? >> >> Thanks >> >> On Jun 19, 6:47 pm, Brandon Thomson <[email protected]> >> wrote: >> > Filtering in memory works best if you have the all the data aggregated >> > into one or a few entities since there is a very high api-cpu overhead >> > per entity fetch regardless of the amount of data. Either way you go >> > this would be expensive. >> > >> > That said I'd split into multiple queries so you can sort the current >> > results after each fetch and skip fetching the rest if you find you >> > have enough results to display something useful to the user. >> > >> > Regards, >> > Brandon Thomson >> > >> > >> > >> > On Sat, Jun 19, 2010 at 8:28 AM, coltsith <[email protected]> wrote: >> > > Tim, >> > >> > > thanks for responding. >> > >> > > Yes, I know the user won't want to look at all the games. However, I >> > > need to filter and sort the results. For example one way I sort is by >> > > which games have players with the closest rating to you. >> > >> > > So for that, I need to fetch all games, and custom sort through all >> > > them based on their ratings being closest to the calling player's >> > > rating. >> > >> > > Also, I need to have an inequality filter on a property that I don't >> > > sort by. I don't want to return games that were created by the calling >> > > player: >> > >> > > where gameOwnerId != playerId && order by gameCreateDate etc etc. >> > >> > > The current query limitations restrict me too much. >> > >> > > On Jun 19, 1:09 pm, Tim Hoffman <[email protected]> wrote: >> > >> I think you should ask yourself, will anyone really page through >> 20000 >> > >> games >> > >> or even 1000 games? >> > >> > >> On Jun 19, 7:39 pm, coltsith <[email protected]> wrote: >> > >> > >> > In my game server, I let the user browse the open games. They can >> > >> > choose to filter them, or not put any filters at all. >> > >> > >> > My question is, what's the best way to load the games from the >> > >> > datastore. Let's say there's like 20,000. >> > >> > >> > Should I fetch them all in 1 query (no more 1000 limit) or fetch in >> > >> > increments of like 2000 each and combine them into a big list? >> > >> > >> > Thanks >> > >> > > -- >> > > You received this message because you are subscribed to the Google >> Groups "Google App Engine" group. >> > > To post to this group, send email to >> [email protected]. >> > > To unsubscribe from this group, send email to >> [email protected]<google-appengine%[email protected]> >> . >> > > For more options, visit this group athttp:// >> groups.google.com/group/google-appengine?hl=en. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]<google-appengine%[email protected]> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
