hello again, one more thing: is it possible to reduce the amount of data that is returned dramatically by doing all the aggregations (e.g. by grouping and summing) completely within the datastore? currently i am experimenting with doing full table scans (but i can only retrieve 1000 items in one request..) and aggregating the data within the application (on server side).
kind regards, ingo 2010/7/16 Ingo Jaeckel <[email protected]>: > hello everyone, > > i want to provide detailed reports in my app. those reports require > data aggregated across full tables. in the client i want to plot the > change of values over time. this then requires to select very huge > data sets. however, in the app engine there is the restriction to get > 1000 items in one request. so what is the best approach to do queries > with big result sets? > > i can imagine that the client could split the big request into 10 > smaller ones to collect, merge and display the data. but i think this > is complex, error prone and slow. i would rather like to do this on > the server side. but do i have to split the requests to work around > the 1000 items limit or is there another pattern that developers > should use to handle huge datasets. > > basically i want to know two things: > 1) how can i retrieve huge amounts of data regarding the get 1000 items limit? > 2) is the app engine performance good enough to allow showing reports > on the client side i.e. can app engine retrieve and analyse > (avg,min,max,grouping) huge amounts of data (10k - 1million rows) fast > enough? > > kind regards, > ingo > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
