The biggest challenge to using the app engine is adjusting to the fact that the datastore, despite it's SQL-like outward appearance, is basically just an object store. Accessing or updating an individual "record" has equivalent performance to updating a file on disk.
Addressing your specific concerns: - Users: That sort of scalability shouldn't be a problem. The most important thing is to limit "write" contention on centralized resources. - Centralized parent site: This is the hardest part. The 1000-item limit is a little bit misleading, in my experience; you start having performance problems well before you get that high. It's almost impossible to display 400 of anything at once on the Google app engine; you have to find ways to show less while still making the rest available. It does support indexing, so you can use the equivalent of a "WHERE" filter to limit results to a specific tag, for example, and you also have the equivalent to "ORDER BY", which when used together, make simple next / previous paging possible. Another problem is Python: the language goes all out to let you do just about anything imaginable, the consequence is that virtually nothing can be checked at compile-time... you have to test extensively to make sure all the code is being exercised to avoid "live" problems. -Ryan On Mar 9, 5:45 am, "[email protected]" <[email protected]> wrote: > Hi, > > So basically my background is java and scripting languages web > application development, RDMS, that sort of thing.I have a project I'd > like to build with GAE because it is expected to be very popular and > needs to scale. > > I'm used to scalability issues because I build serveral large sites > which ran into issues that went beyond the sort of thing you can solve > by 'better indexing your queries' - so I do understand the need for a > mindset change and a different approach to architecture > (denormalization, etc). I also do understand that there are > limitations that are inherent to GAE and I'll have to live with it. > > However, I am concerned that these limits will prevent us to even > build the application - in particular, here's a scenario I'd like to > know if it is possible to deploy or not: > - Users (probably around 50,000 for starters, to scale to 500-600K), > create 'mini websites'. Each of these sites contains various types of > media (image gallery, blog, etc). > - A 'centralized', parent site give visitors the chance to 'browse' > these sites. For example, the specs require that a page holding 'the > last 400 blog posts posted by any of the mini-sites owners' be > displayed. It should also be possible to filter all the mini sites > content by tag . So for example, if I clicked the 'cats' tag, I'd get > a recordset containing all the content that was ever posted about > cats, regardless of source (blog, images, etc). It means searching > through dozens of millions of rows for each 'table' - and there's a > 'table' for each type of content (because a blog post is structured > differently from an image gallery). > > Since I read so much about limitations on large queries, etc, I was > wondering if the above is even something I could consider for > implementation on GAE. > > Thank you. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
