As we all know we can not use join, aggregation functions, group by or multiple inequalities in datastore query. Google's argument is these relational feature are not scalable. But there has to be some use cases like ad hoc request to generate some report. We then have to prepare / test our own mapreduce / pipeline scripts and deploy to production environment, which is usually error prone and not trivial.
Project Yaac(Yet Another Admin Console) is targeted to solve this common use case. Yaac is not a standalone java or python library, instead it's a complete application which is designed to be deployed to your own domain (non-default version of course). Yaac provides Extended GQL(EGQL), which supports aggregation function / group by / complex nested expressions / multiple inequality filters / *not*, *or* clause in where condition. For instance, in a football datastore, you can execute following EGQL query: select HOME_TEAM, AWAY_TEAM, COUNT(1), sum(FTHG > FTAG), sum(FTHG = FTAG), sum(FTHG < FTAG) from MATCH where HOME_TEAM = "Arsenal" group by HOME_TEAM, AWAY_TEAM having sum(FTHG > FTAG) > sum(FTHG = FTAG) + sum(FTHG < FTAG) More details visit this page: http://code.google.com/p/yaac/wiki/EGQLReference Because it's an early experimental verion, some part of the language may subject to change in the future. There is also a sandbox environment http://sandbox.yetanotheradminconsole.appspot.com/. Anyone with a Google account can login and play with Sandbox. Simply go to Datastore --> Extended GQL then execute above query. I have uploaded 86000 football matches (all matches played in 10 leagues for last 18 seasons). For example, to retrieve all matches played in Arsenal's home against Man United after year 2000 and at least one teams scored in the game. We can execute following query: select HOME_TEAM, AWAY_TEAM, format(MATCH_DATE, "MMM dd, yyyy"), FTHG, FTAG from MATCH where HOME_TEAM = "Arsenal" and AWAY_TEAM = "Man United" and MATCH_DATE > datetime("20000101") and FTHG + FTAG > 0 (FTHG means full time home goal :) ) It only tooks several steps to deploy Yaac to your own domain, check out http://code.google.com/p/yaac/wiki/DeployGuideline Again, it's a very very early experimental release, you may notice that the performance is extremely slow. This is because all queries are executed by pipeline API in a niave way, sequential scanning over whole datastore. I will spend more time to optimize it these days. Please try to set batch size to a smaller value if you are not performing query with aggregation / group by, otherwise it's very easy for a result message to hit channel service single message size limit (32KB). This will also be fixed soon. I will also try to provide a better error / exception messages when there is an error in query syntax or during query execution Any comments? Feedbacks? Feature requests are all welcome! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/bwwGQ0MGjJ0J. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
