I really like this project and have been hoping someone would do it for a long time. The need to run arbitrary reports on the datastore is clear, and the current tools are a pain. So this is a great start. My feedback:
- It's still buggy. I ran a few queries and only some of them worked, and the others just get stuck in processing. - Definitely change the name. "Yet another..." makes it feel cheap. - I don't like the concept of running it as a separate version. I prefer to drop the code in a separate folder in my project and access it with it's own path (e.g. /yaac), just like appstats and mapreduce. - I would suggest focusing on EGQL and making it really solid. Remove the other features (blobstore, memcache, taskqueues...etc) to avoid distractions until EGQL is robust. Expanding the scope at this early stage will only spread your efforts too thin and you end up with many half-baked features that no one wants to use. Good luck, Waleed On Mon, Oct 31, 2011 at 9:24 AM, Max <[email protected]> wrote: > As we all know we can not use join, aggregation functions, group by or > multiple inequalities in datastore query. Google's argument is these > relational feature are not scalable. But there has to be some use cases > like ad hoc request to generate some report. We then have to prepare / test > our own mapreduce / pipeline scripts and deploy to production environment, > which is usually error prone and not trivial. > > Project Yaac(Yet Another Admin Console) is targeted to solve this common > use case. Yaac is not a standalone java or python library, instead it's a > complete application which is designed to be deployed to your own domain > (non-default version of course). Yaac provides Extended GQL(EGQL), which > supports aggregation function / group by / complex nested expressions / > multiple inequality filters / *not*, *or* clause in where condition. For > instance, in a football datastore, you can execute following EGQL query: > > select HOME_TEAM, AWAY_TEAM, COUNT(1), sum(FTHG > FTAG), sum(FTHG = > FTAG), sum(FTHG < FTAG) > from MATCH > where HOME_TEAM = "Arsenal" > group by HOME_TEAM, AWAY_TEAM > having sum(FTHG > FTAG) > sum(FTHG = FTAG) + sum(FTHG < FTAG) > > More details visit this page: > http://code.google.com/p/yaac/wiki/EGQLReference Because it's an early > experimental verion, some part of the language may subject to change in the > future. > > There is also a sandbox environment > http://sandbox.yetanotheradminconsole.appspot.com/. Anyone with a Google > account can login and play with Sandbox. Simply go to Datastore --> > Extended GQL then execute above query. I have uploaded 86000 football > matches (all matches played in 10 leagues for last 18 seasons). For > example, to retrieve all matches played in Arsenal's home against Man > United after year 2000 and at least one teams scored in the game. We can > execute following query: > > select HOME_TEAM, AWAY_TEAM, format(MATCH_DATE, "MMM dd, yyyy"), FTHG, > FTAG > from MATCH > where HOME_TEAM = "Arsenal" and AWAY_TEAM = "Man United" and MATCH_DATE > > datetime("20000101") and FTHG + FTAG > 0 > > (FTHG means full time home goal :) ) > > It only tooks several steps to deploy Yaac to your own domain, check out > http://code.google.com/p/yaac/wiki/DeployGuideline > > Again, it's a very very early experimental release, you may notice that > the performance is extremely slow. This is because all queries are executed > by pipeline API in a niave way, sequential scanning over whole datastore. I > will spend more time to optimize it these days. > > Please try to set batch size to a smaller value if you are not performing > query with aggregation / group by, otherwise it's very easy for a result > message to hit channel service single message size limit (32KB). This will > also be fixed soon. > > I will also try to provide a better error / exception messages when there > is an error in query syntax or during query execution > > Any comments? Feedbacks? Feature requests are all welcome! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/bwwGQ0MGjJ0J. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
