Fun fact: we're really just a bunch of normal people who dig our jobs and enjoy problem solving.
Seems like you'd be able to build something like this with the Servers API, though there are edge conditions around failure that are difficult. Ultimately, this would be challenging and reliability could be a dealbreaker for a lot of people (otherwise you could just use Memcache incr and flushing). Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Sat, Mar 19, 2011 at 5:43 AM, Joshua Smith <[email protected]>wrote: > Just have some of those famous google PhDs noodle on it a while. I'm sure > they can come up with some insanely clever way to make a scalable counter. > Perhaps simplify the problem domain a little. For example, a > direct-it-to-a-single-machine solution can scale just fine if you make it > asynchronous. The requestor asks for the next ID and provides a URL handler > to finish the transaction when one becomes available. > > > On Mar 18, 2011, at 8:22 PM, Nick Johnson (Google) wrote: > > Hi folks, > > The problem - and part of the reason App Engine doesn't provide it - is > that there's no scalable way to provide an arbitrarily scalable monotonic > counter. Pretty much by definition, it requires a single coordinating > machine that issues all the IDs, which limits how far it can scale. App > Engine's approach to issuing IDs for entities is a way of avoiding that, at > the expense of monotonicity. > > On Fri, Mar 18, 2011 at 7:03 AM, Joshua Smith <[email protected]>wrote: > >> I second this. I think the "there are lots and lots of use cases" is a >> lazy excuse for not providing a good implementation of the most typical one. >> >> "Generate the next sequential ID" is a function universally provided by >> SQL databases. Also, SQL databases always provide an efficient count(rows) >> function, whereas GAE does not. So I don't think it's unreasonable to >> expect google to back-fill this deficiency as David asks. >> > > Actually, I'm not aware of any relational databases that provide an > efficient count() implementation other than on complete tables (which is > rarely very useful) - counting the results in an arbitrary result set > requires iterating over all the rows, which is what App Engine does. > > -Nick Johnson > > >> >> -Joshua >> >> On Mar 17, 2011, at 10:14 PM, Thomas Wiradikusuma wrote: >> >> > But by saying that, you agree that it's indeed difficult and it can be >> > implemented in various ways, which actually only lead to one thing: a >> > counter. >> > >> > If GAE has it built-in, lazy people like me don't have to scratch head >> > to implement one that works efficiently (and keep wondering whether >> > there's better implementation elsewhere), because the wheel is already >> > invented. >> > >> > But I agree that covering all use cases are hard. >> > >> > On Mar 17, 10:49 am, Robert Kluin <[email protected]> wrote: >> >> Implementing a scalable counter is pretty hard, and different uses >> >> have different requirements on the behavior of counters. Some will >> >> apps might demand sequential numbers, others monotonically increasing >> >> (but not necessarily sequential) numbers, some might also need to >> >> support decrementing the counter. There are several techniques you >> >> can use to create a counters, depending on your needs and the traffic >> >> your app gets. I suspect that's why Google does not provide it as >> >> service -- it won't satisfy everyone's needs. >> >> >> >> Robert >> >> >> >> On Wed, Mar 16, 2011 at 22:38, Thomas Wiradikusuma >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> >> <[email protected]> wrote: >> >>> I agree with David. >> >> >> >>> It's a bit "higher level" compared to other GAE services, but from >> >>> application perspective, it's just another plumbing. >> >>> Just like MapReduce in GAE and AppStats, they sit on top of existing >> >>> services, yet they are considered low level for the rest of us. >> >> >> >>> Just my 2 cents :) >> >> >> >>> On Mar 17, 2:52 am, "Ikai Lan (Google)" <[email protected]> wrote: >> >>>> So there are different schools of thoughts on this issue. There are >> those of >> >>>> us who think that App Engine should be as bare metal as >> possible,there are >> >>>> those of us who think App Engine should provide every single possible >> API, >> >>>> and there are people who argue for various positions in between. >> >> >> >>>> I personally think this makes sense as a configurable option or a >> library, >> >>>> but then that requires that we have a better place to surface App >> Engine >> >>>> libraries. There are a lot of great libraries today, but even now, >> it's not >> >>>> obvious to developers how to find Googler maintained projects like >> App >> >>>> Engine Mapper (http://code.google.com/p/appengine-mapreduce/) or the >> >>>> Pipeline API (http://code.google.com/p/appengine-pipeline/). Don't >> even get >> >>>> me started on non-Googler maintained projects like Fantasm ( >> http://code.google.com/appengine/articles/fantasm.html). >> >> >> >>>> Ikai Lan >> >>>> Developer Programs Engineer, Google App Engine >> >>>> Blog:http://googleappengine.blogspot.com >> >>>> Twitter:http://twitter.com/app_engine >> >>>> Reddit:http://www.reddit.com/r/appengine >> >> >> >>>> On Tue, Mar 15, 2011 at 10:25 PM, David Keyes <[email protected]> >> wrote: >> >>>>> Been using gae/j for a while now. One big missing piece IMO is a >> service >> >>>>> that would allow me to manage global distributed counters. Yes I >> know that >> >>>>> there are patterns for doing this (e.g. sharding), but it is such a >> common >> >>>>> use case that I think a highly optimized service would be ideal. >> >> >> >>>>> The perfect solution would allow me to have read-only access to the >> >>>>> counters via the datastore. >> >> >> >>>>> Thoughts? >> >> >> >>>>> -- >> >>>>> You received this message because you are subscribed to the Google >> Groups >> >>>>> "Google App Engine" group. >> >>>>> To post to this group, send email to >> [email protected]. >> >>>>> To unsubscribe from this group, send email to >> >>>>> [email protected]. >> >>>>> For more options, visit this group at >> >>>>> http://groups.google.com/group/google-appengine?hl=en. >> >> >> >>> -- >> >>> You received this message because you are subscribed to the Google >> Groups "Google App Engine" group. >> >>> To post to this group, send email to >> [email protected]. >> >>> To unsubscribe from this group, send email to >> [email protected]. >> >>> For more options, visit this group athttp:// >> groups.google.com/group/google-appengine?hl=en. >> > >> > -- >> > You received this message because you are subscribed to the Google >> Groups "Google App Engine" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> [email protected]. >> > For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> > > > -- > Nick Johnson, Developer Programs Engineer, App Engine > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
