I'm guessing that due to the nature of tablet splitting, there is a lot of fragmentation, and some low-level background process comes along once in a while and reclaims space.
I've often wanted a feature that would let me "mark entities as deleted" so I could be part of this process (which may be just a figment of my imagination) and have my entities go away on Google's schedule and for very cheap/free. Of course, I would need to be responsible for ignoring these things in query results, etc., which often is not a problem because they are orphan entities that otherwise wouldn't be queried for anyway. Something like: keys = MyEntity.query(ancestor=my_parent).fetch(1000, keys_only=True) ndb.delete_multi(keys, *low_priority=True*) # would mark both the entities and the index entries for background deletion My alternative, often, is to just let dead stuff keep turning on spindles, which is asinine. j On Monday, 3 December 2012 10:25:38 UTC-6, alex wrote: > > > Sometimes I wonder if Google should simply double the price of a write > > operation and make delete operations "free" > > what I wonder is when App Engine will get a Spanner implementation :) > > > but they *are* that expensive since all data will eventually get > deleted. > > not unless you disable => delete the app. Am I missing something here? > > > On Mon, Dec 3, 2012 at 5:08 PM, Jeff Schnitzer > <[email protected]<javascript:>> > wrote: > > Truth is, there is no way to "efficiently" truncate a table because of > the > > nature of BigTable - your data isn't stored in separate tables that can > be > > dropped individually. Every row is stored in one big table, and those > rows > > need to be deleted individually. > > > > Sometimes I wonder if Google should simply double the price of a write > > operation and make delete operations "free". Of course, it makes write > > operations look really expensive... but they *are* that expensive since > all > > data will eventually get deleted. > > > > Jeff > > > > > > On Sun, Dec 2, 2012 at 7:32 PM, Carl Schroeder > > <[email protected]<javascript:>> > > > wrote: > >> > >> I am not sure if it is the most efficient, but what I do is: > >> Remove all indexes associated with the model. This minimizes writes > >> associated with deletes. > >> Then create a cron task that deletes n entities per day according to > how > >> much quota I feel like using. > >> > >> GAE really needs a "remove all entity and indexes for entity" function. > >> > >> > >> On Sunday, December 2, 2012 3:23:07 PM UTC-8, observer247 wrote: > >>> > >>> I have a model (table) which has accumulated a few GBs of data. I do > not > >>> need any of the data and so want to truncate the table. > >>> > >>> Yesterday I tried to DELETE ENTITIES using Datastore Admin but it hit > my > >>> billing limit immediately. > >>> > >>> What is the cheapest way to truncate a table in Google App Engine ? > >>> > >>> PS: I am using python > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To view this discussion on the web visit > >> https://groups.google.com/d/msg/google-appengine/-/YBOQusnZSRcJ. > >> > >> To post to this group, send email to > >> [email protected]<javascript:>. > > >> To unsubscribe from this group, send email to > >> [email protected] <javascript:>. > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > > > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "Google App Engine" group. > > To post to this group, send email to > > [email protected]<javascript:>. > > > To unsubscribe from this group, send email to > > [email protected] <javascript:>. > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/a9noS2lFQoQJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
