Thanks for the info Ikai. I starred issue 2740, hopefully we will be able to view index statistics sometime in the future. Being able to delete the datastore without having to delete each entity would be nice too, it can become expensive very quickly to make mistakes.
On Nov 9, 7:44 pm, "Ikai Lan (Google)" <[email protected]> wrote: > No, it won't auto clear indexes. You'll need to use something like the > Mapper API to read all the entities, modifies them and re-saves them. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > On Tue, Nov 9, 2010 at 2:44 PM, Erik <[email protected]> wrote: > > > Thanks for the information and tips Robert, somehow I missed that > > AppEngine automatically indexes properties. > > Simply adding the unindexed extension to the JDO class should > > eventually clear the auto-generated indexes? > > > Cheers & thanks again, > > -Erik > > > On Nov 9, 10:52 am, Robert Kluin <[email protected]> wrote: > > > Hi Erik, > > > As far as I know the persistence manager would not create any extra > > entities. > > > > It sounds like you might want to explicitly disable indexing on any > > > fields you will not be querying on. See the 'Properties that Aren't > > > Indexed' section on the 'Queries and Indexes' page. > >http://code.google.com/appengine/docs/java/datastore/queriesandindexe... > > > > You might also want to use shorter kind and property names. Kind > > > and property names are stored with every entity, so it can add up > > > pretty fast. > > > > Robert > > > > On Tue, Nov 9, 2010 at 08:05, Erik <[email protected]> wrote: > > > > Hi Robert, > > > > > Thanks for the response, I did wait several days before sending that > > > > message, but shortly afterwards the quota cleared to zero. I am using > > > > a persistence manager with jdo, would these create temporary > > > > entities? > > > > > I need to explicitly index my keys in descending order for mapreduce, > > > > and thought I needed to index my collections but as I am not > > > > performing any queries don't think that is necessary. My persistent > > > > objects consist of a single property with type of long collection, > > > > entities are fetched by id. > > > > > My latest 30MB dataset has already grown to consume 710MB, that is an > > > > enormous increase. Datastore statistics say only 131MB is being used > > > > with 71% of that as metadata. > > > > > Any other ideas? > > > > > Thanks, > > > > -Erik > > > > > On Nov 8, 1:03 pm, Robert Kluin <[email protected]> wrote: > > > >> Hi Erik, > > > >> Several common sources of datastore stats / quota number funkiness: > > > >> 1) The numbers are not updated in real time. Sometimes it can > > > >> take a day for the numbers to get updated. > > > >> 2) Because of 1, if you are using a session library (or something > > > >> similar) that create lots of temporary entities the numbers can > > > >> 'appear' to be out-of-sync. > > > > >> I assume when you say switched to explicit indexes, you mean you > > > >> explicitly disabled indexing of any properties that do not need > > > >> indexes? > > > > >> Also, you might want to star issue 2740. > > > >> http://code.google.com/p/googleappengine/issues/detail?id=2740 > > > > >> Robert > > > > >> On Sat, Nov 6, 2010 at 19:29, Erik <[email protected]> wrote: > > > > >> > Hello all, > > > > >> > I have been uploading a dataset which is composed of 100MB of CSV > > > >> > values. During the process of uploading with bulkloader, which > > never > > > >> > completed, the datastore expanded to consume 2GB of usage. I > > decided > > > >> > to explicitly index my data and clear the datastore before a > > re-import > > > >> > using the blobstore/mapreduce method. However, after clearing the > > > >> > datastore there is still a persistent .37GB of data usage remaining, > > > >> > while datastore statistics say 53KB is being consumed by 174 > > objects, > > > >> > and nothing shows in datastore viewer. > > > > >> > Any ideas on how I can recover the .37GB of quota which is > > > >> > mysteriously being used in my empty datastore? > > > > >> > Many thanks, > > > >> > -Erik > > > > >> > -- > > > >> > You received this message because you are subscribed to the Google > > Groups "Google App Engine" group. > > > >> > To post to this group, send email to > > [email protected]. > > > >> > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > > . > > > >> > For more options, visit this group athttp:// > > groups.google.com/group/google-appengine?hl=en. > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups "Google App Engine" group. > > > > To post to this group, send email to [email protected] > > . > > > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > > . > > > > For more options, visit this group athttp:// > > groups.google.com/group/google-appengine?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
