Hello again,
the above fix breaks delete when items is already a list (such as when
deleting a single entity). The correct fix would therefore be:
def delete(self):
self._db['_lastsql'] = 'DELETE WHERE %s' % self.where
(items, tablename, fields) = self._select()
tableobj = self._db[tablename]._tableobj
counter = self.items_count(items)
if not isinstance(items,list):
leftitems = items.fetch(1000)
while len(leftitems):
gae.delete(leftitems)
leftitems = items.fetch(1000)
else:
gae.delete(items)
return counter
Christophe
On May 5, 1:02 am, Christophe Plat <[email protected]> wrote:
> Hi Massimo,
>
> the issue is that delete can't work directly on the
> google.appengine.ext.db.Query object. In the delete function, items is
> an instance of that Query object, and has to be transformed into a
> list of google.appengine.ext.db.Models - which is what the fetch
> function does.
> In fact, it doesn't have to be limited to 1000: the fetch function
> takes an argument that can have any value - the example said 1000,
> but any other value would be valid.
>
> After playing around a bit, I reconsidered the fix to be:
>
> def delete(self):
> self._db['_lastsql'] = 'DELETE WHERE %s' % self.where
> (items, tablename, fields) = self._select()
> tableobj = self._db[tablename]._tableobj
> counter = self.items_count(items)
> leftitems = items.fetch(1000)
> while len(leftitems):
> gae.delete(leftitems)
> leftitems = items.fetch(1000)
> return counter
>
> so that it removes items by 1000 increments. That should do the trick
> for now, but I see room for performance improvement:
>
> - allow the select function to retrieve keys only instead of the
> entire entity, as the documentation explains that it's a lot faster
> (and we don't need the full entity for a delete)
>
> - counter = self.items_count(items) could be computed via
> len(leftitems) and the 1000 increments
>
> Christophe
>
> On May 5, 12:10 am, mdipierro <[email protected]> wrote:
>
>
>
> > Thank you for reporting this. Please check my fix in trunk.
>
> > Massimo
>
> > On May 4, 4:16 pm, Christophe Plat <[email protected]> wrote:
>
> > > This issue on web2py is related to GAE issue 3119 (http://
> > > code.google.com/p/googleappengine/issues/detail?id=3119).
> > > I'm using web2py 1.77.3.
>
> > > In gql.py, line 702 to 709:
>
> > > def delete(self):
> > > self._db['_lastsql'] = 'DELETE WHERE %s' % self.where
> > > (items, tablename, fields) = self._select()
> > > tableobj = self._db[tablename]._tableobj
> > > counter = self.items_count(items)
> > > if counter:
> > > gae.delete(items)
> > > return counter
>
> > > According to comment 8 on issue 3119, the delete function no longer
> > > works on Query object, which items is an instance of. It suggests to
> > > change the code to either:
>
> > > db.delete(<query>.fetch(1000))
>
> > > or
>
> > > db.delete(list(<query>))
>
> > > with a preference to fetch(1000). In other words: a fix for this issue
> > > is to update line 708 to:
>
> > > gae.delete(items.fetch(1000))
>
> > > It leaves open the issue of queries returning more than 1000 results,
> > > though.
>
> > > Best regards,
> > > Christophe
