Also, GAE has a brand new, under documented, feature called deferreds, which allows you to create deferred functions which are automatically pickled and then executed later on. It uses the task queue, but presents a simpler interface for simple tasks.
http://code.google.com/appengine/articles/deferred.html That link gives an example of how to use a Mapper and a deferred to perform batch db operations. Robin On Oct 15, 4:17 pm, Chris S <[email protected]> wrote: > Thanks for the tip I'll have to look at Task Queues I don't know > anything about them. > > For anyone else looking to do what I was (cron a 'kind' deletion). > Mine did complete. I was able to simply check my Cron tab and when > the process didn't fail that meant all items were deleted and I could > stop the cron. Got the job done, though I'm sure there are better > ways like the Task Queue above at least this way you can do it slowly > and not eat up to much CPU time. In the future I hope to upload new > data first, and then slowly delete old data to avoid having to buy > extra CPU time. > > On Oct 15, 3:30 pm, Robin B <[email protected]> wrote: > > > You could look at the task queue to build a more dynamic version of > > what you are doing with cron polling. > > > Also tasks can execute in parallel, so instead of 30,000/200 units of > > time to delete, you can do (30,000/200)/N where N is the level of > > concurrency you set for your task queue. > > > Robin > > > On Oct 14, 10:09 am, Chris S <[email protected]> wrote: > > > > I'm fine with having to Cron the job. I would just like to manually > > > limit the number deleted in a single cron. That way I won't get an > > > error message on every cron for the next two days while I'm deleting > > > 30,000 enteries 200 at a time. Just doesn't sit right with me > > > executing a command and letting it run until it just errors out. > > > > On Oct 14, 10:00 am, mdipierro <[email protected]> wrote: > > > > > There is never going to e better solution they limit how many records > > > > you can access per request. You have to hit the page many times until > > > > all records are deleted, or do it from outside the web app but I do > > > > not know how. > > > > > Massimo > > > > > On Oct 14, 9:53 am, Chris S <[email protected]> wrote: > > > > > > That was 'better' but same problem. With the change you've given it > > > > > still errors out but 199 deletions were performed (previously nothing > > > > > was deleted). It's not good but at least it's deleting something > > > > > until there is a better solution. > > > > > > On Oct 14, 9:39 am, mdipierro <[email protected]> wrote: > > > > > > > Not sure. Try this: > > > > > > > myset=db(db.item2.id>0) > > > > > > while myset.delete(): pass > > > > > > > Massimo > > > > > > > On Oct 14, 9:29 am, Chris S <[email protected]> wrote: > > > > > > > > Setting up Cron jobs on GAE is simple enough. But I'm not > > > > > > > understanding something with creating a cron that can delete table > > > > > > > entries. > > > > > > > > I was attempting to make the cron remove all items in a selection > > > > > > > with: > > > > > > > db.delete(sqldb.item2) > > > > > > > return 'Delete complete' > > > > > > > > The problem is it fetches the 1000 limit and deleting that many > > > > > > > items > > > > > > > exceeds the 30-second process limit on GAE. > > > > > > > > So I was trying to reduce the number of items deleted in each cron > > > > > > > with: > > > > > > > rows=db.(db.item2.id>0).select(limitby=(0,200)) > > > > > > > > That returns a row object which doesn't seem to take the delete > > > > > > > command. Only a 'set' can be called with delete. > > > > > > > myset=db(db.item2.id>0) > > > > > > > myset.delete() > > > > > > > > This works just fine as far as deleting goes, but the set can't be > > > > > > > limited like the selection can. > > > > > > > > I'm sure I'm just missing something, but how do I perform a delete > > > > > > > command on something other than the max allowed selection of 1000? --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "web2py-users" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/web2py?hl=en -~----------~----~----~----~------~----~------~--~---
