On-disk persistence for jobs and logs is generally considered a very good thing. For logs, it's the reason you want them. I think it would be a misdesign if invoking cancel erased all your logs, and your jobs (which is there so you can restart a crawl after stopping it).
Why don't you just write a cronjob that cleans this up every night? If you want it deleted immediately, just write a tiny API where a call to your API generally wraps the scrapyd web interface? In the case of cancel. your script could delete the required fields after getting cancellation confirmation from the scrapyd API. A scrapyd web API middleware, if you will. On Wed, Oct 15, 2014 at 7:31 AM, Hugo Maugey <hugo.mau...@gmail.com> wrote: > when I call : >> > > curl http://localhost:6800/cancel.json -d project=backrub -d > job=169a003c546f11e49251002590d225a2 > > here is the answer : > > {"status": "ok", "prevstate": null} > > but the job (and its items and logs) are still there ... > > -- > You received this message because you are subscribed to the Google Groups > "scrapy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to scrapy-users+unsubscr...@googlegroups.com. > To post to this group, send email to scrapy-users@googlegroups.com. > Visit this group at http://groups.google.com/group/scrapy-users. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "scrapy-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to scrapy-users+unsubscr...@googlegroups.com. To post to this group, send email to scrapy-users@googlegroups.com. Visit this group at http://groups.google.com/group/scrapy-users. For more options, visit https://groups.google.com/d/optout.
