I am using SQLite. The TIMEOUT record in the scheduler_run table eventually
shows up. I can't be sure when since it normally happens over night. I
don't know how to raise a timeout but if you tell me I can try that. That
would help narrow down the issue because if the DB is locked I would not
see the scheduler_run record inserted and my scheduler_task record would
still be marked running. Is this correct?
>From the docs I was under the impression that retry_failed would only work
if the task actually raised an exception. I will certainly add that and
hope that it works.
Would changing the database out for postgres or MySQL possibly solve the
root issue? I would obviously rather the task did not timeout.
I will enable the debug logging. Thanks so much for your help and for your
work on the scheduler Niphlod.
On Thursday, December 13, 2012 1:31:38 AM UTC-8, Niphlod wrote:
>
> are you using sqlite ? It's possible that the timeout is reached because
> database is locked and the scheduler_run record (that you confirm is
> missing) can't be inserted. Also, just to be sure, did you try raising the
> timeout?
>
> If I cannot fix it I will have to find a different solution for directory
>> monitoring and not use the scheduler.
>>
>
> Can't you use a task with retry_failed ?
>
> Next step is enabling debug logging of the scheduler to see what goes on
> when the task timeouts
>
> On Thursday, December 13, 2012 7:17:29 AM UTC+1, Mike D wrote:
>>
>> Hello,
>>
>> I am using the web2py scheduler to run a task to monitor a directory.
>> Here's some code describing what I'm doing:
>>
>> def get_configured_logger():
>>
>> logger = logging.getLogger("ss_server")
>>
>> if (len(logger.handlers) == 0):
>>
>> handler = logging.handlers.RotatingFileHandler("/path/to/log/log.
>> txt", maxBytes=1024*1024*10, backupCount=2)
>>
>> handler.setLevel(logging.DEBUG)
>>
>> logger.addHandler(handler)
>>
>> logger.setLevel(logging.DEBUG)
>>
>> logger.debug("Logger created")
>>
>> return logger
>>
>>
>> def write_to_log(s):
>>
>> l = get_configured_logger()
>>
>> l.info(s)
>>
>> ...
>>
>>
>> def searchForFiles():
>>
>> print("serach for files")
>>
>> write_to_log("searching for files")
>>
>> print os.getcwd()
>>
>> write_to_log("creating watermark")
>>
>> watermark_opaque = Image.open('./path/to/image/watermark.png')
>>
>> watermark = reduce_opacity(watermark_opaque, 0.7)
>>
>> write_to_log("done creating watermark")
>>
>> write_to_log("globbing files")
>>
>> files = glob.glob(INPUT_DIR + "*.jpg")
>>
>> write_to_log("files globbed")
>>
>> for filename in files:
>>
>> write_to_log("getting basename for " + filename)
>>
>> filename = os.path.basename(filename)
>>
>> write_to_log("splitting filename")
>>
>> parts = filename.split('-')
>>
>> write_to_log("checking filename")
>>
>> if (len(parts) == 6):
>>
>> try:
>>
>> print("processing file: " + filename)
>>
>> write_to_log("processing file: " + filename)
>>
>> im = Image.open(INPUT_DIR + filename)
>>
>> write_to_log("adding watermark")
>>
>> im.paste(watermark,(im.size[0] - watermark.size[0] - 20
>> ,im.size[1] - watermark.size[1] - 20),watermark)
>>
>> im.save(INPUT_DIR + filename, "JPEG", quality=100)
>>
>> write_to_log("added watermark")
>>
>> write_to_log("creating scaled images")
>>
>> createScaledImage(64, INPUT_DIR + filename, THUMBS_DIR)
>>
>> createScaledImage(600, INPUT_DIR + filename, SMALL_DIR)
>>
>> write_to_log("done creating scaled images")
>>
>> pic_id = processFile(filename)
>>
>> print("processed file successfully")
>>
>> write_to_log("processed file successfully")
>>
>> write_to_log("renaming files")
>>
>> shutil.move(INPUT_DIR + filename, PROCESSED_DIR + "%d" %
>> pic_id + ".jpg")
>>
>> shutil.move(THUMBS_DIR + filename, THUMBS_DIR + "%d" %
>> pic_id + ".jpg")
>>
>> shutil.move(SMALL_DIR + filename, SMALL_DIR + "%d" %
>> pic_id + ".jpg")
>>
>> write_to_log("done renaming files")
>>
>> except IOError:
>>
>> #this is likely due to a partial file, so let it finish
>> writing and try
>>
>> #again next time
>>
>> print("IO Error")
>>
>> write_to_log("IO Error")
>>
>> pass
>>
>> except:
>>
>> etype, eval, etb = sys.exc_info()
>>
>> print("error processing file: ", eval.message)
>>
>> write_to_log("error: " + eval.message)
>>
>> shutil.move(INPUT_DIR + filename, ERRORS_DIR + filename)
>>
>> write_to_log("end of loop")
>>
>>
>>
>> write_to_log("ending function")
>>
>> return "success"
>>
>>
>>
>> myscheduler = Scheduler(db, tasks=dict(searchForFiles=searchForFiles))
>>
>>
>> I have a task inserted into my scheduler_task table with the following
>> properties:
>> function name: searchForFiles
>> repeats: 0
>> timeout: 240
>> sync output: 15
>>
>> Everything works fine except I get random TIMEOUT runs 1-2 times per day.
>> Furthermore these runs will happen in the middle of the night when nothing
>> has been added to the directory. When a TIMEOUT happens, the last line in
>> my log file is "ending function". Also, there is no output at all in the
>> scheduler_run record for the run that was marked as TIMEOUT.
>>
>> For these reasons I do not think the timeout is happening inside my code,
>> I believe it is somehow happening between when the scheduler tries to pick
>> up task and when it calls my function. I could totally be wrong though. The
>> reason I have added in such granular logging is to find exactly where the
>> timeout was occurring so I could fix it but I cannot find the problem
>> anywhere.
>>
>> Please let me know if anyone has any ideas on what could be causing this
>> issue. Any help would be very much appreciated. If I cannot fix it I will
>> have to find a different solution for directory monitoring and not use the
>> scheduler.
>>
>> Thanks again,
>> Mike
>>
>
--
