I'll move queue_task() in controller, but the fact is that (till now) I
always used queue_task() in model with no issues, and actually there is no
request other then the ones I trigger visiting my site (site is demo, no
"audience")...
On Tuesday, June 4, 2013 9:13:54 PM UTC+2, Niphlod wrote:
>
> that's probably because your code queueing the task is inside models, that
> are executed for every request.
> Of course, models are executed also in the "fake" environment that
> executes the task (so you have at hand, e.g., the database).
> Move the "queue_task()" to a controller and queue it only one time, it'll
> work fine.
>
> On Tuesday, June 4, 2013 8:59:45 PM UTC+2, operE Aperte wrote:
>>
>> Hi,
>>
>> I use scheduler from few months with - basically - no big issues. In the
>> last days I saw a strange behaviour: tasks queued multiple time when in my
>> shceduler.py I have only one scheduler.queue_task instruction. At first I
>> thougth of multiple scheduler instances thrown by cron jobs (I even don't
>> know if this is possible), but I checked and I have only one scheduler
>> instance running.
>>
>> So I tested this:
>>
>>
>> db.py:
>> db =
>> DAL('postgres://user:[email protected]/oadb',pool_size=1,check_reserved=['all'],
>>
>> migrate=True)
>>
>> scheduler.py:
>> db.define_table('testtable',Field('field1','integer'),migrate=True)
>>
>>
>> def test_insert(tablename,value):
>> db[tablename].insert(field1=value)
>> db.commit()
>> return True
>>
>>
>> from gluon.scheduler import Scheduler
>> scheduler = Scheduler(db)
>>
>>
>> scheduler.queue_task('test_insert',
>> pargs=["testtable",1],
>> task_name='insert test',
>> group_name='main',
>> application_name='scrapers'
>> )
>>
>> When I run from the shell prompt this:
>> /usr/local/bin/python $HOME/webapps/oawa/web2py/web2py.py
>> --password='<recycle>' --scheduler=scrapers -D 0
>> I got this log:
>>
>> web2py Web Framework
>> Created by Massimo Di Pierro, Copyright 2007-2013
>> Version 2.4.7-stable+timestamp.2013.05.24.17.48.47
>> Database drivers available: SQLite(sqlite3), MySQL(pymysql),
>> MySQL(MySQLdb), PostgreSQL(psycopg2), PostgreSQL(pg8000), MSSQL(pyodbc),
>> DB2(pyodbc), Teradata(pyodbc), Ingres(pyodbc), IMAP(imaplib)
>> starting single-scheduler for "scrapers"...
>> DEBUG:web2py.dal:Your database version does not support the JSON data
>> type (using TEXT instead)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:defining tables
>> (migrate=True)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:thread building own
>> DAL object
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.dal:Your database version does not support the JSON data
>> type (using TEXT instead)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:defining tables
>> (migrate=False)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: freeing workers
>> that have not sent heartbeat
>> INFO:web2py.scheduler.web394.webfaction.com#385289:TICKER: I'm a ticker
>> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
>> (ACTIVE)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
>> (ACTIVE)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
>> (ACTIVE)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
>> (ACTIVE)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
>> (ACTIVE)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: freeing workers
>> that have not sent heartbeat
>> INFO:web2py.scheduler.web394.webfaction.com#385289:TICKER: I'm a ticker
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:Assigning tasks...
>> INFO:web2py.scheduler.web394.webfaction.com#385289:TICKER: workers are 1
>> INFO:web2py.scheduler.web394.webfaction.com#385289:TICKER: tasks are 50
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:Tasks assigned...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
>> (ACTIVE)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165100
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new
>> scheduler_run record
>> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165100
>> "insert test" scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
>> scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
>> COMPLETED
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
>> failed
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
>> report in db (COMPLETED)
>> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
>> (COMPLETED)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165110
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new
>> scheduler_run record
>> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165110
>> "insert test" scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
>> scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
>> COMPLETED
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
>> failed
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
>> report in db (COMPLETED)
>> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
>> (COMPLETED)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165113
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new
>> scheduler_run record
>> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165113
>> "insert test" scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
>> scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
>> COMPLETED
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
>> failed
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
>> report in db (COMPLETED)
>> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
>> (COMPLETED)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165109
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new
>> scheduler_run record
>> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165109
>> "insert test" scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
>> scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
>> COMPLETED
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
>> failed
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
>> report in db (COMPLETED)
>> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
>> (COMPLETED)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165114
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new
>> scheduler_run record
>> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165114
>> "insert test" scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
>> scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
>> COMPLETED
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
>> failed
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
>> report in db (COMPLETED)
>> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
>> (COMPLETED)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165112
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new
>> scheduler_run record
>> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165112
>> "insert test" scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
>> scrapers.test_insert
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
>> COMPLETED
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
>> failed
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
>> report in db (COMPLETED)
>> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
>> (COMPLETED)
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165107
>> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new
>> scheduler_run record
>> ...
>> keeps logging this way thousands of time, filling scheduler_task table
>> with more the 160.000 rows.
>>
>> This does not happen if I comment db.commit() line, in this case there is
>> (of course) no data insertion in testtable, the task is completed but there
>> is only 1 task in scheduler_task table.
>>
>> My setup:
>> 2.4.7-stable+timestamp.2013.05.24.17.48.47
>> (Running on Apache/2.2.17 (Unix) mod_wsgi/3.3 Python/2.7.5)
>> PostgreSQL 9.1.9
>>
>> hosting: webfaction.com
>>
>> I really don't know what it's not working properly.
>>
>>
>>
>>
--
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
