that's probably because your code queueing the task is inside models, that
are executed for every request.
Of course, models are executed also in the "fake" environment that executes
the task (so you have at hand, e.g., the database).
Move the "queue_task()" to a controller and queue it only one time, it'll
work fine.
On Tuesday, June 4, 2013 8:59:45 PM UTC+2, operE Aperte wrote:
>
> Hi,
>
> I use scheduler from few months with - basically - no big issues. In the
> last days I saw a strange behaviour: tasks queued multiple time when in my
> shceduler.py I have only one scheduler.queue_task instruction. At first I
> thougth of multiple scheduler instances thrown by cron jobs (I even don't
> know if this is possible), but I checked and I have only one scheduler
> instance running.
>
> So I tested this:
>
>
> db.py:
> db =
> DAL('postgres://user:[email protected]/oadb',pool_size=1,check_reserved=['all'],
>
> migrate=True)
>
> scheduler.py:
> db.define_table('testtable',Field('field1','integer'),migrate=True)
>
>
> def test_insert(tablename,value):
> db[tablename].insert(field1=value)
> db.commit()
> return True
>
>
> from gluon.scheduler import Scheduler
> scheduler = Scheduler(db)
>
>
> scheduler.queue_task('test_insert',
> pargs=["testtable",1],
> task_name='insert test',
> group_name='main',
> application_name='scrapers'
> )
>
> When I run from the shell prompt this:
> /usr/local/bin/python $HOME/webapps/oawa/web2py/web2py.py
> --password='<recycle>' --scheduler=scrapers -D 0
> I got this log:
>
> web2py Web Framework
> Created by Massimo Di Pierro, Copyright 2007-2013
> Version 2.4.7-stable+timestamp.2013.05.24.17.48.47
> Database drivers available: SQLite(sqlite3), MySQL(pymysql),
> MySQL(MySQLdb), PostgreSQL(psycopg2), PostgreSQL(pg8000), MSSQL(pyodbc),
> DB2(pyodbc), Teradata(pyodbc), Ingres(pyodbc), IMAP(imaplib)
> starting single-scheduler for "scrapers"...
> DEBUG:web2py.dal:Your database version does not support the JSON data type
> (using TEXT instead)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:defining tables
> (migrate=True)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:thread building own
> DAL object
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.dal:Your database version does not support the JSON data type
> (using TEXT instead)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:defining tables
> (migrate=False)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: freeing workers
> that have not sent heartbeat
> INFO:web2py.scheduler.web394.webfaction.com#385289:TICKER: I'm a ticker
> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
> (ACTIVE)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
> (ACTIVE)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
> (ACTIVE)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
> (ACTIVE)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> INFO:web2py.scheduler.web394.webfaction.com#385289:nothing to do
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
> (ACTIVE)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: freeing workers
> that have not sent heartbeat
> INFO:web2py.scheduler.web394.webfaction.com#385289:TICKER: I'm a ticker
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:Assigning tasks...
> INFO:web2py.scheduler.web394.webfaction.com#385289:TICKER: workers are 1
> INFO:web2py.scheduler.web394.webfaction.com#385289:TICKER: tasks are 50
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:Tasks assigned...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:sleeping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:........recordingheartbeat
> (ACTIVE)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165100
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new scheduler_run
> record
> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165100
> "insert test" scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
> scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
> COMPLETED
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
> failed
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
> report in db (COMPLETED)
> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
> (COMPLETED)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165110
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new scheduler_run
> record
> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165110
> "insert test" scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
> scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
> COMPLETED
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
> failed
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
> report in db (COMPLETED)
> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
> (COMPLETED)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165113
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new scheduler_run
> record
> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165113
> "insert test" scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
> scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
> COMPLETED
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
> failed
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
> report in db (COMPLETED)
> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
> (COMPLETED)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165109
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new scheduler_run
> record
> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165109
> "insert test" scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
> scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
> COMPLETED
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
> failed
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
> report in db (COMPLETED)
> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
> (COMPLETED)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165114
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new scheduler_run
> record
> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165114
> "insert test" scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
> scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
> COMPLETED
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
> failed
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
> report in db (COMPLETED)
> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
> (COMPLETED)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165112
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new scheduler_run
> record
> INFO:web2py.scheduler.web394.webfaction.com#385289:new task 165112
> "insert test" scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task allocated:
> scrapers.test_insert
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task starting
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task started
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new task report:
> COMPLETED
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: result: true
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: task completed or
> failed
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: recording task
> report in db (COMPLETED)
> INFO:web2py.scheduler.web394.webfaction.com#385289:task completed
> (COMPLETED)
> DEBUG:web2py.scheduler.web394.webfaction.com#385289:looping...
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: work to do 165107
> DEBUG:web2py.scheduler.web394.webfaction.com#385289: new scheduler_run
> record
> ...
> keeps logging this way thousands of time, filling scheduler_task table
> with more the 160.000 rows.
>
> This does not happen if I comment db.commit() line, in this case there is
> (of course) no data insertion in testtable, the task is completed but there
> is only 1 task in scheduler_task table.
>
> My setup:
> 2.4.7-stable+timestamp.2013.05.24.17.48.47
> (Running on Apache/2.2.17 (Unix) mod_wsgi/3.3 Python/2.7.5)
> PostgreSQL 9.1.9
>
> hosting: webfaction.com
>
> I really don't know what it's not working properly.
>
>
>
>
--
---
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/groups/opt_out.
