[web2py] Re: Web2py Scheduler, db query is not executing.

Have you tried adding a db.commit() to your delete_after_hours() function? -Jim On Tuesday, October 5, 2021 at 8:49:44 AM UTC-5 Andrew wrote: > Hello, I am trying to run a database query using the web2py scheduler, but > for some reason I can't get the query to commit. I've tested the

[web2py] Re: Web2py scheduler

Please install and use psycopg2. There are known issues with pg8000. On Tuesday, 6 February 2018 08:17:51 UTC-6, Simona Chovancová wrote: > > solved, ran worker in different application than the task was sent from > > On Tuesday, February 6, 2018 at 2:34:24 PM UTC+1, Simona Chovancová wrote: >>

[web2py] Re: Web2py scheduler

solved, ran worker in different application than the task was sent from On Tuesday, February 6, 2018 at 2:34:24 PM UTC+1, Simona Chovancová wrote: > > Hello, > > I'm having trouble using the scheduler. I have managed to set it up to > queue tasks and successfully complete them. Although, that

[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory

I have tested it and I can now confirm that the issue is with pg8000. I think its a good idea to mention this in the WIKI. On Friday, July 29, 2016 at 12:12:07 AM UTC+5:30, Niphlod wrote: > > could you please try with psycopg2 ? > > On Thursday, July 28, 2016 at 6:53:10 PM UTC+2, Abhishek Ram

[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory

The pg8000 driver has problems. Sorry. We should remove it from web2py. On Thursday, 28 July 2016 11:53:10 UTC-5, Abhishek Ram wrote: > > I am using the default driver pg8000 > > On Thursday, July 28, 2016 at 7:47:41 PM UTC+5:30, Niphlod wrote: >> >> what driver are you using ? it's rather

[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory

could you please try with psycopg2 ? On Thursday, July 28, 2016 at 6:53:10 PM UTC+2, Abhishek Ram wrote: > > I am using the default driver pg8000 > > On Thursday, July 28, 2016 at 7:47:41 PM UTC+5:30, Niphlod wrote: >> >> what driver are you using ? it's rather strange that postrgresql acts up

[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory

I am using the default driver pg8000 On Thursday, July 28, 2016 at 7:47:41 PM UTC+5:30, Niphlod wrote: > > what driver are you using ? it's rather strange that postrgresql acts up > and eat all the available memory, if you didn't tinker with the defaults > settings of postgresql... > > On

[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory

what driver are you using ? it's rather strange that postrgresql acts up and eat all the available memory, if you didn't tinker with the defaults settings of postgresql... On Wednesday, July 27, 2016 at 5:09:34 AM UTC+2, Abhishek Ram wrote: > > Well the memory leak was from the postgres

[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory

Well the memory leak was from the postgres processes spawned by the web2py scheduler process. I am sure about this as I have seen TOP and confirmed that it was these processes that were consuming the memory. I even tried with a basic app and the postgres processes continued to consume memory.

[web2py] Re: Web2py scheduler linked postgres processes consuming too much memory

you're seeing a totally normal thing: postgresql spawns a different process for each connection. Those processes in linux are fork()s, so even if you SEEM to notice a skyrocket in memory utilization summing all different processes (e.g. in "top"), in reality it's not sucking up every bit of

[web2py] Re: web2py scheduler service dead - how to prevent it?

On Monday, May 30, 2016 at 3:15:18 AM UTC-7, Mirek Zvolský wrote: > > Please help > > Debian Jessie, postgres, web2py-scheduler > > How can I prevent following crash - inactive scheduler? > Is the real problem that you're using pg8000? I think there's been a lot of discussion of that

[web2py] Re: web2py scheduler service dead - how to prevent it?

I have found this https://www.digitalocean.com/community/tutorials/how-to-configure-a-linux-service-to-start-automatically-after-a-crash-or-reboot-part-1-practical-examples Service section should contain following: [Service] Restart=always It looks like it works, so I hope it will be well.

[web2py] Re: web2py scheduler with huge output or return values always timeout (even if the task finishes)

results get stored to a tempfile. it has been recently fixed. Huge "prints" are still a problem and are discouraged, but now you can have a result as big as you wish. On Thursday, May 12, 2016 at 5:19:04 AM UTC+2, Andre Kozaczka wrote: > > I'm curious what workarounds folks have come up with

[web2py] Re: web2py scheduler with huge output or return values always timeout (even if the task finishes)

I'm curious what workarounds folks have come up with regarding this issue. On Monday, February 29, 2016 at 1:41:41 PM UTC-5, Boris Aramis Aguilar Rodríguez wrote: > > Hi, there is an issue driving me crazy with the web2py scheduler: > > If you return something that has a huge size then it will

[web2py] Re: web2py-scheduler

On Monday, March 21, 2016 at 3:58:25 PM UTC-7, Dave S wrote: > > > > On Monday, March 21, 2016 at 3:31:06 PM UTC-7, phoeb...@gmail.com wrote: >> >> hi >> i want 10 min after click on submit button , my program do a function and >> in that 10min my program do another functions , how can i

[web2py] Re: web2py-scheduler

On Monday, March 21, 2016 at 3:31:06 PM UTC-7, phoeb...@gmail.com wrote: > > hi > i want 10 min after click on submit button , my program do a function and > in that 10min my program do another functions , how can i implement it ? > To queue task to run ten minutes after the submit, your

Re: [web2py] Re: web2py scheduler with huge output or return values always timeout (even if the task finishes)

aw hell, it's true. huge outputs are effectively blocking. don't know if it's fixable though. I don't see any obvious error in the implementation that could prevent it On Monday, February 29, 2016 at 9:26:38 PM UTC+1, Niphlod wrote: > > uhm, not sure about that though. the order of

Re: [web2py] Re: web2py scheduler with huge output or return values always timeout (even if the task finishes)

Please do try :) I might be wrong :P and it could be something else; but I tried to narrow it down as much as I could. El feb 29, 2016 2:26 PM, "Niphlod" escribió: > uhm, not sure about that though. the order of operations in the scheduler > respects the docs, and even if the

Re: [web2py] Re: web2py scheduler with huge output or return values always timeout (even if the task finishes)

uhm, not sure about that though. the order of operations in the scheduler respects the docs, and even if the bugreport has the 128k statement, the docs don't report it let me try. On Monday, February 29, 2016 at 9:14:41 PM UTC+1, Boris Aramis Aguilar Rodríguez wrote: > > It happends

Re: [web2py] Re: web2py scheduler with huge output or return values always timeout (even if the task finishes)

It happends not only while printing but also when returning values on a function as noted on the example :) so yes i do avoid printing but this also happens when returning more than 128kb of data (as noted by the bug reports) due to the fact (it seems) that the max data a process can comunicate to

[web2py] Re: web2py scheduler with huge output or return values always timeout (even if the task finishes)

don't know if it's documented on the book but nevertheless there has been a few times it popped up on this group. As output is buffered, especially on Windows but on unixes flavoured OSes as well, the "printed" output should be limited. Of course it never has been a real issue because if you

Re: [web2py] Re: web2py Scheduler with Postgres creates idle in transaction connections

in my environment, I can't see any idle in transaction that's why I patched the scheduler and asked to do that test. -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) - https://code.google.com/p/web2py/issues/list

[web2py] Re: web2py Scheduler with Postgres creates idle in transaction connections

I actually get an error using your trick: Traceback (most recent call last): File gluon/scheduler.py, line 1512, in module main() File gluon/scheduler.py, line 1506, in main utc_time=options.utc_time) File gluon/scheduler.py, line 588, in __init__ self.define_tables(db,

[web2py] Re: web2py Scheduler with Postgres creates idle in transaction connections

uhm, the problem is more subtle. The main process of the scheduler is like a shell opened on that application... it needs to reads models to see the scheduler definition. I guess that this means that it will also connect to all the databases defined in models, even if it will

[web2py] Re: web2py Scheduler with Postgres creates idle in transaction connections

uhm^3. The code is quite unfixable as it is (launching scheduler with web2py.py -K appname). Remember that the only thing I'm trying to fix is the main process using idle connections to all databases defined in the models of you app. However, there's a small unknown trick: the scheduler can be

Re: [web2py] Re: web2py Scheduler with Postgres creates idle in transaction connections

idle connection are one thing and they can be fine. idle in transaction, usually happens when there is a missing commit and the transaction stays open, delivering all sorts of locking over the long term usually a commit or rollback before the fork should solve the problem. 2015-02-18 23:09

[web2py] Re: web2py scheduler error when I start the worker...

did you recently upgrade to 2.9.11 without letting scheduler recreating the tables ? tables definitions have changed, and you need to let web2py adjust (i.e. look for migrate_enabled=False in your DAL call) On Tuesday, October 21, 2014 10:03:39 PM UTC+2, José Leite wrote: Hello, I just

[web2py] Re: web2py scheduler period and repeats result not expected

On Thursday, June 26, 2014 11:17:21 PM UTC+2, 黄祥 wrote: yes, followed your step, i understood the logic flow of the web2py scheduler and made it work, thank you so much, simone for the pointers. just wondering, the precision for second is not exact on created_on, tested using period = 5

[web2py] Re: web2py scheduler period and repeats result not expected

a, i understood, the puzzle completed now. thank you so much, simone, for detail explaination about web2py scheduler. thanks and best regards, stifan -- Resources: - http://web2py.com - http://web2py.com/book (Documentation) - http://github.com/web2py/web2py (Source code) -

[web2py] Re: web2py scheduler period and repeats result not expected

@guys..if you queue a task inside models, you're queuing a new task at every request ! ;-D On Thursday, June 26, 2014 6:17:54 AM UTC+2, 黄祥 wrote: i've already tried, but got the same result + another strange output during start web2py. here is the detail step i took. 1. create new

[web2py] Re: web2py scheduler period and repeats result not expected

a, i c, so that's why scheduler_task table is have a lot of record. thank you so much simone. btw, what i want to learn in here is to have a scheduler to create a new record in database every day, how can i achieve it using testing example above? what i have tried is : *models/db.py*

[web2py] Re: web2py scheduler period and repeats result not expected

you're missing a BG point. Either you need to queue a task as a result of some user action (i.e. send a reminder, calculate/aggregate/refresh some data, etc) -- usually it's a one-time-only activity, in which case, the following pattern applies #models\scheduler.py from gluon.scheduler

[web2py] Re: web2py scheduler period and repeats result not expected

yeah, i realize that something i'm lacked off, unfortunately, i don't know where is it. tried again using your code but still no luck started from scratch *models\db_wizard_0.py* db.define_table('asdf', Field('asdf'), auth.signature ) *models\scheduler.py* from gluon.scheduler import

[web2py] Re: web2py scheduler period and repeats result not expected

ok. steps. web2py.py -a yourpassword go to /appname/test/queue_task go to /appname/appadmin/select/db?query=db.scheduler_task.id0 if nothing is there, then in controller test.py the function queue_task is not queuing the task. if a single row is there, then, open a terminal and start the

[web2py] Re: web2py scheduler period and repeats result not expected

yes, followed your step, i understood the logic flow of the web2py scheduler and made it work, thank you so much, simone for the pointers. just wondering, the precision for second is not exact on created_on, tested using period = 5 and period = 60 here is the steps i took : *cmd* cd

[web2py] Re: web2py scheduler period and repeats result not expected

Except that I pass the function, rather than a string, my usage is the similiar to yours: scheduler.queue_task(demo1, repeats = 0, period = 180,prevent_drift=True) On Thursday, June 26, 2014 11:38:13 AM UTC+10, 黄祥 wrote: hi, i tested and learned from web2py

[web2py] Re: web2py scheduler period and repeats result not expected

i've already tried, but got the same result + another strange output during start web2py. here is the detail step i took. 1. create new web2py app (copas the welcome folder and rename it into test) 2. copas the scheduler.py into models (test\models\scheduler.py) *test\models\scheduler.py*

[web2py] Re: web2py scheduler

You could try adding standard python logging statements to try to figure out what's happening. Edit web2py-root/logging.conf and add an entry for your app following the instructions in the file. Then in scheduler.py: import logging def your_scheduled_function(): logger =

[web2py] Re: web2py scheduler

see http://web2py.com/books/default/chapter/29/04/the-core#markmin_queue_task_sig On Sunday, February 2, 2014 1:05:30 PM UTC+1, Jayadevan M wrote: When I queue a taks using - scheduler.queue_task('demo1', pvars=dict(a=1,b=2), repeats = 0, period = 180) how do I know

[web2py] Re: web2py scheduler

Looks like I had to execute db.commit(); before the task showed up in table. That was the missing part. On Sunday, February 2, 2014 6:13:30 PM UTC+5:30, Niphlod wrote: see http://web2py.com/books/default/chapter/29/04/the-core#markmin_queue_task_sig On Sunday, February 2, 2014 1:05:30

[web2py] Re: web2py scheduler

OK. I shouldn't have 'jobs/send_email_invites' , just 'send_email_invites' On Sunday, February 2, 2014 8:57:18 PM UTC+5:30, Jayadevan M wrote: I have a file job.py under models. In that I have def send_email_invites(): I added a task that says job/send_email_invites. The task errors out

Re: [web2py] Re: web2py scheduler not working

Why can't you add that line in cron? I don't get it. Scheduler WILL be running if you do it, you just need a full path to web2py.py. Did you even try? What wasn't working? Anyway there is another way to start scheduler in background:

Re: [web2py] Re: web2py scheduler not working

@ranjith, I think that you may be confusing the difference between Unix/Linux cron https://en.wikipedia.org/wiki/Cron and the cronhttp://web2py.com/books/default/chapter/29/04/the-core?search=cron#Cron provided by web2py which are two different things. What @Marin is meaning is that you'd

Re: [web2py] Re: web2py scheduler not working

Is that not possible? On Wednesday, 23 October 2013 11:13:53 UTC+5:30, ranjith wrote: Thanks for the response. Other than the below script, it was mentioned in the document that scheduler workers can be started via cron@reboot The scheduler does not use cron, although one can use cron

[web2py] Re: web2py scheduler not working

I tried changing the worker table status to active/pick and others. It didnt work On Tuesday, 22 October 2013 23:53:24 UTC+5:30, ranjith wrote: Hi, I have a model file called scheduler.py which has from gluon.scheduler import Scheduler def messageTask():

Re: [web2py] Re: web2py scheduler not working

You are not supposed to add entries to scheduler_worker. You need to run scheduler in the background. python web2py.py -K myapp Detailed explanation here: http://web2py.com/books/default/chapter/29/04/the-core#Scheduler Marin On Tue, Oct 22, 2013 at 8:25 PM, ranjith ranjith2...@gmail.com

Re: [web2py] Re: web2py scheduler not working

Thanks for the response. Other than the below script, it was mentioned in the document that scheduler workers can be started via cron@reboot The scheduler does not use cron, although one can use cron @reboot to start the worker nodes. I am just starting to work on web2py. So, sorry for

[web2py] Re: Web2py scheduler timeout

I'm not sure what narrow is by the period is 30 seconds. I'm seeing weirder behavior now. The task completes successfully but it quits being executed at a random time. The only way for me to fix it is to restart the scheduler process. I enabled the debug logging and here are the last two runs

[web2py] Re: Web2py scheduler timeout

you could attach the log to the message, so the thread doesn't get long:P I see 2 rounds only COMPLETED, then a huge gap 2012-12-16 21:39:30,330 - web2py.scheduler - INFO - task completed (COMPLETED) 2012-12-16 23:12:23,296 - web2py.scheduler - DEBUG - new scheduler_run record

[web2py] Re: Web2py scheduler timeout

Sorry about that. I just tried to attach the log and it keeps giving me an error. The scheduler_task record is marked QUEUED and the last scheduler_run record is marked COMPLETED. The task just isn't being run for some reason. On Monday, December 17, 2012 12:29:34 PM UTC-8, Niphlod wrote:

[web2py] Re: Web2py scheduler timeout

I have now tried with two different browsers to attach the log with no success. I keep getting error (340) occurred while communicating with the server. Do you want me to just email you the log? On Monday, December 17, 2012 12:45:45 PM UTC-8, Mike D wrote: Sorry about that. I just tried to

[web2py] Re: Web2py scheduler timeout

no probl, the email is kinda obvious, at gmail.com On Monday, December 17, 2012 9:54:38 PM UTC+1, Mike D wrote: I have now tried with two different browsers to attach the log with no success. I keep getting error (340) occurred while communicating with the server. Do you want me to just

[web2py] Re: Web2py scheduler timeout

are you using sqlite ? It's possible that the timeout is reached because database is locked and the scheduler_run record (that you confirm is missing) can't be inserted. Also, just to be sure, did you try raising the timeout? If I cannot fix it I will have to find a different solution for

[web2py] Re: Web2py scheduler timeout

I am using SQLite. The TIMEOUT record in the scheduler_run table eventually shows up. I can't be sure when since it normally happens over night. I don't know how to raise a timeout but if you tell me I can try that. That would help narrow down the issue because if the DB is locked I would not

[web2py] Re: Web2py scheduler timeout

All my statements were made under the assumption that the scheduler_run table showed absolutely no trace of the TIMOUTted task. In any case, running the scheduler on SQLite is safe only for 1 or 2 workers and not with a zillion of tasks. Concurrency was never a friend of SQLite. BTW, at this

[web2py] Re: Web2py scheduler timeout

I have only this one task. I am certainly going to change the retry_failed and hopefully that will be a sufficient solution. I actually tried your other solution a while ago and that monitoring task actually ended up in a TIMEOUT state as well. Sad face. Any idea on that one? Do you think that

[web2py] Re: Web2py scheduler timeout

unless your tasks have a very narrow period they shouldn't go on timeout on SQLite (remember to commit if acting on the tables within a task). W2P_TVSeries uses 500-600 tasks with a watcher task coordinating those and it doesn't block. Upgrading to Mysql or PostgreSQL will fix the issue if the

[web2py] Re: Web2py Scheduler

web2py has a built-in scheduler: http://web2py.com/books/default/chapter/29/4?search=scheduler#Scheduler-(experimental) On Feb 27, 3:43 am, VIREN PATEL viren...@gmail.com wrote: Hi, I need to implement the Scheduler and Job Queues at the application level. Please suggest some good Schedulers.