Re: How I run background processes

[Rajesh D]

On Jul 21, 9:10 am, Philippe Josse  wrote:
> Hi there,
>
> I would like to have your feedback on the way I am running daemon scripts
> for my Django/ Python app. I know this is definitely not the "one best way",
> but this is a solution I came up with that seemed simple to implement. (I am
> a real newbie!)
>
> My constraints:
>  - getting a few processs constantly repeating themselves
>  - getting an email notification if one of the process crashes
>
> I set up a cron job that is launching a "daemon manager" script every
> minute. This script is querying a PostGreSQL database to retrieve boolean
> values associated to the state of my background processes (running / not
> running).  If the database indicates the process is not running, then the
> cron script launches it.
>
> Boolean values are updated by each process when it starts and finishes.
> Currently, it seems to work fine - but my background processes are very
> quick to execute. When my user base will grow, they may last longer and it
> will be essential that background processes do not run concurrently.
>
> Would that solution work? I was worried by potential caching issues with the
> database that may prevent my cron script to retrieve the latest boolean
> value. Is there any real risk?

What happens if your process starts but crashes before it can tell the
DB that it has crashed/finished? Your monitor job will think the
process is still active.

Here's another way that doesn't involve the DB:

When your process starts, have it bind to a known TCP port using the
Python socket[1] library. If the bind call fails, it means an instance
of your process is still running so just exit out of the current
instance. If the bind call succeeds, assume that no other instance is
active so continue running the rest of the process. Even if your
process crashes without explicitly releasing the socket, the OS will
unbind it for you.

[1] http://docs.python.org/library/socket.html

-RD
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: How I run background processes

[cheeming]

I think the other situation is that the old cron job is not done yet
and the new cron job gets started up.
Having a flag in the DB would be able to fix that as well.


On Jul 21, 9:33 pm, Andrew Fong  wrote:
> >> I was worried by potential caching issues with the database that
> >> may prevent my cron script to retrieve the latest boolean value.
>
> It sounds as if you're worried about a race condition when scale up --
> e.g. let's say you need to run a lot of background processes and
> decide to split them up among several machines, each with their own
> cron jobs. If Machine A's cron job and Machine B's cron job both check
> the database at the same time, they'll both see that the process isn't
> running and start it up -- resulting in two copies of the process
> instead of just one.
>
> Am I correct in saying that's what you're concerned about?
>
> If so, a simple solution would be to just wrap that part of the code
> in a transaction. This will ensure Machine B doesn't get a response
> back from the database until Machine A finishes updating it.
>
> Aside from that, this seems fine.
>
> -- Andrew
>
> On Jul 21, 9:10 am, Philippe Josse  wrote:
>
> > Hi there,
>
> > I would like to have your feedback on the way I am running daemon scripts
> > for my Django/ Python app. I know this is definitely not the "one best way",
> > but this is a solution I came up with that seemed simple to implement. (I am
> > a real newbie!)
>
> > My constraints:
> >  - getting a few processs constantly repeating themselves
> >  - getting an email notification if one of the process crashes
>
> > I set up a cron job that is launching a "daemon manager" script every
> > minute. This script is querying a PostGreSQL database to retrieve boolean
> > values associated to the state of my background processes (running / not
> > running).  If the database indicates the process is not running, then the
> > cron script launches it.
>
> > Boolean values are updated by each process when it starts and finishes.
> > Currently, it seems to work fine - but my background processes are very
> > quick to execute. When my user base will grow, they may last longer and it
> > will be essential that background processes do not run concurrently.
>
> > Would that solution work? I was worried by potential caching issues with the
> > database that may prevent my cron script to retrieve the latest boolean
> > value. Is there any real risk?
>
> > Thanks for your feedback!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

Re: How I run background processes

[Andrew Fong]

>> I was worried by potential caching issues with the database that
>> may prevent my cron script to retrieve the latest boolean value.

It sounds as if you're worried about a race condition when scale up --
e.g. let's say you need to run a lot of background processes and
decide to split them up among several machines, each with their own
cron jobs. If Machine A's cron job and Machine B's cron job both check
the database at the same time, they'll both see that the process isn't
running and start it up -- resulting in two copies of the process
instead of just one.

Am I correct in saying that's what you're concerned about?

If so, a simple solution would be to just wrap that part of the code
in a transaction. This will ensure Machine B doesn't get a response
back from the database until Machine A finishes updating it.

Aside from that, this seems fine.

-- Andrew

On Jul 21, 9:10 am, Philippe Josse  wrote:
> Hi there,
>
> I would like to have your feedback on the way I am running daemon scripts
> for my Django/ Python app. I know this is definitely not the "one best way",
> but this is a solution I came up with that seemed simple to implement. (I am
> a real newbie!)
>
> My constraints:
>  - getting a few processs constantly repeating themselves
>  - getting an email notification if one of the process crashes
>
> I set up a cron job that is launching a "daemon manager" script every
> minute. This script is querying a PostGreSQL database to retrieve boolean
> values associated to the state of my background processes (running / not
> running).  If the database indicates the process is not running, then the
> cron script launches it.
>
> Boolean values are updated by each process when it starts and finishes.
> Currently, it seems to work fine - but my background processes are very
> quick to execute. When my user base will grow, they may last longer and it
> will be essential that background processes do not run concurrently.
>
> Would that solution work? I was worried by potential caching issues with the
> database that may prevent my cron script to retrieve the latest boolean
> value. Is there any real risk?
>
> Thanks for your feedback!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---

How I run background processes

[Philippe Josse]

Hi there,

I would like to have your feedback on the way I am running daemon scripts
for my Django/ Python app. I know this is definitely not the "one best way",
but this is a solution I came up with that seemed simple to implement. (I am
a real newbie!)

My constraints:
 - getting a few processs constantly repeating themselves
 - getting an email notification if one of the process crashes

I set up a cron job that is launching a "daemon manager" script every
minute. This script is querying a PostGreSQL database to retrieve boolean
values associated to the state of my background processes (running / not
running).  If the database indicates the process is not running, then the
cron script launches it.

Boolean values are updated by each process when it starts and finishes.
Currently, it seems to work fine - but my background processes are very
quick to execute. When my user base will grow, they may last longer and it
will be essential that background processes do not run concurrently.

Would that solution work? I was worried by potential caching issues with the
database that may prevent my cron script to retrieve the latest boolean
value. Is there any real risk?


Thanks for your feedback!

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Django users" group.
To post to this group, send email to django-users@googlegroups.com
To unsubscribe from this group, send email to 
django-users+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/django-users?hl=en
-~--~~~~--~~--~--~---