Daemon loses __file__ reference after a while.
Hello, I have a daemon process that runs for a considerable amount of time (weeks on end) without any problems. At some point I start getting the exception: Exception info: Traceback (most recent call last): File "scheduler.py", line 376, in applyrule result = execrule(rule_code) File "scheduler.py", line 521, in execrule rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename'] NameError: name '__file__' is not defined This section of the code is executed in this process *all the time*, but suddenly stops working. I have been searching for similar issues online, but only come accross people having problems because they run the script interactively. This is not the case here. I am running python from a virtual-env installation from a stock Red Hat EL 6.2 installation: (virtual-env)[user@host ~]$ python --version Python 2.6.6 (virtual-env)[user@host ~]$ cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.2 (Santiago) I would greatly appreciate any pointers on where to start looking to find the problem. Ian. -- http://mail.python.org/mailman/listinfo/python-list
Daemon loses __file__ reference after a while
Hello all, I have a deamon process that runs for some considerable time (weeks) without any problems. At some point it starts throwing the following exception: File "/some/path/scheduler.py", line 376, in applyrule result = execrule(rule_code) File "/some/path/scheduler.py", line 521, in execrule rulepath = os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename'] NameError: name '__file__' is not defined This section is executed *all the time* but somehow stops working after a while. I have searched the web and this group, but can only find references to this occurring when run interactively, which is not the case here. When I restart the process the problem, at least temporarily, disappears. I am running the script in a virtual-env on a stock Red Hat EL 6.2 installation: (my-env)[user@host ~]$ python --version Python 2.6.6 (my-env)[user@host ~]$ cat /etc/redhat-release Red Hat Enterprise Linux Server release 6.2 (Santiago) Thank you very much in advance for any pointers as to where to start looking to find the problem. Ian. (If this post occurs twice, I apologize. Google groups was complaining about my post taking long to process and to wait a few minutes and try again if it didn't show up, which as far as I can determine, it didn't.) -- http://mail.python.org/mailman/listinfo/python-list
Re: Daemon loses __file__ reference after a while.
On Tuesday, July 24, 2012 2:24:31 PM UTC+2, Ervin Hegedüs wrote: > hello, > > On Tue, Jul 24, 2012 at 04:48:42AM -0700, ivdn...@gmail.com wrote: > > Hello, > > > > I have a daemon process that runs for a considerable amount of time > (weeks on end) without any problems. At some point I start getting the > exception: > > > > Exception info: Traceback (most recent call last): > > File "scheduler.py", line 376, in applyrule > > result = execrule(rule_code) > > File "scheduler.py", line 521, in execrule > > rulepath = > os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename'] > > NameError: name '__file__' is not defined > > > > This section of the code is executed in this process *all the time*, but > suddenly stops working. I have been searching for similar issues online, but > only come accross people having problems because they run the script > interactively. This is not the case here. > > could you send the relevant part of the code? > > I mean: how do you daemonize your process? It's done by a double fork: ## First fork() pid = os.fork() if pid != 0: sys.exit(0) # parent exits. ## create new session os.setsid() ## ignore SIGHUP signal.signal(signal.SIGHUP, signal.SIG_IGN) ## Second fork() pid = os.fork() if pid != 0: sys.exit(0) # First child exits. ## Change working directory to the home directory. homedir = pwd.getpwuid(os.geteuid())[5] os.chdir(homedir) os.umask(0) for fd in range(0, 1024): try: os.close(fd) except: pass # fd not open, ignore this exception. The original C version of this code is from W.R. Stevens' daemon_init() routine in "UNIX Network Programming Volume 1, second edition" > > > I am running python from a virtual-env installation from a stock Red Hat > EL 6.2 installation: > > > > (virtual-env)[user@host ~]$ python --version > > Python 2.6.6 > > (virtual-env)[user@host ~]$ cat /etc/redhat-release > > Red Hat Enterprise Linux Server release 6.2 (Santiago) > > If you use fork(), it drops all file descriptors, and creates new > ones - may be then loss the __file__...? I doubt this would be it, or it would stop working immediately, since daemonization is done upon startup of the process. File descriptors are closed immediately upon startup, it doesn't seem to affect the reference to the __file__ string (which is not a file object, but a str object) > > > a. > > > -- > I � UTF-8 On Tuesday, July 24, 2012 2:24:31 PM UTC+2, Ervin Hegedüs wrote: > hello, > > On Tue, Jul 24, 2012 at 04:48:42AM -0700, ivdn...@gmail.com wrote: > > Hello, > > > > I have a daemon process that runs for a considerable amount of time > (weeks on end) without any problems. At some point I start getting the > exception: > > > > Exception info: Traceback (most recent call last): > > File "scheduler.py", line 376, in applyrule > > result = execrule(rule_code) > > File "scheduler.py", line 521, in execrule > > rulepath = > os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename'] > > NameError: name '__file__' is not defined > > > > This section of the code is executed in this process *all the time*, but > suddenly stops working. I have been searching for similar issues online, but > only come accross people having problems because they run the script > interactively. This is not the case here. > > could you send the relevant part of the code? > > I mean: how do you daemonize your process? > > > I am running python from a virtual-env installation from a stock Red Hat > EL 6.2 installation: > > > > (virtual-env)[user@host ~]$ python --version > > Python 2.6.6 > > (virtual-env)[user@host ~]$ cat /etc/redhat-release > > Red Hat Enterprise Linux Server release 6.2 (Santiago) > > If you use fork(), it drops all file descriptors, and creates new > ones - may be then loss the __file__...? > > > a. > > > -- > I � UTF-8 -- http://mail.python.org/mailman/listinfo/python-list
Re: Daemon loses __file__ reference after a while
On Tuesday, July 24, 2012 2:29:19 PM UTC+2, Laszlo Nagy wrote: > On 2012-07-24 14:17, ivdn...@gmail.com wrote: > > Hello all, > > > > I have a deamon process that runs for some considerable time (weeks) > without any problems. At some point it starts throwing the following > exception: > > > >File "/some/path/scheduler.py", line 376, in applyrule > > result = execrule(rule_code) > >File "/some/path/scheduler.py", line 521, in execrule > > rulepath = > os.path.dirname(__file__)+"/"+'/'.join(rule['modules'])+"/"+rule['rulename'] > > NameError: name '__file__' is not defined > It is not a direct solution to your problem, but can you save the value > of os.path.dirname(__file__) into another variable? That might be a workaround that I'm seriously pondering as well. Thank you. Ian. (sorry for google messing up my posts) -- http://mail.python.org/mailman/listinfo/python-list
unpickling derived LogRecord in python 2.7 from python2.6
Hello all,
I have a service that runs in python 2.6.4. This service sends
LogRecords to a log monitoring app on my workstation running python
2.7. The LogRecord class is derived:
class LogRecord(logging.LogRecord):
def __init__(self, name, level, fn, lno, user, hostname, msg,
args, exc_info, func=None):
if sys.version_info[1] > 4:
logging.LogRecord.__init__(self, name, level, fn, lno,
msg, args, exc_info, func)
else:
logging.LogRecord.__init__(self, name, level, fn, lno,
msg, args, exc_info)
Now when I try to unpickle it:
record = cPickle.loads(data)
I get a TypeError exception:
TypeError: ('__init__() takes at least 8 arguments (1 given)', , ())
I've searched the web and this group, but most results are old. It
worked when my workstation still ran python 2.6.
Thank you,
Ian.
--
http://mail.python.org/mailman/listinfo/python-list
Re: unpickling derived LogRecord in python 2.7 from python2.6
On Apr 28, 9:22 am, Peter Otten <__pete...@web.de> wrote: > Vinay Sajip wrote: > > On Apr 27, 5:41 pm, Peter Otten <__pete...@web.de> wrote: > > >> The Problem is that as of Python 2.7logging.LogRecord has become a > >> newstyle class which is pickled/unpickled differently. I don't know if > >> there is an official way to do the conversion, but here's what I've > >> hacked up. The script can read pickles written with 2.6 in 2.7, but not > >> the other way round. > >> [code snipped] > > > I don't know about "official", but another way of doing this is to > > pickle just the LogRecord's __dict__ and send that over the wire. The > > logging package contains a function makeLogRecord(d) where d is a > > dict. > > You are right, my approach is too complicated and only needed when the OP > cannot modify the sending script -- which is unlikely. > > > This is the approach used by the examples in the library documentation > > which pickle events for sending across a network: > > >http://docs.python.org/howto/logging-cookbook.html#sending-and-receiv... > > logging-events-across-a-network > > > > > The built-in SocketHandler pickles the LogRecord's __dict__ rather > > than the LogRecord itself, precisely because of the improved > > interoperability over pickling the instance directly. > > As a minimal change ensuring that the logging.LogRecord subclass used by the > OP is a newstyle class in 2.6 with > > class LogRecord(logging.LogRecord, object): > #... > > should work, too. I tried this, but it didn't work. Pickling the __dict__ and then use makeLogRecord does the trick. Thank you very much for the excellent help, Ian. -- http://mail.python.org/mailman/listinfo/python-list
