Re: Example of python service running under systemd?
Hi Michael, On Thu, Sep 11, 2014 at 08:03:54PM -0600, Michael Torrie wrote: What I want is to have this startup, after my board has it’s networking layer up and running (and hopefully a valid ip address by then), and to just keep running forever may be you think about the fork(), eg: No, you you don't need to do this. Systemd can handle all of that for you. Read up on the docs on creating systemd services. Here's a little blog post that has some good examples, both a non-daemonizing service and a daemonizing service: http://patrakov.blogspot.com/2011/01/writing-systemd-service-files.html Any executable file can be turned into a daemon service with systemd (whether or not it forks itself into the background). Thus any python script can easily be run from systemd. thanks a lot, I didn't hear about that feature. Cheers, Ervin -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
Hi Chris, On Fri, Sep 12, 2014 at 12:29:27PM +1000, Chris Angelico wrote: On Fri, Sep 12, 2014 at 12:03 PM, Michael Torrie torr...@gmail.com wrote: Any executable file can be turned into a daemon service with systemd (whether or not it forks itself into the background). Thus any python script can easily be run from systemd. I strongly recommend making a non-daemonizing service. It's so much easier to debug - there's one mode of operation, the script just runs. You can then run that directly in a terminal, or via tmux, or via systemd - and I've done all three with Yosemite. In fact, I think I have instances here on the LAN that are doing all three, right now! is there any other reason outside the debugging? Of course, I've handled that in a simple way: parser = optparse.OptionParser() parser.add_option(-d, --debug, action=count, dest=debug_mode, help=Start process in debug mode, not forking.) (options, args) = parser.parse_args() debug_mode = True if options.debug_mode is None: debug_mode = False try: pid = os.fork() if pid 0: And of course, I've handled the signals, logfiles and so on... So, now I can run my app with -d, then it will not do the fork(), I'll see all messages and feedbacks. Elsewhere, the process will run in background. Anyway, thanks all comments from others. May be the life is easier with systemd, but that was my 5-minutes-finger-exercise :) Thanks again, a. -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On Fri, Sep 12, 2014 at 4:18 PM, Ervin Hegedüs airw...@gmail.com wrote: is there any other reason outside the debugging? Of course, I've handled that in a simple way: parser = optparse.OptionParser() parser.add_option(-d, --debug, action=count, dest=debug_mode, help=Start process in debug mode, not forking.) (options, args) = parser.parse_args() debug_mode = True if options.debug_mode is None: debug_mode = False try: pid = os.fork() if pid 0: And of course, I've handled the signals, logfiles and so on... 1) You don't need all of the above code. 2) You don't need to test all of that code. And that code is significantly abbreviated. In reality it's quite a bit longer. Having less code branches is itself an advantage. If I can accomplish everything with simple top-down code, why go for a -d option and then an alternative method that forks, forks again, handles signals, etc, etc, etc? (Although handling signals may still be important, if I want some kind of more orderly shutdown on SIGTERM, or if I want SIGHUP to do some sort of reload - not usually in Python, but my Pike code generally takes SIGHUP to mean reload your code from the disk.) The simpler, the better. Less code = less bugs. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Python stdout goes where under systemd? (Was: Example of python service running under systemd?)
Thanks all for the help/advice. I’m getting there.
To experiment/learn, I made a simple python program (/Foo/cyclic.py):
#!/usr/bin/env python3
import time
while True:
time.sleep(5)
with open('sound', 'r') as file:
currentValue = file.read()
otherValue = 'tick' if currentValue == 'tock' else 'tock'
with open('sound', 'w') as file:
file.write(otherValue)
print(currentValue, '-', otherValue)
Run from the command line, this tick-tocks nicely, both outputting, as well as
updating the ‘/Foo/sound’ file on a 5 second period.
I then created a simple .service file:
[Unit]
Description=Foo for learning service
After=network-online.target
[Service]
Type=simple
ExecStart=/Foo/cyclic.py
WorkingDirectory=/Foo
StandardOutput=journal
[Install]
WantedBy=multi-user.target
I chose to be “explicit” with some of the default options (Type and
StandardOutput).
I finally executed:
systemctl --system daemon-reload
systemctl enable foo
systemctl start foo
It seems to work. Almost. The file is being updated regularly (watch cat
/Foo/sound shows the change happening). But I can’t seem to find the output
from my print() statement. journalctl -f doesn’t show anything. Nor does tail
-f /var/log/syslog or any of the others. It just seems to be going nowhere? Is
there something I need to do special to get the print() output going somewhere
logable?
--
https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On Friday, September 12, 2014 1:48:37 AM UTC+8, Travis Griggs wrote: I've been reading lots of systemd docs. And blogs. Etc. At this point, I think I would benefit from learning by example... Does anyone have an example .service file that they use to launch a long running service written as a python program? If there is any example of what you changed to your python program itself, that to would be really instructional for me. Please check the examples in wxpython and boa. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python stdout goes where under systemd? (Was: Example of python service running under systemd?)
On Sep 12, 2014, at 12:05 PM, Travis Griggs travisgri...@gmail.com wrote:
Thanks all for the help/advice. I’m getting there.
To experiment/learn, I made a simple python program (/Foo/cyclic.py):
#!/usr/bin/env python3
import time
while True:
time.sleep(5)
with open('sound', 'r') as file:
currentValue = file.read()
otherValue = 'tick' if currentValue == 'tock' else 'tock'
with open('sound', 'w') as file:
file.write(otherValue)
print(currentValue, '-', otherValue)
Run from the command line, this tick-tocks nicely, both outputting, as well
as updating the ‘/Foo/sound’ file on a 5 second period.
I then created a simple .service file:
[Unit]
Description=Foo for learning service
After=network-online.target
[Service]
Type=simple
ExecStart=/Foo/cyclic.py
WorkingDirectory=/Foo
StandardOutput=journal
[Install]
WantedBy=multi-user.target
I chose to be “explicit” with some of the default options (Type and
StandardOutput).
I finally executed:
systemctl --system daemon-reload
systemctl enable foo
systemctl start foo
It seems to work. Almost. The file is being updated regularly (watch cat
/Foo/sound shows the change happening). But I can’t seem to find the output
from my print() statement. journalctl -f doesn’t show anything. Nor does tail
-f /var/log/syslog or any of the others. It just seems to be going nowhere?
Is there something I need to do special to get the print() output going
somewhere logable?
Arghhh… I’ll answer my own question here. I wasn’t patient enough, when I
checked after lunch, I found I had a mountain of tick/tock entries in
journalctl -f. Python print() is buffered, so it wasn’t showing up except in
huge blocks. Changed the .service file to start with -u and everything works as
expected now.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Python stdout goes where under systemd? (Was: Example of python service running under systemd?)
On Sat, Sep 13, 2014 at 7:45 AM, Travis Griggs travisgri...@gmail.com wrote: Python print() is buffered, so it wasn’t showing up except in huge blocks. Changed the .service file to start with -u and everything works as expected now. Ah, yes, that'll happen any time stdout isn't connected to a tty. Nothing to do with systemd or journal. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On 09/12/2014 02:05 PM, CHIN Dihedral wrote: Please check the examples in wxpython and boa. Oh funny. Just when I think the bot is ready to pass a turing test we get a regression. -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On Sat, Sep 13, 2014 at 12:36 PM, Michael Torrie torr...@gmail.com wrote: Oh funny. Just when I think the bot is ready to pass a turing test we get a regression. Ah, the Turing test... everyone loves it. I had some really naughty fun with that name a while ago. In my DD world themed on Wonderland, there are two rival manufacturers who produce a drinkable form of intelligence. One form is drained from real people, stolen from our world (they send them back with empty liquor bottles and nobody notices the difference - or else they just take from middle management, and again, nobody notices), and the other produces an artificial form, which is dangerously addictive. The test of addictiveness is how it tours, and when eventually a safer form of manufactured stuff was developed, it was hailed as artificial intelligence that passes the touring test... Yeah, they listened to rumours in the tavern, and that was their pun-ishment... ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Example of python service running under systemd?
I’ve been reading lots of systemd docs. And blogs. Etc. At this point, I think I would benefit from learning by example… Does anyone have an example .service file that they use to launch a long running service written as a python program? If there is any example of what you changed to your python program itself, that to would be really instructional for me. -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On Thu, Sep 11, 2014 at 7:48 PM, Travis Griggs travisgri...@gmail.com wrote: I’ve been reading lots of systemd docs. And blogs. Etc. At this point, I think I would benefit from learning by example… Does anyone have an example .service file that they use to launch a long running service written as a python program? If there is any example of what you changed to your python program itself, that to would be really instructional for me. -- https://mail.python.org/mailman/listinfo/python-list Depends what you want. If you are running a Python web app, you can use uWSGI Emperor, which plugs into systemd nicely and provides a .service file in the docs (which also ships with the Arch Linux package). Otherwise, there are various ways, and this all depends on how you structure this and your needs. Use Type=simple and you won’t need any changes to your program, or use one of the more magical methods and implement them. Describe your needs to get more details. -- Chris “Kwpolska” Warrick http://chriswarrick.com/ PGP: 5EAAEA16 stop html mail | always bottom-post | only UTF-8 makes sense -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On Sep 11, 2014, at 11:18 AM, Chris “Kwpolska” Warrick kwpol...@gmail.com wrote: Depends what you want. Mine is not a web service. My main.py looks like this: #!/usr/bin/env python3 import cycle import pushTelemetry from threading import Thread def main(): Thread(target=pushTelemetry.udpLoop).start() Thread(target=cycle.cycleLoop).start() if __name__ == '__main__': main() It basically creates two threads, one which does some local processing and control, the other which periodically does reporting via udp packets. I use the dual threads because they both work with a shared serial port at times, so I have to synchronize access through that. What I want is to have this startup, after my board has it’s networking layer up and running (and hopefully a valid ip address by then), and to just keep running forever -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
Hi Travis, On Thu, Sep 11, 2014 at 02:06:48PM -0700, Travis Griggs wrote: On Sep 11, 2014, at 11:18 AM, Chris “Kwpolska” Warrick kwpol...@gmail.com wrote: Depends what you want. Mine is not a web service. My main.py looks like this: #!/usr/bin/env python3 import cycle import pushTelemetry from threading import Thread def main(): Thread(target=pushTelemetry.udpLoop).start() Thread(target=cycle.cycleLoop).start() if __name__ == '__main__': main() It basically creates two threads, one which does some local processing and control, the other which periodically does reporting via udp packets. I use the dual threads because they both work with a shared serial port at times, so I have to synchronize access through that. What I want is to have this startup, after my board has it’s networking layer up and running (and hopefully a valid ip address by then), and to just keep running forever may be you think about the fork(), eg: if __name__ == __main__: ...other codes, eg. drop root privileges, ... ...check arguments... try: pid = os.fork() if pid 0: #print Daemon started (pid: %d) % (pid) sys.exit(0) except OSError, e: print sys.stderr, fork #1 failed: %d (%s) % (e.errno, e.strerror) sys.exit(1) os.chdir(/) os.setsid() os.umask(0) # do second fork try: pid = os.fork() if pid 0: #print Daemon started (pid: %d) % (pid) sys.exit(0) except OSError, e: print sys.stderr, fork #2 failed: %d (%s) % (e.errno, e.strerror) sys.exit(1) main() regards, a. -- I � UTF-8 -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On Sep 11, 2014, at 2:29 PM, Ervin Hegedüs airw...@gmail.com wrote: Hi Travis, On Thu, Sep 11, 2014 at 02:06:48PM -0700, Travis Griggs wrote: On Sep 11, 2014, at 11:18 AM, Chris “Kwpolska” Warrick kwpol...@gmail.com wrote: Depends what you want. Mine is not a web service. My main.py looks like this: #!/usr/bin/env python3 import cycle import pushTelemetry from threading import Thread def main(): Thread(target=pushTelemetry.udpLoop).start() Thread(target=cycle.cycleLoop).start() if __name__ == '__main__': main() It basically creates two threads, one which does some local processing and control, the other which periodically does reporting via udp packets. I use the dual threads because they both work with a shared serial port at times, so I have to synchronize access through that. What I want is to have this startup, after my board has it’s networking layer up and running (and hopefully a valid ip address by then), and to just keep running forever may be you think about the fork(), eg: if __name__ == __main__: ...other codes, eg. drop root privileges, ... ...check arguments... try: pid = os.fork() if pid 0: #print Daemon started (pid: %d) % (pid) sys.exit(0) except OSError, e: print sys.stderr, fork #1 failed: %d (%s) % (e.errno, e.strerror) sys.exit(1) os.chdir(/) os.setsid() os.umask(0) # do second fork try: pid = os.fork() if pid 0: #print Daemon started (pid: %d) % (pid) sys.exit(0) except OSError, e: print sys.stderr, fork #2 failed: %d (%s) % (e.errno, e.strerror) sys.exit(1) main() OK, I’m probably going to show my naivety about something simple here… I thought a “fork” essentially created a memory copy of the original process and let it go off running. The problem is, in the bowels of the code executing in those loops, I access a single instance of a threading.RLock, so that I can avoid both threads trying to do transactions with a single serial port at the same time. If I end up with two copies of the base process, unhooked from their parent, does that RLock still remain valid between the two? I thought since they were complete different copies of the same memory, they would no longer be coordinated. Is this a day where I discover something new? -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On Fri, Sep 12, 2014 at 3:48 AM, Travis Griggs travisgri...@gmail.com wrote:
I’ve been reading lots of systemd docs. And blogs. Etc. At this point, I
think I would benefit from learning by example…
Does anyone have an example .service file that they use to launch a long
running service written as a python program?
If there is any example of what you changed to your python program itself,
that to would be really instructional for me.
Yeah, I did that for the Yosemite Project:
https://github.com/Rosuav/Yosemite
The main program is Python, but all the systemd code comes from a Pike
script that creates a bash script that creates the systemd file. Yes,
that's a little convoluted... Here's the relevant shell script part,
in case you don't want to dig it out of auth.pike:
#!/bin/bash
[ $1 = install ] [ -d /etc/systemd/system ] {
echo [Unit]
Description=Yosemite Project
[Service]
# The user, path, and X display are derived at installation time
# from the attributes of the yos script. Reinstall to reset them.
Environment=DISPLAY=$DISPLAY
User=`stat -c %u $0`
ExecStart=`readlink -e $0`
# If the network isn't available yet, restart until it is.
Restart=on-failure
RestartSec=10
[Install]
WantedBy=multi-user.target
/etc/systemd/system/yos.service
# Note that some of this will fail if systemd is installed
# but isn't the default init system. In that case, well, you
# can't use this method of autostarting. TODO: Provide some
# other ways to autostart (eg for Upstart and/or sysvinit).
systemctl --system daemon-reload
systemctl enable yos.service
echo Installed as yos.service.
systemctl start yos.service
exit
}
# ... blah blah various setup
python Yosemite.py
This has some complications that you probably don't need, like that
the owner of the script should become the user that runs it (chances
are you can hard-code this, or run it as root and have it drop
privileges itself, or something), and it needs to restart on failure,
as it has to establish an sshfs mount before starting the main
program. But it's a start.
Note the three commands just before the script exits. Unlike sysvinit
and upstart, systemd needs to be told to reload service files, and
then you need to enable the service before it'll run on startup (and I
like to start it immediately, to see that it's working properly). Took
me a few hang-ups to figure that part out. Fortunately systemd can
notice and give a warning if you change files and don't daemon-reload
it; I'm not sure why it can't simply reload automatically (AIUI
Upstart uses inotify on /etc/init - no idea why systemd can't do the
same).
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On 09/11/2014 03:29 PM, Ervin Hegedüs wrote: It basically creates two threads, one which does some local processing and control, the other which periodically does reporting via udp packets. I use the dual threads because they both work with a shared serial port at times, so I have to synchronize access through that. What I want is to have this startup, after my board has it’s networking layer up and running (and hopefully a valid ip address by then), and to just keep running forever may be you think about the fork(), eg: No, you you don't need to do this. Systemd can handle all of that for you. Read up on the docs on creating systemd services. Here's a little blog post that has some good examples, both a non-daemonizing service and a daemonizing service: http://patrakov.blogspot.com/2011/01/writing-systemd-service-files.html Any executable file can be turned into a daemon service with systemd (whether or not it forks itself into the background). Thus any python script can easily be run from systemd. -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On Fri, Sep 12, 2014 at 12:03 PM, Michael Torrie torr...@gmail.com wrote: No, you you don't need to do this. Systemd can handle all of that for you. Read up on the docs on creating systemd services. Here's a little blog post that has some good examples, both a non-daemonizing service and a daemonizing service: http://patrakov.blogspot.com/2011/01/writing-systemd-service-files.html Any executable file can be turned into a daemon service with systemd (whether or not it forks itself into the background). Thus any python script can easily be run from systemd. I strongly recommend making a non-daemonizing service. It's so much easier to debug - there's one mode of operation, the script just runs. You can then run that directly in a terminal, or via tmux, or via systemd - and I've done all three with Yosemite. In fact, I think I have instances here on the LAN that are doing all three, right now! ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Example of python service running under systemd?
On 09/11/2014 08:29 PM, Chris Angelico wrote: On Fri, Sep 12, 2014 at 12:03 PM, Michael Torrie torr...@gmail.com wrote: No, you you don't need to do this. Systemd can handle all of that for you. Read up on the docs on creating systemd services. Here's a little blog post that has some good examples, both a non-daemonizing service and a daemonizing service: http://patrakov.blogspot.com/2011/01/writing-systemd-service-files.html Any executable file can be turned into a daemon service with systemd (whether or not it forks itself into the background). Thus any python script can easily be run from systemd. I strongly recommend making a non-daemonizing service. It's so much easier to debug - there's one mode of operation, the script just runs. You can then run that directly in a terminal, or via tmux, or via systemd - and I've done all three with Yosemite. In fact, I think I have instances here on the LAN that are doing all three, right now! Agreed 100%. If you want to run on a system without systemd, I recommend using supervisord[1] for turning your program into a daemon. It also does not require the app to fork. [1] http://supervisord.org/ -- https://mail.python.org/mailman/listinfo/python-list
