I've migrated the opkg-networking server-post-install script as an example (the
previous script was broken)
File: packages/networking/scripts/server-post-install
Regards,
Olivier.
--
Olivier LAHAYE
CEA DRT/LIST/DM2I/DIR
________________________________
De : LAHAYE Olivier
Date d'envoi : vendredi 11 octobre 2013 16:53
À : oscar-devel@lists.sourceforge.net
Objet : [PROVENANCE INTERNET] [Oscar-devel] RE : linux system
servicesmanagement cleanup and codeuniformisation.
Warning: long message ;-)
Thanks a lot DongInn for your help proposal, the task is huge.
Though, I don't know how we could proceed without committing.
Or maybe I can commit and we agree not to rebuild so the repos are not broken.
Just tell me if you think that commit now is a good idea.
The philosophy of the change is the following: handle systemd (systemctl),
initscripts (chkconfig/service), manual (links in /etc/init.d) using a single
set of functions. It is required for the following reasons: (1) current code
does not work on systemd based systems. It has also limited support on debian
side (no chkconfog, thus a printf asking to enable is displayed), and (2) the
code is not consistent, sometimes it calls the script, sometimes there is a
call to system_service with hardcoded path to service, sometimes only rpm side
is handled, and so on.
What I've done so far:
============================
1) In OS_Detect I've added a fields 'service_mgt' that tells what the distro is
using for service management (it can be "systemd" based, "initscripts" based or
"manual").
The following files have been updated (not committed yet)
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/CentOS.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/Debian.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/Fedora.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/Mandriva.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/RedHat.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/SLES.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/ScientificLinux.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/SuSE.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/Ubuntu.pm
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Detect/YDL.pm
Note: another approach could be to set this in OS_Settings, but it may require
to have more config files as we cannot tell in fedora, if version >=15, the
service_mgt=systemd else =initscripts.
for the code, only systemSettings.pm would require a change:
instead of testing $id->{service_mgt} after OS_Detect::open(), we could use
OS_Settings and read a service_mgt variable.
============================
2) Then,
In
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Settings/{debian,debian5,debian6,default,rhel,suse,ubuntu1204}
(maybe we need more)
I've added and updated some values.(not commited yet)
Now, there are "_service" variables that gives the sevice real name. example
dns_service is called named. So if we need to restart the dns_service, we will
restart the "named" service using appropriate command (from OS_Detect)
the values so far are (file default; other files will override adequate values
for the OS flavour):
# Init directory specific to this distribution
init=/etc/init.d
# Specifies the name of few packages (not used?)
nfs_package=nfs
ssh_package=sshd
portmap_package=portmap
dns_package=bind
# Specifies the name of few deamons (obsolete)
ssh_daemon=/etc/init.d/sshd
dhcp_daemon=/etc/init.d/dhcpd
mysql_daemon=/etc/init.d/mysqld
monitord_daemon=/etc/init.d/systemimager-server-monitord
rsyncd_daemon=/etc/init.d/systemimager-server-rsyncd
# Specifies the name of few services
ssh_service=sshd
dhcp_service=dhcpd
mysql_service=mysqld
postresql_service=postgresql
monitor_service=systemimager-server-monitord
rsync_service=systemimager-server-rsyncd
tftp_service=atftpd
ftp_service=ftpd
http_service=httpd
dns_service=named
# Specifies few configuration files
exports_file=/etc/exports
dhcp_configfile=/etc/dhcp/dhcpd.conf
dns_configfile=/etc/named.conf
============================
3) Then, in all places in the code (many places) where services are restarted,
stopped, enabled, ... using /etc/init.d/<script> or using system_service (old
call that assumes we have the path and uses <name>_daemon variable), we must
update the code in the following way.
So far, I've fixed few places (not commited yet). Many files in oscar and
pkgsrc needs update.
Example: if we need to restart the service responsible for DNS (found in
opkg-networking-server: postinstall)
# Enabling DNS service
OSCAR::SystemServices::enable_system_services( ("dns") );
# Note: "dns" can be the generic name of a service (must be known by the
OS_Settings) or a real name found in /etc/init.d/<real_name> or
/lib/systemd/system/<real_name>.service.
# If we are sure that the DNS is provided by named service, then we could have
written:
OSCAR::SystemServices::enable_system_services( ("named") );
# but it is preferable to use the generic name so we can change this upon
distros (ftp can be ncftpd or ftpd or whatever ftp server the distro prefers).
# This function reads a array of services to enable.
# Stop and start the service. (instead of system('/etc/init.d/named stop');
system('/etc/init.d/named start') and such.
OSCAR::SystemServices::system_service(OSCAR::SystemServicesDefs::STOP(),"dns");
OSCAR::SystemServices::system_service(OSCAR::SystemServicesDefs::START(),"dns");
===========
4) Test:
- systemd distros: fedora-15+
- initscripts distros: rhel6
- manual: all debian and compatible distros (systemd and chconfig and service
commands are not mandatory so far, thus we handle the service in the old school
way by managing links in /etc/init.d. (enabling not yet perfect, but better
than nothing as it was not handeled by previous funtions)).
Sorry for so long mail, I hope that my explanation is understandable :)
PS: The list of files modified so far not yet commited:
M lib/OSCAR/OCA/OS_Detect/CentOS.pm
M lib/OSCAR/OCA/OS_Detect/Debian.pm
M lib/OSCAR/OCA/OS_Detect/Fedora.pm
M lib/OSCAR/OCA/OS_Detect/Mandriva.pm
M lib/OSCAR/OCA/OS_Detect/RedHat.pm
M lib/OSCAR/OCA/OS_Detect/SLES.pm
M lib/OSCAR/OCA/OS_Detect/ScientificLinux.pm
M lib/OSCAR/OCA/OS_Detect/SuSE.pm
M lib/OSCAR/OCA/OS_Detect/Ubuntu.pm
M lib/OSCAR/OCA/OS_Detect/YDL.pm
M lib/OSCAR/OCA/OS_Settings/Makefile
M lib/OSCAR/OCA/OS_Settings/README
M lib/OSCAR/OCA/OS_Settings/debian
A lib/OSCAR/OCA/OS_Settings/debian5
A lib/OSCAR/OCA/OS_Settings/debian6
M lib/OSCAR/OCA/OS_Settings/default
D lib/OSCAR/OCA/OS_Settings/fedora7
A lib/OSCAR/OCA/OS_Settings/rhel
M lib/OSCAR/OCA/OS_Settings/ubuntu1204
M lib/OSCAR/OCA/OS_Settings.pm
M lib/OSCAR/SystemServices.pm
M packages/ganglia/scripts/server-post-install
# Files using the old "system_service" call:
#in oscar
lib/OSCAR/NodeMgt.pm
lib/OSCAR/SystemServices.pm
packages/sis/scripts/server-post-install
packages/networking/scripts/server-post-install
scripts/oscar_wizard
scripts/install_server
#in pkgsrc
oda/trunk/lib/Database.pm
oda/trunk/lib/ODA/Bootstrap.pm
oda/trunk/bin/oda
network-configurator/trunk/bin/network-configurator
# Files using /etc/init.d scripts
# in oscar
doc/oscarpkg-howto/pkg-example.tex
lib/OSCAR/MAC.pm
lib/OSCAR/NodeMgt.pm
lib/OSCAR/OCA/OS_Settings/debian
lib/OSCAR/OCA/OS_Settings/debian5
lib/OSCAR/OCA/OS_Settings/default
lib/OSCAR/SISBuildSetup.pm
lib/OSCAR/SystemServices.pm
packages/ganglia/scripts/api-post-deploy
packages/ganglia/scripts/edit_ganglia_conf
packages/ganglia/scripts/server-post-install
packages/loghost/scripts/api-post-deploy
packages/maui/scripts/api-post-image
packages/maui/testing/test_root
packages/mta-config/scripts/api-post-deploy
packages/mta-config/scripts/enable.client.mail-locally
packages/mta-config/scripts/server-post-install
packages/netbootmgr/scripts/server-post-install
packages/nfs/scripts/server-post-install
packages/ntpconfig/doc/install.tex
packages/ntpconfig/scripts/api-post-deploy
packages/ntpconfig/scripts/client-post-install
packages/ntpconfig/scripts/server-post-install
packages/oda/scripts/api-post-deploy
packages/oda/scripts/api-post-install
packages/pfilter/scripts/api-post-clientdef
packages/sge/scripts/server-post-install
packages/sis/scripts/si_monitor.patch
packages/slurm/scripts/api-post-deploy
packages/slurm/scripts/server-post-install
packages/slurm/testing/test_root
packages/torque/scripts/api-post-deploy
packages/torque/scripts/server-post-install
packages/torque/testing/test_root
scripts/allow_client_access
scripts/install_server
scripts/setup_pxe
scripts/update_live_macs
# in pkgsrc
network-configurator/trunk/lib/Network_Configurator.pm
oda/trunk/scripts/api-post-deploy
oda/trunk/scripts/api-pre-install
opkg/oscar-nat/trunk/opkg/scripts/server-post-install
opkg/torque/trunk/debian/init.d.ex
opkg/torque/trunk/debian/rules
opkg/torque/trunk/debian/torque-default.ex
oscar-v/trunk/lib/VirtualCluster.pm
packman/trunk/PackMan.pm
rapt/trunk/rapt
systemconfigurator/trunk/bin/systemconfigurator
systemconfigurator/trunk/lib/Network/RawNetwork.pm
systemconfigurator/trunk/t/networking_rn.t
yume/trunk/yume
Olivier.
--
Olivier LAHAYE
CEA DRT/LIST/DM2I/DIR
________________________________
De : DongInn Kim [di...@cs.indiana.edu]
Date d'envoi : vendredi 11 octobre 2013 15:54
À : oscar-devel@lists.sourceforge.net
Objet : Re: [Oscar-devel] RE : [PROVENANCE INTERNET] linux system services
management cleanup and codeuniformisation.
Hi Olivier,
How can I test it?
I can test to verify your changes at rhel6 and ubuntu1204.
Any help to migrate the below files?
Regards,
--
- DongInn
On Oct 11, 2013, at 8:40 AM, LAHAYE Olivier
<olivier.lah...@cea.fr<mailto:olivier.lah...@cea.fr>> wrote:
Progress on this migration: (not yet commited until all is working (tested at
least on rhel6,fedora17,ubuntu1204)
SystemServices.pm is rewritten, and I'm now in the process of fixing all
scripts that relies on /etc/init.d to start/top/restart/get status of daemon
(fails on fedora-15+ and more).
now, the correct code to get status of a daemon (enabled, disabled, ...) and
start,stop, restart it will be done this way: (example: opkg-networking-server
: server-post-install)
use strict;
use OSCAR::OCA::OS_Settings;
use OSCAR::SystemServices;
use Carp;
# dns_service is defined in
/usr/share/perl5/vendor_perl/OSCAR/OCA/OS_Settings/{default,compatdistro,compatdistrover,distro,distrover,ident)
# (on FC-17: default, rhel,rhel6,fedora,fedora17,fedora-17-0)
my $named = OSCAR::OCA::OS_Settings::getitem("dns_service");
# If undefined, we assume that the $service is named "named".
$named = "named" if (not defined $named);
# Enabling DNS service
OSCAR::SystemServices::enable_system_services( ($named) );
# Restarting DNS service (avoid restart as it fails if not started on some
configuration).
OSCAR::SystemServices::system_service(OSCAR::SystemServicesDefs::STOP(),$named);
OSCAR::SystemServices::system_service(OSCAR::SystemServicesDefs::START(),$named);
Still need to migrate those files.
./ganglia/scripts/api-post-deploy
./ganglia/scripts/edit_ganglia_conf
./ganglia/scripts/server-post-install
./loghost/scripts/api-post-deploy
./maui/scripts/api-post-image
./maui/testing/test_root
./mta-config/scripts/api-post-deploy
./mta-config/scripts/enable.client.mail-locally
./mta-config/scripts/server-post-install
./netbootmgr/scripts/multi-arch-prepare
./netbootmgr/scripts/server-post-install
./nfs/scripts/server-post-install
./ntpconfig/doc/install.tex
./ntpconfig/scripts/api-post-deploy
./ntpconfig/scripts/client-post-install
./ntpconfig/scripts/server-post-install
./oda/scripts/api-post-deploy
./oda/scripts/api-post-install
./pfilter/scripts/api-post-clientdef
./sge/scripts/server-post-install
./sis/config.xml
./sis/scripts/api-post-image
./sis/scripts/si_monitor.patch
./slurm/scripts/api-post-deploy
./slurm/scripts/server-post-install
./slurm/testing/test_root
./torque/scripts/api-post-deploy
./torque/scripts/server-post-install
./torque/testing/test_root
--
Olivier LAHAYE
CEA DRT/LIST/DM2I/DIR
________________________________
De : LAHAYE Olivier
Date d'envoi : mercredi 9 octobre 2013 17:06
À : oscar-devel@lists.sourceforge.net<mailto:oscar-devel@lists.sourceforge.net>
Objet : [PROVENANCE INTERNET] [Oscar-devel] linux system services management
cleanup and codeuniformisation.
In order to be able to handle newer distros (fedora17+, ...) that uses
exclusively systemd, I need to fix how system services are handeled (start,
stop, restart, enable, disable, ...) (example: on fedora17+, /etc/init.d/gmond
doesn't exists. (it's /lib/systemd/system/gmond.service that is used)).
Instead of doing if else statement each time a service needs to be started or
enabled, I've seen that the base stuffs are already in place
(SystemService.pm). Unfortunately, it's not quite ready to support systemd, so
I need to fix things on the following components:
1/ OSCAR::OCA::OS_Detect (Added a service_mgt field: containing either systemd,
initscripts, manual).
Set it in the distro specific pm file. typically, debian is manual (start
scripts by calling /etc/init.d/service. redhat is initscripts (use the service
command), fedora15+ is "systemd" (systemctl used to enable, disable, start, ...)
2/ OSCAR::OCA::OS_Settings: will remove absolute path for the init script. will
keep only the name. Then we will read the os specific http_daemon and start it
using SystemService::
We should be able to avoid this in the future: (from ganglia opkg:
edit_ganglia_conf)
> # autoload apache in levels 345 and start it up
> my $HTTPD;
> for my $srv ("httpd", "apache2", "apache") {
> if (-x "/etc/init.d/$srv") {
> $HTTPD=$srv;
> last;
> }
> }
SystemServices.pm: Instead of gessing the right command to use based on sitro
packaging system, we use the os->{service_mgt} from OS_Detect::open. the we
clean up SystemServices.pm and now starting or restarting a service will be
usable.
Then use this code instead of checking -x /etc/init.d/daemon or blindly use
chkconfig wich fails on ubuntu as it's not mandatory for the system to have
/sbin/chkconfig installed.
Hopefully, it should be ok in a few days provided I don't find any code
elsewhere that can't be updated
Olivier.
--
Olivier LAHAYE
CEA DRT/LIST/DM2I/DIR
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk_______________________________________________
Oscar-devel mailing list
Oscar-devel@lists.sourceforge.net<mailto:Oscar-devel@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/oscar-devel
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60134071&iu=/4140/ostg.clktrk
_______________________________________________
Oscar-devel mailing list
Oscar-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oscar-devel
