Hi, Strange. It’s actually normal that ZooKeeper process gets stopped with the kill signal. That’s "business as usual”, but why does systemd have problems with that and since when has this been happening?
Andor > On 2020. Mar 4., at 15:44, Anastasios Lisgaras > <tasosnumber...@yahoo.gr.INVALID> wrote: > > Hello zookeeper community, > > After the last system update I had a problem with the zookeeper-server. > The problem was about the same as this: > https://issues.apache.org/jira/browse/BIGTOP-3302 > So, I went to the file : > /etc/init.d/zookeeper-server > > and I replaced "su" with "runuser". If you want to see it : > > # cat /etc/init.d/zookeeper-server : https://termbin.com/gupi > # cat /usr/lib/zookeeper/bin/zkServer.sh : https://termbin.com/3xpb > > After this change the server runs flawlessly, but when i try to stop it > via "systemctl" the system (systemd) complains. > > ``` > # systemctl status zookeeper-server > ● zookeeper-server.service - LSB: ZooKeeper is a centralized service for > maintaining configuration information, naming, providing distributed > synchronization, and providing group services. > Loaded: loaded (/etc/rc.d/init.d/zookeeper-server; bad; vendor > preset: disabled) > Active: active (running) since Wed 2020-03-04 16:16:51 EET; 2s ago > Docs: man:systemd-sysv-generator(8) > Process: 30509 ExecStart=/etc/rc.d/init.d/zookeeper-server start > (code=exited, status=0/SUCCESS) > Main PID: 30557 (java) > CGroup: /system.slice/zookeeper-server.service > └─30557 /usr/lib/jvm/jre-openjdk/bin/java > -Dzookeeper.datadir.autocreate=false > -Dzookeeper.log.dir=/var/log/zookeeper -Dzookee... > > Mar 04 16:16:50 myserver systemd[1]: Starting LSB: ZooKeeper is a > centralized service for maintaining configuration information,...ices.... > Mar 04 16:16:50 myserver runuser[30541]: pam_unix(runuser:session): > session opened for user zookeeper by (uid=0) > Mar 04 16:16:50 myserver zookeeper-server[30509]: JMX enabled by default > Mar 04 16:16:50 myserver zookeeper-server[30509]: Using config: > /etc/zookeeper/conf/zoo.cfg > Mar 04 16:16:51 myserver zookeeper-server[30509]: Starting zookeeper ... > STARTED > Mar 04 16:16:51 myserver runuser[30541]: pam_unix(runuser:session): > session closed for user zookeeper > Mar 04 16:16:51 myserver systemd[1]: Started LSB: ZooKeeper is a > centralized service for maintaining configuration information, ...rvices.. > Hint: Some lines were ellipsized, use -l to show in full. > > > # systemctl stop zookeeper-server > > > # systemctl status zookeeper-server -l > ● zookeeper-server.service - LSB: ZooKeeper is a centralized service for > maintaining configuration information, naming, providing distributed > synchronization, and providing group services. > Loaded: loaded (/etc/rc.d/init.d/zookeeper-server; bad; vendor > preset: disabled) > Active: failed (Result: signal) since Wed 2020-03-04 16:17:00 EET; 4s ago > Docs: man:systemd-sysv-generator(8) > Process: 30605 ExecStop=/etc/rc.d/init.d/zookeeper-server stop > (code=exited, status=0/SUCCESS) > Process: 30509 ExecStart=/etc/rc.d/init.d/zookeeper-server start > (code=exited, status=0/SUCCESS) > Main PID: 30557 (code=killed, signal=KILL) > > Mar 04 16:16:51 myserver systemd[1]: Started LSB: ZooKeeper is a > centralized service for maintaining configuration information, naming, > providing distributed synchronization, and providing group services.. > Mar 04 16:17:00 myserver systemd[1]: Stopping LSB: ZooKeeper is a > centralized service for maintaining configuration information, naming, > providing distributed synchronization, and providing group services.... > Mar 04 16:17:00 myserver runuser[30642]: pam_unix(runuser:session): > session opened for user zookeeper by (uid=0) > Mar 04 16:17:00 myserver zookeeper-server[30605]: JMX enabled by default > Mar 04 16:17:00 myserver zookeeper-server[30605]: Using config: > /etc/zookeeper/conf/zoo.cfg > Mar 04 16:17:00 myserver zookeeper-server[30605]: Stopping zookeeper ... > STOPPED > Mar 04 16:17:00 myserver systemd[1]: zookeeper-server.service: main > process exited, code=killed, status=9/KILL > Mar 04 16:17:00 myserver systemd[1]: Stopped LSB: ZooKeeper is a > centralized service for maintaining configuration information, naming, > providing distributed synchronization, and providing group services.. > Mar 04 16:17:00 myserver systemd[1]: Unit zookeeper-server.service > entered failed state. > Mar 04 16:17:00 myserver systemd[1]: zookeeper-server.service failed. > > > # systemctl is-active zookeeper-server > failed > > > # systemctl is-failed zookeeper-server > failed > > > # ps -aux | grep zookeeper > root 31181 0.0 0.0 112712 968 pts/0 S+ 16:21 0:00 grep > --color=auto zookeeper > > > # systemctl restart zookeeper-server -l > > > # systemctl status zookeeper-server -l > ● zookeeper-server.service - LSB: ZooKeeper is a centralized service for > maintaining configuration information, naming, providing distributed > synchronization, and providing group services. > Loaded: loaded (/etc/rc.d/init.d/zookeeper-server; bad; vendor > preset: disabled) > Active: active (running) since Wed 2020-03-04 16:18:37 EET; 2s ago > Docs: man:systemd-sysv-generator(8) > Process: 30605 ExecStop=/etc/rc.d/init.d/zookeeper-server stop > (code=exited, status=0/SUCCESS) > Process: 30746 ExecStart=/etc/rc.d/init.d/zookeeper-server start > (code=exited, status=0/SUCCESS) > Main PID: 30794 (java) > CGroup: /system.slice/zookeeper-server.service > └─30794 /usr/lib/jvm/jre-openjdk/bin/java > -Dzookeeper.datadir.autocreate=false > -Dzookeeper.log.dir=/var/log/zookeeper > -Dzookeeper.root.logger=INFO,ROLLINGFILE -cp > /usr/lib/zookeeper/bin/../build/classes:/usr/lib/zookeeper/bin/../build/lib/*.jar:/usr/lib/zookeeper/bin/../lib/slf4j-log4j12.jar:/usr/lib/zookeeper/bin/../lib/slf4j-log4j12-1.7.5.jar:/usr/lib/zookeeper/bin/../lib/slf4j-api-1.7.5.jar:/usr/lib/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/usr/lib/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/lib/zookeeper/bin/../lib/jline-2.11.jar:/usr/lib/zookeeper/bin/../zookeeper-3.4.5-cdh5.16.2.jar:/usr/lib/zookeeper/bin/../src/java/lib/*.jar:/etc/zookeeper/conf::/etc/zookeeper/conf:/usr/lib/zookeeper/*:/usr/lib/zookeeper/lib/* > -Dzookeeper.log.threshold=INFO -Dcom.sun.management.jmxremote > -Dcom.sun.management.jmxremote.local.only=false > org.apache.zookeeper.server.quorum.QuorumPeerMain > /etc/zookeeper/conf/zoo.cfg > > Mar 04 16:18:35 myserver systemd[1]: Starting LSB: ZooKeeper is a > centralized service for maintaining configuration information, naming, > providing distributed synchronization, and providing group services.... > Mar 04 16:18:36 myserver runuser[30778]: pam_unix(runuser:session): > session opened for user zookeeper by (uid=0) > Mar 04 16:18:36 myserver zookeeper-server[30746]: JMX enabled by default > Mar 04 16:18:36 myserver zookeeper-server[30746]: Using config: > /etc/zookeeper/conf/zoo.cfg > Mar 04 16:18:37 myserver zookeeper-server[30746]: Starting zookeeper ... > STARTED > Mar 04 16:18:37 myserver systemd[1]: Started LSB: ZooKeeper is a > centralized service for maintaining configuration information, naming, > providing distributed synchronization, and providing group services.. > > > # ps -aux | grep zookeeper > zookeep+ 30794 1.0 1.8 3499256 72648 ? Sl 16:18 0:02 > /usr/lib/jvm/jre-openjdk/bin/java -Dzookeeper.datadir.autocreate=false > -Dzookeeper.log.dir=/var/log/zookeeper > -Dzookeeper.root.logger=INFO,ROLLINGFILE -cp > /usr/lib/zookeeper/bin/../build/classes:/usr/lib/zookeeper/bin/../build/lib/*.jar:/usr/lib/zookeeper/bin/../lib/slf4j-log4j12.jar:/usr/lib/zookeeper/bin/../lib/slf4j-log4j12-1.7.5.jar:/usr/lib/zookeeper/bin/../lib/slf4j-api-1.7.5.jar:/usr/lib/zookeeper/bin/../lib/netty-3.10.5.Final.jar:/usr/lib/zookeeper/bin/../lib/log4j-1.2.16.jar:/usr/lib/zookeeper/bin/../lib/jline-2.11.jar:/usr/lib/zookeeper/bin/../zookeeper-3.4.5-cdh5.16.2.jar:/usr/lib/zookeeper/bin/../src/java/lib/*.jar:/etc/zookeeper/conf::/etc/zookeeper/conf:/usr/lib/zookeeper/*:/usr/lib/zookeeper/lib/* > -Dzookeeper.log.threshold=INFO -Dcom.sun.management.jmxremote > -Dcom.sun.management.jmxremote.local.only=false > org.apache.zookeeper.server.quorum.QuorumPeerMain > /etc/zookeeper/conf/zoo.cfg > root 31105 0.0 0.0 112712 968 pts/0 R+ 16:21 0:00 grep > --color=auto zookeeper > ``` > > As you can see, starting, and restarting works properly. > When I stop the zookeeper-server, then the systemctl in the status > indicates a problem, but the zookeeper-server actually stops. > > Can you please help me to fix this problem ? > Because this problem (about systemd ) is relatively recent, the > development team is aware of it ? > > OS : CentOS Linux release 7.7.1908 (Core) > > # yum info zookeeper-server > Loaded plugins: fastestmirror > Loading mirror speeds from cached hostfile > * base: ftp.ntua.gr > * epel: mirrors.daticum.com > * extras: ftp.ntua.gr > * updates: ftp.ntua.gr > Installed Packages > Name : zookeeper-server > Arch : x86_64 > Version : 3.4.5+cdh5.16.2+159 > Release : 1.cdh5.16.2.p0.23.el7 > Size : 5.0 k > Repo : installed > From repo : cloudera-cdh5 > Summary : The Hadoop Zookeeper server > URL : http://zookeeper.apache.org/ > License : ASL 2.0 > Description : This package starts the zookeeper server on startup > > > # yum info kafka > Loaded plugins: fastestmirror > Loading mirror speeds from cached hostfile > * base: ftp.ntua.gr > * epel: mirror.hosting90.cz > * extras: ftp.ntua.gr > * updates: ftp.ntua.gr > Installed Packages > Name : kafka > Arch : noarch > Version : 2.2.1+kafka4.1.0 > Release : 2.4.1.0.p0.6.el7 > Size : 91 M > Repo : installed > From repo : cloudera-kafka > Summary : Apache Kafka is publish-subscribe messaging rethought as a > distributed commit log. > URL : http://kafka.apache.org/ > License : ASL 2.0 > Description : Apache Kafka is publish-subscribe messaging rethought as a > distributed commit log. > : A single Kafka broker can handle hundreds of megabytes of > reads and writes per second > : from thousands of clients. It can be elastically and > transparently expanded without downtime. > : Data streams are partitioned and spread over a cluster of > machines to allow data streams > : larger than the capability of any single machine and to > allow clusters of co-ordinated consumers > > > # systemctl --version > systemd 219 > +PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP > +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 -SECCOMP +BLKID +ELFUTILS > +KMOD +IDN > > > # systemctl cat zookeeper-server.service > # /run/systemd/generator.late/zookeeper-server.service > # Automatically generated by systemd-sysv-generator[Unit] > Documentation=man:systemd-sysv-generator(8) > SourcePath=/etc/rc.d/init.d/zookeeper-server > Description=LSB: ZooKeeper is a centralized service for maintaining > configuration information, naming, providing distributed synchronization > Before=runlevel2.target > Before=runlevel3.target > Before=runlevel4.target > Before=runlevel5.target > Before=shutdown.target > After=network-online.target > After=nss-lookup.target > After=network-online.target > Wants=network-online.target > Conflicts=shutdown.target[Service] > Type=forking > Restart=no > TimeoutSec=5min > IgnoreSIGPIPE=no > KillMode=process > GuessMainPID=no > RemainAfterExit=no > PIDFile=/var/run/zookeeper/zookeeper-server.pid > ExecStart=/etc/rc.d/init.d/zookeeper-server start > ExecStop=/etc/rc.d/init.d/zookeeper-server stop > > > Thanks in advance for your time! > > -- > Kind regards, > Tasos >
