Hi Asra,
It looks like monit keeps trying to start Chronos, but it keeps failing.
The following log means that Chronos has captured a diagnostics package.
[04-May-2016 07:47:28 UTC] Running extra diags scripts:
/usr/share/clearwater/clearwater-diags-monitor/scripts/chronos_diags
/usr/share/clearwater/clearwater-diags-monitor/scripts/sprout_base_diags
This will contain more details of what may have gone wrong, including a core
dump from the Chronos process. You should be able to find it at
/var/clearwater-diags-monitor/dumps/. If there are several dumps, with
different timestamps, it would be good to have one from around the time you
were seeing issues.
I just noticed that you asked some other questions in another email thread:
-The file var/lib/snmp/Homestead is missing even after i restart the
Clearwater_infrastructure.! How can i restore it?
I think this should be benign. Can you please send the log or output that is
making you worried about this?
-About Setting up S-CSCF configuration
{
"s-cscfs" : [
{ "server" :
"sip:scscf.<sprout_domain>:5054;transport=TCP<sip:scscf.%3csprout_domain%3e:5054;transport=TCP>",
"priority" : 0,
"weight" : 100,
"capabilities" : [<comma separated capabilities>]
}
]
}
I don't know what to write in Capabilities .Could someone give me an example,
please
Project Clearwater is able to run out-of-the-box without I-CSCF enabled, and it
looks like that is what you are doing at the moment. I suggest getting your
deployment running without I-CSCF first as it will be simpler, then adding
I-CSCF when you need it. That means adding the shared configuration at
http://clearwater.readthedocs.io/en/stable/Manual_Install.html?highlight=icscf#provide-shared-configuration,
and setting up this file.
When you do need to set up capabilities, you will need to make sure that the
capabilities match the set of capabilities defined in your HSS. If you are not
using an HSS (and using Ellis for provisioning), then you can leave the list of
capabilities blank.
Yours,
Chris
From: Clearwater [mailto:[email protected]] On
Behalf Of [email protected]
Sent: 04 May 2016 09:01
To: [email protected]
Subject: [Project Clearwater] Clearwater Sprout / Ralf : issue with chronos
process
Hello Chris,
To complete the previous message about the issue with 'Chronos' process (it
seems the process failed to start), here are extracts from logs from your
system:
Can you help us to find the root of this issue ?
/var/log/monit.log
[UTC May 4 07:47:45] info : 'chronos_process' trying to restart
[UTC May 4 07:47:45] info : 'chronos_process' restart: /bin/bash
[UTC May 4 07:48:15] error : 'chronos_process' failed to start
[UTC May 4 07:48:15] info : 'chronos_process' start: /bin/bash
[UTC May 4 07:48:45] error : 'chronos_process' failed to start
[UTC May 4 07:48:55] error : 'chronos_process' process is not running
[UTC May 4 07:48:55] info : 'chronos_process' trying to restart
[UTC May 4 07:48:55] info : 'chronos_process' restart: /bin/bash
[UTC May 4 07:49:25] error : 'chronos_process' failed to start
[UTC May 4 07:49:25] info : 'chronos_process' start: /bin/bash
/var/log/syslog
May 4 07:45:47 sprout queue-manager[11251]: dropped request: 'issue-alarm
queue-manager 9001.1'
May 4 07:45:49 sprout queue-manager[11251]: dropped request: 'issue-alarm
queue-manager 9002.1'
May 4 07:45:56 sprout issue_alarm.py: dropped request: 'issue-alarm monit
3000.4'
May 4 07:45:57 sprout chronos: 3001 - Description: Fatal - Chronos has exited
or crashed with signal Aborted. @@Cause: Chronos has encountered a fatal
software error or has been terminated. @@Effect: The application will exit and
restart until the problem is fixed. If Chronos processes are running correctly
on other servers in the cluster, they will provide service, although statistics
may be incorrect. @@Action: Ensure that Chronos has been installed correctly
and that it has valid configuration.
/var/log/Clearwater-diags-monitor.log
tar: 20160504074631Z.sprout.chronos.temp/root/run/acpid.socket: socket ignored
tar: 20160504074631Z.sprout.chronos.temp/root/run/clearwater/stats/astaire:
socket ignored
tar: 20160504074631Z.sprout.chronos.temp/root/run/clearwater/stats/sprout:
socket ignored
tar: 20160504074631Z.sprout.chronos.temp/root/run/udev/control: socket ignored
tar: 20160504074631Z.sprout.chronos.temp/root/run/dbus/system_bus_socket:
socket ignored
[04-May-2016 07:46:53 UTC] Diagnostic archive
/var/clearwater-diags-monitor/dumps/20160504074631Z.sprout.chronos.temp.tar.gz
created
[04-May-2016 07:46:53 UTC] Deleting dump 20160504073303Z.sprout.chronos.tar.gz
[04-May-2016 07:47:23 UTC] Processing trigger files: core.chronos.1462347997
core.chronos.1462348027
[04-May-2016 07:47:23 UTC] Dump triggered by chronos
[04-May-2016 07:47:23 UTC] Gathering dump 20160504074723Z.sprout.chronos.temp
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2334 0 2334 0 0 451k 0 --:--:-- --:--:-- --:--:-- 569k
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2334 0 2334 0 0 526k 0 --:--:-- --:--:-- --:--:-- 569k
[04-May-2016 07:47:28 UTC] Running extra diags scripts:
/usr/share/clearwater/clearwater-diags-monitor/scripts/chronos_diags
/usr/share/clearwater/clearwater-diags-monitor/scripts/sprout_base_diags
[04-May-2016 07:47:38 UTC] core.chronos.1462347997 has been closed
[04-May-2016 07:47:38 UTC] core.chronos.1462348027 has been closed
[04-May-2016 07:47:38 UTC] Moving core.chronos.1462347997 to dump
[04-May-2016 07:47:38 UTC] Moving core.chronos.1462348027 to dump
Also we tried to launch 'chronos' manually (/usr/bin/chronos), please find here
the report:
.chronos
terminate called after throwing an instance of
'boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<boost::program_options::multiple_occurrences>
>'
what(): option 'address' cannot be specified more than once
Signal 6 caught
Basic stack dump:
./chronos[0x447a5d]
./chronos[0x4475dd]
./chronos[0x47b16c]
/lib/x86_64-linux-gnu/libc.so.6(+0x36d40)[0x7fbd5ea6ad40]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39)[0x7fbd5ea6acc9]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x148)[0x7fbd5ea6e0d8]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZN9__gnu_cxx27__verbose_terminate_handlerEv+0x155)[0x7fbd5f15f535]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x5e6d6)[0x7fbd5f15d6d6]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0x5e703)[0x7fbd5f15d703]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(__cxa_rethrow+0x46)[0x7fbd5f15d976]
/usr/lib/x86_64-linux-gnu/libboost_program_options.so.1.54.0(_ZN5boost15program_options5storeERKNS0_20basic_parsed_optionsIcEERNS0_13variables_mapEb+0x1bc1)[0x7fbd60935f21]
./chronos[0x4346b9]
./chronos[0x43414d]
./chronos[0x419140]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5)[0x7fbd5ea55ec5]
./chronos[0x41a02e]
Advanced stack dump (requires gdb):
[New LWP 31779]
[New LWP 31778]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007fbd5eaf4b99 in __libc_waitpid (pid=31780,
stat_loc=stat_loc@entry=0x7ffd37053460, options=options@entry=0) at
../sysdeps/unix/sysv/linux/waitpid.c:40
40 ../sysdeps/unix/sysv/linux/waitpid.c: No such file or directory.
Thread 3 (Thread 0x7fbd572cd700 (LWP 31778)):
#0 sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1 0x0000000000448a24 in ?? ()
#2 0x00007fbd6100d182 in start_thread (arg=0x7fbd572cd700) at
pthread_create.c:312
#3 0x00007fbd5eb2e47d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Thread 2 (Thread 0x7fbd56acc700 (LWP 31779)):
#0 sem_wait () at ../nptl/sysdeps/unix/sysv/linux/x86_64/sem_wait.S:85
#1 0x0000000000448a74 in ?? ()
#2 0x00007fbd6100d182 in start_thread (arg=0x7fbd56acc700) at
pthread_create.c:312
#3 0x00007fbd5eb2e47d in clone () at
../sysdeps/unix/sysv/linux/x86_64/clone.S:111
Thread 1 (Thread 0x7fbd61626780 (LWP 31777)):
#0 0x00007fbd5eaf4b99 in __libc_waitpid (pid=31780,
stat_loc=stat_loc@entry=0x7ffd37053460, options=options@entry=0) at
../sysdeps/unix/sysv/linux/waitpid.c:40
#1 0x00007fbd5ea7a2e2 in do_system (line=<optimized out>) at
../sysdeps/posix/system.c:148
#2 0x0000000000447b13 in ?? ()
#3 0x00000000004475dd in ?? ()
#4 0x000000000047b16c in ?? ()
#5 <signal handler called>
#6 0x00007fbd5ea6acc9 in __GI_raise (sig=sig@entry=6) at
../nptl/sysdeps/unix/sysv/linux/raise.c:56
#7 0x00007fbd5ea6e0d8 in __GI_abort () at abort.c:89
#8 0x00007fbd5f15f535 in __gnu_cxx::__verbose_terminate_handler() () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#9 0x00007fbd5f15d6d6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#10 0x00007fbd5f15d703 in std::terminate() () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#11 0x00007fbd5f15d976 in __cxa_rethrow () from
/usr/lib/x86_64-linux-gnu/libstdc++.so.6
#12 0x00007fbd60935f21 in
boost::program_options::store(boost::program_options::basic_parsed_options<char>
const&, boost::program_options::variables_map&, bool) () from
/usr/lib/x86_64-linux-gnu/libboost_program_options.so.1.54.0
#13 0x00000000004346b9 in ?? ()
#14 0x000000000043414d in ?? ()
#15 0x0000000000419140 in ?? ()
#16 0x00007fbd5ea55ec5 in __libc_start_main (main=0x418ff0, argc=1,
argv=0x7ffd37057368, init=<optimized out>, fini=<optimized out>,
rtld_fini=<optimized out>, stack_end=0x7ffd37057358) at libc-start.c:287
#17 0x000000000041a02e in ?? ()
Aborted
Best regards,
_________________________________________________________________________________________________________________________
Ce message et ses pieces jointes peuvent contenir des informations
confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce
message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages
electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou
falsifie. Merci.
This message and its attachments may contain confidential or privileged
information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete
this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been
modified, changed or falsified.
Thank you.
_______________________________________________
Clearwater mailing list
[email protected]
http://lists.projectclearwater.org/mailman/listinfo/clearwater_lists.projectclearwater.org
