Hi,
I'll keep the required information, please scroll down.
On 27.02.2013 13:35, Serge Noiraud wrote:
> icinga 1.8.4
> icinga-web 1.8.2
>
> icinga configuration : --enable-idoutils --enable-event-broker ...
>
> icinga.cfg
> ...
> cfg_dir=/products/monitoring/configuration/icinga/modules
> ...
> event_broker_options=-1
>
> in the modules directory I have the following :
> idoutils.cfg ( the standard file )
> define module{
> module_name idomod
> module_type neb
> path /products/monitoring/icinga/lib/idomod.so
> args
> config_file=/products/monitoring/configuration/icinga/idomod.cfg
> }
>
> and remote.cfg :
> define module{
> module_name remote
> module_type neb
> path /products/monitoring/icinga/lib/idomod.so
> args
> config_file=/products/monitoring/configuration/icinga/remote.cfg
> }
>
> the idomod.cfg contains :
>
> instance_name=default
> output_type=unixsocket
> output=/products/monitoring/var/ido.sock
> tcp_port=5668
> use_ssl=0
> output_buffer_items=5000
> buffer_file=/products/monitoring/var/idomod.tmp
> file_rotation_interval=14400
> file_rotation_timeout=60
> reconnect_interval=15
> reconnect_warning_interval=15
> data_processing_options=67108669
> config_output_options=2
> debug_level=0
> debug_verbosity=2
> debug_file=/products/monitoring/var/idomod.debug
> max_debug_file_size=1000000
>
> The remote.cfg contains :
>
> instance_name=instance1
> output_type=tcpsocket
> output=172.45.18.1 # central address
> tcp_port=5668
> use_ssl=0
> output_buffer_items=5000
> buffer_file=/products/monitoring/var/idomod.tmp
> file_rotation_interval=14400
> file_rotation_timeout=60
> reconnect_interval=15
> reconnect_warning_interval=15
> data_processing_options=67108669
> config_output_options=2
> debug_level=-1
> debug_verbosity=2
> debug_file=/products/monitoring/var/idomod.debug
> max_debug_file_size=1000000
>
> In the first debug log I only collect the second idomod connection and
> then it stop.
>
> /var/log/messages : As you can see, the two idomod modules are
> successfully loaded
>
> Feb 27 10:59:37 opium icinga: Icinga 1.8.4 starting... (PID=26497)
> Feb 27 10:59:37 opium icinga: Local time is Wed Feb 27 10:59:37 CET 2013
> Feb 27 10:59:37 opium icinga: LOG VERSION: 2.0
> Feb 27 10:59:37 opium icinga: idomod: IDOMOD 1.8.4 (01-13-2013)
> Copyright(c) 2005-2008 Ethan Galstad, Copyright(c) 2009-2012 Icinga
> Development Team (https://www.icinga.org)
> Feb 27 10:59:37 opium icinga: idomod: Successfully connected to data
> sink. 0 queued items to flush.
> Feb 27 10:59:37 opium icinga: Event broker module 'IDOMOD' version
> '1.8.4' from '/products/monitoring/icinga/lib/idomod.so' initialized
> successfully.
> Feb 27 10:59:37 opium icinga: idomod: IDOMOD 1.8.4 (01-13-2013)
> Copyright(c) 2005-2008 Ethan Galstad, Copyright(c) 2009-2012 Icinga
> Development Team (https://www.icinga.org)
> Feb 27 10:59:37 opium ido2db: Client connected, data available.
> Feb 27 10:59:37 opium icinga: idomod: Successfully connected to data
> sink. 0 queued items to flush.
> Feb 27 10:59:37 opium icinga: Event broker module
> '/products/monitoring/icinga/lib/idomod.so' initialized successfully.
> Feb 27 10:59:37 opium ido2db: Handling client connection...
> Feb 27 10:59:37 opium icinga: livestatus: Livestatus 1.2.0p3 by
> Mathias Kettner. Socket: '/products/monitoring/var/rw/livestatus'
> Feb 27 10:59:37 opium icinga: livestatus: Please visit us at
> http://mathias-kettner.de/
> Feb 27 10:59:37 opium icinga: livestatus: Hint: please try out OMD -
> the Open Monitoring Distribution
> Feb 27 10:59:37 opium icinga: livestatus: Please visit OMD at
> http://omdistro.org
> Feb 27 10:59:37 opium ido2db: Successfully connected to mysql database
> Feb 27 10:59:37 opium icinga: livestatus: Removed old left over socket
> file /products/monitoring/var/rw/livestatus
> Feb 27 10:59:37 opium icinga: livestatus: Finished initialization.
> Further log messages go to /products/monitoring/var/logs/livestatus.log
> Feb 27 10:59:37 opium icinga: Event broker module
> '/products/monitoring/mk-livestatus/lib/mk-livestatus/livestatus.o'
> initialized successfully.
> Feb 27 10:59:40 opium icinga: Finished daemonizing... (New PID=26502)
>
> idomod.debud :
> All is working perfectly and logged.
>
> remote.debug :
>
> [1361959177.239303] [001.2] [pid=26497] idomod_open_debug_log()
> [1361959177.239331] [001.2] [pid=26497] idomod_init() start
> [1361959177.239334] [001.2] [pid=26497] idomod_sink_buffer_init() start
> [1361959177.239356] [001.2] [pid=26497] idomod_sink_buffer_init() end
> [1361959177.239360] [001.2] [pid=26497] idomod_load_unprocessed_data()
> start
> [1361959177.239370] [001.2] [pid=26497] idomod_write_to_sink() start
> [1361959177.239374] [001.2] [pid=26497] idomod_write_to_sink(
> )
> [1361959177.239377] [001.2] [pid=26497] idomod_open_sink() start
> [1361959177.293828] [001.2] [pid=26497] idomod_open_sink() end
> [1361959177.293875] [001.2] [pid=26497] idomod_hello_sink() start
> [1361959177.293898] [001.2] [pid=26497] idomod_write_to_sink() start
> [1361959177.293904] [001.2] [pid=26497] idomod_write_to_sink(
>
> HELLO
> PROTOCOL: 2
> AGENT: IDOMOD
> AGENTVERSION: 1.8.4
> STARTTIME: 1361959177
> DISPOSITION: REALTIME
> CONNECTION: TCPSOCKET
> CONNECTTYPE: INITIAL
> INSTANCENAME: instance1
> STARTDATADUMP
>
> )
> [1361959177.294096] [001.2] [pid=26497] idomod_write_to_sink() end
> [1361959177.294111] [001.2] [pid=26497] idomod_hello_sink() end
> [1361959177.294342] [001.2] [pid=26497] idomod_sink_buffer_items()
> [1361959177.294538] [001.2] [pid=26497] idomod_write_to_sink() end
> [1361959177.294554] [001.2] [pid=26497] idomod_register_callbacks() start
> [1361959177.294564] [001.2] [pid=26497] idomod_register_callbacks() end
> [1361959177.294570] [001.2] [pid=26497] idomod_init() end
> [1361959177.294725] [001.2] [pid=26497] idomod_broker_data() start
> [1361959177.294782] [001.2] [pid=26497] idomod_write_to_sink() start
> [1361959177.294793] [001.2] [pid=26497] idomod_write_to_sink(
> 202:
> 1=300
> 2=0
> 3=0
> 4=1361959177.294704
> 73=1361959177
> 74=262144
> 72=Event broker module 'IDOMOD' version '1.8.4' from
> '/products/monitoring/icinga/lib/idomod.so' initialized successfully.
> 999
>
> )
> [1361959177.294801] [001.2] [pid=26497] idomod_sink_buffer_items()
> [1361959177.294985] [001.2] [pid=26497] idomod_write_to_sink() end
> [1361959177.295002] [001.2] [pid=26497] idomod_broker_data() end
> [1361959177.295193] [001.2] [pid=26497] idomod_broker_data() start
> [1361959177.295206] [001.2] [pid=26497] idomod_write_to_sink() start
> [1361959177.295214] [001.2] [pid=26497] idomod_write_to_sink(
> 202:
> 1=300
> 2=0
> 3=0
> 4=1361959177.295182
> 73=1361959177
> 74=262144
> 72=idomod: IDOMOD 1.8.4 (01-13-2013) Copyright(c) 2005-2008 Ethan
> Galstad, Copyright(c) 2009-2012 Icinga Development Team
> (https://www.icinga.org)
> 999
>
> )
> [1361959177.295221] [001.2] [pid=26497] idomod_sink_buffer_items()
> [1361959177.295371] [001.2] [pid=26497] idomod_write_to_sink() end
> [1361959177.295387] [001.2] [pid=26497] idomod_broker_data() end
> Then nothing after the above line for the remote.debug file all others
> lines continue in the idomod.debug file
Sorry for leaving this behind (had this in mind for a long time), but
today I've debugged a different issue with neb modules and now found the
reason for that issue.
It's a bit complicated to explain, but long story short - it's a bug,
and I am willing to fix it with 1.9.1, even if this would add some
behavioural change again.
https://dev.icinga.org/issues/4199
Basically, when dlopen() loads a neb module into memory, it checks
agains the filename and only lets the first occurence register its
symbols (for functions, etc). The other loaded symbols are mainly
ignored, which then leads to the problem that only the first module
handle wins all the the callback data. The reason why the initial api
handshake takes place is a bit harder to understand - during neb module
load, the init function gets called (the one from the first module!),
but since the core passes the arguments (idomod.cfg and different
instance_name!) to that function call, there's actually more than one
handshake with ido2db. But as remarked, after that the callbacks do not
happen, as there's no such identifier passed around here (can't happen
anyways).
In order to fix that, copying the different idomod.so binaries into temp
files and putting some random names will trick dlopen() into keeping
function symbols for every module loaded into memory, and therefore
having the callback functions receiving data from the core, having the
idoutils database updated with multiple instances.
So this will fix your issue you're experiencing - you should really have
opened a bug for this. If you are willing to test, a bugfix against
current 1.9.0 is currently living in git.
https://git.icinga.org/?p=icinga-core.git;a=shortlog;h=refs/heads/fix/dlopen-mult-sym-4199
Feedback welcome!
kind regards,
Michael
>
> -
> Serge
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_feb
>
>
> _______________________________________________
> icinga-users mailing list
> icinga-users@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/icinga-users
--
DI (FH) Michael Friedrich
mail: michael.friedr...@gmail.com
twitter: https://twitter.com/dnsmichi
jabber: dnsmi...@jabber.ccc.de
irc: irc.freenode.net/icinga dnsmichi
icinga open source monitoring
position: lead core developer
url: https://www.icinga.org
------------------------------------------------------------------------------
AlienVault Unified Security Management (USM) platform delivers complete
security visibility with the essential security capabilities. Easily and
efficiently configure, manage, and operate all of your security controls
from a single console and one unified framework. Download a free trial.
http://p.sf.net/sfu/alienvault_d2d
_______________________________________________
icinga-users mailing list
icinga-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/icinga-users
