Hi, the hudson tests failed, so i committed another modification. Now i get special characters on the command line
OMD[stest]:~$ printf 'GET comments\nColumns: comment_id comment\nOutputFormat: csv\n' | unixcat tmp/run/live 66;îcî un cômméntère 67;This host has been scheduled for fixed downtime from 2011-12-08 21:44:01 to 2011-12-08 23:44:01. Notifications for the host will not be sent out during that time period. and the unittests look good, too. Gerhard > -----Ursprüngliche Nachricht----- > Von: Gerhard Lausser [mailto:gerhard.laus...@consol.de] > Gesendet: Donnerstag, 8. Dezember 2011 23:14 > An: shinken-devel@lists.sourceforge.net > Betreff: Re: [Shinken-devel] Slight problem with livestatus > modulewithdirect queries > > Hi, > > as you can see from the screenshot, accents are no problem in > the Thruk gui. > That is because it uses OutputFormat:json in the livestatus > queries. And the > json module seems to handle these special characters automagically. > Now i made this: > > stdout_encoding = sys.getdefaultencoding() > ... > elif isinstance(x, basestring): > > l.append(x.encode(stdout_encoding, 'replace')) > else: > l.append(str(x)) > > when i run the broker as a daemon, stdout_encoding is > "ascii", so this code > is practically the same as your "dirty" workaround. You > should see your > accents. > > Gerhard > > > > -----Ursprüngliche Nachricht----- > > Von: Denis GERMAIN [mailto:dt.germ...@gmail.com] > > Gesendet: Donnerstag, 8. Dezember 2011 14:56 > > An: shinken-devel@lists.sourceforge.net > > Betreff: Re: [Shinken-devel] Slight problem with livestatus > > module withdirect queries > > > > For now i used the following workaround, which is really > > dirty but seems to work. I post it in case someone uses it > > also (even though it's highly improbable) > > > > > > cd > /usr/lib/python2.6/site-packages/shinken/modules/livestatus_broker/ > > cp livestatus_response.py livestatus_response.py.old > > vi livestatus_response.py > > line 92 > > -- l.append(str(x)) > > ++ if type(x) == int: > > ++ l.append(str(x)) > > ++ else: > > ++ x = x.encode('ascii', 'replace') > > ++ l.append(str(x)) > > I had to make a difference between int which don't have the > > .encode() method and string types > > > > Denis GERMAIN > > > > > > 2011/12/7 Denis GERMAIN <dt.germ...@gmail.com> > > > > > > Hi, > > > > I have a new issue, partly related to this script. The > > script in itself is now working fine, thanks to your > > modifications, but we are getting a Traceback on Livestatus > > broker because we have special French caracters in our > > comments for hosts down (é, è, à, etc...) > > > > This didn't happen before because we couldn't add these > > caracters at all in Livestatus until our last Shinken upgrade > > (there has been some additions to prevent these to create > > bugs in the latest version of Shinken) > > > > I don't know what kind of "protection" has been added > > in the code to prevent these special caracters from crashing > > the broker, but if I knew it I could add it here also, in > > order to avoid the error I have here (see attached traceback) > > > > Regards > > > > Denis GERMAIN > > > > > > 2011/11/3 Denis GERMAIN <dt.germ...@gmail.com> > > > > > > Hi > > > > This indeed seems to be a valid solution, I > > launched it many times, with no more issue on the broker. I'm > > going to put back my script in production, thanks to your help. > > > > Many thanks Gerhard > > > > Denis > > > > > > 2011/11/2 Gerhard Lausser <gerhard.laus...@consol.de> > > > > > > Hi, > > > > > > > > maybe the Nagios livestatus module is > > more robust. I took your script and it crashed, but only when > > I had hosts without a comment. I rewrote it a little bit and > > it works now (CentOS5.6,64,Python 2.4.3) > > > > > > > > def main_function(): > > > > host_table = send_query("GET > > hosts\nFilter: state = 0\nColumns: name > > comments\nOutputFormat: csv\n") > > > > for host in host_table: > > > > host_name, > > host_comment_ids = host > > > > > > > > try: > > > > > > host_last_comment_id = host_comment_ids.split(',')[-1] #use > > only last comment > > > > > > host_last_comment_id = str(int(host_last_comment_id)) # if > > it is not a number (or empty), then it raises an exception > > > > comment = > > send_query("GET comments\nColumns: comment_id > > comment\nFilter: comment_id = "+host_last_comment_id+"\n") > > > > print comment > > > > except Exception: > > > > print host_name + " > > has no comments" > > > > > > > > OMD[denis]:~$ python denis.py > > > > [['16', 'kaas']] > > > > [['2', 'schmarrn']] > > > > omd11 has no comments > > > > [['4', 'schmarrn']] > > > > [['5', 'schmarrn']] > > > > [['6', 'schmarrn']] > > > > [['7', 'schmarrn']] > > > > [['17', 'kaas-omd-2']] > > > > [['9', 'schmarrn']] > > > > [['10', 'schmarrn']] > > > > [['11', 'schmarrn']] > > > > [['12', 'schmarrn']] > > > > [['13', 'schmarrn']] > > > > [['14', 'schmarrn']] > > > > [['15', 'schmarrn']] > > > > > > > > Even if I add lots of main_function() > > calls to the script, I get valid output. > > > > Can you please try it with my modifications? > > > > > > > > Gerhard > > > > > > > > Von: Denis GERMAIN > > [mailto:dt.germ...@gmail.com] > > Gesendet: Mittwoch, 2. November 2011 18:08 > > An: shinken-devel@lists.sourceforge.net > > Betreff: Re: [Shinken-devel] Slight > > problem with livestatus module with direct queries > > > > > > > > Hi again, > > > > I might have been a little optimistic, > > because my prod crashed again. > > > > This time I used python code which has > > been adapted from the website from Livestatus conceptor, so I > > guess that this time the queries are correctly done > > > > This error is really strange, because > > it seems to be realted to the number of request I do in a > > short period of time. I managed to reproduce the error on my > > dev environment *relief*. Here, everything works fine until > > the broker crashes. > > > > I might be totally wrong here as I > > don't know much of Livestatus coding in Shinken, but is it > > possible that the maximum number of socket has been reached > > or something like that? > > > > Attached, you can see the scenario I > > used to reproduce the error : > > I first added comments on running hosts > > (in order to have a certain amount of hosts with comments on them) > > Then I launched my script > > Livestatus answered correctly for some > > hosts, then crashed, like on prod env > > > > If this is indeed a problem with max > > number of sockets in livestatus module, I could recode my > > script in order to avoid making too many queries (getting all > > hosts, then getting all comments, only 2 queries here), but > > I'd like to be sure it's really that in order to avoid > > further issues on prod environnement > > > > Regards > > > > Denis GERMAIN > > > > 2011/10/27 Olivier Hanesse > > <olivier.hane...@gmail.com> > > > > hi, > > > > > > > > I used to work with netcat too, and I > > got some issues in the past with some netcat version. > > > > On debian for example, you have 2 versions : > > > > > > > > netcat-openbsd - TCP/IP swiss army knife > > > > netcat-traditional - TCP/IP swiss army knife > > > > > > > > One was working, but not the other one. > > > > Maybe you could give a try and install > > the other version. > > > > > > > > Regards > > > > > > > > Olivier > > > > > > > > 2011/10/27 Denis GERMAIN <dt.germ...@gmail.com> > > > > Hi, > > > > Thanks for the answer, I had forgotten > > that we used netcat instead of unixcat. I have still to > > confirm this, but i'm think we have a problem because we're > > not correctly closing the connection with netcat. > > > > I'm going to code it in python in order > > to do this in a proper way. > > > > I'll keep you posted if this is not the > > issue but I'm pretty sure that this is our problem. > > > > Thanks! > > > > Denis > > > > 2011/10/26 Gerhard Lausser > > <gerhard.laus...@consol.de> > > > > Hi, > > > > > > > > which version of Shinken do you use? > > > > I have here a 0.8 and cant reproduce > > your error. Theres one host (omd-disco) which I made DOWN by > > submitting a passive command and which has two comments attached. > > > > > > > > OMD[disco]:~$ printf "GET > > hosts\nFilter: state = 1\nColumns: name comments\n"|unixcat > > /omd/sites/disco/tmp/run/live > > > > omd-disco;3,4 > > > > > > > > OMD[disco]:~$ printf 'GET > > comments\nColumns: comment_id comment\n' | unixcat > > /omd/sites/disco/tmp/run/live > > > > 3;i bi hi > > > > 4;i kos ned oft gnou sogn, i bi hi > > > > > > > > OMD[disco]:~$ printf 'GET > > comments\nColumns: comment_id comment\nFilter: comment_id = > > 3\n' | unixcat /omd/sites/disco/tmp/run/live > > > > 3;i bi hi > > > > > > > > Gerhard > > > > > > > > > > > > Von: Denis GERMAIN > > [mailto:dt.germ...@gmail.com] > > Gesendet: Mittwoch, 26. Oktober 2011 16:40 > > An: shinken-devel@lists.sourceforge.net > > Betreff: [Shinken-devel] Slight problem > > with livestatus module with direct queries > > > > > > > > Hello list, > > > > For historic reporting purpose, we have > > a developper in our team that directly execute queries to the > > livestatus module via commands like > > printf 'GET hosts\nFilter: state = 1' | > > nc 10.197.64.97 50000 > /data/ftptrace/nagios/thruk.txt > > printf 'GET comments\nFilter comment_id > > = '$commid | nc 10.197.64.97 50000 | tail -1 | awk -F";" > '{print $2}' > > > > This gives us informations about hosts > > down which are then reformatted in a dusty old report sent to > > our client. > > At the beginning we used only the first > > request for our report, as comments were directly available > > from the host information given by the query. After an > > update, we didn't realised that the comments were not > > directly in the query result, but only the comment ID (which > > I grant you is much more "clean") > > > > 1) Problem is that now, we need to > > query for the comments (selecting good comment ID) and that > > it constantly break livestatus module (which doesn't always > > manage to relaunch itself) > > > > 2) We also noted that when we use the > > Column statement (to select only the columns we need), the > > livestatus module instantly crash > > > > I realise that our problem is a little > > exotic, as the vast majority of livestatus users only use it > > through Thruk, but if something could be done (or as already > > been done), we would be grateful (our developper mostly :-p) > > > > Denis GERMAIN > > > > PS: An example traceback. I don't have > > much more yet, we are running Shinken on production (I > > already see admins frowning eyes, but it's stable enough if > > you don't try strange things like us) and my manager don't > > really like us crashing it on purpose. We'll try to reproduce > > on preproduction ASAP, but if this can help pinpointing the issue... > > > > [1319536549] [broker-1] Traceback (most > > recent call last): > > File > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b > roker/livestatus_broker.py", line 828, in main > > self.do_main() > > File > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b > roker/livestatus_broker.py", line 1052, in do_main > > response, keepalive = > > self.livestatus.handle_request(open_connections[socketid]['buffer']) > > File > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b > roker/livestatus.py", line 129, in handle_request > > request.parse_input(data) > > File > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b > roker/livestatus_request.py", line 96, in parse_input > > query.parse_input('\n'.join(query_cmds)) > > File > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b > roker/livestatus_query.py", line 259, in parse_input > > > self.filter_stack.put(self.make_filter(operator, attribute, > > reference)) > > File > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b > roker/livestatus_query.py", line 786, in make_filter > > converter = self.find_converter(attribute) > > File > > "/usr/lib/python2.6/site-packages/shinken/modules/livestatus_b > roker/livestatus_query.py", line 102, in find_converter > > out_map = LSout_map[self.out_map_name] > > KeyError: 'hosts' > > > > > > > > > > -------------------------------------------------------------- > > ---------------- > > The demand for IT networking > > professionals continues to grow, and the > > demand for specialized networking > > skills is growing even more rapidly. > > Take a complimentary Learning@Cisco > > Self-Assessment and learn > > about Cisco certifications, training, > > and career opportunities. > > http://p.sf.net/sfu/cisco-dev2dev > > _______________________________________________ > > Shinken-devel mailing list > > Shinken-devel@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/shinken-devel > > > > > > > > > > -------------------------------------------------------------- > > ---------------- > > The demand for IT networking > > professionals continues to grow, and the > > demand for specialized networking > > skills is growing even more rapidly. > > Take a complimentary Learning@Cisco > > Self-Assessment and learn > > about Cisco certifications, training, > > and career opportunities. > > http://p.sf.net/sfu/cisco-dev2dev > > _______________________________________________ > > Shinken-devel mailing list > > Shinken-devel@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/shinken-devel > > > > > > > > > > > > -------------------------------------------------------------- > > ---------------- > > The demand for IT networking > > professionals continues to grow, and the > > demand for specialized networking > > skills is growing even more rapidly. > > Take a complimentary Learning@Cisco > > Self-Assessment and learn > > about Cisco certifications, training, > > and career opportunities. > > http://p.sf.net/sfu/cisco-dev2dev > > _______________________________________________ > > Shinken-devel mailing list > > Shinken-devel@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/shinken-devel > > > > > > > > > > > > -------------------------------------------------------------- > > ---------------- > > RSA® Conference 2012 > > Save $700 by Nov 18 > > Register now! > > http://p.sf.net/sfu/rsa-sfdev2dev1 > > > > _______________________________________________ > > Shinken-devel mailing list > > Shinken-devel@lists.sourceforge.net > > > > https://lists.sourceforge.net/lists/listinfo/shinken-devel > > > > > > > > > > > > > > > > __________ Hinweis von ESET NOD32 Antivirus, > > Signaturdatenbank-Version 6695 (20111208) __________ > > > > > > E-Mail wurde geprüft mit ESET NOD32 Antivirus. > > > > > > http://www.eset.com > > > > > > > __________ Hinweis von ESET NOD32 Antivirus, > Signaturdatenbank-Version 6695 > (20111208) __________ > > E-Mail wurde geprüft mit ESET NOD32 Antivirus. > > http://www.eset.com > > > > __________ Hinweis von ESET NOD32 Antivirus, > Signaturdatenbank-Version 6695 (20111208) __________ > > E-Mail wurde geprüft mit ESET NOD32 Antivirus. > > http://www.eset.com > > __________ Hinweis von ESET NOD32 Antivirus, Signaturdatenbank-Version 6695 (20111208) __________ E-Mail wurde geprüft mit ESET NOD32 Antivirus. http://www.eset.com ------------------------------------------------------------------------------ Cloud Services Checklist: Pricing and Packaging Optimization This white paper is intended to serve as a reference, checklist and point of discussion for anyone considering optimizing the pricing and packaging model of a cloud services business. Read Now! http://www.accelacomm.com/jaw/sfnl/114/51491232/ _______________________________________________ Shinken-devel mailing list Shinken-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shinken-devel
