Hi All, The current code on my branch (upgrade-tools-1.12), contains the following:
1) The online version of the tool for fixing SNMP interface directories (NMS-6056). This is the only implementation enabled. 2) The offline version of the tool for fixing SNMP interface directories (NMS-6056). This is currently disabled, as it requires to request the ifPhysAddr from all the interfaces from all nodes and update the DB prior execute the same operation as the online version. 3) The offline version of the tool for fixing the JMX JRBs/RRDs, data sources, etc. (NMS-1539, NMS-3485, NMS-4592, NMS-4612, NMS-5247, NMS-5279, NMS-5824). This is currently disabled because it requires more tests and implement something to fix the graph templates and metric definitions without any other external tool. About the online update: In theory, if Queued is trying to update a file at the same time the tool is trying to do the same, Queued is going to throw an exception and maybe a small hole will be on the graphs (because the data won't be stored for that timestamp). Other than that I'm not expecting something else but I could be wrong. For all the people that wants to try it on a test server, please let me know and I can give the details about how to do that. Les, about your questions: On Oct 10, 2013, at 12:26 PM, Les Mikesell <lesmikes...@gmail.com> wrote: > On Thu, Oct 10, 2013 at 10:23 AM, Alejandro Galue <aga...@opennms.org> wrote: >> >> My feeling is that if we merge the RRD files online, this could cause >> hundreds (or thousands) of errors on Queued (and queued.log). The worst case >> is what could happen if Queued is updating a file while the Upgrading task >> is replacing the file at the same time (here is where I'm not sure what >> could happen). > > Is there any way to tell Queued to flush a specific file and skip > updates until further notice - and then re-open it? No. >> I'm working on both solutions at the same time, I mean, the offline version >> and the online version of the SNMP Interface Upgrader. Then, we can decide >> which should be the final version to use (as I've created an annotation to >> ignore certain implementations of the OnmsUpgrade interface at runtime). > > What kind of timing do you anticipate? I have about 65k rrd files > with storegroup - about 20GB to rewrite. The current implementation basically dumps the data as XML for each old and new file (see "rrdtool dump"), no matter if you're using JRobin or RRDtool, and then navigate through each datapoint defined on each RRA and perform the merge if the data on the old file is not null. I don't have a large environment to test, but the operation is really quick in both cases (i.e., JRobin and RRDtool), but I don't have numbers or enough data to answer how long it should take. >> (B) Can this approach and/or tool be generalized for other rrd edit >> tasks like removing spikes or merging histories of continuing services >> that OpenNMS sees on different devices/interfaces? >> >> >> I believe so, this can be an interesting use case, anyone who wants to >> extend the updater to perform additional operations can do that without >> problems, you just need to create a class (on any package inside >> org.opennms.upgrade) that implement the OnmsUpgrade interface, and place the >> JAR on /opt/opennms/lib, and that's it. > > I was thinking more of a way to tell opennms you are going to > externally modify one of the rrd files and when you are done than > embedding everything. I don't think that this tool can do that, but the code that I made to manipulate RRDtool files and JRobin files could help. I made the JAXB version of an RRDtool dump and RRDtool xport. Because JRobin is a small sub-set of all the features that RRDtool supports, the JAXB representation is different in both cases. For RRDtool files, I've created a RRDv3 object. For JRobin I've created a RRDv1 object. I've also added methods and helper functions to migrate from RRDv3 to RRDv1 (of course, if you are not using advanced features like computed data sources, or aberrant behavior detection), from RRDv1 to RRDv3, for merge several single-ds RRDs/JRBs into a multi-ds, to split a mult-ds file into a single-ds files, etc. Basically, the code that I made with the dump/xport representation can give us the opportunity to convert from JRobin world to RRDtool world and viceversa no matter if you're using storeByGroup or not, and because it was implemented with JAXB we could make a ReST API to export raw data (dump) or pre-processed data (xport) no matter if you're using RRDtool or JRobin. Of course, these are just ideas about what we can do with the new RRD code that I made. Alejandro.
signature.asc
Description: Message signed with OpenPGP using GPGMail
------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60135991&iu=/4140/ostg.clktrk
_______________________________________________ Please read the OpenNMS Mailing List FAQ: http://www.opennms.org/index.php/Mailing_List_FAQ opennms-devel mailing list To *unsubscribe* or change your subscription options, see the bottom of this page: https://lists.sourceforge.net/lists/listinfo/opennms-devel
