Hi Peter, Today Peter Stamfest wrote:
> Tobias Oetiker <t...@oetiker.ch> schrieb am 23.02.2010 23:28:53: > > > Hi Peter, > > > > wow ... sounds impressive. Have you done performance evaluations ? > > > > Only insofar, as it is fast. Accessing RRDs in /dev/shm is just "wow". I > wanted to do my first real deployment of the software yesterday, and then > I stumbled across rrdcached. And my motivation just vanished :-( > > > If you could apply your genius to rrdcached and maybe make it even > > better based on the insights gained from the rrdfs implementation? > > Well, I had some plans for rrdfs: > > - Provide a memory only version of rrd_update_r. Then I could get rid of > the staging area and just cache RRDs entirely in memory - while still > providing file-level access to them. > - The write ahead logging rrdfs is able to do would allow for proper > restart of a crashed/killed/whatever rrdfs in such a scenario (but it > would also help for the staging-area case). > - Make it NFS capable: Access RRDs handled by rrdfs from a remote host > using normal file access. This requires a recent kernel and has some > drawbacks, though. But this would allow for HUGE RRD repositories - > distributed over any number of NFS servers. > - It would even be possible to integrate more of rrdtool commands into > rrdfs. Just imagine a mechanism to just access a png file by name and have > it automagically computed from memory. It would even be possible to > provide "pseudo" directories for every RRD, eg. containing RRD data as CSV > "files". The sky is the limit (or the avoidance of feature creep)... > > > I don't know the inner workings of rrdcached, but rrdfs could actually > become part of it, I think (#ifdef'd on the availability of FUSE). The > question is: Is this something you would accept? > > > So a plan could be: > - factor out file access from rrd_update_r and provide a rrd_update_mem_r > as a base for rrd_update_r and become memory only (with the transparent > permanent storage handling). > - integrate the IPC handling from rrdcached into rrdfs or vice-versa. > This MIGHT be as simple as just modifying the main() from rrdfs to fire up > a single pthread and have that take care of rrdfs and all the rest would > be handled by the existing rrdcached (plus the backend storage/update > caching would have to be integrated - in that area I'm using a hashtable > keyed on the file name essentially containing a dynamic array of updates > per file). well I guess doing fuse based stuff is funky but since rrdtool is used on a wide variety of OSes I am not as enthusiastic about it as I would be about our crossplatfom rrdcached :-) ... eventhough the interface is different, the functionality is pretty similar ... what is missing are the following things * authentication/authorization and encryption for network links * in-memory updates, so that functions like updatev would work too. when it comes to in-memory updates there is probably a tie-in with the planned move from a machine specific data format to a cross-platform dataformat which will require some additional abstraction layer anyway. cheers tobi > > peter > > > > > > > Today Peter Stamfest wrote: > > > > [.. removed ..] > > > _________________________________________________________________________ > Dipl.-Ing. Peter Stamfest UNIX, Networking & Computing Consultant > Tel: +43/699/10711205 Software Development - Internetservices > E-Mail: pe...@stamfest.at WWW: http://stamfest.at/ > -- Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland http://it.oetiker.ch t...@oetiker.ch ++41 62 775 9902 / sb: -9900 _______________________________________________ rrd-developers mailing list rrd-developers@lists.oetiker.ch https://lists.oetiker.ch/cgi-bin/listinfo/rrd-developers