[Sorry if the discussion on RRD is off topic with respect to OWFS]
I think that when you are collecting up to a few hundreds different metrics,
sqlite + any scripting language + AJAX is fine (and probably the best option.)
When you go into the range of 1k up to 100k different metrics, collected at 15s
intervals, then something like ganglia + rrd (or better ganglia + graphite)
comes into play.
This does not mean that if you like the rrdtool way you cannot employ it also
on small scale projects, for collecting just a few metrics at 300s intervals.
As Colin correctly says, rrd is very rigid, everything is hardcoded into the
.rrd file, and you need to carefully plan what data do you want to store, at
which frequencies, and how long it should persist. Therefore it is also very
important that you have a single file for each sensor in your network:
otherwise when you add a new sensor to your network you have to recreate the
.rrd file, and probably throw away your past data. (Not completely true, of
course: you can always dump the rrd content into xml, map the data to the new
.rrd structure and populate the new .rrd file; but this procedure is a real
pain…)
S.
> On 30 Mar 2015, at 19:41, Colin Reese <[email protected]> wrote:
>
> Speed? Wrt what? Db reads/writes? Db queries are vanishingly small time wise,
> I've found, especially compared to the ajax POST or GET.
>
> To answer the OP's question, for graphing I use jqplot. This way my client
> does all the processing, and they can customize their plots on the fly rather
> than serving a static image. There are other packages that perform similar
> functions.
>
> For the data acquisition, I just run a python daemon with sqlite databases.
> Nice part here is that you can store acquisition parameters in tables and
> change them as you see fit, rather than hard-coding them. This also totally
> decouples acquisition and visualization.
>
> To get data to the client there are also many options, but I use ajax/wsgi,
> either with mod_wsgi on Apache or uwsgi on nginx.
>
> C
>
>
>
>
> On Mar 30, 2015, at 10:05 AM, Stefano Miccoli <[email protected]
> <mailto:[email protected]>> wrote:
>
>> The only reason I can see is speed, and compatibility with (big) apps like
>> cacti, nagios, or ganglia.
>>
>> S.
>>
>>> On 30 Mar 2015, at 01:00, Colin Reese <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>> Out of curiosity, why would you use this? Using a database with accurate
>>> time stamps and log sizing is trivial, and apis for beautiful html plots
>>> are easy to come by.
>>>
>>> C
>>>
>>>
>>>
>>> On Mar 29, 2015, at 3:42 PM, Stefano Miccoli <[email protected]
>>> <mailto:[email protected]>> wrote:
>>>
>>>>
>>>>> On 29 Mar 2015, at 15:32, [email protected]
>>>>> <mailto:[email protected]> wrote:
>>>>>
>>>>> rrdtool create house.rrd --start N --step 300 \DS:garageIN:GAUGE:600:U:U \
>>>>> DS:garageRTN:GAUGE:600:U:U \
>>>>> DS:bedroomIN:GAUGE:600:U:U \
>>>>> DS:bedroomRTN:GAUGE:600:U:U \
>>>>> DS:floorIN:GAUGE:600:U:U \
>>>>> DS:floorRTN:GAUGE:600:U:U \
>>>>> DS:watertankIN:GAUGE:600:U:U \
>>>>> DS:watertankRTN:GAUGE:600:U:U \
>>>>> DS:loopIN:GAUGE:600:U:U \
>>>>> DS:loopRTN:GAUGE:600:U:U \
>>>>> DS:headerIN:GAUGE:600:U:U \
>>>>> DS:headerRTN:GAUGE:600:U:U \
>>>>> DS:basementIN:GAUGE:600:U:U \
>>>>> DS:basementRTN:GAUGE:600:U:U \
>>>>> DS:mechanicalRM:GAUGE:600:U:U \
>>>>> RRA:AVERAGE:0.5:1:12 \
>>>>> RRA:AVERAGE:0.5:1:288 \
>>>>> RRA:AVERAGE:0.5:12:168 \
>>>>> RRA:AVERAGE:0.5:12:720 \
>>>>> RRA:AVERAGE:0.5:288:365
>>>>
>>>> In fact having rrdtool working is quite complicated.
>>>>
>>>> Thirst thing to note: you should run “rrdtool update” at about a 300s
>>>> interval (—step 300). If you fail to run ‘rrdupdate’ for 600s (see 600 in
>>>> 'DS:garageIN:GAUGE:600:U:U’) the data point will be marked ‘unknown’ (and
>>>> nothing will show up in your graphs.)
>>>>
>>>> Second thing: RRA:AVERAGE:0.5:1:12 and RRA:AVERAGE:0.5:12:168 are
>>>> redundant, since you save also longer RRA’s at the same frequency
>>>> (RRA:AVERAGE:0.5:1:288 and RRA:AVERAGE:0.5:12:720).
>>>>
>>>> Third thing: if you are interested in plotting MIN/MAX graphs, you should
>>>> also add RRA:MIN… and RRA:MAX definitions in your rrd.
>>>>
>>>> Now suppose that the values in OWFS are correct. The best way to check
>>>> what got into your rrd is “rrdtool dump house.rrd” which will output and
>>>> XML file with the exact content of your rrd. If you are interested in the
>>>> last 24h of data, you can also issue ‘rrdtool fetch house.rrd AVERAGE
>>>> —start -24h and so on. Once you confirm that data is correctly stored in
>>>> rrd, then you can start plotting graphs.
>>>>
>>>> A final comment: AFAIK, while having a single rrd file for all your
>>>> sensors is fine, this is rather an uncommon choice. Usually you define a
>>>> file for each sensor, in order to break the read/update loop in smaller
>>>> chunks; e.g.
>>>>
>>>> rrdtool update Garage_Return.rrd N:`cat
>>>> /mnt/owfs/uncached/Garage_Return/temperature`
>>>>
>>>> and so on… possibly in a bash for-loop over all sensors.
>>>>
>>>> Bye,
>>>>
>>>> Stefano
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>>> sponsored
>>>> by Intel and developed in partnership with Slashdot Media, is your hub for
>>>> all
>>>> things parallel software development, from weekly thought leadership blogs
>>>> to
>>>> news, videos, case studies, tutorials and more. Take a look and join the
>>>> conversation now. http://goparallel.sourceforge.net/
>>>> <http://goparallel.sourceforge.net/>_______________________________________________
>>>> Owfs-developers mailing list
>>>> [email protected]
>>>> <mailto:[email protected]>
>>>> https://lists.sourceforge.net/lists/listinfo/owfs-developers
>>>> <https://lists.sourceforge.net/lists/listinfo/owfs-developers>
>>> ------------------------------------------------------------------------------
>>> Dive into the World of Parallel Programming The Go Parallel Website,
>>> sponsored
>>> by Intel and developed in partnership with Slashdot Media, is your hub for
>>> all
>>> things parallel software development, from weekly thought leadership blogs
>>> to
>>> news, videos, case studies, tutorials and more. Take a look and join the
>>> conversation now.
>>> http://goparallel.sourceforge.net/_______________________________________________
>>>
>>> <http://goparallel.sourceforge.net/_______________________________________________>
>>> Owfs-developers mailing list
>>> [email protected]
>>> <mailto:[email protected]>
>>> https://lists.sourceforge.net/lists/listinfo/owfs-developers
>>> <https://lists.sourceforge.net/lists/listinfo/owfs-developers>
>>
>> ------------------------------------------------------------------------------
>> Dive into the World of Parallel Programming The Go Parallel Website,
>> sponsored
>> by Intel and developed in partnership with Slashdot Media, is your hub for
>> all
>> things parallel software development, from weekly thought leadership blogs to
>> news, videos, case studies, tutorials and more. Take a look and join the
>> conversation now. http://goparallel.sourceforge.net/
>> <http://goparallel.sourceforge.net/>_______________________________________________
>> Owfs-developers mailing list
>> [email protected]
>> <mailto:[email protected]>
>> https://lists.sourceforge.net/lists/listinfo/owfs-developers
>> <https://lists.sourceforge.net/lists/listinfo/owfs-developers>
> ------------------------------------------------------------------------------
> Dive into the World of Parallel Programming The Go Parallel Website, sponsored
> by Intel and developed in partnership with Slashdot Media, is your hub for all
> things parallel software development, from weekly thought leadership blogs to
> news, videos, case studies, tutorials and more. Take a look and join the
> conversation now.
> http://goparallel.sourceforge.net/_______________________________________________
> Owfs-developers mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/owfs-developers
------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Owfs-developers mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/owfs-developers
