It may be worth having a look into highcharts/highstock which does a lot of optimizations related to large data sets:

http://www.highcharts.com/stock/demo/data-grouping
http://www.highcharts.com/stock/demo/lazy-loading

I'm using highcharts in our apps successfully with qooxdoo, but not with such large data sets.

Regards
Dietrich


Am 14.07.2015 um 14:36 schrieb Vishal:
Phyo, Thanks for the links.

Dear friends,

I want to switch to a general discussion on handling large data sets for drawing charts. I have used jqplot metergauge renderer with live data and have also included easing functions to make the gauge look like an actual analog meter. Myself and product reviewers are quite satisfied with the end result. So from that I experience, I think creating live charts (running) charts with jqplot should satisfy our purpose. But the meter gauge renderer takes exactly 1 data point everytime (x = 1, and y = actual location of the pointer)

The big difference with line charts is that the data set contains multiple points. Currently my requirement is about *5000 points / series*. There are upto*7 series* that need to be visible at any time. Ofcourse the actual data set that needs to be stored in memory (either at the browser or at the server) is larger..its about 200,000 points/series for 7 such series.

My strategy is to have an Array() (created like so: this.series1 = [];) for each actual data set. The max length is always chosen by the user from the chart controls. Then we make slices from this data set and send them to chart.redraw(). However, when the number of data samples exceeds the actual data set length, we need to do something like this, everytime we get new data from the server.

What do you think of this?

dataset.shift();
dataset.push(new_data);

the above is prohibitively long for most interactions. On Chrome console, doing the above for a 200000 element array is of the order of 1s or more average case...and sometimes when there are too many tabs open...it has taken more than 10s.

The real issue I think is that Array() operations like push(), shift() even though highly optimized, are still are O(N) operations. if Array() was implemented using a linked list...could this be easier. if yes, are there any options like this?

Thanks and best regards,
Vishal Sapre




Thanks and best regards,
Vishal Sapre

---
Please DONT print this email, unless you really need to. Save Energy & Paper. Save the Earth.

On Mon, Jul 13, 2015 at 10:10 PM, Tobias Oetiker <t...@oetiker.ch <mailto:t...@oetiker.ch>> wrote:

    Today Phyo Arkar wrote:

    > https://github.com/oetiker/RrdGraphJS

    note that the documentation part is maynly for the Png version ...

    cheers
    tobi

    >
    > On Mon, Jul 13, 2015 at 8:29 PM, Vishal <vsapr...@gmail.com
    <mailto:vsapr...@gmail.com>> wrote:
    >
    > > Thanks Tobias,
    > >
    > > Is there a demo for this grapher that you mention.
    > >
    > > Best regards,
    > > Vishal Sapre
    > >
    > >
    > >
    > > Thanks and best regards,
    > > Vishal Sapre
    > >
    > > ---
    > > Please DONT print this email, unless you really need to. Save
    Energy &
    > > Paper. Save the Earth.
    > >
    > > On Mon, Jul 13, 2015 at 1:54 PM, Tobias Oetiker
    <t...@oetiker.ch <mailto:t...@oetiker.ch>> wrote:
    > >
    > >> Hi Vishal,
    > >>
    > >> I have not ventured into using jqplot for anything 'live'
    > >>
    > >> I have written a live grapher with qooxdoo pretty
    specialized, but
    > >> maybe this can give you some insporation ...
    > >>
    > >>
    https://github.com/oetiker/RrdGraphJS/blob/master/public/rrdGraphSvg.js
    > >>
    > >> it is based on D3.js since this seems to be much more
    flexible than
    > >> jqplot.
    > >>
    > >> cheers
    > >> tobi
    > >>
    > >> Yesterday Vishal wrote:
    > >>
    > >> > Dear Friends,
    > >> >
    > >> > I have been using qxjqplot for running meters (gauges)
    > >> > ​in my single page qx.desktop app.
    > >> >
    > >> >
    > >> > Now I wish to use a charting library to show running live
    data (like a
    > >> time
    > >> > series) on a chart. Going forward with qxjqplot, I have
    come to a point
    > >> > where data can be shown on a chart very well. However,
    since the data is
    > >> > always coming in fresh, I can only show a part of it while
    keeping most
    > >> of
    > >> > the data in a background array. The user may want to stop
    the charting,
    > >> and
    > >> > go back to some point (using a scroll bar). There is no
    such facility in
    > >> > jqplot, while there is something called a zoom
    proxy...which is like
    > >> > this: http://www.jqplot.com/deploy/dist/examples/zoomProxy.html
    > >> > <http://www.jqplot.com/deploy/dist/examples/zoomProxy.html>
    > >> > However, it does not really work like a scroll bar. in
    addition the
    > >> > controller plot itself will be getting filled in all the time.
    > >> > To use this function I have to call something like this:
    > >> > $.jqplot.Cursor.zoomProxy(targetPlot, controllerPlot);
    > >> >
    > >> > How do i call this function from qx? The qxjqplot.MPlot
    mixin does not
    > >> > expose jquery.jqplot object, or does it?
    > >> >
    > >> > Can you please point me to the right direction so as to use
    the above
    > >> > functionality from qxjqplot?
    > >> >
    > >> > What would be your suggestion to have an x axis scroll bar
    to move about
    > >> > the time axis?
    > >> >
    > >> > I also looked at the qxdygraph option. Unfortunately there
    is no hint
    > >> as to
    > >> > how to use DyGraphs with qooxdoo. The demo given for
    qxdygraph is good,
    > >> but
    > >> > I dont see an option to look at the actual demo code.
    > >> >
    > >> > Your help would be seminal for me to complete this tool.
    > >> >
    > >> > Thanks and best regards,
    > >> > Vishal Sapre
    > >> >
    > >> > p.s: Please find attached my current chart tab with some
    random data in
    > >> it.
    > >> >
    > >>
    > >> --
    > >> Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten,
    Switzerland
    > >> www.oetiker.ch <http://www.oetiker.ch> t...@oetiker.ch
    <mailto:t...@oetiker.ch> +41 62 775 9902 <tel:%2B41%2062%20775%209902>
    > >>
    > >>
    > >>
    
------------------------------------------------------------------------------
    > >> Don't Limit Your Business. Reach for the Cloud.
    > >> GigeNET's Cloud Solutions provide you with the tools and
    support that
    > >> you need to offload your IT needs and focus on growing your
    business.
    > >> Configured For All Businesses. Start Your Cloud Today.
    > >> https://www.gigenetcloud.com/
    > >> _______________________________________________
    > >> qooxdoo-devel mailing list
    > >> qooxdoo-devel@lists.sourceforge.net
    <mailto:qooxdoo-devel@lists.sourceforge.net>
    > >> https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
    > >>
    > >>
    > >
    > >
    > >
    
------------------------------------------------------------------------------
    > > Don't Limit Your Business. Reach for the Cloud.
    > > GigeNET's Cloud Solutions provide you with the tools and
    support that
    > > you need to offload your IT needs and focus on growing your
    business.
    > > Configured For All Businesses. Start Your Cloud Today.
    > > https://www.gigenetcloud.com/
    > > _______________________________________________
    > > qooxdoo-devel mailing list
    > > qooxdoo-devel@lists.sourceforge.net
    <mailto:qooxdoo-devel@lists.sourceforge.net>
    > > https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel
    > >
    > >
    >

    --
    Tobi Oetiker, OETIKER+PARTNER AG, Aarweg 15 CH-4600 Olten, Switzerland
    www.oetiker.ch <http://www.oetiker.ch> t...@oetiker.ch
    <mailto:t...@oetiker.ch> +41 62 775 9902 <tel:%2B41%2062%20775%209902>

    
------------------------------------------------------------------------------
    Don't Limit Your Business. Reach for the Cloud.
    GigeNET's Cloud Solutions provide you with the tools and support that
    you need to offload your IT needs and focus on growing your business.
    Configured For All Businesses. Start Your Cloud Today.
    https://www.gigenetcloud.com/
    _______________________________________________
    qooxdoo-devel mailing list
    qooxdoo-devel@lists.sourceforge.net
    <mailto:qooxdoo-devel@lists.sourceforge.net>
    https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel




------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/


_______________________________________________
qooxdoo-devel mailing list
qooxdoo-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel

------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
_______________________________________________
qooxdoo-devel mailing list
qooxdoo-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/qooxdoo-devel

Reply via email to