Hi Jim, Yes, LOL ... and it gets worse when we try to plot a graph for one year (about 500,000 observatons). We are plotting values received from sensors (eg temperature, vibration, humidity, gps location, gps velocity, wind speed, current, voltage, light). We take readings each minute. I have produced a dashboard which uses the following techniques: 1. Each sensor graph is in its own window ... so can be looking at 10 graphs at the same time for instance. 2. I average data to reduce the dataset size 3. I employ drill down 4. I have included a slider to look at reduced time frames The client rejected this solution. I did some tests in flex and found results similar to you Jim. When I tried to plot 40,000 points I gave up waiting for it after 6 minutes. 10,000 points took about 20 secs the first time (including reading from the local sqlite data store). The client and I have now agreed that:
1. If we have a large number of observations from a sensor and plot all of those on a graph over a period of time, then whatever optimisation we use to improve performance should produce an identical looking graph to the first. (I think that identical is probably to strong a word here) 2. We will add a confif paramter for each sensor such that the sensor readings can be changed to occur at multiples of 1 minute. For example, every 10 minutes. So, I will still end up with data sets to large for flex, and given that I want to display many graphs at once and will also be looking at representing many sensors on the one graph this contributes to the issue. So, I do need to sample the dataset somehow to make it work. But have to try to meet the goal expressed in point 1 above. I think a combination of your points 3 and 4 might be a good approached combined with some algorithm to ensure that my top level curve does not miss an max's and min's and as close as possible approximates a full detail curve. A few challenges there! Thanks again, Mark _____ From: flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] On Behalf Of jim.abbott45 Sent: Wednesday, November 19, 2008 8:27 AM To: flexcoders@yahoogroups.com Subject: [flexcoders] Re: Creating a Chart With 50,000 Data Points 50,000? LOL. ;-) On a more serious note, I have to agree with Fotis and Ricky that 50K data points is too many, both from a (Flash 9 VM at least) performance perspective and probably also from an Information Visualization perspective (unless, maybe, your users have 300 dpi monitors). There are several standard interaction design and information visualization techniques which may be useful to you . . . 1) Aggregation (as already suggested, take 'N' data points, average them, and then only display the average value). 2) Filtering (by sliders on an axis, by date/time, by structured or open-ended queries, ...) 3) Progressive rendering (sample the entire data set at intervals of 'N', render those points, then go back and get the data which is mid-point between the original points, add the new points to the curve, repeat until desired resolution obtained--or on user demand) 4) Non-linear zooming (like idea 1), but allow user to zoom in on a section of the curve, when they do, add the additional data points to the curve for the zoomed-in region) On a more pragmatic note, you certainly _can_ plot more than 2,000 points. For example, I've personally used Flex to create charts with over 14,000 points in them, so I know that at least that number is definitely *possible*. However, the overall rendering time was (as I recall) over 3 minutes long. More troubling was the fact that the entire Flex UI become very sluggish once a chart that large was displayed. I've seen good chart rendering speed ( ~= 2S ) and no UI sluggishness, up to about 1500 points. I've also been able to render up to 6,000 points, but it took about 10S for the rendering and the UI started to become sluggish. From your posting, it sounds like you're not seeing those rendering speeds. I'd recommend that you experiment with how you are using the charting objects/API. My experience was that I was able to see some very noticeable speed-ups simply be optimizing how I called the charting API (especially w.r.t. how I loaded the series data). In particular, if your charts are interactive, try to cache anything you can that doesn't necessarily change during an interaction (i.e., axis objects). Last thought: the new Flash 10 VM (Astro) has support for larger bitmaps and also can now do hardware-based rendering, apparently. I haven't tried either of those features myself, but using the v10 VM _might_ be another way to get a performance boost, etc. Good luck! Jim --- In [EMAIL PROTECTED] <mailto:flexcoders%40yahoogroups.com> ups.com, "Mark Easton" <[EMAIL PROTECTED]> wrote: > > Hi, > > It appears that Flex Charts cannot handle generating charts with large > DataSets. We tried with 50,000 data points and it thrashed away without > producing a result after 6 minutes. It was able to plot 2,000 points in > about 25 seconds. > > What is the recommended approach for creating charts from large data sets. > The best I can think of is to write some code that will reduce the data set > in size yet still provide enough data to represent the graph accurately. > > Thoughts? > > Ta. > Mark > No virus found in this incoming message. Checked by AVG - http://www.avg.com Version: 8.0.175 / Virus Database: 270.9.4/1795 - Release Date: 11/18/2008 11:23 AM
