Hi, On Fri, Aug 28, 2015 at 9:18 PM, Srinath Perera <[email protected]> wrote:
> Correct approch (for Viz) would be to sample it down rather than taking > first 1000 ( but not for DAL data retrival case). Any chance we can do that? > That also, should be done case by case for graph types I guess, for example, it doesn't make sense to sample data for a bar graphs, like some bars won't be visible! .. Cheers, Anjana. > > On Fri, Aug 28, 2015 at 7:12 PM, Dunith Dhanushka <[email protected]> wrote: > >> Hi Anjana, >> >> I'll work on the above change. >> >> Regards, >> Dunith >> >> On Fri, Aug 28, 2015 at 8:22 AM, Anjana Fernando <[email protected]> wrote: >> >>> Hi Srinath, >>> >>> I was checking on this, and it seems, the issue is, if you've a large >>> result set, and for the gadget, if you select this large result set as its >>> source data, it will go and read everything. For example, if you use the >>> smart home sample and just put all the raw records as the X axis of a bar >>> chart, it will try to read all thousands of data records and try to render >>> it. But this operation takes a bit of time, to do the JSON serialization >>> and send it to the front end. And what makes this far more worse is, the >>> dashboard automatically refreshes the data, and this happens in an >>> asynchronous manner, and it doesn't wait for the earlier cycle to be >>> finished. So these requests effectively builds up and eats up the CPU in >>> the backend and can end up going OOM. >>> >>> So I guess, first, we have to fix the refreshing of the gadgets somehow, >>> so it will wait for the earlier request to finish, before making new ones. >>> Then after that, we should also provide a sensible upper limit to the data >>> records queries, so people won't mistakenly query millions of records from >>> a raw data table. We have to do this carefully, because for a thing like a >>> scatter plot, we may need to read thousands of records to render the image. >>> So for things like that, it maybe easier for it to be rendered in the >>> backend and sent to the front end. And also, Maninda also pointed out, >>> since we have a natural refresh interval for a dashboard, we should also >>> cache the data in the end backend, with a cache validity period similar to >>> the dashboard interval, so we don't have to read data from the physical >>> databases all the time. >>> >>> So as an immediate fix, shall we try to change the dashboard refresh >>> logic and also put a upper limit to the backend data retrieval to about >>> 1000 (by changing "LAST_INDEX" variable from -1 to 100 in analytics.jag at >>> carbon-dashboards). @Dunith, will you be able to do these changes. >>> >>> Cheers, >>> Anjana. >>> >>> >>> On Fri, Aug 28, 2015 at 3:07 PM, Anjana Fernando <[email protected]> >>> wrote: >>> >>>> Hi Srinath, >>>> >>>> Yeah, I also could reproduce the issue. Will check it out. >>>> >>>> Cheers, >>>> Anjana. >>>> >>>> On Fri, Aug 28, 2015 at 2:38 PM, Srinath Perera <[email protected]> >>>> wrote: >>>> >>>>> This was a batch chart. >>>>> >>>>> On Fri, Aug 28, 2015 at 2:32 PM, Damith Wickramasinghe < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> AFAIK That message "Data is not available for plotting. The chart >>>>>> will be loaded once the dashboard receives events" normally comes for >>>>>> real >>>>>> time flow gadgets. >>>>>> >>>>>> Regards, >>>>>> Damith. >>>>>> >>>>>> On Fri, Aug 28, 2015 at 2:22 PM, Srinath Perera <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Yes, I could see the preview ok. >>>>>>> >>>>>>> --Srinath >>>>>>> >>>>>>> On Fri, Aug 28, 2015 at 2:19 PM, Anjana Fernando <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Srinath, >>>>>>>> >>>>>>>> Did you have data in that table before you created the gadget? .. >>>>>>>> >>>>>>>> Cheers, >>>>>>>> Anjana. >>>>>>>> >>>>>>>> On Fri, Aug 28, 2015 at 2:16 PM, Srinath Perera <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> It is fixed with new pack. >>>>>>>>> >>>>>>>>> However, when I create a Gadget ( based on beatch data) and add >>>>>>>>> teh Gadget, it says "Data is not available for plotting. The chart >>>>>>>>> will be >>>>>>>>> loaded once the dashboard receives events". >>>>>>>>> >>>>>>>>> Thanks >>>>>>>>> Srinath >>>>>>>>> >>>>>>>>> On Fri, Aug 28, 2015 at 8:58 AM, Srinath Perera <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Yesterday packs had the same problem >>>>>>>>>> >>>>>>>>>> On Thu, Aug 27, 2015 at 4:20 PM, Maninda Edirisooriya < >>>>>>>>>> [email protected]> wrote: >>>>>>>>>> >>>>>>>>>>> This was fixed in https://wso2.org/jira/browse/DAS-84 on >>>>>>>>>>> yesterday by Dunith. Now the fix should be there in latest jenkins >>>>>>>>>>> built >>>>>>>>>>> packs. >>>>>>>>>>> Thanks. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> *Maninda Edirisooriya* >>>>>>>>>>> Senior Software Engineer >>>>>>>>>>> >>>>>>>>>>> *WSO2, Inc.*lean.enterprise.middleware. >>>>>>>>>>> >>>>>>>>>>> *Blog* : http://maninda.blogspot.com/ >>>>>>>>>>> *E-mail* : [email protected] >>>>>>>>>>> *Skype* : @manindae >>>>>>>>>>> *Twitter* : @maninda >>>>>>>>>>> >>>>>>>>>>> On Thu, Aug 27, 2015 at 4:16 PM, Srinath Perera < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> is this fixed? >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Aug 25, 2015 at 4:41 PM, Maninda Edirisooriya < >>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Aug 25, 2015 at 3:28 PM, Srinath Perera < >>>>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Using yesterday's pack >>>>>>>>>>>>>> >>>>>>>>>>>>>> 1. Can we make creating a receiver part of the creating/ >>>>>>>>>>>>>> editing new streams flow? e.g. by asking what transports to >>>>>>>>>>>>>> expose and >>>>>>>>>>>>>> automatically creating it. >>>>>>>>>>>>>> >>>>>>>>>>>>>> 2. In the gadget generation wizard, x, y axis drop downs are >>>>>>>>>>>>>> now not be populated. >>>>>>>>>>>>>> >>>>>>>>>>>>> Yes this issue was observed in latest packs. I am looking at >>>>>>>>>>>>> it. >>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> [image: Inline image 1] >>>>>>>>>>>>>> >>>>>>>>>>>>>> 3. When you go into Gadget design view, there is no button to >>>>>>>>>>>>>> come back. >>>>>>>>>>>>>> >>>>>>>>>>>>>> [image: Inline image 2] >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> Srinath >>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> ============================ >>>>>>>>>>>>>> Blog: http://srinathsview.blogspot.com >>>>>>>>>>>>>> twitter:@srinath_perera >>>>>>>>>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>>>>>> Phone: 0772360902 >>>>>>>>>>>>>> >>>>>>>>>>>>>> _______________________________________________ >>>>>>>>>>>>>> Dev mailing list >>>>>>>>>>>>>> [email protected] >>>>>>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> ============================ >>>>>>>>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>>>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>>>> Phone: 0772360902 >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> ============================ >>>>>>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>>> Phone: 0772360902 >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> ============================ >>>>>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>>>> Phone: 0772360902 >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> *Anjana Fernando* >>>>>>>> Senior Technical Lead >>>>>>>> WSO2 Inc. | http://wso2.com >>>>>>>> lean . enterprise . middleware >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> ============================ >>>>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>>>> Site: http://people.apache.org/~hemapani/ >>>>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>>>> Phone: 0772360902 >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Software Engineer >>>>>> WSO2 Inc.; http://wso2.com >>>>>> <http://www.google.com/url?q=http%3A%2F%2Fwso2.com&sa=D&sntz=1&usg=AFQjCNEZvyc0uMD1HhBaEGCBxs6e9fBObg> >>>>>> lean.enterprise.middleware >>>>>> >>>>>> mobile: *+94728671315 <%2B94728671315>* >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> ============================ >>>>> Blog: http://srinathsview.blogspot.com twitter:@srinath_perera >>>>> Site: http://people.apache.org/~hemapani/ >>>>> Photos: http://www.flickr.com/photos/hemapani/ >>>>> Phone: 0772360902 >>>>> >>>> >>>> >>>> >>>> -- >>>> *Anjana Fernando* >>>> Senior Technical Lead >>>> WSO2 Inc. | http://wso2.com >>>> lean . enterprise . middleware >>>> >>> >>> >>> >>> -- >>> *Anjana Fernando* >>> Senior Technical Lead >>> WSO2 Inc. | http://wso2.com >>> lean . enterprise . middleware >>> >> >> >> >> -- >> Regards, >> >> Dunith Dhanushka, >> Senior Software Engineer >> WSO2 Inc, >> >> Mobile - +94 71 8615744 >> Blog - dunithd.wordpress.com <http://blog.dunith.com> >> Twitter - @dunithd <http://twitter.com/dunithd> >> > > > > -- > ============================ > Blog: http://srinathsview.blogspot.com twitter:@srinath_perera > Site: http://people.apache.org/~hemapani/ > Photos: http://www.flickr.com/photos/hemapani/ > Phone: 0772360902 > -- *Anjana Fernando* Senior Technical Lead WSO2 Inc. | http://wso2.com lean . enterprise . middleware
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
