Re: [flexcoders] DataGrid limit on amount of records or data?
Hi Jason. Merrill, Jason wrote: No, I guess I can't think of cases when showing 10,000 records with no filtering would be important in a Flex app. What examples were you thinking of? I am thinking of a tree. When I expand a node I might expand all the children down to the leaf node. Funny then, as per the topic title, I thought we were talking about DataGrid. Well. Im a newbie to flex and want to eventually move a whole heap of stuff to flex. To begin Id like to represent one of our company's structures as a tree in flex. As a starting point Ive tried to list that structure in a Datagrid. I hope that explains more where Im coming from. Sorry about the confusion. Kr. Luke. Jason Merrill *Bank of America * GTO LLD Solutions Design Development eTools Multimedia *Bank of America Flash Platform Developer Community* -- Luke Vanderfluit Analyst / Web Programmer e3Learning.com.au 08 8221 6422
RE: [flexcoders] DataGrid limit on amount of records or data?
No, I guess I can't think of cases when showing 10,000 records with no filtering would be important in a Flex app. What examples were you thinking of? I am thinking of a tree. When I expand a node I might expand all the children down to the leaf node. Funny then, as per the topic title, I thought we were talking about DataGrid. Jason Merrill Bank of America GTO LLD Solutions Design Development eTools Multimedia Bank of America Flash Platform Developer Community
RE: [flexcoders] DataGrid limit on amount of records or data?
You need to determine whether you're having a problem fetching the data from the server, or displaying the data in the DataGrid. Gordon Smith Adobe Flex SDK Team From: flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Luke Vanderfluit Sent: Sunday, February 10, 2008 12:34 AM To: flexcoders@yahoogroups.com Subject: Re: [flexcoders] DataGrid limit on amount of records or data? Hi Alex. Thanks very very much for responding. Alex Harui wrote: The only limit should be the amount of memory, not some number. What problem did you see when you got above 5000? The problem was that the data simply did not render. I tested with 2 different datasets, gradually increasing the number of rows in the database table. One dataset rendered correctly up to 4128 rows in the database. Then increasing the data with one row caused nothing (blank page) to be displayed. This was retried with a server (tomcat) restart and without in both cases the same result. No errors or debug messages. The other dataset showed the same behaviour however the cut off point was up around 6030 records. After I had reached the upper limit where no data was displayed ( a blank page) I shrunk the data back a few rows in both cases and reloaded the page with data displaying again. What do you think? Shouldnt I be able to display that many rows if I want? Kind regards. Luke. -- *From:* flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com [mailto:flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com ] *On Behalf Of *Luke Vanderfluit *Sent:* Saturday, February 09, 2008 4:36 PM *To:* flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com ; [EMAIL PROTECTED] mailto:javaflexcoders%40yahoogroups.com *Subject:* [flexcoders] DataGrid limit on amount of records or data? Hi. Im using flex with a hibernate backend. I have tables in a db that Id like to show using Datagrid. I have done tests and found that there is a limit to the number of records shown in the datagrid. Is this a limitation of the datagrid? How do I get around this? I currently am able to show around 5000 records. When I have tables of say 1 or 10 records, how do I show them? Thanks. Kind regards. Luke.
RE: [flexcoders] DataGrid limit on amount of records or data?
As a side note, taking a step back, why would you want to present that many records at a time to the user? It seems anything above a few hundred is overly cumbersome to wade through, and should perhaps instead be filtered further. No? True in most cases I guess, but I can think of a few use cases where Id want to load a severe amount of data. Can you? (-; No, I guess I can't think of cases when showing 10,000 records with no filtering would be important in a Flex app. What examples were you thinking of? Jason Merrill Bank of America GTO LLD Solutions Design Development eTools Multimedia Bank of America Flash Platform Developer Community
Re: [flexcoders] DataGrid limit on amount of records or data?
Hi Seth. Seth Hodgson wrote: What type of channel/endpoint are you using between the client and server? Its rtmp, which is the default channel in data-management-config.xml default-channels channel ref=my-rtmp/ /default-channels If you're using RTMP, that protocol currently has a maximum underlying chunk size of 10M. If you're trying to return more than this amount of data in a single result the player will actually terminate the connection. AMF serialization handles the result object graph in a single pass, so there's no way to auto-magically split a serialized result that's larger than this out into separate result messages/chunks. OK that sounds like a reasonable explanation. If this is what you're running into, you can work around it by fetching your data set to the client in smaller pages, where each page doesn't approach this size limit individually. Or use one of the HTTP-based channels/endpoints which don't impose this resource limit. Or, as others have mentioned, for the use cases you have that require huge data sets on the client, try to compute a slimmer result on the server if that's reasonable rather than pulling your entire domain, or large portions of it, down to the client at once. Say you're wanting to generate a graph of 10K data points - rather than pulling down all 10K full objects, you could make a separate remote object call to the server to a method that just pulled out the specific property you're graphing from the database and returns a simple list of those values. Granted, if you're plotting or processing most of the properties in these objects this type of optimization would make much less sense. Best, Seth -- Luke Vanderfluit Analyst / Web Programmer e3Learning.com.au 08 8221 6422
RE: [flexcoders] DataGrid limit on amount of records or data?
I continue to be skeptical that it's a DataGrid problem. I have no
problem getting a DataGrid to display 1,000,000 small items:
?xml version=1.0 encoding=utf-8?
mx:Application xmlns:mx=http://www.adobe.com/2006/mxml;
mx:Script
![CDATA[
private function dg_initialize(event:Event):void
{
const N:int = 100;
var dp:Array = new Array(N);
for (var i:int = 0; i N; i++)
{
dp[i] = { i: i, Log(i): Math.log(i) };
}
dg.dataProvider = dp;
}
]]
/mx:Script
mx:DataGrid id=dg initialize=dg_initialize(event)/
/mx:Application
The fact that JSP/HTML works isn't relevant. The issue is whether all of
the data items are being successfully fetched by HTTPService or whatever
you're using.
- Gordon
From: flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] On
Behalf Of Luke Vanderfluit
Sent: Monday, February 11, 2008 4:08 PM
To: flexcoders@yahoogroups.com
Subject: Re: [flexcoders] DataGrid limit on amount of records or data?
Hi.
I really appreciate everyones response (-:
Gordon Smith wrote:
You need to determine whether you're having a problem fetching the
data
from the server, or displaying the data in the DataGrid.
The problem is in the rendering.
I am able, with a simple jsp to fetch the data and display it in
html.
Kr.
Luke.
Gordon Smith
Adobe Flex SDK Team
--
*From:* flexcoders@yahoogroups.com
mailto:flexcoders%40yahoogroups.com
[mailto:flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com
]
*On Behalf Of *Luke Vanderfluit
*Sent:* Sunday, February 10, 2008 12:34 AM
*To:* flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com
*Subject:* Re: [flexcoders] DataGrid limit on amount of records or
data?
Hi Alex.
Thanks very very much for responding.
Alex Harui wrote:
The only limit should be the amount of memory, not some number. What
problem did you see when you got above 5000?
The problem was that the data simply did not render.
I tested with 2 different datasets, gradually increasing the number of
rows in the database table.
One dataset rendered correctly up to 4128 rows in the database.
Then increasing the data with one row caused nothing (blank page) to
be
displayed. This was retried with a server (tomcat) restart and without
in both cases the same result. No errors or debug messages.
The other dataset showed the same behaviour however the cut off point
was up around 6030 records.
After I had reached the upper limit where no data was displayed ( a
blank page) I shrunk the data back a few rows in both cases and
reloaded
the page with data displaying again.
What do you think?
Shouldnt I be able to display that many rows if I want?
Kind regards.
Luke.
--
*From:* flexcoders@yahoogroups.com
mailto:flexcoders%40yahoogroups.com
mailto:flexcoders%40yahoogroups.com
[mailto:flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com
mailto:flexcoders%40yahoogroups.com]
*On Behalf Of *Luke Vanderfluit
*Sent:* Saturday, February 09, 2008 4:36 PM
*To:* flexcoders@yahoogroups.com
mailto:flexcoders%40yahoogroups.com
mailto:flexcoders%40yahoogroups.com; [EMAIL PROTECTED]
mailto:javaflexcoders%40yahoogroups.com
mailto:javaflexcoders%40yahoogroups.com
*Subject:* [flexcoders] DataGrid limit on amount of records or data?
Hi.
Im using flex with a hibernate backend.
I have tables in a db that Id like to show using Datagrid.
I have done tests and found that there is a limit to the number of
records shown in the datagrid.
Is this a limitation of the datagrid?
How do I get around this?
I currently am able to show around 5000 records.
When I have tables of say 1 or 10 records, how do I show
them?
Thanks.
Kind regards.
Luke.
--
Luke Vanderfluit
Analyst / Web Programmer
e3Learning.com.au
08 8221 6422
Re: [flexcoders] DataGrid limit on amount of records or data?
Hi Jason. Merrill, Jason wrote: No, I guess I can't think of cases when showing 10,000 records with no filtering would be important in a Flex app. What examples were you thinking of? I am thinking of a tree. When I expand a node I might expand all the children down to the leaf node. Kr. Luke. -- Luke Vanderfluit Analyst / Web Programmer e3Learning.com.au 08 8221 6422
Re: [flexcoders] DataGrid limit on amount of records or data?
Hi. I really appreciate everyones response (-: Gordon Smith wrote: You need to determine whether you're having a problem fetching the data from the server, or displaying the data in the DataGrid. The problem is in the rendering. I am able, with a simple jsp to fetch the data and display it in html. Kr. Luke. Gordon Smith Adobe Flex SDK Team *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On Behalf Of *Luke Vanderfluit *Sent:* Sunday, February 10, 2008 12:34 AM *To:* flexcoders@yahoogroups.com *Subject:* Re: [flexcoders] DataGrid limit on amount of records or data? Hi Alex. Thanks very very much for responding. Alex Harui wrote: The only limit should be the amount of memory, not some number. What problem did you see when you got above 5000? The problem was that the data simply did not render. I tested with 2 different datasets, gradually increasing the number of rows in the database table. One dataset rendered correctly up to 4128 rows in the database. Then increasing the data with one row caused nothing (blank page) to be displayed. This was retried with a server (tomcat) restart and without in both cases the same result. No errors or debug messages. The other dataset showed the same behaviour however the cut off point was up around 6030 records. After I had reached the upper limit where no data was displayed ( a blank page) I shrunk the data back a few rows in both cases and reloaded the page with data displaying again. What do you think? Shouldnt I be able to display that many rows if I want? Kind regards. Luke. -- *From:* flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com [mailto:flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com] *On Behalf Of *Luke Vanderfluit *Sent:* Saturday, February 09, 2008 4:36 PM *To:* flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com; [EMAIL PROTECTED] mailto:javaflexcoders%40yahoogroups.com *Subject:* [flexcoders] DataGrid limit on amount of records or data? Hi. Im using flex with a hibernate backend. I have tables in a db that Id like to show using Datagrid. I have done tests and found that there is a limit to the number of records shown in the datagrid. Is this a limitation of the datagrid? How do I get around this? I currently am able to show around 5000 records. When I have tables of say 1 or 10 records, how do I show them? Thanks. Kind regards. Luke. -- Luke Vanderfluit Analyst / Web Programmer e3Learning.com.au 08 8221 6422
Re: [flexcoders] DataGrid limit on amount of records or data?
Hi Alex. Thanks very very much for responding. Alex Harui wrote: The only limit should be the amount of memory, not some number. What problem did you see when you got above 5000? The problem was that the data simply did not render. I tested with 2 different datasets, gradually increasing the number of rows in the database table. One dataset rendered correctly up to 4128 rows in the database. Then increasing the data with one row caused nothing (blank page) to be displayed. This was retried with a server (tomcat) restart and without in both cases the same result. No errors or debug messages. The other dataset showed the same behaviour however the cut off point was up around 6030 records. After I had reached the upper limit where no data was displayed ( a blank page) I shrunk the data back a few rows in both cases and reloaded the page with data displaying again. What do you think? Shouldnt I be able to display that many rows if I want? Kind regards. Luke. *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On Behalf Of *Luke Vanderfluit *Sent:* Saturday, February 09, 2008 4:36 PM *To:* flexcoders@yahoogroups.com; [EMAIL PROTECTED] *Subject:* [flexcoders] DataGrid limit on amount of records or data? Hi. Im using flex with a hibernate backend. I have tables in a db that Id like to show using Datagrid. I have done tests and found that there is a limit to the number of records shown in the datagrid. Is this a limitation of the datagrid? How do I get around this? I currently am able to show around 5000 records. When I have tables of say 1 or 10 records, how do I show them? Thanks. Kind regards. Luke.
Re: [flexcoders] DataGrid limit on amount of records or data?
Hi Jason. Thanks for your response. Merrill, Jason wrote: As a side note, taking a step back, why would you want to present that many records at a time to the user? It seems anything above a few hundred is overly cumbersome to wade through, and should perhaps instead be filtered further. No? True in most cases I guess, but I can think of a few use cases where Id want to load a severe amount of data. Can you? (-; Kind regards. Luke. Jason Merrill *Bank of America * GTO LLD Solutions Design Development eTools Multimedia *Bank of America Flash Platform Developer Community* *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On Behalf Of *Alex Harui *Sent:* Saturday, February 09, 2008 11:12 PM *To:* flexcoders@yahoogroups.com; [EMAIL PROTECTED] *Subject:* RE: [flexcoders] DataGrid limit on amount of records or data? The only limit should be the amount of memory, not some number. What problem did you see when you got above 5000? *From:* flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] *On Behalf Of *Luke Vanderfluit *Sent:* Saturday, February 09, 2008 4:36 PM *To:* flexcoders@yahoogroups.com; [EMAIL PROTECTED] *Subject:* [flexcoders] DataGrid limit on amount of records or data? Hi. Im using flex with a hibernate backend. I have tables in a db that Id like to show using Datagrid. I have done tests and found that there is a limit to the number of records shown in the datagrid. Is this a limitation of the datagrid? How do I get around this? I currently am able to show around 5000 records. When I have tables of say 1 or 10 records, how do I show them? Thanks. Kind regards. Luke.
RE: [flexcoders] DataGrid limit on amount of records or data?
What type of channel/endpoint are you using between the client and server? If you're using RTMP, that protocol currently has a maximum underlying chunk size of 10M. If you're trying to return more than this amount of data in a single result the player will actually terminate the connection. AMF serialization handles the result object graph in a single pass, so there's no way to auto-magically split a serialized result that's larger than this out into separate result messages/chunks. If this is what you're running into, you can work around it by fetching your data set to the client in smaller pages, where each page doesn't approach this size limit individually. Or use one of the HTTP-based channels/endpoints which don't impose this resource limit. Or, as others have mentioned, for the use cases you have that require huge data sets on the client, try to compute a slimmer result on the server if that's reasonable rather than pulling your entire domain, or large portions of it, down to the client at once. Say you're wanting to generate a graph of 10K data points - rather than pulling down all 10K full objects, you could make a separate remote object call to the server to a method that just pulled out the specific property you're graphing from the database and returns a simple list of those values. Granted, if you're plotting or processing most of the properties in these objects this type of optimization would make much less sense. Best, Seth From: flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Luke Vanderfluit Sent: Sunday, February 10, 2008 12:34 AM To: flexcoders@yahoogroups.com Subject: Re: [flexcoders] DataGrid limit on amount of records or data? Hi Alex. Thanks very very much for responding. Alex Harui wrote: The only limit should be the amount of memory, not some number. What problem did you see when you got above 5000? The problem was that the data simply did not render. I tested with 2 different datasets, gradually increasing the number of rows in the database table. One dataset rendered correctly up to 4128 rows in the database. Then increasing the data with one row caused nothing (blank page) to be displayed. This was retried with a server (tomcat) restart and without in both cases the same result. No errors or debug messages. The other dataset showed the same behaviour however the cut off point was up around 6030 records. After I had reached the upper limit where no data was displayed ( a blank page) I shrunk the data back a few rows in both cases and reloaded the page with data displaying again. What do you think? Shouldnt I be able to display that many rows if I want? Kind regards. Luke. -- *From:* flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com [mailto:flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com ] *On Behalf Of *Luke Vanderfluit *Sent:* Saturday, February 09, 2008 4:36 PM *To:* flexcoders@yahoogroups.com mailto:flexcoders%40yahoogroups.com ; [EMAIL PROTECTED] mailto:javaflexcoders%40yahoogroups.com *Subject:* [flexcoders] DataGrid limit on amount of records or data? Hi. Im using flex with a hibernate backend. I have tables in a db that Id like to show using Datagrid. I have done tests and found that there is a limit to the number of records shown in the datagrid. Is this a limitation of the datagrid? How do I get around this? I currently am able to show around 5000 records. When I have tables of say 1 or 10 records, how do I show them? Thanks. Kind regards. Luke.
Re: [flexcoders] DataGrid limit on amount of records or data?
Hey Luke, Luke Vanderfluit wrote: The problem was that the data simply did not render. I tested with 2 different datasets, gradually increasing the number of rows in the database table. One dataset rendered correctly up to 4128 rows in the database. Then increasing the data with one row caused nothing (blank page) to be displayed. This was retried with a server (tomcat) restart and without in both cases the same result. No errors or debug messages. The other dataset showed the same behaviour however the cut off point was up around 6030 records. After I had reached the upper limit where no data was displayed ( a blank page) I shrunk the data back a few rows in both cases and reloaded the page with data displaying again. Perhaps your request timed out or something? FWIW I've loaded 50,000+ rows(with about 10-15 columns) into a datagrid before(just as an experiment) from an xml file, not generated from the DB. I suspect the problem is some sort of server side issue given that you received a blank page. HTH. cheers, shaun
Re: [flexcoders] DataGrid limit on amount of records or data?
I think that is a the limit for the flash player, i remb someone made a post a while back on the same thing. Alot of people just said to do a paged since there is no way someone is going to look at 5000 or 10 records. So maybe load up 1000 then page the DataGrid. - Original Message From: Luke Vanderfluit [EMAIL PROTECTED] To: flexcoders@yahoogroups.com; [EMAIL PROTECTED] Sent: Saturday, February 9, 2008 6:36:14 PM Subject: [flexcoders] DataGrid limit on amount of records or data? Hi. Im using flex with a hibernate backend. I have tables in a db that Id like to show using Datagrid. I have done tests and found that there is a limit to the number of records shown in the datagrid. Is this a limitation of the datagrid? How do I get around this? I currently am able to show around 5000 records. When I have tables of say 1 or 10 records, how do I show them? Thanks. Kind regards. Luke. Be a better friend, newshound, and know-it-all with Yahoo! Mobile. Try it now. http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
RE: [flexcoders] DataGrid limit on amount of records or data?
The only limit should be the amount of memory, not some number. What problem did you see when you got above 5000? From: flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Luke Vanderfluit Sent: Saturday, February 09, 2008 4:36 PM To: flexcoders@yahoogroups.com; [EMAIL PROTECTED] Subject: [flexcoders] DataGrid limit on amount of records or data? Hi. Im using flex with a hibernate backend. I have tables in a db that Id like to show using Datagrid. I have done tests and found that there is a limit to the number of records shown in the datagrid. Is this a limitation of the datagrid? How do I get around this? I currently am able to show around 5000 records. When I have tables of say 1 or 10 records, how do I show them? Thanks. Kind regards. Luke.
RE: [flexcoders] DataGrid limit on amount of records or data?
As a side note, taking a step back, why would you want to present that many records at a time to the user? It seems anything above a few hundred is overly cumbersome to wade through, and should perhaps instead be filtered further. No? Jason Merrill Bank of America GTO LLD Solutions Design Development eTools Multimedia Bank of America Flash Platform Developer Community From: flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Alex Harui Sent: Saturday, February 09, 2008 11:12 PM To: flexcoders@yahoogroups.com; [EMAIL PROTECTED] Subject: RE: [flexcoders] DataGrid limit on amount of records or data? The only limit should be the amount of memory, not some number. What problem did you see when you got above 5000? From: flexcoders@yahoogroups.com [mailto:[EMAIL PROTECTED] On Behalf Of Luke Vanderfluit Sent: Saturday, February 09, 2008 4:36 PM To: flexcoders@yahoogroups.com; [EMAIL PROTECTED] Subject: [flexcoders] DataGrid limit on amount of records or data? Hi. Im using flex with a hibernate backend. I have tables in a db that Id like to show using Datagrid. I have done tests and found that there is a limit to the number of records shown in the datagrid. Is this a limitation of the datagrid? How do I get around this? I currently am able to show around 5000 records. When I have tables of say 1 or 10 records, how do I show them? Thanks. Kind regards. Luke.
