Re: [flexcoders] Flex grid to serve 100K-1mil rows
On Friday 09 Nov 2007, letterpigeon wrote: Hi all, requirement is for the grid to be able to support up to 100K+ or even 1 million rows Why ? It's utterly impossible for a user to cope with this much data at once, so what's the point ? -- Tom Chiverton Helping to dynamically incubate essential CEOs on: http://thefalken.livejournal.com This email is sent for and on behalf of Halliwells LLP. Halliwells LLP is a limited liability partnership registered in England and Wales under registered number OC307980 whose registered office address is at St James's Court Brown Street Manchester M2 2JF. A list of members is available for inspection at the registered office. Any reference to a partner in relation to Halliwells LLP means a member of Halliwells LLP. Regulated by The Solicitors Regulation Authority. CONFIDENTIALITY This email is intended only for the use of the addressee named above and may be confidential or legally privileged. If you are not the addressee you must not read it and must not use any information contained in nor copy it nor inform any person other than Halliwells LLP or the addressee of its existence or contents. If you have received this email in error please delete it and notify Halliwells LLP IT Department on 0870 365 2500. For more information about Halliwells LLP visit www.halliwells.com. -- Flexcoders Mailing List FAQ: http://groups.yahoo.com/group/flexcoders/files/flexcodersFAQ.txt Search Archives: http://www.mail-archive.com/flexcoders%40yahoogroups.com Yahoo! Groups Links * To visit your group on the web, go to: http://groups.yahoo.com/group/flexcoders/ * Your email settings: Individual Email | Traditional * To change settings online go to: http://groups.yahoo.com/group/flexcoders/join (Yahoo! ID required) * To change settings via email: mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] * To unsubscribe from this group, send an email to: [EMAIL PROTECTED] * Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
Re: [flexcoders] Flex grid to serve 100K-1mil rows
On Nov 12, 2007 4:35 AM, Tom Chiverton [EMAIL PROTECTED] wrote: On Friday 09 Nov 2007, letterpigeon wrote: Hi all, requirement is for the grid to be able to support up to 100K+ or even 1 million rows Why ? It's utterly impossible for a user to cope with this much data at once, so what's the point ? In general, the question of where to process data is totally legit. If you read carefully, he did not say he wanted to *show* 100k rows at once. He said he wanted to be able to do mass operations on it on the client side. Without understanding his application and architecture I cannot opine on whether it is better to do this on the client or server. But the idea, in applications that may need to scale, of doing more work on the client, is a good one. If you have lots of clients you can indeed save money on server costs by having the clients help with the work. Hank
Re: [flexcoders] Flex grid to serve 100K-1mil rows
On Monday 12 Nov 2007, hank williams wrote: applications that may need to scale, of doing more work on the client, is a good one. If you have lots of clients you can indeed save money on server costs by having the clients help with the work. I'd still want to plumb for sending an operation, parameters and list/range of ids to the server though, as others have mentioned depending on what the operations are it might take minutes on the client, and the user experience here could be bad. -- Tom Chiverton Helping to apprehensively unleash B2C CEOs on: http://thefalken.livejournal.com This email is sent for and on behalf of Halliwells LLP. Halliwells LLP is a limited liability partnership registered in England and Wales under registered number OC307980 whose registered office address is at St James's Court Brown Street Manchester M2 2JF. A list of members is available for inspection at the registered office. Any reference to a partner in relation to Halliwells LLP means a member of Halliwells LLP. Regulated by The Solicitors Regulation Authority. CONFIDENTIALITY This email is intended only for the use of the addressee named above and may be confidential or legally privileged. If you are not the addressee you must not read it and must not use any information contained in nor copy it nor inform any person other than Halliwells LLP or the addressee of its existence or contents. If you have received this email in error please delete it and notify Halliwells LLP IT Department on 0870 365 2500. For more information about Halliwells LLP visit www.halliwells.com. -- Flexcoders Mailing List FAQ: http://groups.yahoo.com/group/flexcoders/files/flexcodersFAQ.txt Search Archives: http://www.mail-archive.com/flexcoders%40yahoogroups.com Yahoo! Groups Links * To visit your group on the web, go to: http://groups.yahoo.com/group/flexcoders/ * Your email settings: Individual Email | Traditional * To change settings online go to: http://groups.yahoo.com/group/flexcoders/join (Yahoo! ID required) * To change settings via email: mailto:[EMAIL PROTECTED] mailto:[EMAIL PROTECTED] * To unsubscribe from this group, send an email to: [EMAIL PROTECTED] * Your use of Yahoo! Groups is subject to: http://docs.yahoo.com/info/terms/
Re: [flexcoders] Flex grid to serve 100K-1mil rows
On Nov 12, 2007 4:56 AM, Tom Chiverton [EMAIL PROTECTED] wrote: On Monday 12 Nov 2007, hank williams wrote: applications that may need to scale, of doing more work on the client, is a good one. If you have lots of clients you can indeed save money on server costs by having the clients help with the work. I'd still want to plumb for sending an operation, parameters and list/range of ids to the server though, as others have mentioned depending on what the operations are it might take minutes on the client, and the user experience here could be bad. There are lots of depending ons here, but in general, there is nothing about flash or AS3 that makes processing data slow. In fact it can be quite fast. The issue is how to architect it on the client side. For example it probably doesnt make sense to load all this data into a datagrid. But it can still be on the client in stoage focused data structures. Really flash is entirely capable of loading and handling large datasets. Adobe has announced their desire to move all their apps to flex, and in general, I view flex as just as capable as a regular desktop development environment for which one would never say oh you cant load all that data into excel for example. Flex isnt as fast as C++ but it really is getting there and for most applications the speed difference doesnt matter. Just like today we do not hesitate to do stuff in java vs C++. The point is you can build highly performant client side apps in flex, and with things like AIR being introduced, we need to get comfortable with that idea. Hank -- Tom Chiverton Helping to apprehensively unleash B2C CEOs on: http://thefalken.livejournal.com This email is sent for and on behalf of Halliwells LLP. Halliwells LLP is a limited liability partnership registered in England and Wales under registered number OC307980 whose registered office address is at St James's Court Brown Street Manchester M2 2JF. A list of members is available for inspection at the registered office. Any reference to a partner in relation to Halliwells LLP means a member of Halliwells LLP. Regulated by The Solicitors Regulation Authority. CONFIDENTIALITY This email is intended only for the use of the addressee named above and may be confidential or legally privileged. If you are not the addressee you must not read it and must not use any information contained in nor copy it nor inform any person other than Halliwells LLP or the addressee of its existence or contents. If you have received this email in error please delete it and notify Halliwells LLP IT Department on 0870 365 2500. For more information about Halliwells LLP visit www.halliwells.com. -- Flexcoders Mailing List FAQ: http://groups.yahoo.com/group/flexcoders/files/flexcodersFAQ.txt Search Archives: http://www.mail-archive.com/flexcoders%40yahoogroups.com Yahoo! Groups Links
[flexcoders] Flex grid to serve 100K-1mil rows
Hi all, We've built a flex grid that supports mass action on its data right now (column-wise search replace). But it just doesn't scale right now. It can hold up to around 3K+ rows in the grid, but it's taking a good few mins to load that up from the java backend servlet. The requirement is for the grid to be able to support up to 100K+ or even 1 million rows (may be not displaying them all at once, but when the mass action/search is taken, it should be performed on the complete dataset, not just what is being served up on the grid). We're just looking for ideas how this could be achieved, i.e.: if pagination is needed, what is the best way to do? client side pagination v.s. server? Could this be a good use case for Flex Data Service? etc. Any pointers/ideas/experience would be greatly appreicated. Thanks in advance. Regards, Ban
Re: [flexcoders] Flex grid to serve 100K-1mil rows
- Original Message - From: letterpigeon [EMAIL PROTECTED] To: flexcoders@yahoogroups.com Sent: Friday, November 09, 2007 7:03 PM Subject: [flexcoders] Flex grid to serve 100K-1mil rows Hi all, We've built a flex grid that supports mass action on its data right now (column-wise search replace). But it just doesn't scale right now. It can hold up to around 3K+ rows in the grid, but it's taking a good few mins to load that up from the java backend servlet. The requirement is for the grid to be able to support up to 100K+ or even 1 million rows (may be not displaying them all at once, but when the mass action/search is taken, it should be performed on the complete dataset, not just what is being served up on the grid). We're just looking for ideas how this could be achieved, i.e.: if pagination is needed, what is the best way to do? client side pagination v.s. server? Could this be a good use case for Flex Data Service? etc. Any pointers/ideas/experience would be greatly appreicated. Thanks in advance. Imagine sitting using this application and scrolling through 3K rows. You'll be there awhile. Now imagine scrolling through 100K rows. No user will do that. My suggestion would be to implement filters for the data and use those to reduce the traffic to the client by filtering on the server. Implement your current search and replace on the server and refresh the (now reduced data) on the client. If you have to have large datasets, a pagination algorithm and caching is probably the way to go. Paul Regards, Ban
Re: [flexcoders] Flex grid to serve 100K-1mil rows
In data intensive apps I've had best results by pushing as much functionality as possible on to the DB server and using FDS (or WebORB) to retrieve as small a subset of data as you can get away with. If you're loading large amounts of data (I've worked with ~100k rows) then IMHO FDS or equivalent is a must, the load times are orders of magnitude quicker. There is a data management part to FDS which supports paging of data, the problem with it is that it pulls huge data sets from the DB and holds them in memory on the FDS server, which can be problematic for very large sets. We ended up creating a 'paging' system that pulls chunks of data from the DB as required and sends them back via Remote Object. Our DB people tell me that they can optimise the queries so that the DB anticipates future page requests and pre-caches them or some such... if you have a DB specialist I'd talk to them first! Stephen. On 9 Nov 2007, at 19:03, letterpigeon wrote: Hi all, We've built a flex grid that supports mass action on its data right now (column-wise search replace). But it just doesn't scale right now. It can hold up to around 3K+ rows in the grid, but it's taking a good few mins to load that up from the java backend servlet. The requirement is for the grid to be able to support up to 100K+ or even 1 million rows (may be not displaying them all at once, but when the mass action/search is taken, it should be performed on the complete dataset, not just what is being served up on the grid). We're just looking for ideas how this could be achieved, i.e.: if pagination is needed, what is the best way to do? client side pagination v.s. server? Could this be a good use case for Flex Data Service? etc. Any pointers/ideas/experience would be greatly appreicated. Thanks in advance. Regards,
