Re: [Flashcoders] Massive XML files and Flash
XPath != *native* E4X i realise this, correct xml parsing is still faster while xpath is a comfort library to use when speed is not essential. xfactorstudio is the best and most complient xpath. have not used MM xpath since 7.0, but its api is extremely limited the best sollution thus far without/minimil server side code is xslt. e4x will mature with beta and release. currently (without using much of it) i find it slow (since my only experience with it is reading bout it and playing with it maybe) . the best sollution is to use remoting and server code and if possible hire a contractor for 2 weeks to backend ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
So then... does anyone know of any solutions to speed up the 'parsing' process? Is there a way to pause the parser, do something else, and then resume parsing? (It sure would be nice if we could do two things at once) Is anyone really having trouble with the XML Object's parser or is it developer defined traversal routine pulling from the DOM that is taking up time? I'm also working with massive amounts of XML data and cannot find a way to speed up my parsing routine. (For one set it takes 10+ seconds to release it's grip on the processor. Not allowing the user to do anything, of course) This will all move to a database in the near future, but the speed issue is not in Macromedia's XML object parsing the data, it's in my traversal of it. I've tried a few different techniques, yet, it's clearly not an issue of the XML objects onLoad parse. -sp On 1/7/06, Johannes Nel [EMAIL PROTECTED] wrote: XPath != *native* E4X i realise this, correct xml parsing is still faster while xpath is a comfort library to use when speed is not essential. xfactorstudio is the best and most complient xpath. have not used MM xpath since 7.0, but its api is extremely limited the best sollution thus far without/minimil server side code is xslt. e4x will mature with beta and release. currently (without using much of it) i find it slow (since my only experience with it is reading bout it and playing with it maybe) . the best sollution is to use remoting and server code and if possible hire a contractor for 2 weeks to backend ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Spencer Markowski The Able Few [EMAIL PROTECTED] 314.631.5576 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
This has been discussed countless times, and everytime all reach the same conclusion; you cannot do anything when XML.parseXML is run. The function has been improved in 3 releases (5, 6, and 7), but it's still single-threaded, and does not scale to meet larger XML datasets. If during testing it doesn't work out, you'll have to throw smaller chunks of XML at it over time. The typical scenario is: - show loading bar Parsing... - 1 frame later, parse - 1 frame after parsed, hide loading bar For optimizing your traversel search the archives, this has been discussed 10 billion times. However, to repeat: - use attributes vs. nested nodes - make references to everything. Flash 7 is optimized for local registers (local variables inside of functions) - additionally, those references prevent re-parsing, such as childNodes: var nodes:Array = this.firstChild.childNodes; Then use nodes throughout the rest of your code vs. re-declaring childNodes - use backwards while loops while(i--) - where applicable use nextChild vs. childNodes - Original Message - From: Spencer Markowski [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Saturday, January 07, 2006 9:42 AM Subject: Re: [Flashcoders] Massive XML files and Flash So then... does anyone know of any solutions to speed up the 'parsing' process? Is there a way to pause the parser, do something else, and then resume parsing? (It sure would be nice if we could do two things at once) Is anyone really having trouble with the XML Object's parser or is it developer defined traversal routine pulling from the DOM that is taking up time? I'm also working with massive amounts of XML data and cannot find a way to speed up my parsing routine. (For one set it takes 10+ seconds to release it's grip on the processor. Not allowing the user to do anything, of course) This will all move to a database in the near future, but the speed issue is not in Macromedia's XML object parsing the data, it's in my traversal of it. I've tried a few different techniques, yet, it's clearly not an issue of the XML objects onLoad parse. -sp On 1/7/06, Johannes Nel [EMAIL PROTECTED] wrote: XPath != *native* E4X i realise this, correct xml parsing is still faster while xpath is a comfort library to use when speed is not essential. xfactorstudio is the best and most complient xpath. have not used MM xpath since 7.0, but its api is extremely limited the best sollution thus far without/minimil server side code is xslt. e4x will mature with beta and release. currently (without using much of it) i find it slow (since my only experience with it is reading bout it and playing with it maybe) . the best sollution is to use remoting and server code and if possible hire a contractor for 2 weeks to backend ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Spencer Markowski The Able Few [EMAIL PROTECTED] 314.631.5576 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
RE: [Flashcoders] Massive XML files and Flash
I had a project recently where I was parsing huge XML datasets... To get the speed and memory efficiency I needed, I ended up avoiding the XML object altogether. I loaded the XML as a String instead (using LoadVars), and split it into page sized chunks that I could parse on-demand later. String-splitting by node name is tricky, but doable. It's not very scalable: if the format of the xml changes, I'll probably have to rewrite my parser. But it got around a nasty problem where the XML object wasn't being cleared from memory... as I recall, it was using over 40 MB of RAM for 1 MB of XML data!! I'll be glad when I can release an AS3 version and take advantage of E4X. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of JesterXL Sent: Saturday, January 07, 2006 9:52 AM To: Flashcoders mailing list Subject: Re: [Flashcoders] Massive XML files and Flash This has been discussed countless times, and everytime all reach the same conclusion; you cannot do anything when XML.parseXML is run. The function has been improved in 3 releases (5, 6, and 7), but it's still single-threaded, and does not scale to meet larger XML datasets. If during testing it doesn't work out, you'll have to throw smaller chunks of XML at it over time. The typical scenario is: - show loading bar Parsing... - 1 frame later, parse - 1 frame after parsed, hide loading bar For optimizing your traversel search the archives, this has been discussed 10 billion times. However, to repeat: - use attributes vs. nested nodes - make references to everything. Flash 7 is optimized for local registers (local variables inside of functions) - additionally, those references prevent re-parsing, such as childNodes: var nodes:Array = this.firstChild.childNodes; Then use nodes throughout the rest of your code vs. re-declaring childNodes - use backwards while loops while(i--) - where applicable use nextChild vs. childNodes - Original Message - From: Spencer Markowski [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Saturday, January 07, 2006 9:42 AM Subject: Re: [Flashcoders] Massive XML files and Flash So then... does anyone know of any solutions to speed up the 'parsing' process? Is there a way to pause the parser, do something else, and then resume parsing? (It sure would be nice if we could do two things at once) Is anyone really having trouble with the XML Object's parser or is it developer defined traversal routine pulling from the DOM that is taking up time? I'm also working with massive amounts of XML data and cannot find a way to speed up my parsing routine. (For one set it takes 10+ seconds to release it's grip on the processor. Not allowing the user to do anything, of course) This will all move to a database in the near future, but the speed issue is not in Macromedia's XML object parsing the data, it's in my traversal of it. I've tried a few different techniques, yet, it's clearly not an issue of the XML objects onLoad parse. -sp On 1/7/06, Johannes Nel [EMAIL PROTECTED] wrote: XPath != *native* E4X i realise this, correct xml parsing is still faster while xpath is a comfort library to use when speed is not essential. xfactorstudio is the best and most complient xpath. have not used MM xpath since 7.0, but its api is extremely limited the best sollution thus far without/minimil server side code is xslt. e4x will mature with beta and release. currently (without using much of it) i find it slow (since my only experience with it is reading bout it and playing with it maybe) . the best sollution is to use remoting and server code and if possible hire a contractor for 2 weeks to backend ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Spencer Markowski The Able Few [EMAIL PROTECTED] 314.631.5576 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
I'll be glad when I can release an AS3 version and take advantage of E4X. Even then you'll have issues with the amount of time it takes to parse large datasets. However, with native RegEx in 8.5 as well, you can write much more robust string splitting routines that are less dependant on the exact format and more tolerant to minor changes. Not to mention much easier to update if the format changes drastically. ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
I'm also working with massive amounts of XML data and cannot find a way to speed up my parsing routine. (For one set it takes 10+ seconds to release it's grip on the processor. Not allowing the user to do anything, of course) Like I said, break it up across frames and show a progress bar. 10 seconds isn't a long time to wait if there is a progress bar and the rest of the app is working, but it's a loong time to wait if the whole app is locked up. ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
How would you break it up? Brookd At 12:05 PM 1/7/2006, you wrote: I'm also working with massive amounts of XML data and cannot find a way to speed up my parsing routine. (For one set it takes 10+ seconds to release it's grip on the processor. Not allowing the user to do anything, of course) Like I said, break it up across frames and show a progress bar. 10 seconds isn't a long time to wait if there is a progress bar and the rest of the app is working, but it's a loong time to wait if the whole app is locked up. ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
How would you break it up? Overwrite the onData event, take the data as a string and pass it off to a frame loop that parses x number of records per frame. You can teak the number of records per frame to fit the app, depending on how long parsing takes. The easiest way to do that is to not use XML at all, but use a format that is easier to parse, like comma delimited or fixed-width. Check the archives, it's been discussed at length before. ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
In my current project, I need to be able to pause my traversal function in order to load another nested xml file, which may contain dependencies the current loaded parent xml file is depended on. This almost can't be done by multiple passes or by breaking up data. So what I did was as suggested by many folks on this list: 1. Have a class(TaskManager in my case) extends MovieClip, which can run a function assigned by client in onEnterFrame event handler on demand. 2. Break up traversal recursive loop into a chain of function calls by delegating each call to the TaskManager object. I think this idea can apply to any time-consuming operation , for example, breaking up your parsing routines. The concept is to scatter heavy workload over frames instead of sticking on one frame. Doug - Original Message - From: Spencer Markowski [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Saturday, January 07, 2006 6:42 AM Subject: Re: [Flashcoders] Massive XML files and Flash So then... does anyone know of any solutions to speed up the 'parsing' process? Is there a way to pause the parser, do something else, and then resume parsing? (It sure would be nice if we could do two things at once) Is anyone really having trouble with the XML Object's parser or is it developer defined traversal routine pulling from the DOM that is taking up time? I'm also working with massive amounts of XML data and cannot find a way to speed up my parsing routine. (For one set it takes 10+ seconds to release it's grip on the processor. Not allowing the user to do anything, of course) This will all move to a database in the near future, but the speed issue is not in Macromedia's XML object parsing the data, it's in my traversal of it. I've tried a few different techniques, yet, it's clearly not an issue of the XML objects onLoad parse. -sp On 1/7/06, Johannes Nel [EMAIL PROTECTED] wrote: XPath != *native* E4X i realise this, correct xml parsing is still faster while xpath is a comfort library to use when speed is not essential. xfactorstudio is the best and most complient xpath. have not used MM xpath since 7.0, but its api is extremely limited the best sollution thus far without/minimil server side code is xslt. e4x will mature with beta and release. currently (without using much of it) i find it slow (since my only experience with it is reading bout it and playing with it maybe) . the best sollution is to use remoting and server code and if possible hire a contractor for 2 weeks to backend ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Spencer Markowski The Able Few [EMAIL PROTECTED] 314.631.5576 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
RE: [Flashcoders] Massive XML files and Flash
I would suggest looking into setInterval as an architecturally cleaner way of doing this... Has the advantage that the interval is not dependant on the movie's framerate, and has less overhead than instantiating a movieclip. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of MyName Sent: Saturday, January 07, 2006 8:03 PM To: Flashcoders mailing list Subject: Re: [Flashcoders] Massive XML files and Flash In my current project, I need to be able to pause my traversal function in order to load another nested xml file, which may contain dependencies the current loaded parent xml file is depended on. This almost can't be done by multiple passes or by breaking up data. So what I did was as suggested by many folks on this list: 1. Have a class(TaskManager in my case) extends MovieClip, which can run a function assigned by client in onEnterFrame event handler on demand. 2. Break up traversal recursive loop into a chain of function calls by delegating each call to the TaskManager object. I think this idea can apply to any time-consuming operation , for example, breaking up your parsing routines. The concept is to scatter heavy workload over frames instead of sticking on one frame. Doug - Original Message - From: Spencer Markowski [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Saturday, January 07, 2006 6:42 AM Subject: Re: [Flashcoders] Massive XML files and Flash So then... does anyone know of any solutions to speed up the 'parsing' process? Is there a way to pause the parser, do something else, and then resume parsing? (It sure would be nice if we could do two things at once) Is anyone really having trouble with the XML Object's parser or is it developer defined traversal routine pulling from the DOM that is taking up time? I'm also working with massive amounts of XML data and cannot find a way to speed up my parsing routine. (For one set it takes 10+ seconds to release it's grip on the processor. Not allowing the user to do anything, of course) This will all move to a database in the near future, but the speed issue is not in Macromedia's XML object parsing the data, it's in my traversal of it. I've tried a few different techniques, yet, it's clearly not an issue of the XML objects onLoad parse. -sp On 1/7/06, Johannes Nel [EMAIL PROTECTED] wrote: XPath != *native* E4X i realise this, correct xml parsing is still faster while xpath is a comfort library to use when speed is not essential. xfactorstudio is the best and most complient xpath. have not used MM xpath since 7.0, but its api is extremely limited the best sollution thus far without/minimil server side code is xslt. e4x will mature with beta and release. currently (without using much of it) i find it slow (since my only experience with it is reading bout it and playing with it maybe) . the best sollution is to use remoting and server code and if possible hire a contractor for 2 weeks to backend ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Spencer Markowski The Able Few [EMAIL PROTECTED] 314.631.5576 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Xpath looks pretty sweet, anyone used it for simillar data sizes? On 06/01/06, ryanm [EMAIL PROTECTED] wrote: I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. When I found myself in this situation I did 2 things: 1. Don't use XML, it is way too heavy for this much data. I found that by using a double-delimeted or fixed-width data format, the file size was reduced by as much as 70%. In the end, I went with fixed width because I could parse it faster (by avoiding calling split() thousands of times). Now, I still used the XML object, but instead of letting it parse the file, I overwrote the onData event and used my own parsing function, which generated objects directly instead of parsing it out to an XML object. Essentially, the XML object just read the data in and dumped it to my parsing function. 2. Don't try to parse it all at once. What I did was dump it all into a buffer when it was loaded, and then fire off a parsing function that parsed 250 records per frame. I found that number through trial and error, you can find your own balance. The important thing was, the application didn't stop functioning while the records were being parsed, you could go to other areas of the app and use it normally, and when you went to the section that required the data, you got a progress bar showing how many records had been parsed. My parsing function was semi-complicated. It took the whole dataset in as a string and split it on my record delimiter, and this array became my buffer. This way I knew how many records there were to parse, and approximately how long it would take to parse them. It then sliced 250 records off the top of the buffer on every frame and passed them to the serialization function, which took them, serialized them, and inserted them into my database object. My parsing function also built several indexes while it was parsing the records, to make lookups faster once the database was ready. My application was a database of hotels, which were sortable by a number of criteria, so the parsing routine looked for those attributes of each hotel as it parsed, and when it saw a new value for one of those criteria, it made a new entry in the appropriate index for it. I made very heavy use of the object collection syntax, for example: Index[Location][USA][Texas][Dallas] ...referred to an array of hotel ids which were in Dallas, Texas, USA, which could be used to find a hotel like this: // 0 is the first index in the array of ids hotelID = Index[Location][USA][Texas][Dallas][0]; return(Database[hotelID]); In the end, it took about 5 times as much code to import, parse, and index the database than the whole rest of the application, but it worked, it was relatively fast, and it met the requirements I was given. I would've preferred for it to work from a web server, selecting what I needed from the database, but the client required that it work offline from a database that shipped with the cd, as well as be able to download an updated database from their website, and this was the best solution I could find in Flash that worked on both PC and Mac (no 3rd party wrappers). Unfortunately it had to parse the whole database every time you ran the app, but it would get the newest version from the web if you were online and it gave you the option to store it (in an ungodly-sized shared object) if you wanted to. Anyway, that's how I did it, whether or not it was successful is a matter of opinion. ;-) ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
If I understand it correctly, the whole XML file needs to be loaded for you to use XPath, so it won't help you much if you just want the section of the XML data to utilize... I'd go with what JesterXL said... Make smaller chunks of XML file and load them as needed... CyanBlue - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Friday, January 06, 2006 7:33 AM Subject: Re: [Flashcoders] Massive XML files and Flash Xpath looks pretty sweet, anyone used it for simillar data sizes? ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
RE: [Flashcoders] Massive XML files and Flash
True - that's how I've used it anyway - you still have to load in an XML file into an XML object. Then you can call specific nodes. So XPath doesn't create an additional object representation of the XML file like many other parsers do, but it does have to load in the entire XML file first. Jason Merrill | E-Learning Solutions | icfconsulting.com -Original Message- From: [EMAIL PROTECTED] [mailto:flashcoders- [EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED] Sent: Friday, January 06, 2006 8:20 AM To: Flashcoders mailing list Subject: Re: [Flashcoders] Massive XML files and Flash If I understand it correctly, the whole XML file needs to be loaded for you to use XPath, so it won't help you much if you just want the section of the XML data to utilize... I'd go with what JesterXL said... Make smaller chunks of XML file and load them as needed... CyanBlue - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Friday, January 06, 2006 7:33 AM Subject: Re: [Flashcoders] Massive XML files and Flash Xpath looks pretty sweet, anyone used it for simillar data sizes? ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders NOTICE: This message is for the designated recipient only and may contain privileged or confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of this e-mail by you is prohibited. ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Jonathan Clarke wrote: Xpath looks pretty sweet, anyone used it for simillar data sizes? Too avoid memory and performance issues most people use a SAX-liek parser instead of a DOM one, to avoid building the xml file in object tree. Normally XPath is implementated on top of a DOM parser, but you can use SAX ofcourse too. Anyway 8-10mb files should do fine. ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Xpath looks pretty sweet, anyone used it for simillar data sizes? XPath won't help with the initial load and parse. What you're going to get is a long pause with your processor usage at 100% while the XML object parses the string. The more records/larger the file, the longer the pause. XPath also adds some overhead, which may be minimal, but with the amount of data you're talking about, it could quickly become significant. Besides, I'm not fooling with any 3rd party workarounds, not with E4X on the way. From the Adobemedia flex2 FAQ: http://www.macromedia.com/software/flex/productinfo/faq/flex2_faq.html - What new features will Flash Player 8.5 introduce for application developers? Flash Player 8.5 delivers a number of features that contribute to developer productivity, including a new, more powerful version of the ActionScript programming language, improved runtime errors that help in debugging, and ***native support for ECMAScript for XML (E4X) and regular expressions***. Flash Player 8.5 also delivers significant improvements in performance and a reduced memory footprint, enabling more responsive user interfaces. - Here's an IBM article on E4X: http://www-128.ibm.com/developerworks/library/ws-ajax1/ RegEx + E4X means insanely fast custom parsing and conversion, plus insanely fast searches and filters for XML. The need for XPath is about to be a thing of the past (whenever they get around to releasing 8.5, that is). ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
RE: [Flashcoders] Massive XML files and Flash
Besides, I'm not fooling with any 3rd party workarounds, not with E4X on the way. From the Adobemedia flex2 FAQ: Yeah, not to be a downer, but AS 3 is going to require the 8.5 or 9 player, so that's only good if your audience has it... I've been excited about Flex 2, but the excitement is tempered by the fact that even if it comes out this spring, I won't be using it for real projects at least another 18-24 months. Jason Merrill | E-Learning Solutions | icfconsulting.com NOTICE: This message is for the designated recipient only and may contain privileged or confidential information. If you have received it in error, please notify the sender immediately and delete the original. Any other use of this e-mail by you is prohibited. ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Besides, I'm not fooling with any 3rd party workarounds, not with E4X on the way. this has been around for ages...add to this that e4x is currently in alpha and slow, and the fact that xpath is a w3c spec and even macromedia implement a much thinner version of this in F7 then this statement sounds a bit absurd you will find xpath is very slow on large data sets, and writng certain types of queries make it even slower. i would really consider using a differant format (excel can export cvs, you can parse that much much faster). we tend to use xpath everywhere, check where the bottlenecks are and rewrite those parsing routines using node.firstSibling type access where needed. -- j:pn ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Yeah, not to be a downer, but AS 3 is going to require the 8.5 or 9 player, so that's only good if your audience has it... I've been excited about Flex 2, but the excitement is tempered by the fact that even if it comes out this spring, I won't be using it for real projects at least another 18-24 months. Yeah, but for web-based projects I'll be using the server for what it's good at and forget about parsing that much data. Bringing the whole database to the client doesn't work well anyway. For stand alones, I'll get to determine the player version, and that's where I'll end up needing E4X and RegEx, not just for the convenience, but out of necessity. ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
xpath has been around for a while On 1/6/06, ryanm [EMAIL PROTECTED] wrote: this has been around for ages...add to this that e4x is currently in alpha and slow, and the fact that xpath is a w3c spec and even macromedia implement a much thinner version of this in F7 then this statement sounds a bit absurd Uhhh... what? Yes, the E4X spec has been around a while, but is poorly implemented, at best, in most platforms. There was never any E4X in any version of Flash before, and the implementation in Flex 2 is pretty good. Native E4X and RegEx are both beautiful things. you will find xpath is very slow on large data sets, and writng certain types of queries make it even slower. i would really consider using a differant format (excel can export cvs, you can parse that much much faster). we tend to use xpath everywhere, check where the bottlenecks are and rewrite those parsing routines using node.firstSibling type access where needed. Which is essentially what I said in the first place... ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- j:pn ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
xpath has been around for a while XPath != *native* E4X XPath only bears a passing resemblence to E4X, and is nothing like it in terms of performance. You should check out http://labs.macromedia.com and download the Flex2 alpha. ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
RegEx + E4X means insanely fast custom parsing and conversion, plus insanely fast searches and filters for XML. The need for XPath is about to be a thing of the past (whenever they get around to releasing 8.5, that is). I did some benchmarks comparing fp8 XML vs fp8.5 flash.xml.XML vs fp8.5 E4X (recursive traversal of an xml document) where the fp8.5 version turned out to be 20x faster than it's fp8 counterpart while the E4X version was only twice as fast. I also did benchmarks comparing RegExp vs char-by-char parsing on my CSS parser and char-by-char appeared to be faster. Now these may not be common cases, but they show that there is no such thing as an ultimate way to do things. If i need comfort and ease of use i'd go for E4X or RegExp, if i need speed i do it the oldschool way, and if i need to be less language dependent and don't care much about speed i stick with XPath. In fact i'd love to see a solid XPath processor written in AS3. cheers, claus. ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Clarke, It will take you same amount of time to learn(database) and implement a server side solution to your problem then to go with handling all xml in Actionscript.x. If you go with the 100% ActionScript solution then you will get: 1. low Scalability 2. low Stability 3. high maintenance 4. bad user experience (sluggish, 100% cpu,..) Server side solution will go long way. On 1/6/06, Claus Wahlers [EMAIL PROTECTED] wrote: ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Have you tried to work with your data across three xml files with xsl? Using xsl with a server side scripting language, you could get that data in the format you need before loading it with the AS XML() object. Then, thanks to Flash, you can use the native XPath() object to select elements and their attributes for use in your production. Regards, Derek Lords From: Jonathan Clarke [EMAIL PROTECTED] Reply-To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Subject: Re: [Flashcoders] Massive XML files and Flash Date: Thu, 5 Jan 2006 23:17:42 -0400 Thanks Jester, but databases are not my forte, to do it quickly I'd like to just use ActionScript only. Is there anyway I can just load specific sections of the XML or do I have to load the whole file into memory? On 05/01/06, JesterXL [EMAIL PROTECTED] wrote: Does it have to stay in that format? Meaning, can you convert it and throw it into a database? Paging is still an effective methodology, and if you can throw it in a database, you can then do what Flash does best and load what you need on the fly. - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Thursday, January 05, 2006 7:21 PM Subject: [Flashcoders] Massive XML files and Flash Hi, I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. I'm sure there are lots of more suitable applications and languages, but I the only one I know is Actionscript. Luckily, I know it very well and am more than confident at parsing XML; however, I've never delt with data on this scale before. So, I'm interested to know if anyone has any experience of dealing with this sort of build, and whether it is feasible to expect Flash to handle it. More on the data: I need to cross-reference and check data accross 3 xml files. They are all big, but the biggest, by way of an example in Excel form, has 25,000 records each with around 30 fields, often very string heavy. Any tips would be greatly appreciated? -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
RE: [Flashcoders] Massive XML files and Flash
Notice Flash 8 includes Xpath as a native, check livedocs or your help documentation for details. But the XPath tool from xfactorstudio is an old standby. Regards, Derek Lords From: Steve Lloyd [EMAIL PROTECTED] Reply-To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com To: 'Flashcoders mailing list' flashcoders@chattyfig.figleaf.com Subject: RE: [Flashcoders] Massive XML files and Flash Date: Fri, 6 Jan 2006 11:37:52 +0800 XPath. Beautiful! I've been looking for something like that! Thanks, will give it a shot! -Steve -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andy Johnston Sent: Friday, 6 January 2006 11:23 AM To: Flashcoders mailing list Subject: Re: [Flashcoders] Massive XML files and Flash I've found this as version of xpath very very handy.. easy to implement and use also.. http://www.xfactorstudio.com/ActionScript/AS2/XPath/ Thanks Jester, but databases are not my forte, to do it quickly I'd like to just use ActionScript only. Is there anyway I can just load specific sections of the XML or do I have to load the whole file into memory? On 05/01/06, JesterXL [EMAIL PROTECTED] wrote: Does it have to stay in that format? Meaning, can you convert it and throw it into a database? Paging is still an effective methodology, and if you can throw it in a database, you can then do what Flash does best and load what you need on the fly. - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Thursday, January 05, 2006 7:21 PM Subject: [Flashcoders] Massive XML files and Flash Hi, I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. I'm sure there are lots of more suitable applications and languages, but I the only one I know is Actionscript. Luckily, I know it very well and am more than confident at parsing XML; however, I've never delt with data on this scale before. So, I'm interested to know if anyone has any experience of dealing with this sort of build, and whether it is feasible to expect Flash to handle it. More on the data: I need to cross-reference and check data accross 3 xml files. They are all big, but the biggest, by way of an example in Excel form, has 25,000 records each with around 30 fields, often very string heavy. Any tips would be greatly appreciated? -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Does it have to stay in that format? Meaning, can you convert it and throw it into a database? Paging is still an effective methodology, and if you can throw it in a database, you can then do what Flash does best and load what you need on the fly. - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Thursday, January 05, 2006 7:21 PM Subject: [Flashcoders] Massive XML files and Flash Hi, I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. I'm sure there are lots of more suitable applications and languages, but I the only one I know is Actionscript. Luckily, I know it very well and am more than confident at parsing XML; however, I've never delt with data on this scale before. So, I'm interested to know if anyone has any experience of dealing with this sort of build, and whether it is feasible to expect Flash to handle it. More on the data: I need to cross-reference and check data accross 3 xml files. They are all big, but the biggest, by way of an example in Excel form, has 25,000 records each with around 30 fields, often very string heavy. Any tips would be greatly appreciated? -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Thanks Jester, but databases are not my forte, to do it quickly I'd like to just use ActionScript only. Is there anyway I can just load specific sections of the XML or do I have to load the whole file into memory? On 05/01/06, JesterXL [EMAIL PROTECTED] wrote: Does it have to stay in that format? Meaning, can you convert it and throw it into a database? Paging is still an effective methodology, and if you can throw it in a database, you can then do what Flash does best and load what you need on the fly. - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Thursday, January 05, 2006 7:21 PM Subject: [Flashcoders] Massive XML files and Flash Hi, I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. I'm sure there are lots of more suitable applications and languages, but I the only one I know is Actionscript. Luckily, I know it very well and am more than confident at parsing XML; however, I've never delt with data on this scale before. So, I'm interested to know if anyone has any experience of dealing with this sort of build, and whether it is feasible to expect Flash to handle it. More on the data: I need to cross-reference and check data accross 3 xml files. They are all big, but the biggest, by way of an example in Excel form, has 25,000 records each with around 30 fields, often very string heavy. Any tips would be greatly appreciated? -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
I've found this as version of xpath very very handy.. easy to implement and use also.. http://www.xfactorstudio.com/ActionScript/AS2/XPath/ Thanks Jester, but databases are not my forte, to do it quickly I'd like to just use ActionScript only. Is there anyway I can just load specific sections of the XML or do I have to load the whole file into memory? On 05/01/06, JesterXL [EMAIL PROTECTED] wrote: Does it have to stay in that format? Meaning, can you convert it and throw it into a database? Paging is still an effective methodology, and if you can throw it in a database, you can then do what Flash does best and load what you need on the fly. - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Thursday, January 05, 2006 7:21 PM Subject: [Flashcoders] Massive XML files and Flash Hi, I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. I'm sure there are lots of more suitable applications and languages, but I the only one I know is Actionscript. Luckily, I know it very well and am more than confident at parsing XML; however, I've never delt with data on this scale before. So, I'm interested to know if anyone has any experience of dealing with this sort of build, and whether it is feasible to expect Flash to handle it. More on the data: I need to cross-reference and check data accross 3 xml files. They are all big, but the biggest, by way of an example in Excel form, has 25,000 records each with around 30 fields, often very string heavy. Any tips would be greatly appreciated? -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
Well, you can override the XML.onData. This is fired before XML's onLoad, and it'll give you the raw XML string data, unparsed. You can then manually parse this. Are these XML files created yet? Like, you say Excel document. If you can, creating a dictionary file that points to a series of XML files is tons better than 1 big beast. The problem is, even XML.onData still has that honkin' string in it, and using the String methods on it will still chug. Your best bet is to load in the index file, and only load those smaller files. Even if the the smaller file consists of thousands of records, you can simply wait a frame after she loads, use a String.indexOf on a certain node, say record and repeat over a series of frames to record indices of where all the nodes are. In the next pass, you could then break it up into chunks. Since strings are copy by value, you could make a copy of the string you want to parse, just parse that chunk, convert to something meaningful like an array full of objects, discard the string, and repeat on the next frame. You could even save those objects you've created as sharedobjects, eliminating them from memory. - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Thursday, January 05, 2006 10:17 PM Subject: Re: [Flashcoders] Massive XML files and Flash Thanks Jester, but databases are not my forte, to do it quickly I'd like to just use ActionScript only. Is there anyway I can just load specific sections of the XML or do I have to load the whole file into memory? On 05/01/06, JesterXL [EMAIL PROTECTED] wrote: Does it have to stay in that format? Meaning, can you convert it and throw it into a database? Paging is still an effective methodology, and if you can throw it in a database, you can then do what Flash does best and load what you need on the fly. - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Thursday, January 05, 2006 7:21 PM Subject: [Flashcoders] Massive XML files and Flash Hi, I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. I'm sure there are lots of more suitable applications and languages, but I the only one I know is Actionscript. Luckily, I know it very well and am more than confident at parsing XML; however, I've never delt with data on this scale before. So, I'm interested to know if anyone has any experience of dealing with this sort of build, and whether it is feasible to expect Flash to handle it. More on the data: I need to cross-reference and check data accross 3 xml files. They are all big, but the biggest, by way of an example in Excel form, has 25,000 records each with around 30 fields, often very string heavy. Any tips would be greatly appreciated? -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
RE: [Flashcoders] Massive XML files and Flash
XPath. Beautiful! I've been looking for something like that! Thanks, will give it a shot! -Steve -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Andy Johnston Sent: Friday, 6 January 2006 11:23 AM To: Flashcoders mailing list Subject: Re: [Flashcoders] Massive XML files and Flash I've found this as version of xpath very very handy.. easy to implement and use also.. http://www.xfactorstudio.com/ActionScript/AS2/XPath/ Thanks Jester, but databases are not my forte, to do it quickly I'd like to just use ActionScript only. Is there anyway I can just load specific sections of the XML or do I have to load the whole file into memory? On 05/01/06, JesterXL [EMAIL PROTECTED] wrote: Does it have to stay in that format? Meaning, can you convert it and throw it into a database? Paging is still an effective methodology, and if you can throw it in a database, you can then do what Flash does best and load what you need on the fly. - Original Message - From: Jonathan Clarke [EMAIL PROTECTED] To: Flashcoders mailing list flashcoders@chattyfig.figleaf.com Sent: Thursday, January 05, 2006 7:21 PM Subject: [Flashcoders] Massive XML files and Flash Hi, I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. I'm sure there are lots of more suitable applications and languages, but I the only one I know is Actionscript. Luckily, I know it very well and am more than confident at parsing XML; however, I've never delt with data on this scale before. So, I'm interested to know if anyone has any experience of dealing with this sort of build, and whether it is feasible to expect Flash to handle it. More on the data: I need to cross-reference and check data accross 3 xml files. They are all big, but the biggest, by way of an example in Excel form, has 25,000 records each with around 30 fields, often very string heavy. Any tips would be greatly appreciated? -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders -- Jonathan Clarke 1976 Ltd http://19seventysix.co.uk e: [EMAIL PROTECTED] m (UK): +44 773 646 1954 m (Barbados): +1246 259 9475 ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
Re: [Flashcoders] Massive XML files and Flash
I find myself in a situation where I need to build a tool to analyse lots of xml data. Thousands of records containing a lot of strings as well as numericals. When I found myself in this situation I did 2 things: 1. Don't use XML, it is way too heavy for this much data. I found that by using a double-delimeted or fixed-width data format, the file size was reduced by as much as 70%. In the end, I went with fixed width because I could parse it faster (by avoiding calling split() thousands of times). Now, I still used the XML object, but instead of letting it parse the file, I overwrote the onData event and used my own parsing function, which generated objects directly instead of parsing it out to an XML object. Essentially, the XML object just read the data in and dumped it to my parsing function. 2. Don't try to parse it all at once. What I did was dump it all into a buffer when it was loaded, and then fire off a parsing function that parsed 250 records per frame. I found that number through trial and error, you can find your own balance. The important thing was, the application didn't stop functioning while the records were being parsed, you could go to other areas of the app and use it normally, and when you went to the section that required the data, you got a progress bar showing how many records had been parsed. My parsing function was semi-complicated. It took the whole dataset in as a string and split it on my record delimiter, and this array became my buffer. This way I knew how many records there were to parse, and approximately how long it would take to parse them. It then sliced 250 records off the top of the buffer on every frame and passed them to the serialization function, which took them, serialized them, and inserted them into my database object. My parsing function also built several indexes while it was parsing the records, to make lookups faster once the database was ready. My application was a database of hotels, which were sortable by a number of criteria, so the parsing routine looked for those attributes of each hotel as it parsed, and when it saw a new value for one of those criteria, it made a new entry in the appropriate index for it. I made very heavy use of the object collection syntax, for example: Index[Location][USA][Texas][Dallas] ...referred to an array of hotel ids which were in Dallas, Texas, USA, which could be used to find a hotel like this: // 0 is the first index in the array of ids hotelID = Index[Location][USA][Texas][Dallas][0]; return(Database[hotelID]); In the end, it took about 5 times as much code to import, parse, and index the database than the whole rest of the application, but it worked, it was relatively fast, and it met the requirements I was given. I would've preferred for it to work from a web server, selecting what I needed from the database, but the client required that it work offline from a database that shipped with the cd, as well as be able to download an updated database from their website, and this was the best solution I could find in Flash that worked on both PC and Mac (no 3rd party wrappers). Unfortunately it had to parse the whole database every time you ran the app, but it would get the newest version from the web if you were online and it gave you the option to store it (in an ungodly-sized shared object) if you wanted to. Anyway, that's how I did it, whether or not it was successful is a matter of opinion. ;-) ryanm ___ Flashcoders mailing list Flashcoders@chattyfig.figleaf.com http://chattyfig.figleaf.com/mailman/listinfo/flashcoders
