Hi Dan. I can't guarantee by how much this will improve your Red5 usage, but if you're using video files with lots of keyframes, this can make a huge difference. For example, I have a 4 hour video, 1.69 Gigabytes big, with keyframes at every 4 frames (this is a 300Kbps video). This amounts to 115 217 keyframes. The binary index file for this is 1.3 Mb big, and can be generated on my Pentium M 1.86 Ghz laptop with 1Gb of ram in about 3 to 5 minutes (depending on processor load. On average about 60% memory usage on non-red5-related stuff - its a windows laptop...) To play the file back after the index is generated, or seeking around in it, is not an issue.
I tried running the same movie through the current Red5 (head of the tree) and it ran out of memory and crashed java after a couple of minutes. I tried the same on a beefier machine (2.2 Ghz, 2 Gb of ram) and got the same result. I'm rewriting a substantial part of my code at the moment, since I'm aiming to make this multi-threaded, so that you don't block the user's streaming while building the index, which is the current interaction. Ideally I want to be able to build indexes on the fly, even for files being downloaded to disk by another process (perhaps a live stream?) and incrementally build the index as it is needed. This is taking some time and sweat (since i'm not an expert at multi-threading and want to keep it simple!) but my goal is to finish this by Friday, so that I can spend the weekend integrating it with the current Red5 tree. Hopefully a patch will be into the Jira bugtracker by next week. Of course this is not the "silver bullet" that's going to solve everyone's issues, and really this only affects Video On Demand content coming from disk. I'm also hoping to get input from the current Red5 devs to make sure this code is bullet-proof. It's always a long way from proving that you can improve it to actually having bug-free code (if such a thing exists). But, if you have any video that is longer than youtube-content, you'll hopefully get at least a little performance increase once this patch is up. - Niels Joubert UC Berkeley http://njoubert.blogspot.com/ Dan Rossi wrote: > This is insane, I think this may solve some of our problems aswell with > our larger videos which are over 200MB each. > > Niels Joubert wrote: > >> Hello Adam and other Red5 Users >> >> This is a known issues with Red5's IO capability, and I am currently >> working on a patch that will fix all these issues. >> >> Red5 builds index files that represents the keyframes and corresponding >> byte-offsets of the media you are playing. You can open up the index >> files with a XML editor and see this. The problem with this approach is >> that XML is ridiculously slow for long lists of data (we're talking >> about 10 000 + entries) and Red5 keeps all these lists in memory, >> synchronizing it with the disk-based version. >> >> When you open a file for the first time, Red5 runs through the whole >> file, finding keyframes (or, in the case of music, EVERY frame), and >> corresponding byte offsets, making a list in memory of this, and finally >> writing it to disk. >> >> When you open a file that has an index, on your first seek, Red5 loads >> the whole XML file into memory (For a 4 hour video, anything between 3 >> and 10 MB of XML) before pushing the seek - which it does by seeking >> linearly, from the start of the file, to find the closes keyframe. This >> means that each file being opened can take 10MB of memory easily if >> you're using big video files. >> >> Unfortunately this is a slowdown and causes memory bloat with big files. >> I have patched Red5 to use a fast binary-based disk index file, that >> keeps as little information as possible in memory, and does smarter >> seeking. I routinely throw full-length movies in 1080p at it and it >> happily spins through them. This patch is almost ready for inclusion in >> Red5, at the moment i'm working on making it multi-threaded, so that >> there's no wait when a file loads, and the index gets built in the >> background. >> >> Hopefully this will be integrated soon, but at this point in time, your >> best bet is to connect, and wait until the index file is built (it keeps >> building even if the client gets disconnected) and connect again if the >> index file is done building. Look at the changing filesize on disk of >> the index to see when it's done. >> >> good luck, we'll have this part of Red5 blazing soon enough! My initial >> tests show that I can open 4000 files at the same time with a memory >> overhead of about 40Mb. >> >> Adam wrote: >> >> >>> Oh, and I tested with Red5 pulled out of the Trunk today, >>> so I should be using the latest version here. It's failing >>> on my Mac, and also on the Windows machine (though probably >>> not with the trunk release over there). >>> >>> Thanks again, >>> >>> Adam...... >>> >>> On Wed, Aug 08, 2007 at 04:10:20PM +0100, Adam wrote: >>> >>> >>> >>>> So nearing a functionally complete version of my software, >>>> and actually using it for it's intended purpose a little >>>> rather than doing quick tests, we've come across a large >>>> problem. Or at least a problem with large files. >>>> >>>> Playing an audio-only .flv file works well, until the >>>> file is more than about 45 minutes long. Then it really >>>> slows things down during some initializating phase before >>>> reporting "Closing RTMPMinaConnection from 127.0.0.1:49372 >>>> to localhost (in: 3427, out: 3262) due to too much >>>> inactivity (1186585160652)" >>>> >>>> Here's a stripped down program that shows my problem here: >>>> ------------------------------------------------------------------- >>>> ncStatus=function(info){ >>>> trace("Net Connection Status Change: Level"+info.level+" >>>> Code:"+info.code); >>>> if(info.code=="NetConnection.Connect.Success"){ >>>> trace("Connection Complete - playing"); >>>> _root.aud_ns.play("test.flv"); >>>> } >>>> } >>>> >>>> trace("Createing New Netstream"); >>>> _root.nc = new NetConnection(); >>>> _root.nc.onStatus = ncStatus; >>>> _root.nc.connect("rtmp://localhost/test"); >>>> _root.aud_ns = new NetStream(_root.nc); >>>> ------------------------------------------------------------------- >>>> >>>> Nothing too complicated there as you can see. When the >>>> webapps/test/test.flv file is less than 30 minutes long, >>>> it works without trouble. When it's greater than 60 minutes >>>> it consistantly fails with that error. >>>> >>>> Is this a known bug? Should I put in a bug report? Anyone >>>> got any clues on any workarounds I might be able to use? >>>> I'm supposed to be demoing this project in about 3 weeks. >>>> Eeep. >>>> >>>> Thanks for your help, >>>> >>>> Adam............ >>>> >>>> >>>> >>> _______________________________________________ >>> Red5 mailing list >>> [email protected] >>> http://osflash.org/mailman/listinfo/red5_osflash.org >>> >>> >>> >> _______________________________________________ >> Red5 mailing list >> [email protected] >> http://osflash.org/mailman/listinfo/red5_osflash.org >> >> >> > > > _______________________________________________ > Red5 mailing list > [email protected] > http://osflash.org/mailman/listinfo/red5_osflash.org > _______________________________________________ Red5 mailing list [email protected] http://osflash.org/mailman/listinfo/red5_osflash.org
