Martijn van Beek wrote: > In conclusion as far as I understand the possibilities the best > solution at the moment is using a network storage solution in > combination with shares. Maybe also in combination with a pooling > system (a popular video is distributed to more disks for load balancing). > > How does a hardware load balancer handle this? Do you have multiple > disks with the same content? The SAN isnt also clustered that is too expensive, but its backed up. Its a 2TB raided rack. All frontend machines load the drive(s) over a network drive via the controller machine over the gigabit network so the files are generated as file:////controllermachine/thesandrive/thepath/thefile.flv. I'm not too sure how to check for disk I/O and lower level performance in this situation.
There is no caching currently and the java processes are quite high on each machine with and without the JVM GC flags. > > On 6/19/07, *Dan Rossi* < [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> wrote: > > Steve speaking of caches when will that be put back in. What is > the most > stable solution there, it really didnt go so well back in november :) > > > Gong wrote: > > Martijn, > > > > On 6/19/07, *Martijn van Beek* <[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>>> wrote: > > > > To everybody who has network issues with the link I posted. > Please > > give me details like: browser, flash player version, firewalls, > > etc. Network failures aren't logged if you can't reach the > server. > > > > Mark, the problem is the I/O that's why I'm posting this. > But I'm > > glad that FMS has the same problem ;). > > > > > > I guess that you are referring to FMS without clustering. The > > Edge/Origin solution of FMS provides a cache system on Edge so that > > the VOD can be directly distributed to clients on Edge, thus > alleviate > > the load from Storage server accessed by Origin. > > > > The only solution I can think about is the creation of a system > > which divides the load over multiple disks/systems. > Expensive but > > effective. > > > > > > On 6/19/07, *Mark de Jong [NetMasters BV]* < > > [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]> > <mailto: [EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>> > wrote: > > > > Hi Martijn, > > > > Be sure to check out the I/O of the server. We've had > the same > > issues on a > > edge-server (yes, edge/origin) from a client from us and > I/O > > was the > > bottleneck for this. They have a lot of seeking (and > hundreds > > of files) and > > it was VERY I/O intensive. I don't now how Red5 handles this > > but it is worth > > checking this out first before going on to next steps. > > > > Kind regards, > > > > Mark de Jong > > > > ________________________________ > > > > Van: [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> [mailto: > > [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>>] Namens > > Martijn van Beek > > Verzonden: maandag 18 juni 2007 10:32 > > Aan: [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > > Onderwerp: Re: [Red5] Load balancing for live streams > > > > > > I'm not streaming live data but stored flv movies. And the > > problem then > > appears when a lot of people access the file at the same > time > > but all at a > > different position (in time). > > > > > > On 6/17/07, Steven Gong < [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>>> wrote: > > > > Orion, > > > > > > On 6/17/07, Orion Letizi < [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>> > > <mailto: [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>>> > > > wrote: > > > > > > While it may be impractical to cluster the > > actual AV data in > > the stream, > > perhaps the current state (metadata) of the > > stream could be > > clustered so > > that, as long as the AV data was > available on > > disk at each > > server, any > > server could pick up the stream and start > > serving it based > > on the metadata > > about the stream ( e.g., offset into the > data > > file, current > > position of > > buffers, stuff like that). > > > > > > Yep, that's true if the VOD streaming is pulled > by the > > client. But > > as we also need to track the client buffer, we are using a > > scheduled task to > > push the VOD content to the client. > > > > I was thinking of sharing the connection buffer > across > > the cluster > > if the connection is of type RTMPTConnection. When the > > connection is of type > > RTMPT, we use a buffer for each connection to save the > packets > > that will be > > sent to the client. These packets includes AV packets, RSO > > packets, > > invocation result etc. When the client's request is > > distributed to one of > > the node, the server retrieves packets in the buffer and > sends > > to the > > client. So all kinds of services, regardless of VOD, Live or > > RSO, can be > > shared without the need to share the whole connection or > > stream objects. > > > > What do you think? > > > > > > > > > > Of course, I share Steve Harris's lack of > > knowledge of the > > internals of > > Red5, so my observation might be dumb, > > nonsensical, or both > > for which I > > apologize in advance. > > > > > > Steve, > > > > On 6/16/07, sharrissf < > > [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>> > > <mailto: [EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]> > > <mailto:[EMAIL PROTECTED] > <mailto:[EMAIL PROTECTED]>>> > wrote: > > > > > > > > > Assuming the stream fits in memory and > just > > has multiple > > people accessing > > > it > > > I suspect it could be clustered with > > Terracotta. Though I > > would need to > > > know > > > more details to know for sure as I'm not > > that familiar > > with how they work > > > and are implemented. My thinking is that > > everyone in this > > case is > > > essentially sharing the same stream so > > really only one > > instance ends up in > > > each jvm and all the meta-data associated > > with each user > > is only in the > > > jvm > > > where the person is connected. > > > > > > The live stream is pushed to the > subscribers > > just like what > > Remote > > SharedObject does so if configured > properly I > > believe it can > > be clustered by > > TC. But I doubt that this is practically > > realistic because > > (1) The amount of AV data is much bigger > than > > that of RSO. > > So much more data > > will be transfered across the nodes. > > (2) When the amount of data arises, the > > transfer latency > > will also arise and > > the real time requirement of live > streaming is > > compromised. > > > > Anyway we can implement it technically and > > whether it's > > practical will be > > decided by the application. > > > > -- > > View this message in context: > > > > http://www.nabble.com/Load-balancing-for-live-streams-tf3926799.html#a111556 > > <http://www.nabble.com/Load-balancing-for-live-streams-tf3926799.html#a111556> > > > > <http://www.nabble.com/Load-balancing-for-live-streams-tf3926799.html#a111556> > > 12 > > Sent from the Red5 - English mailing list > > archive at > > Nabble.com <http://Nabble.com> <http://Nabble.com>. > > > > > > > _______________________________________________ > > Red5 mailing list > > [email protected] > <mailto:[email protected]> <mailto:[email protected] > <mailto:[email protected]>> > > > http://osflash.org/mailman/listinfo/red5_osflash.org > > > > > > > > > > > > -- > > I cannot tell why this heart languishes in > silence. It > > is for small > > needs it never asks, or knows or remembers. -- Tagore > > > > Best Regards > > Steven Gong > > > > _______________________________________________ > > Red5 mailing list > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > > http://osflash.org/mailman/listinfo/red5_osflash.org > > > > > > > > > > > > > > _______________________________________________ > > Red5 mailing list > > [email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > > http://osflash.org/mailman/listinfo/red5_osflash.org > <http://osflash.org/mailman/listinfo/red5_osflash.org> > > > > > > > > _______________________________________________ > > Red5 mailing list > > [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>> > > http://osflash.org/mailman/listinfo/red5_osflash.org > > > > > > > > > > -- > > I cannot tell why this heart languishes in silence. It is for small > > needs it never asks, or knows or remembers. -- Tagore > > > > Best Regards > > Steven Gong > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > Red5 mailing list > > [email protected] <mailto:[email protected]> > > http://osflash.org/mailman/listinfo/red5_osflash.org > > > > > _______________________________________________ > Red5 mailing list > [email protected] <mailto:[email protected]> > http://osflash.org/mailman/listinfo/red5_osflash.org > <http://osflash.org/mailman/listinfo/red5_osflash.org> > > > ------------------------------------------------------------------------ > > _______________________________________________ > Red5 mailing list > [email protected] > http://osflash.org/mailman/listinfo/red5_osflash.org > _______________________________________________ Red5 mailing list [email protected] http://osflash.org/mailman/listinfo/red5_osflash.org
