One quick point of clarification about GOP size....I think it is probably clear, but just in case it isn't: we are sending I-frames in our NAL 5s -- no differentials. The stream looks like this
[7][8][5][7][8][5][7][8][5]... Each NAL 5 is an i-frame, 23K each. I hope that clarifies. Brad On Mar 21, 2012, at 6:38 PM, Brad O'Hearne wrote: > Ross, > > Thanks for the reply. I appreciate the pointer to the additional information. > I did read it and followed its advice, but in a nutshell, I'm still > experiencing around 60% packet loss though doing very little processing in > the afterGettingFrame() method callback in my MediaSink subclass. The FAQ > makes the statement: > > "It's important to understand that because a LIVE555 Streaming Media > application runs as a single thread - never writing to, or reading from, > sockets concurrently - if packet loss: occurs, then it must be happening > either (i) on the network, or (ii) in the operating system of the sender or > receiver. There's nothing in our code that can be 'losing' packets." > > I do not dispute that, but I need to know more about the implications of > Live555 behavior, because even though it might not explicitly lose packets, > the implications of its approach may result in the same for any app actually > doing any processing. I hope there's something I'm missing, which can improve > performance. I'll explain. > > Earlier in the thread, I mentioned that the big picture of my use case is > receiving an H.264 video stream via RTSP with Live555, then decoding the > received data with ffmpeg, and displaying the resulting video frame to the > screen. I originally thought the data loss was occurring in the ffmpeg > decoding and rendering to the screen, until I removed decoding and rendering > entirely, and discovered the frame loss was experienced before even leaving > the afterGettingFrame method. Here's the specifics of what I did: > > 1. I followed the FAQ's recommendations, which increased the receiveBuffer to > (VLC's recommended) 2000000. (Note, I also tried 400000, same results as > 2000000, also note there are no truncated bytes). > > 2. I removed all ffmpeg decoding, rendering to the screen, and all processing > whatsoever from the afterGettingFrame() callback on the MediaSink subclass, > except for logging counts of frames received. Combined with the receiveBuffer > increase in 1, this resulted in receipt of 28-29fps, which tracks with the > 30fps being sent by the server. At this point, I was fairly certain that I > had the Live555 side of the processing pipeline figured out, and there was > nothing left to do. > > 3. <Abridged here, it has taken me days to discover this> I added back in the > minimal processing I was doing in the afterGettingFrame() callback of the > MediaSink subclass *but didn't add any decoding/displaying* -- all I am doing > is copying a 4-byte start code and and the data in the client buffer to > another buffer to send it downstream. Specifically these two chunks of code: > > // add the start code > len = 4; > memcpy((newFrame + offset), start_code, len); > offset += len; > > // add the buffer data > memcpy((newFrame + offset), fReceiveBuffer, frameSize); > > when the code above was commented out, I get no frame loss, around 28-29fps. > When the code is included, I only get 10-14fps, i.e. around 60% frame loss. > Note, nothing is being done in afterGettingFrame() or anywhere else with this > buffer...after this code above, continuePlaying() is called. The reason I put > this code in was for the sole purpose of gauging what effect if any this had > on frame loss. Clearly, it has a huge impact. This now probably warrants some > explanation of our stream being processed. > > Our H.264 stream is very simple. gop size is 1, each one with 3 sequential > NAL units being sent repeatedly: > > NAL 7 | NAL 8 | NAL 5 > > NAL units 7 and 8 are 4-9 bytes each whose general purpose is just to > communicate frame size. NAL unit 5 is consistently around 23K and is the > frame image data. So in reference to the code snippets above, when > afterGettingFrame() is called, I copy both a start code and the actual data > into a new buffer. (Now in actual processing I'll copy prop records info as > well, and then pass this on to ffmpeg, but this was all commented out for > this test). That memcpy is basically copying 15 bytes of data for the first > two afterGettingFrame calls and 23K on the third afterGettingFrame call for > what essentially is one video frame. > > This brings us to the heart of the matter: the only processing being done > here is just simply the creation of a new buffer. Why would this result in a > 60% frame loss? No decoding or displaying of any kind is even being > attempted. > > My assumption is that I'm missing or misunderstanding something fundamental > about Live555, and there's a knob or switch to be turned or flipped somewhere > that will solve the problem. Because as it is, I'm not seeing how any > processing at all of this stream is realistically feasible (even handing off > the processing) without losing over half the stream data. > > So I return to the statement in the FAQ -- if the culprit in frame loss is > essentially afterGettingFrame() processing, is there any metric or > quantification on whether any real processing (or even handing off of data) > can actually be done without prohibitive data loss? There's got to be a way > to receive a video stream and actually do something minimal with it without > losing so much data. > > Ross, yours (and anyone else's) expertise and ideas here are greatly > appreciated. > > Thanks, > > Brad > > Brad O'Hearne > Founder / Lead Developer > Big Hill Software LLC > http://www.bighillsoftware.com > > On Mar 10, 2012, at 11:08 PM, Ross Finlayson wrote: > >>> 1. What factors / culprits could be causing this massive frame loss? >>> >>> 2. Is there anything in the base RTSPClient or MediaSink subclasses (or >>> elsewhere) that can be configured to reduce frame loss? >> >> See <http://www.live555.com/liveMedia/faq.html#packet-loss> >> >> You should also, of course, make sure that your receiver ("MediaSink" >> subclass)'s buffer is big enough for the incoming NAL units. (You can do >> this by checking whether "numTruncatedBytes" is ever >0.) >> >> >> Ross Finlayson >> Live Networks, Inc. >> http://www.live555.com/ >> >> _______________________________________________ >> live-devel mailing list >> [email protected] >> http://lists.live555.com/mailman/listinfo/live-devel > > _______________________________________________ > live-devel mailing list > [email protected] > http://lists.live555.com/mailman/listinfo/live-devel
_______________________________________________ live-devel mailing list [email protected] http://lists.live555.com/mailman/listinfo/live-devel
