Ross,
Thanks for the reply. I appreciate the pointer to the additional information. I
did read it and followed its advice, but in a nutshell, I'm still experiencing
around 60% packet loss though doing very little processing in the
afterGettingFrame() method callback in my MediaSink subclass. The FAQ makes the
statement:
"It's important to understand that because a LIVE555 Streaming Media
application runs as a single thread - never writing to, or reading from,
sockets concurrently - if packet loss occurs, then it must be happening either
(i) on the network, or (ii) in the operating system of the sender or receiver.
There's nothing in our code that can be 'losing' packets."
I do not dispute that, but I need to know more about the implications of
Live555 behavior, because even though it might not explicitly lose packets, the
implications of its approach may result in the same for any app actually doing
any processing. I hope there's something I'm missing, which can improve
performance. I'll explain.
Earlier in the thread, I mentioned that the big picture of my use case is
receiving an H.264 video stream via RTSP with Live555, then decoding the
received data with ffmpeg, and displaying the resulting video frame to the
screen. I originally thought the data loss was occurring in the ffmpeg decoding
and rendering to the screen, until I removed decoding and rendering entirely,
and discovered the frame loss was experienced before even leaving the
afterGettingFrame method. Here's the specifics of what I did:
1. I followed the FAQ's recommendations, which increased the receiveBuffer to
(VLC's recommended) 2000000. (Note, I also tried 400000, same results as
2000000, also note there are no truncated bytes).
2. I removed all ffmpeg decoding, rendering to the screen, and all processing
whatsoever from the afterGettingFrame() callback on the MediaSink subclass,
except for logging counts of frames received. Combined with the receiveBuffer
increase in 1, this resulted in receipt of 28-29fps, which tracks with the
30fps being sent by the server. At this point, I was fairly certain that I had
the Live555 side of the processing pipeline figured out, and there was nothing
left to do.
3. <Abridged here, it has taken me days to discover this> I added back in the
minimal processing I was doing in the afterGettingFrame() callback of the
MediaSink subclass *but didn't add any decoding/displaying* -- all I am doing
is copying a 4-byte start code and and the data in the client buffer to another
buffer to send it downstream. Specifically these two chunks of code:
// add the start code
len = 4;
memcpy((newFrame + offset), start_code, len);
offset += len;
// add the buffer data
memcpy((newFrame + offset), fReceiveBuffer, frameSize);
when the code above was commented out, I get no frame loss, around 28-29fps.
When the code is included, I only get 10-14fps, i.e. around 60% frame loss.
Note, nothing is being done in afterGettingFrame() or anywhere else with this
buffer...after this code above, continuePlaying() is called. The reason I put
this code in was for the sole purpose of gauging what effect if any this had on
frame loss. Clearly, it has a huge impact. This now probably warrants some
explanation of our stream being processed.
Our H.264 stream is very simple. gop size is 1, each one with 3 sequential NAL
units being sent repeatedly:
NAL 7 | NAL 8 | NAL 5
NAL units 7 and 8 are 4-9 bytes each whose general purpose is just to
communicate frame size. NAL unit 5 is consistently around 23K and is the frame
image data. So in reference to the code snippets above, when
afterGettingFrame() is called, I copy both a start code and the actual data
into a new buffer. (Now in actual processing I'll copy prop records info as
well, and then pass this on to ffmpeg, but this was all commented out for this
test). That memcpy is basically copying 15 bytes of data for the first two
afterGettingFrame calls and 23K on the third afterGettingFrame call for what
essentially is one video frame.
This brings us to the heart of the matter: the only processing being done here
is just simply the creation of a new buffer. Why would this result in a 60%
frame loss? No decoding or displaying of any kind is even being attempted.
My assumption is that I'm missing or misunderstanding something fundamental
about Live555, and there's a knob or switch to be turned or flipped somewhere
that will solve the problem. Because as it is, I'm not seeing how any
processing at all of this stream is realistically feasible (even handing off
the processing) without losing over half the stream data.
So I return to the statement in the FAQ -- if the culprit in frame loss is
essentially afterGettingFrame() processing, is there any metric or
quantification on whether any real processing (or even handing off of data) can
actually be done without prohibitive data loss? There's got to be a way to
receive a video stream and actually do something minimal with it without losing
so much data.
Ross, yours (and anyone else's) expertise and ideas here are greatly
appreciated.
Thanks,
Brad
Brad O'Hearne
Founder / Lead Developer
Big Hill Software LLC
http://www.bighillsoftware.com
On Mar 10, 2012, at 11:08 PM, Ross Finlayson wrote:
>> 1. What factors / culprits could be causing this massive frame loss?
>>
>> 2. Is there anything in the base RTSPClient or MediaSink subclasses (or
>> elsewhere) that can be configured to reduce frame loss?
>
> See <http://www.live555.com/liveMedia/faq.html#packet-loss>
>
> You should also, of course, make sure that your receiver ("MediaSink"
> subclass)'s buffer is big enough for the incoming NAL units. (You can do
> this by checking whether "numTruncatedBytes" is ever >0.)
>
>
> Ross Finlayson
> Live Networks, Inc.
> http://www.live555.com/
>
> _______________________________________________
> live-devel mailing list
> [email protected]
> http://lists.live555.com/mailman/listinfo/live-devel
_______________________________________________
live-devel mailing list
[email protected]
http://lists.live555.com/mailman/listinfo/live-devel
