On 11 August 2010 02:01, Martin Storsjö <mar...@martin.st> wrote:
> On Wed, 11 Aug 2010, Josh Allmann wrote:
>
>> Attaching...
>
> The packetizer looks ok to me. Perhaps still adding a warning message at
> startup that the spec isn't finalized yet and may still change, as Luca A
> pointed out?
>
Done. The best place to print this was in sdp.c. Also added a similar
note when initializing the depacketizer.
I will take responsibility for maintaining this through spec updates,
even after gsoc. Hopefully it will be officially ratified as a rfc
soon enough.
> For the depacketizer:
>
>> +static int vp8_handle_packet(AVFormatContext *ctx,
>> + PayloadContext *vp8,
>> + AVStream *st,
>> + AVPacket *pkt,
>> + uint32_t *timestamp,
>> + const uint8_t *buf,
>> + int len, int flags)
>> +{
>> + int start_packet, end_packet, has_au;
>> + if (!buf)
>> + return AVERROR_INVALIDDATA;
>> +
>> + start_packet = *buf & 1;
>> + end_packet = flags & RTP_FLAG_MARKER;
>> + has_au = *buf & 2;
>> + buf++;
>> + len--;
>> +
>> + if (start_packet) {
>> + int res;
>> + if (vp8->data) { // drop previous frame if needed
>> + uint8_t *tmp;
>> + url_close_dyn_buf(vp8->data, &tmp);
>> + vp8->data = NULL;
>> + av_free(tmp);
>> + }
>
> Instead of dropping the previous frame, perhaps we should return the data
> we have buffered so far? If this contains the whole first data partition,
> we will be able to decode the rest of the GOP even if that particular
> frame is broken.
>
> If end_packet isn't set, this is relatively simple - close the previous
> dyn buffer and set the data into pkt, then buffer the new data into a new
> dyn buf. You also need to swap vp8->timestamp and *timestamp in that case
> - the timestamp for the data you've returned is in vp8->timestamp, which
> you need to set into *timestamp so that the returned packet gets the right
> timestamp, but for the new data you buffer up, you need to store the
> current *timestamp.
>
> If end_packet is set, we'd need to return two packets - both the previous,
> incomplete one, and the new one. In that case, you'd need to do something
> like what I described above, but return 1 so that the code outside knows
> to request another packet, then make the code handle the case when called
> with buf = NULL.
>
I think I addressed this in a sensible way. I haven't actually tested
this behavior, though.
>> + if ((res = url_open_dyn_buf(&vp8->data)) < 0)
>> + return res;
>> + vp8->is_keyframe = *buf & 1;
>> + vp8->timestamp = *timestamp;
>> + }
>> +
>> + if (!vp8->data || vp8->timestamp != *timestamp) {
>> + av_log(ctx, AV_LOG_WARNING,
>> + "Received no start marker; dropping frame\n");
>> + return AVERROR(EAGAIN);
>> + }
>
> If the timestamps differ and we actually have some old data buffered, do
> we want to return it here? Since the timestamps differ in that case, we
> wouldn't be interested in returning the currently processed data, though,
> since we have missed the start of that frame anyway. (On the other hand,
> in that case, we've already screwed up the GOP, so is there any point in
> returning the previous packet at all?)
>
In this case, we will have definitely lost the first data partition in
the frame, so I don't see the point in returning old data.
>> +
>> + // cycle through VP8AU headers if needed
>> + while (len) {
>> + int au_len = len;
>> + if (has_au && len > 2) {
>> + au_len = AV_RB16(buf) & 0x7ff; // use lower 11 bits
>> + buf += 2;
>> + len -= 2;
>> + if (buf + au_len > buf + len) {
>> + av_log(ctx, AV_LOG_ERROR, "Invalid VP8AU length\n");
>> + return AVERROR_INVALIDDATA;
>> + }
>> + }
>
> I'm not totally sure of the intent behind VP8 aggregate units - if each
> unit is a separate frame, we should split them here, I think. But from the
> discussion on the spec:
>
I don't either, apparently VP8AUs are for real time/low latency
communications. Error detection/correction is my best guess.
>> > If I understand correctly, it means that you can only aggregate
>> > packets from the same video frame ? If it is the case, it should be
>> > clearer. That said, can't they be just merged ?
>>
>> Yes. When people asked for having the ability to aggregate VP8 data with
>> same timestamp, I think they were looking to the future. For different
>> applications it could be advantageous.
>
> So in that case, merging the AUs like this probably is ok.
>
Also, from Section 1.1 of the proposal (timestamp part):
"There is at most one media sample or partial media sample per RTP packet."
Which implies that multiple VP8AUs in a packet must belong to the same frame.
> Perhaps a warning (either code comment or runtime warning) in this
> codepath in some way, that the VP8AU code is untested?
>
Fixed, put it in comments to avoid spamming the console in the event
we do encounter a stream with VP8AUs.
Also, I changed the VP8AU header to use the full 16 bits of the field
as length, since that seems to be the consensus on the list.
Josh
From 249bfd1cea5715f81f29486f1ee9c12ae849fc56 Mon Sep 17 00:00:00 2001
From: Josh Allmann <joshua.allm...@gmail.com>
Date: Wed, 28 Jul 2010 00:30:09 -0700
Subject: [PATCH 1/2] Add RTP packetization of VP8.
---
libavformat/rtpenc.c | 29 +++++++++++++++++++++++++++++
libavformat/sdp.c | 5 +++++
2 files changed, 34 insertions(+), 0 deletions(-)
diff --git a/libavformat/rtpenc.c b/libavformat/rtpenc.c
index 9335e8c..6caf220 100644
--- a/libavformat/rtpenc.c
+++ b/libavformat/rtpenc.c
@@ -55,6 +55,7 @@ static int is_supported(enum CodecID id)
case CODEC_ID_AMR_WB:
case CODEC_ID_VORBIS:
case CODEC_ID_THEORA:
+ case CODEC_ID_VP8:
return 1;
default:
return 0;
@@ -301,6 +302,31 @@ static void rtp_send_mpegaudio(AVFormatContext *s1,
}
}
+/* Based on a draft spec for VP8 RTP.
+ * (https://groups.google.com/a/webmproject.org/group/webm-discuss/msg/eef6d312ac5c09c3?dmode=source&output=gplain) */
+static void rtp_send_vp8(AVFormatContext *s1, const uint8_t *buf, int size)
+{
+ RTPMuxContext *s = s1->priv_data;
+ int len, max_packet_size;
+
+ s->buf_ptr = s->buf;
+ s->timestamp = s->cur_timestamp;
+ max_packet_size = s->max_payload_size - 1; // minus one for header byte
+
+ *s->buf_ptr++ = 1; // 0b1 indicates start of frame
+ while (size > 0) {
+ len = FFMIN(size, max_packet_size);
+
+ memcpy(s->buf_ptr, buf, len);
+ ff_rtp_send_data(s1, s->buf, len+1, size == len); // marker bit is last packet in frame
+
+ size -= len;
+ buf += len;
+ s->buf_ptr = s->buf;
+ *s->buf_ptr++ = 0; // payload descriptor
+ }
+}
+
static void rtp_send_raw(AVFormatContext *s1,
const uint8_t *buf1, int size)
{
@@ -407,6 +433,9 @@ static int rtp_write_packet(AVFormatContext *s1, AVPacket *pkt)
case CODEC_ID_THEORA:
ff_rtp_send_xiph(s1, pkt->data, size);
break;
+ case CODEC_ID_VP8:
+ rtp_send_vp8(s1, pkt->data, size);
+ break;
default:
/* better than nothing : send the codec raw data */
rtp_send_raw(s1, pkt->data, size);
diff --git a/libavformat/sdp.c b/libavformat/sdp.c
index ab5ef40..099591d 100644
--- a/libavformat/sdp.c
+++ b/libavformat/sdp.c
@@ -412,6 +412,11 @@ static char *sdp_write_media_attributes(char *buff, int size, AVCodecContext *c,
c->width, c->height, pix_fmt, config);
break;
}
+ case CODEC_ID_VP8:
+ av_log(c, AV_LOG_WARNING, "RTP VP8 payload is still experimental\n");
+ av_strlcatf(buff, size, "a=rtpmap:%d VP8/90000\r\n",
+ payload_type);
+ break;
default:
/* Nothing special to do here... */
break;
--
1.7.0.4
From 4a298fd7a2a5f1c822a6caeb89aa00c3b90907a6 Mon Sep 17 00:00:00 2001
From: Josh Allmann <joshua.allm...@gmail.com>
Date: Thu, 29 Jul 2010 03:51:36 -0700
Subject: [PATCH 2/2] Add RTP depacketization of VP8.
---
libavformat/Makefile | 1 +
libavformat/rtpdec.c | 1 +
libavformat/rtpdec_formats.h | 1 +
libavformat/rtpdec_vp8.c | 149 ++++++++++++++++++++++++++++++++++++++++++
4 files changed, 152 insertions(+), 0 deletions(-)
create mode 100644 libavformat/rtpdec_vp8.c
diff --git a/libavformat/Makefile b/libavformat/Makefile
index 16aa0c7..251c463 100644
--- a/libavformat/Makefile
+++ b/libavformat/Makefile
@@ -234,6 +234,7 @@ OBJS-$(CONFIG_SDP_DEMUXER) += rtsp.o \
rtpdec_mpeg4.o \
rtpdec_qdm2.o \
rtpdec_svq3.o \
+ rtpdec_vp8.o \
rtpdec_xiph.o
OBJS-$(CONFIG_SEGAFILM_DEMUXER) += segafilm.o
OBJS-$(CONFIG_SHORTEN_DEMUXER) += raw.o
diff --git a/libavformat/rtpdec.c b/libavformat/rtpdec.c
index c17dc2d..2c5243a 100644
--- a/libavformat/rtpdec.c
+++ b/libavformat/rtpdec.c
@@ -65,6 +65,7 @@ void av_register_rtp_dynamic_payload_handlers(void)
ff_register_dynamic_payload_handler(&ff_theora_dynamic_handler);
ff_register_dynamic_payload_handler(&ff_qdm2_dynamic_handler);
ff_register_dynamic_payload_handler(&ff_svq3_dynamic_handler);
+ ff_register_dynamic_payload_handler(&ff_vp8_dynamic_handler);
ff_register_dynamic_payload_handler(&ff_ms_rtp_asf_pfv_handler);
ff_register_dynamic_payload_handler(&ff_ms_rtp_asf_pfa_handler);
diff --git a/libavformat/rtpdec_formats.h b/libavformat/rtpdec_formats.h
index eac1b52..a96ffa6 100644
--- a/libavformat/rtpdec_formats.h
+++ b/libavformat/rtpdec_formats.h
@@ -44,5 +44,6 @@ extern RTPDynamicProtocolHandler ff_qdm2_dynamic_handler;
extern RTPDynamicProtocolHandler ff_svq3_dynamic_handler;
extern RTPDynamicProtocolHandler ff_theora_dynamic_handler;
extern RTPDynamicProtocolHandler ff_vorbis_dynamic_handler;
+extern RTPDynamicProtocolHandler ff_vp8_dynamic_handler;
#endif /* AVFORMAT_RTPDEC_FORMATS_H */
diff --git a/libavformat/rtpdec_vp8.c b/libavformat/rtpdec_vp8.c
new file mode 100644
index 0000000..536d9fa
--- /dev/null
+++ b/libavformat/rtpdec_vp8.c
@@ -0,0 +1,149 @@
+/*
+ * RTP VP8 Depacketizer
+ * Copyright (c) 2010 Josh Allmann
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+/**
+ * @file
+ * @brief RTP support for the VP8 payload
+ * @author Josh Allmann <joshua.allm...@gmail.com>
+ * (https://groups.google.com/a/webmproject.org/group/webm-discuss/msg/eef6d312ac5c09c3?dmode=source&output=gplain)
+ */
+
+#include "libavcodec/bytestream.h"
+
+#include "rtpdec_formats.h"
+
+struct PayloadContext {
+ ByteIOContext *data;
+ uint32_t timestamp;
+ int is_keyframe;
+};
+
+static void prepare_packet(AVPacket *pkt, PayloadContext *vp8, int stream)
+{
+ av_init_packet(pkt);
+ pkt->stream_index = stream;
+ pkt->flags = vp8->is_keyframe ? AV_PKT_FLAG_KEY : 0;
+ pkt->size = url_close_dyn_buf(vp8->data, &pkt->data);
+ pkt->destruct = av_destruct_packet;
+ vp8->data = NULL;
+}
+
+static int vp8_handle_packet(AVFormatContext *ctx,
+ PayloadContext *vp8,
+ AVStream *st,
+ AVPacket *pkt,
+ uint32_t *timestamp,
+ const uint8_t *buf,
+ int len, int flags)
+{
+ int start_packet, end_packet, has_au, ret = AVERROR(EAGAIN);
+
+ if (!buf) {
+ // only called when vp8_handle_packet returns 1
+ prepare_packet(pkt, vp8, st->index);
+ *timestamp = vp8->timestamp;
+ return 0;
+ }
+
+ start_packet = *buf & 1;
+ end_packet = flags & RTP_FLAG_MARKER;
+ has_au = *buf & 2;
+ buf++;
+ len--;
+
+ if (start_packet) {
+ int res;
+ uint32_t ts = *timestamp;
+ if (vp8->data) {
+ // missing end marker; return old frame anyway. untested
+ prepare_packet(pkt, vp8, st->index);
+ *timestamp = vp8->timestamp; // reset timestamp from old frame
+
+ // if current frame fits into one rtp packet, need to hold
+ // that for the next av_get_packet call
+ ret = end_packet ? 1 : 0;
+ }
+ if ((res = url_open_dyn_buf(&vp8->data)) < 0)
+ return res;
+ vp8->is_keyframe = *buf & 1;
+ vp8->timestamp = ts;
+ }
+
+ if (!vp8->data || vp8->timestamp != *timestamp && ret == AVERROR(EAGAIN)) {
+ av_log(ctx, AV_LOG_WARNING,
+ "Received no start marker; dropping frame\n");
+ return AVERROR(EAGAIN);
+ }
+
+ // cycle through VP8AU headers if needed
+ // not tested with actual VP8AUs
+ while (len) {
+ int au_len = len;
+ if (has_au && len > 2) {
+ au_len = AV_RB16(buf);
+ buf += 2;
+ len -= 2;
+ if (buf + au_len > buf + len) {
+ av_log(ctx, AV_LOG_ERROR, "Invalid VP8AU length\n");
+ return AVERROR_INVALIDDATA;
+ }
+ }
+
+ put_buffer(vp8->data, buf, au_len);
+ buf += au_len;
+ len -= au_len;
+ }
+
+ if (ret != AVERROR(EAGAIN)) // did we miss a end marker?
+ return ret;
+
+ if (end_packet) {
+ prepare_packet(pkt, vp8, st->index);
+ return 0;
+ }
+
+ return AVERROR(EAGAIN);
+}
+
+static PayloadContext *vp8_new_context(void)
+{
+ av_log(NULL, AV_LOG_WARNING, "RTP VP8 payload is still experimental\n");
+ return av_mallocz(sizeof(PayloadContext));
+}
+
+static void vp8_free_context(PayloadContext *vp8)
+{
+ if (vp8->data) {
+ uint8_t *tmp;
+ url_close_dyn_buf(vp8->data, &tmp);
+ av_free(tmp);
+ }
+ av_free(vp8);
+}
+
+RTPDynamicProtocolHandler ff_vp8_dynamic_handler = {
+ .enc_name = "VP8",
+ .codec_type = AVMEDIA_TYPE_VIDEO,
+ .codec_id = CODEC_ID_VP8,
+ .open = vp8_new_context,
+ .close = vp8_free_context,
+ .parse_packet = vp8_handle_packet,
+};
--
1.7.0.4
_______________________________________________
FFmpeg-soc mailing list
FFmpeg-soc@mplayerhq.hu
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-soc
