Re: [Linux-uvc-devel] test results and a patch for timestamps in UVC
Hi Yann, On Tuesday 14 June 2011 12:02:09 Yann Sionneau wrote: Hi Alexey, Thanks a lot for your testing and report ! :) Your test and mine show that webcams seem to be implementing the UVC spec correctly regarding timestamping. I mean : (*) Putting PTS for each packet (*) Putting SCR for each packet (*) Having the same PTS for each packet of the same frame (*) Having different SCR for each packet of the same frame Well, about FID/EOF bugs, I guess a change of PTS can be interpreted as the beginning of a new frame, so it may be useful in case of a webcam having troubles signaling new frames using the usual FID/EOF bits. I am new to this (UVC), so you may need to confirm with someone else (Laurent ?) what I just wrote, but to me it makes sense to interpret a change of PTS as a new frame. That could be done, provided that the camera sends correct PTS values. Reading the code I noticed a few quirks used to workaround webcams not implementing correctly the UVC spec regarding FID and EOF bits. Are there webcams which have both problems ? (not making FID toggle, and not specifying EOF ?) Not that I know of. -- Regards, Laurent Pinchart ___ Linux-uvc-devel mailing list Linux-uvc-devel@lists.berlios.de https://lists.berlios.de/mailman/listinfo/linux-uvc-devel
Re: [Linux-uvc-devel] test results and a patch for timestamps in UVC
Hi, please use uvc_trace(UVC_TRACE_DESCR, or pr_debug instead of printk. So you can enable verbosity only if you need. On Di, 2011-06-07 at 17:56 +0200, Yann Sionneau wrote: Hi again, here is as attached file a new patch that applies to linux 2.6.39 tree (linux-2.6.git, tag v2.6.39). This patch now prints at the end of each stream : (*) total number of PTS (taking into account each packet) (*) total number of SCR (*) total number of ERR (*) number of packets without PTS (nb_missing_pts) (*) number of packets without SCR (nb_missing_scr) It will print as well for each packet (beware, it will hugely flood your syslog) : (*) SCR (*) diff with previous packet's SCR (*) PTS (*) diff with previous packet's PTS Beware, to see the end of stream statistics, you have to scroll up a little bit, it will be somewhere in the end of the per-packet information flood. Some results inline in the e-mail. On 06/06/2011 11:22 PM, Yann Sionneau wrote: Le 06/06/11 19:42, Laurent Pinchart a écrit : Hi Yann, Thanks for the patch. On Friday 03 June 2011 15:48:59 Yann Sionneau wrote: Hi Laurent and the UVC list, Here is a patch [snip] You should take all UVC packets into account, not just the first one for each frame. Yes I don't know why in my head I thought only the first uvc packet would have a header for the frame with timestamps and such, but it makes no sense, each packet has its own header and the start of frame is just signaled by the toggling of the FID :) the important information are - do all frames have a PTS timestamp in their first packet ? It seems so yes, at least on the two webcams I tried with the patch. - do all non-empty packets for a frame have a PTS timestamp, and is it constant through the whole frame as it should be ? Let's modify the patch to count the PTS and SCR of all packets instead of only one per frame in order to sort this out ! I tested with the Logitech HD Pro C910, all packets have a PTS. PTS are indeed constant through the whole frame as it should be. - how many SCR timestamps do we have per frame ? are they constant through the whole frame or do they vary as they should ? Same here ! With the same webcam, all packets have a SCR. The SCR value does vary through each frame, as it should do. The difference between the previous SCR is usually the same, except at a regular interval where there is a jump in SCR values. for example, SCR - SCR_prev would be 7 times the same difference, and then it will change for just one time, and then go back to the old difference for 7 times etc. [snip] Thanks for your review and comments, will submit a new patch ASAP ! -- Regards, Alexey ___ Linux-uvc-devel mailing list Linux-uvc-devel@lists.berlios.de https://lists.berlios.de/mailman/listinfo/linux-uvc-devel
Re: [Linux-uvc-devel] test results and a patch for timestamps in UVC
Hi again,
here is as attached file a new patch that applies to linux 2.6.39 tree
(linux-2.6.git, tag v2.6.39).
This patch now prints at the end of each stream :
(*) total number of PTS (taking into account each packet)
(*) total number of SCR
(*) total number of ERR
(*) number of packets without PTS (nb_missing_pts)
(*) number of packets without SCR (nb_missing_scr)
It will print as well for each packet (beware, it will hugely flood your
syslog) :
(*) SCR
(*) diff with previous packet's SCR
(*) PTS
(*) diff with previous packet's PTS
Beware, to see the end of stream statistics, you have to scroll up a
little bit, it will be somewhere in the end of the per-packet
information flood.
Some results inline in the e-mail.
On 06/06/2011 11:22 PM, Yann Sionneau wrote:
Le 06/06/11 19:42, Laurent Pinchart a écrit :
Hi Yann,
Thanks for the patch.
On Friday 03 June 2011 15:48:59 Yann Sionneau wrote:
Hi Laurent and the UVC list,
Here is a patch [snip]
You should take all UVC packets into account, not just the first one for each
frame.
Yes I don't know why in my head I thought only the first uvc packet
would have a header for the frame with timestamps and such, but it makes
no sense, each packet has its own header and the start of frame is just
signaled by the toggling of the FID :)
the important information are
- do all frames have a PTS timestamp in their first packet ?
It seems so yes, at least on the two webcams I tried with the patch.
- do all non-empty packets for a frame have a PTS timestamp, and is it
constant through the whole frame as it should be ?
Let's modify the patch to count the PTS and SCR of all packets instead
of only one per frame in order to sort this out !
I tested with the Logitech HD Pro C910, all packets have a PTS.
PTS are indeed constant through the whole frame as it should be.
- how many SCR timestamps do we have per frame ? are they constant through the
whole frame or do they vary as they should ?
Same here !
With the same webcam, all packets have a SCR.
The SCR value does vary through each frame, as it should do.
The difference between the previous SCR is usually the same, except at a
regular interval where there is a jump in SCR values.
for example, SCR - SCR_prev would be 7 times the same difference, and
then it will change for just one time, and then go back to the old
difference for 7 times etc.
[snip]
Thanks for your review and comments, will submit a new patch ASAP !
Regards,
Regards,
--
Yann Sionneau
From 1a2adf814a3a55181c46a4bef8ba8ec18f7b492c Mon Sep 17 00:00:00 2001
From: Yann Sionneau y...@minet.net
Date: Tue, 7 Jun 2011 14:19:25 +0200
Subject: [PATCH] add statistics about ERR, PTS and SCR bits
---
drivers/media/video/uvc/uvc_queue.c |5 +++
drivers/media/video/uvc/uvc_video.c | 64 +++
drivers/media/video/uvc/uvcvideo.h | 11 ++
3 files changed, 80 insertions(+), 0 deletions(-)
diff --git a/drivers/media/video/uvc/uvc_queue.c b/drivers/media/video/uvc/uvc_queue.c
index f14581b..2ca2898 100644
--- a/drivers/media/video/uvc/uvc_queue.c
+++ b/drivers/media/video/uvc/uvc_queue.c
@@ -372,6 +372,11 @@ int uvc_dequeue_buffer(struct uvc_video_queue *queue,
}
list_del(buf-stream);
+
+ printk(timestamp == (%ld : %ld)\n,
+ queue-buffer[v4l2_buf-index].buf.timestamp.tv_sec,
+ queue-buffer[v4l2_buf-index].buf.timestamp.tv_usec);
+
__uvc_query_buffer(buf, v4l2_buf);
done:
diff --git a/drivers/media/video/uvc/uvc_video.c b/drivers/media/video/uvc/uvc_video.c
index fc766b9..635708b 100644
--- a/drivers/media/video/uvc/uvc_video.c
+++ b/drivers/media/video/uvc/uvc_video.c
@@ -410,6 +410,7 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
struct uvc_buffer *buf, const __u8 *data, int len)
{
__u8 fid;
+ bool debug_printed = false;
/* Sanity checks:
* - packet must be at least 2 bytes long
@@ -471,10 +472,55 @@ static int uvc_video_decode_start(struct uvc_streaming *stream,
buf-buf.timestamp.tv_sec = ts.tv_sec;
buf-buf.timestamp.tv_usec = ts.tv_nsec / NSEC_PER_USEC;
+ printk([uvcvideo] A new frame starts\n);
+
+ /* reset the number of packets per frame
+ * because we start a new frame
+ */
+ stream-stats.nb_packets = 0;
+
/* TODO: Handle PTS and SCR. */
buf-state = UVC_BUF_STATE_ACTIVE;
}
+ /* inc the number of packets of the current frame */
+ ++(stream-stats.nb_packets);
+
+ debug_printed = (data[1] (UVC_STREAM_PTS | UVC_STREAM_SCR)) ?
+ true : false;
+
+ if (debug_printed)
+ printk([uvcvideo] frame %lu ,
+ (unsigned long)stream-sequence);
+
+ if (data[1] UVC_STREAM_PTS) {
+ __u32 pts;
+ ++(stream-stats.nb_pts);
+ pts = *((uint32_t *)(data + 2));
+ printk(PTS : %lu, diff with prev PTS : %lu ,
+ (unsigned long)pts,
+ (unsigned long)(pts - stream-stats.last_pts));
+ stream-stats.last_pts = pts;
+ } else
+ ++(stream-stats.nb_missing_pts);
+
+ if (data[1] UVC_STREAM_SCR) {
+ __u64 scr;
+ ++(stream-stats.nb_scr);
+
Re: [Linux-uvc-devel] test results and a patch for timestamps in UVC
Hi Yann, Thanks for the patch. On Friday 03 June 2011 15:48:59 Yann Sionneau wrote: Hi Laurent and the UVC list, Here is a patch (that applies against the linux-2.6.39 vanilla tree) that shows the number of frames whose header (the one described at section 2.4.3.3 of the document USB Device Class Definition for Video Devices) contains the SCR, PTS and ERR bits. The patch is attached in the e-mail. The patch works as follow : (*) Apply the patch (*) Compile the patched UVC module (*) Unload the previous module and load the new one (*) Start a video capture (*) Upon stop of the video capture (ioctl STREAMOFF) it will print statistics about the stream in the dmesg (/var/log/syslog) It will show the number of frames whose header contains the bits SCR, PTS and ERR. It will show the total number of frames of the stream as well (the stream-sequence). You should take all UVC packets into account, not just the first one for each frame. the important information are - do all frames have a PTS timestamp in their first packet ? - do all non-empty packets for a frame have a PTS timestamp, and is it constant through the whole frame as it should be ? - how many SCR timestamps do we have per frame ? are they constant through the whole frame or do they vary as they should ? I have tested this patch with 2 webcams so far, one is the Logitech HD Pro C910, the other is the Freetalk skype webcam made by Tatung, model Talk 7190, board R00C, firmware SDK8RC26. The second one is featuring a MG3500 chip from Maxim IC, it does hardware encoding to stream H.264 codec inside a MPEG-2 TS container. During my test I used it without any hardware encoding, it was doing normal raw YUV 4:2:2 streaming. I noticed both webcams are putting PTS and SCR bits on all (or all but one, not sure) frames. Moreover no frame got the ERR bit on (good :)). Here are two examples of what the patch should output : Jun 3 11:58:43 obiwan kernel: [180779.218667] [uvcvideo] stats about previous stream : Jun 3 11:58:43 obiwan kernel: [180779.218672] nb_pts = 140 ; nb_scr = 140 ; nb_err = 0 over 139 frames and Jun 3 11:46:33 obiwan kernel: [180050.637465] [uvcvideo] stats about previous stream : Jun 3 11:46:33 obiwan kernel: [180050.637471] nb_pts = 258 ; nb_scr = 258 ; nb_err = 0 over 257 frames I don't know why I have one extra frame in my pts, scr and err counters than in the stream-sequence. I used guvcview -d /dev/video0 to start the stream, and closed the GTK window to stop the stream. Feel free to test this patch with your webcams are report back your dmesg output (with framerate and the duration of the test please). This test is the first step of a bigger goal : implementing a correct timestamp support. Regards, -- Regards, Laurent Pinchart ___ Linux-uvc-devel mailing list Linux-uvc-devel@lists.berlios.de https://lists.berlios.de/mailman/listinfo/linux-uvc-devel
Re: [Linux-uvc-devel] test results and a patch for timestamps in UVC
Le 06/06/11 19:42, Laurent Pinchart a écrit : Hi Yann, Thanks for the patch. On Friday 03 June 2011 15:48:59 Yann Sionneau wrote: Hi Laurent and the UVC list, Here is a patch (that applies against the linux-2.6.39 vanilla tree) that shows the number of frames whose header (the one described at section 2.4.3.3 of the document USB Device Class Definition for Video Devices) contains the SCR, PTS and ERR bits. The patch is attached in the e-mail. The patch works as follow : (*) Apply the patch (*) Compile the patched UVC module (*) Unload the previous module and load the new one (*) Start a video capture (*) Upon stop of the video capture (ioctl STREAMOFF) it will print statistics about the stream in the dmesg (/var/log/syslog) It will show the number of frames whose header contains the bits SCR, PTS and ERR. It will show the total number of frames of the stream as well (the stream-sequence). You should take all UVC packets into account, not just the first one for each frame. Yes I don't know why in my head I thought only the first uvc packet would have a header for the frame with timestamps and such, but it makes no sense, each packet has its own header and the start of frame is just signaled by the toggling of the FID :) the important information are - do all frames have a PTS timestamp in their first packet ? It seems so yes, at least on the two webcams I tried with the patch. - do all non-empty packets for a frame have a PTS timestamp, and is it constant through the whole frame as it should be ? Let's modify the patch to count the PTS and SCR of all packets instead of only one per frame in order to sort this out ! - how many SCR timestamps do we have per frame ? are they constant through the whole frame or do they vary as they should ? Same here ! I have tested this patch with 2 webcams so far, one is the Logitech HD Pro C910, the other is the Freetalk skype webcam made by Tatung, model Talk 7190, board R00C, firmware SDK8RC26. The second one is featuring a MG3500 chip from Maxim IC, it does hardware encoding to stream H.264 codec inside a MPEG-2 TS container. During my test I used it without any hardware encoding, it was doing normal raw YUV 4:2:2 streaming. I noticed both webcams are putting PTS and SCR bits on all (or all but one, not sure) frames. Moreover no frame got the ERR bit on (good :)). Here are two examples of what the patch should output : Jun 3 11:58:43 obiwan kernel: [180779.218667] [uvcvideo] stats about previous stream : Jun 3 11:58:43 obiwan kernel: [180779.218672] nb_pts = 140 ; nb_scr = 140 ; nb_err = 0 over 139 frames and Jun 3 11:46:33 obiwan kernel: [180050.637465] [uvcvideo] stats about previous stream : Jun 3 11:46:33 obiwan kernel: [180050.637471] nb_pts = 258 ; nb_scr = 258 ; nb_err = 0 over 257 frames I don't know why I have one extra frame in my pts, scr and err counters than in the stream-sequence. I used guvcview -d /dev/video0 to start the stream, and closed the GTK window to stop the stream. Feel free to test this patch with your webcams are report back your dmesg output (with framerate and the duration of the test please). This test is the first step of a bigger goal : implementing a correct timestamp support. Regards, Thanks for your review and comments, will submit a new patch ASAP ! Regards, -- Yann Sionneau ___ Linux-uvc-devel mailing list Linux-uvc-devel@lists.berlios.de https://lists.berlios.de/mailman/listinfo/linux-uvc-devel
