Re: [vdr] [PATCH] Auto sized ringbuffers v3
VDR User wrote: I've recorded/watched a lot of tv in the last couple days and so far I haven't noticed any problems with this patch. I am currently using v3 (the patch I attached to my previous post) with vdr-1.5.2. Do you know how large the buffers usually grow? If you haven't reduced vdrs log level (-l) this info should be somewhere in /var/log/ and you can take a look at the buffer usage w/ eg: $ grep vdr /var/log/* | grep 'ring buffer' | less Did any overflows happen (you can search for 'overflow' in the output of above commands to find them)? I've now checked my vdr logs here (starting ~ a week ago) and not a single buffer overflowed. I would have expected a few problems, but it looks like the tuning worked. (However that vdr has all data moving threads running at real time priority and this may have helped too.) Thanks, artur ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
On 5/20/07, Artur Skawina [EMAIL PROTECTED] wrote: VDR User wrote: I've recorded/watched a lot of tv in the last couple days and so far I haven't noticed any problems with this patch. I am currently using v3 (the patch I attached to my previous post) with vdr-1.5.2. Do you know how large the buffers usually grow? If you haven't reduced vdrs log level (-l) this info should be somewhere in /var/log/ and you can take a look at the buffer usage w/ eg: I use level 3 logging for vdr. I emailed you a copy of my logs so you should see that by the time you read this. ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
VDR User wrote: On 5/20/07, Artur Skawina [EMAIL PROTECTED] wrote: VDR User wrote: I've recorded/watched a lot of tv in the last couple days and so far I haven't noticed any problems with this patch. I am currently using v3 (the patch I attached to my previous post) with vdr-1.5.2. Do you know how large the buffers usually grow? If you haven't reduced vdrs log level (-l) this info should be somewhere in /var/log/ and you can take a look at the buffer usage w/ eg: I use level 3 logging for vdr. I emailed you a copy of my logs so you should see that by the time you read this. Thanks. It seems that 648k was the largest size ever used for a buffer, which is less than 1/3 of the full size (2M for Result,Transfer and TS). The Recorder buffer was typically at 5% of its max size, ie 300k instead of 5M. No overflows w/ v3 of the patch (two with the previous version, which was starting out with just 128k). I get a lot worse results here (buffers grow to a few M), but that's probably because the vdr box is not a dedicated one, but also handles quite a few other services; IO stalls are not uncommon and several simultaneous recordings are sometimes done while streaming via streamdev to another vdr. Still, no overflows. artur ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
On 5/20/07, Artur Skawina [EMAIL PROTECTED] wrote: Thanks. It seems that 648k was the largest size ever used for a buffer, which is less than 1/3 of the full size (2M for Result,Transfer and TS). The Recorder buffer was typically at 5% of its max size, ie 300k instead of 5M. No overflows w/ v3 of the patch (two with the previous version, which was starting out with just 128k). I get a lot worse results here (buffers grow to a few M), but that's probably because the vdr box is not a dedicated one, but also handles quite a few other services; IO stalls are not uncommon and several simultaneous recordings are sometimes done while streaming via streamdev to another vdr. Still, no overflows. My vdr box is dedicated with the only other software running being an ftpd for my lan. No streaming video or anything like that. One dedicated vdr box connected to one tv in my living room, thats it. :) ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
I just wanted to give some feedback on this, im using the patch on my vdr 1.4.7 and it workes without problems for now. Good work, thanks. Dortje In case anyone is having trouble, here's an attached patch of v3 for vdr-1.5.2. ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
I've recorded/watched a lot of tv in the last couple days and so far I haven't noticed any problems with this patch. I am currently using v3 (the patch I attached to my previous post) with vdr-1.5.2. ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
2007/5/15, VDR User [EMAIL PROTECTED]: In case anyone is having trouble, here's an attached patch of v3 for vdr-1.5.2. You mean 1.4 :) Thanks. -- Jaroslaw Swierczynski [EMAIL PROTECTED] www.archlinux.org | www.juvepoland.com ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
Jaroslaw Swierczynski wrote: It seems your patch is against VDR 1.5.x. Could you please backport it to VDR 1.4.x? actually it's vs 1.4.6, but applies to 1.5.2 too. ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
2007/5/15, VDR User [EMAIL PROTECTED]: No, I mean vdr-1.5.2. Artur said he had attached a patch to his post but there was no attachment and since copy paste the patch from his post was causing problems for some reason, I patched a vanilla vdr-1.5.2 (which is what I use) by hand. If it applies to any 1.4.x then I guess that's just a bonus. Sorry, I was confused by the fact that the patch didn't apply to 1.4.7 so I assumed it was against 1.5.x as I didn't think differences between 1.4.x releases were so significant. Artur has cleared up the issue stating he had made the patch with 1.4.6. However now when I'm thinking about it perhaps copypaste was the problem. -- Jaroslaw Swierczynski [EMAIL PROTECTED] www.archlinux.org | www.juvepoland.com ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
It seems your patch is against VDR 1.5.x. Could you please backport it to VDR 1.4.x? -- Jaroslaw Swierczynski [EMAIL PROTECTED] www.archlinux.org | www.juvepoland.com ___ vdr mailing list vdr@linuxtv.org http://www.linuxtv.org/cgi-bin/mailman/listinfo/vdr
Re: [vdr] [PATCH] Auto sized ringbuffers v3
In case anyone is having trouble, here's an attached patch of v3 for vdr-1.5.2.
diff -ruN vdr-1.5.2-orig/dvbdevice.c vdr-1.5.2/dvbdevice.c
--- vdr-1.5.2-orig/dvbdevice.c 2007-05-14 14:58:29.0 -0700
+++ vdr-1.5.2/dvbdevice.c 2007-05-14 15:22:50.0 -0700
@@ -1172,7 +1172,7 @@
CloseDvr();
fd_dvr = DvbOpen(DEV_DVB_DVR, CardIndex(), O_RDONLY | O_NONBLOCK, true);
if (fd_dvr = 0)
- tsBuffer = new cTSBuffer(fd_dvr, MEGABYTE(2), CardIndex() + 1);
+ tsBuffer = new cTSBuffer(fd_dvr, MEGABYTE(8), CardIndex() + 1);
return fd_dvr = 0;
}
diff -ruN vdr-1.5.2-orig/recorder.c vdr-1.5.2/recorder.c
--- vdr-1.5.2-orig/recorder.c 2007-05-14 14:58:29.0 -0700
+++ vdr-1.5.2/recorder.c 2007-05-14 15:27:12.0 -0700
@@ -176,8 +176,22 @@
int Count = remux-Put(b, r);
if (Count)
ringBuffer-Del(Count);
- else
- cCondWait::SleepMs(100); // avoid busy loop when resultBuffer is full in cRemux::Put()
- }
+ else { // avoid busy loop when resultBuffer is full in cRemux::Put()
+ int ms;
+ for (ms=5; Running() ms100; ms+=ms) {
+ cCondWait::SleepMs(ms);
+ if (!Running())
+ return;
+ Count = remux-Put(b, r);
+ if (Count) {
+ ringBuffer-Del(Count);
+ dsyslog(cRecorder::Action() ring buffer consumer slept %d ms, ms);
+ break;
+ }
+ }
+ if (ms=100)
+ dsyslog(cRecorder::Action() ring buffer consumer slept 100 ms);
+ }
+ }
}
}
diff -ruN vdr-1.5.2-orig/remux.c vdr-1.5.2/remux.c
--- vdr-1.5.2-orig/remux.c 2007-05-14 14:58:29.0 -0700
+++ vdr-1.5.2/remux.c 2007-05-14 15:27:45.0 -0700
@@ -1851,7 +1851,7 @@
// --- cRemux
-#define RESULTBUFFERSIZE KILOBYTE(256)
+#define RESULTBUFFERSIZE MEGABYTE(2)
cRemux::cRemux(int VPid, const int *APids, const int *DPids, const int *SPids, bool ExitOnFailure)
{
diff -ruN vdr-1.5.2-orig/ringbuffer.c vdr-1.5.2/ringbuffer.c
--- vdr-1.5.2-orig/ringbuffer.c 2007-05-14 14:58:29.0 -0700
+++ vdr-1.5.2/ringbuffer.c 2007-05-14 15:33:18.0 -0700
@@ -151,13 +151,31 @@
}
#endif
+// cRingBufferLinear are dynamically sized, or at least we can pretend they are ;)
+// We treat 'Size' as the maximum size, but start with a small buffer, which can
+// grow later as it fills up. Memory is always allocated for the full buffer, but
+// the unused RAM portion remains untouched until (if at all) it is actually needed.
+// Note that we can not start with a larger than requested buffer because there are
+// ring buffer users that cause crashes then (eg softdevice mpegdecoder absolutely
+// needs 32/64k).
+
+// Startup size. 64k still causes overflows before buffer starts to grow, 128k doesn't.
+// The buffers grow to 200..500k anyway, so maybe increasing this a bit more would
+// make sense, but let's first see if it's really needed.
+// In fact 128k is on the low side, but let's try not going for 256k yet.
+#define DEFRBSIZE KILOBYTE(192)
+
cRingBufferLinear::cRingBufferLinear(int Size, int Margin, bool Statistics, const char *Description)
-:cRingBuffer(Size, Statistics)
+:cRingBuffer(min(Size,DEFRBSIZE), Statistics)
{
description = Description ? strdup(Description) : NULL;
tail = head = margin = Margin;
gotten = 0;
buffer = NULL;
+ maxsize = Size;
+ growthresh = sizemaxsize ? size/3 : maxsize+1;
+ growbuf = 0;
+ dsyslog(New ring buffer \%s\ size: %d margin: %d, description, Size, Margin );
if (Size 1) { // 'Size - 1' must not be 0!
if (Margin = Size / 2) {
buffer = MALLOC(uchar, Size);
@@ -183,6 +201,7 @@
#ifdef DEBUGRINGBUFFERS
DelDebugRBL(this);
#endif
+ dsyslog(Deleting ring buffer \%s\ size: %d / %d (used %g%% of requested size), description, size, maxsize, size/(double)maxsize*100.0 );
free(buffer);
free(description);
}
@@ -205,8 +224,20 @@
EnablePut();
}
+// Must only be called by the producer (ie Read()/Put()), not the consumer (Get()).
+void cRingBufferLinear::GrowBuffer(void)
+{
+ size = min(size+size/2, maxsize);
+ growthresh = sizemaxsize ? size/3 : maxsize+1;
+ dsyslog(Enlarging ring buffer \%s\: %d bytes (trigger %d, thresh %d),
+ description, size, growbuf, growthresh);
+ growbuf = 0;
+}
+
int cRingBufferLinear::Read(int FileHandle, int Max)
{
+ if (growbuf head=tail sizemaxsize)
+ GrowBuffer();
int Tail = tail;
int diff = Tail - head;
int free = (diff 0) ? diff - 1 : Size() - head;
@@ -219,6 +250,10 @@
Count = safe_read(FileHandle, buffer + head, free);
if (Count 0) {
int Head = head + Count;
+if (Available()+Count = growthresh)
+ growbuf = 2;
+if (growbuf head=tail sizemaxsize)
+ GrowBuffer();
if (Head = Size())
Head = margin;
head = Head;
@@ -245,6 +280,10 @@
int cRingBufferLinear::Put(const uchar *Data, int Count)
{
